Through the years, mining and beneficiation processes in Panasqueira Sn-W mine (Central Portugal) produced large amounts of As-rich mine wastes laid up in huge tailings and open-air impoundments (Barroca Grande and Rio tailings) that are the main source of pollution in the surrounding area once they are exposed to the weathering conditions leading to the formation of acid mine drainage (AMD) and consequently to the contamination of the surrounding environments, particularly soils. The active mine started the exploration during the nineteenth century. This study aims to look at the extension of the soil pollution due to mining activities and tailing erosion by combining data on the degree of soil contamination that allows a better understanding of the dynamics inherent to leaching, transport, and accumulation of some potential toxic elements in soil and their environmental relevance. Soil samples were collected in the surrounding soils of the mine, were digested in aqua regia, and were analyzed for 36 elements by inductively coupled plasma mass spectrometry (ICP-MS). Selected results are that (a) an association of elements like Ag, As, Bi, Cd, Cu, W, and Zn strongly correlated and controlled by the local sulfide mineralization geochemical signature was revealed; (b) the global area discloses significant concentrations of As, Bi, Cd, and W linked to the exchangeable and acid-soluble bearing phases; and (c) wind promotes the mechanical dispersion of the rejected materials, from the milled waste rocks and the mineral processing plant, with subsequent deposition on soils and waters. Arsenic- and sulfide-related heavy metals (such as Cu and Cd) are associated to the fine materials that are transported in suspension by surface waters or associated to the acidic waters, draining these sites and contaminating the local soils. Part of this fraction, especially for As, Cd, and Cu, is temporally retained in solid phases by precipitation of soluble secondary minerals (through the precipitation of hydrated metal sulfates) in warm, dry periods, but such minerals are easily dissolved during rainy periods. Climate is an important instability factor, and the hot and dry summers and cold, rainy, and windy winters in this region are physical phenomena that enhance the good receptivity of these soils to retain some of the metals present in the primary and also the secondary mineralogy. Considering the obtained results from both the sequential chemical extraction and the environmental risk assessment according to the risk assessment code, Ag, Cd, Cu, and Zn are classified with very high risk while As is classified with medium risk.