A large amount of excavated soils with low-level As contamination caused by civil construction projects is of great concern in Japan. This study investigated the chemical speciation and extractability of As in 24 soil samples from the sites affected and unaffected (naturally contaminated) by anthropogenic pollution. The results of As K-edge XANES demonstrated that naturally contaminated soils were grouped into two types: (i) soils containing FeAsS-like and As2S3-like species (ave. 53%, hereafter As-S species) and (ii) soils with no or minor As-S species (ave. 3%). Clear differences were found in As, Fe, and S fractionations by sequential extraction. From naturally contaminated soils enriched with As-S species, more than 50% of As was extracted in the oxidizable fraction. Arsenic was mainly recovered in the reducible fraction for naturally contaminated soils with no or minor As-S species and anthropogenically contaminated soils. The μ-XRF and μ-XAFS revealed that the naturally contaminated soils containing As-S species were abundant in pyrite framboids (∼20 μm in diameter) in which As occurred as multiple oxidation states. The results suggest that framboidal pyrite becomes a source of As in naturally contaminated soils after being excavated and exposed to the surface environment.