The chemical composition of the continental crust cannot be adequately explained by current models for its formation, because it is too rich in Ni and Cr compared to that which can be generated by any of the proposed mechanisms. Estimates of the crust composition are derived from average sediment, while crustal growth is ascribed to amalgamation of differentiated magmatic rocks at continental margins. Here we show that chemical weathering of Ni- and Cr-rich, undifferentiated ultramafic rock equivalent to ~1.3 wt% of today's continental crust compensates for low Ni and Cr in formation models of the continental crust. Ultramafic rock weathering produces a residual that is enriched in Ni and also silica. In the light of potentially large volumes of ultramafic rock and high atmospheric CO2 concentrations during the Archean, chemical weathering must therefore have played a major role in forming compositionally evolved components of the early Earth's crust.