The ancient rock record for Mars has long been at odds with climate modelling. The presence of valley networks, dendritic channels and deltas on ancient terrains points towards running water and fluvial erosion on early Mars1, but climate modelling indicates that long-term warm conditions were not sustainable2. Widespread phyllosilicates and other aqueous minerals on the Martian surface3-6 provide additional evidence that an early wet Martian climate resulted in surface weathering. Some of these phyllosilicates formed in subsurface crustal environments5, with no association with the Martian climate, while other phyllosilicate-rich outcrops exhibit layered morphologies and broad stratigraphies7 consistent with surface formation. Here, we develop a new geochemical model for early Mars to explain the formation of these clay-bearing rocks in warm and wet surface locations. We propose that sporadic, short-term warm and wet environments during a generally cold early Mars enabled phyllosilicate formation without requiring long-term warm and wet conditions. We conclude that Mg-rich clay-bearing rocks with lateral variations in mixed Fe/Mg smectite, chlorite, talc, serpentine and zeolite occurrences formed in subsurface hydrothermal environments, whereas dioctahedral (Al/Fe3+-rich) smectite and widespread vertical horizonation of Fe/Mg smectites, clay assemblages and sulphates formed in variable aqueous environments on the surface of Mars. Our model for aluminosilicate formation on Mars is consistent with the observed geological features, diversity of aqueous mineralogies in ancient surface rocks and state-of-the-art palaeoclimate scenarios.