During the SARS-CoV-2 pandemic, multiple variants escaping pre-existing immunity emerged, causing concerns about continued protection. Here, we use antigenic cartography to analyze patterns of cross-reactivity among a panel of 21 variants and 15 groups of human sera obtained following primary infection with 10 different variants or after mRNA-1273 or mRNA-1273.351 vaccination. We find antigenic differences among pre-Omicron variants caused by substitutions at spike protein positions 417, 452, 484, and 501. Quantifying changes in response breadth over time and with additional vaccine doses, our results show the largest increase between 4 weeks and >3 months post-2nd dose. We find changes in immunodominance of different spike regions depending on the variant an individual was first exposed to, with implications for variant risk assessment and vaccine strain selection.