Small heat shock proteins (sHsps) are molecular chaperones involved in maintaining protein homeostasis; they have also been implicated in protein folding diseases and in cancer. In this protein family, a conserved core domain, the so-called α-crystallin or Hsp20 domain, is flanked by highly variable, nonconserved sequences that are essential for chaperone function. Analysis of 8714 sHsps revealed a broad variation of primary sequences within the superfamily as well as phyla-dependent differences. Significant variations were found in the number of sHsps per genome, their amino acid composition, and the length distribution of the different sequence parts. Reconstruction of the evolutionary tree for the sHsp superfamily shows that the flanking regions fall into several subgroups, indicating that they were remodeled several times in parallel but independent of the evolution of the α-crystallin domain. The evolutionary history of sHsps is thus set apart from that of other protein families in that two exon boundary-independent strategies are combined: the evolution of the conserved α-crystallin domain and the independent evolution of the N- and C-terminal sequences. This scenario allows for increased variability in specific small parts of the protein and thus promotes functional and structural differentiation of sHsps, which is not reflected in the general evolutionary tree of species.