The vertebrates share the ability to produce a skeleton made of mineralized extracellular matrix. However, our understanding of the molecular changes that accompanied their emergence remains scarce. Here, we describe the evolutionary history of the SPARC (secreted protein acidic and rich in cysteine) family, because its vertebrate orthologues are expressed in cartilage, bones and teeth where they have been proposed to bind calcium and act as extracellular collagen chaperones, and because further duplications of specific SPARC members produced the small calcium-binding phosphoproteins (SCPP) family that is crucial for skeletal mineralization to occur. Both phylogeny and synteny conservation analyses reveal that, in the eumetazoan ancestor, a unique ancestral gene duplicated to give rise to SPARC and SPARCB described here for the first time. Independent losses have eliminated one of the two paralogues in cnidarians, protostomes and tetrapods. Hence, only non-tetrapod deuterostomes have conserved both genes. Remarkably, SPARC and SPARCB paralogues are still linked in the amphioxus genome. To shed light on the evolution of the SPARC family members in chordates, we performed a comprehensive analysis of their embryonic expression patterns in amphioxus, tunicates, teleosts, amphibians and mammals. Our results show that in the chordate lineage SPARC and SPARCB family members were recurrently recruited in a variety of unrelated tissues expressing collagen genes. We propose that one of the earliest steps of skeletal evolution involved the co-expression of SPARC paralogues with collagenous proteins.