The spectrally narrow, long-lived luminescence of lanthanide ions makes optical nanomaterials based on these elements uniquely attractive from both a fundamental and applicative standpoint. A highly coveted class of such nanomaterials is represented by colloidal lanthanide-doped semiconductor nanocrystals (LnSNCs). Therein, upon proper design, the poor light absorption intrinsically featured by lanthanides is compensated by the semiconductor moiety, which harvests the optical energy and funnel it to the luminescent metal center. Although a great deal of experimental effort has been invested to produce efficient nanomaterials of that sort, relatively modest results have been obtained thus far. As of late, halide perovskite nanocrystals have surged as materials of choice for doping lanthanides, but they have non-negligible shortcomings in terms of chemical stability, toxicity, and light absorption range. The limited gamut of currently available colloidal LnSNCs is unfortunate, given the tremendous technological impact that these nanomaterials could have in fields like biomedicine and optoelectronics. In this review, we provide an overview of the field of colloidal LnSNCs, while distilling the lessons learnt in terms of material design. The result is a compendium of key aspects to consider when devising and synthesizing this class of nanomaterials, with a keen eye on the foreseeable technological scenarios where they are poised to become front runners.