Fluorescent defects in noncytotoxic diamond nanoparticles are candidates for qubits in quantum computing, optical labels in biomedical imaging, and sensors in magnetometry. For each application these defects need to be optically and thermodynamically stable and included in individual particles at suitable concentrations (singly or in large numbers). In this Letter, we combine simulations, theory, and experiment to provide the first comprehensive and generic prediction of the size, temperature, and nitrogen-concentration-dependent stability of optically active N-V defects in nanodiamonds.