Fluorescent nanodiamonds (FNDs) are extremely photostable markers and nanoscale sensors, which are increasingly used in biomedical applications. Nanoparticle size is a critical parameter in the majority of these applications. Yet, the effect of particle size on FND's fluorescence and colloidal properties is not well understood today. Here, we investigate the fluorescence and colloidal stability of commercially available high-pressure high-temperature FNDs containing nitrogen-vacancy (NV) centers in biological media. Unconjugated FNDs in sizes ranging between 10 nm and 140 nm with an oxidized surface are studied using dynamic light scattering and fluorescence spectroscopy. We determine their colloidal stability in water, fetal bovine serum, Dulbecco's Modified Eagle Medium and complete media. The FNDs' relative fluorescence brightness, the NV charge-state, and the FND fluorescence against media autofluorescence are analyzed as a function of FND size. Our results will enable researchers in biology and beyond to identify the most promising FND particle size for their application.