When quantum dots are used as fluorescent probes or drug tracers for in vivo imaging, the quantum dots in the blood will come into direct contact with vascular endothelial cells. Thus, it is necessary to study whether quantum dots can affect endothelial function after being injected into blood vessels as imaging agents. In recent years, there have been numerous studies on the toxicity of quantum dots. Herein, we focused on five types of quantum dots (Cd-containing quantum dots, CuInS2 quantum dots, black phosphorus quantum dots, MXene quantum dots, and carbon-based quantum dots) for cell imaging and their toxicity in vivo and in vitro. Although current research on the toxicity of quantum dots has not reached a consistent conclusion, it can guide the next step in evaluating their cytotoxicity.