Carbon nanomaterials are the most studied materials in nanotechnology. There have been numerous studies on cytotoxicity assessments of carbon nanomaterials, which, however, often lead to controversy. It is generally considered that chemical and physical properties of carbon nanomaterials should have specific biological outcomes. More recent studies have identified the significance of environmental factors surrounding nanomaterial-treated cells. In this perspective, we mainly review culture medium-associated physicochemical insights on the cytotoxicity of carbon nanomaterials, which are largely based on studies in our laboratory. These studies established the close relationship and interplay among the physicochemical properties of the nanomaterials, culture medium, and their toxicological responses.