The application of terahertz imaging in biomedical diagnostics is limited by the poor imaging contrast due to the strong absorption of terahertz waves by water molecules in the living body. To resolve this problem, gold nanorods (GNRs) have been considered as a possible contrast agent. However, the conventional preparation of GNRs requires using cetyltrimethylammonium bromide (CTAB) as a stabilizing surfactant, which is toxic to animals at high exposures. In this work, a modified seed-mediated growth method was proposed to synthesize silica-coated GNRs, which exhibited better stability and biocompatibility and the same cellular uptake efficiency as compared to those of uncoated GNRs. Moreover, silica-coated GNRs showed higher photothermal efficiency caused by the localized surface plasmon resonance that increased the terahertz reflection amplitude. In particular, the in vitro terahertz imaging of prostate cancer cells demonstrated noticeably higher contrast when silica-coated GNRs were used as a contrast agent. Hence, this study suggests a stable, biocompatible, and efficient contrast-enhancing agent for terahertz imaging, which can expand the scope of its clinical applications.