Photothermal therapy effectively ablates tumors by hyperthermia (>50 °C) under laser irradiation. However, the hyperthermia may inevitably diffuse to the surrounding healthy tissues to induce additional damage. Thus, effective cancer therapy by mild photothermal therapy at low temperatures is greatly desirable. In this study, a nanoagent (COF-GA) was designed to inhibit HSP90 for enhanced photothermal therapy against cancer at low temperatures. The nanoscale covalent organic frameworks (COFs) were able to increase the temperature of the tumor tissue under laser irradiation, which can transfer the energy of laser into heat for cancer cell killing. Gambogic acid (GA), as an inhibitor of HSP90, was used to overcome the heat resistance of tumor, achieving efficient mild-temperature photothermal therapy. As an excellent candidate for the photothermal therapy agent, COF-GA can induce the temperature to elevate as the exposure time increased when irradiated with laser. In vivo tests further demonstrated that the tumor growth was able to be significantly suppressed after being treated with COF-GA. The mild-temperature photothermal therapy exhibits an excellent antitumor efficacy at a relatively low temperature and minimizes the nonspecific thermal damage to normal tissues. This COF-GA nanoagent also enriches our understanding towards the various applications of COFs, particularly in the biomedicine field.