In this study, a nanocomposite consisting of graphitic carbon nitride nanosheets loaded with graphitic carbon nitride quantum dots (CNQDs/CNNNs) was synthesized via a one-step pyrolysis method. This nanocomposite exhibited excellent thermal stability, photobleaching and salt resistance. Then a new fluorescence sensing platform based on CNQDs/CNNNs was constructed, which showed high sensitivity and selectivity towards trace mercury ions (Hg2+). By using X-ray photoelectron spectroscopy, UV-vis diffuse reflectance spectra and density functional theory, the fluorescence response mechanism was elucidated where Hg2+ could interact with CNQDs/CNNNs, causing a structural change in the nanocomposite, further affecting its bandgap structure, and finally leading to fluorescence quenching. The linear range for detecting Hg2+ was found to be 0.025-4.0 μmol L-1, with a detection limit of 7.82 nmol L-1. This strategy provided the advantages of a rapid response and a broad detection range, making it suitable for quantitative detection of Hg2+ in environmental water.