Contamination of the environment with heavy metal ions has been an important concern throughout the world for decades. Driven by the need to detect trace amounts of mercury in environmental samples, this article demonstrates for the first time that nonlinear optical (NLO) properties of MPA-HCys-PDCA-modified gold nanoparticles can be used for rapid, easy and reliable screening of Hg(II) ions in aqueous solution, with high sensitivity (5 ppb) and selectivity over competing analytes. The hyper Rayleigh scattering (HRS) intensity increases 10 times after the addition of 20 ppm Hg(2+) ions to modified gold nanoparticle solution. The mechanism for HRS intensity change has been discussed in detail using particle size-dependent NLO properties as well as a two-state model. Our results show that the HRS assay for monitoring Hg(II) ions using MPA-HCys-PDCA-modified gold nanoparticles has excellent selectivity over alkali, alkaline earth (Li(+), Na(+), K(+), Mg(2+), Ca(2+)), and transition heavy metal ions (Pb(2+), Pb(+), Mn(2+), Fe(2+), Cu(2+), Ni(2+), Zn(2+), Cd(2+)).