Highly flexible nanocomposite hydrogels were prepared by using silica nanoparticles (SNPs) as fillers and multi-functional cross-links to graft hydrophilic poly(acrylic acid) (PAA) by free radical polymerization from an aqueous solution. The SNPs were collected by neighboring polymer chains and dispersed uniformly within a PAA matrix. The mechanical properties of the nanocomposite hydrogels were tailored by the concentration of SNPs according to the percolation model. It was proposed that covalent bonds of adsorbed chains on the filler surface resulted in the formation of a shell of an immobilized glassy layer and trapped entanglements, where the glassy polymer layer greatly enhanced the elastic modulus and the release of trapped entanglements at deformation contributed to the viscoelastic properties.