Z-scan technology was used to study the nonlinear absorption (NLA) and nonlinear refraction (NLR) of silver nanoparticles (Ag NPs) with various sizes under different laser intensities. The results demonstrate that the NLA and NLR of Ag NPs were size-dependent. Specifically, the 10 nm Ag NPs exhibit saturation absorption (SA) and insignificant NLR. The 20 and 40 nm Ag NPs show the coexistence of SA and reverse saturation absorption (RSA). SA is believed to result from ground-state plasma bleaching, whereas RSA originates from excited state absorption (ESA). The 20 nm and 40 nm Ag NPs shows increasing self-defocusing with the increase of laser intensity. It was observed that the energy relaxation of Ag NPs mainly includes two processes of electron-phonon and phonon-phonon couplings on the order of picoseconds.