We improved the piezoelectric property of poly(vinylidene fluoride) (PVDF) by employing graphene. The reduced graphene oxide (rGO)–PVDF nanocomposites were prepared by a solution casting method and the rGO contents ranged from 0.0 wt% to 0.2 wt%. To induce the piezoelectric β-phase crystal structure, the nanocomposite films were drawn in a ratio of 4–5 and polarized by a step-wise poling method. To evaluate the piezoelectric property, the output voltages of the rGO–PVDF nanocomposite films were measured through extensive experimental vibration tests. The experimental results show that the rGO–PVDF nanocomposite film with 0.05 wt% rGO loading possesses the highest output voltage compared with other loadings, which is around 293% of that of the pure PVDF film. Moreover, it can be found that with the increase of the rGO content from 0 wt% to 0.2 wt%, the output voltage tends to have a peak at 0.05 wt%. The main reason for this phenomenon is that a more β-crystalline phase can be formed at those rGO loadings, as confirmed by XRD and FT-IR spectrum analyses.