Piezoelectric composites comprising an active phase of ferroelectric ceramic and a polymer matrix have recently found numerous sensory applications. However, it remains a major challenge to further improve their electromechanical response for advanced applications such as precision control and monitoring systems. We here investigated the incorporation of graphene platelets (GnPs) and multi-walled carbon nanotubes (MWNTs), each with various weight fractions, into PZT (lead zirconate titanate)/epoxy composites to produce three-phase nanocomposites. The nanocomposite films show markedly improved piezoelectric coefficients and electromechanical responses (50%) besides an enhancement of ~200% in stiffness. The carbon nanomaterials strengthened the impact of electric field on the PZT particles by appropriately raising the electrical conductivity of the epoxy. GnPs have been proved to be far more promising in improving the poling behavior and dynamic response than MWNTs. The superior dynamic sensitivity of GnP-reinforced composite may be caused by the GnPs' high load transfer efficiency arising from their two-dimensional geometry and good compatibility with the matrix. The reduced acoustic impedance mismatch resulting from the improved thermal conductance may also contribute to the higher sensitivity of GnP-reinforced composite. This research pointed out the potential of employing GnPs to develop highly sensitive piezoelectric composites for sensing applications.