We propose a highly efficient graphene-on-gap modulator (GOGM) by employing the hybrid plasmonic effect, whose modulation efficiency (up to 1.23 dB/μm after optimization) is ∼12-fold larger than that of the present graphene-on-silicon modulator (∼0.1 dB/μm). The proposed modulator has the advantage of a short modulation length of ∼3.6 μm, a relatively low insertion loss of ∼0.32 dB, and a larger modulation bandwidth of ∼0.48 THz. The physical insight is investigated, showing that both the slow light effect and the overlap between graphene and the mode field contribute. Moreover, an efficient taper coupler has been designed to convert the quasi-transverse electric mode of conventional silicon waveguide to the hybrid plasmonic mode of GOGM, with a high coupling efficiency of 91%. This Letter may promote the design of high-performance on-chip electro-optical modulators.