In this paper, we theoretically propose a novel graphene-based hybrid plasmonic waveguide (GHPW) consisting of a low-index rectangle waveguide between a high-index cylindrical dielectric waveguide and the substrate with coated graphene on the surface. The geometric dependence of the mode characteristics on the proposed structure is analyzed in detail, showing that the proposed GHPW has a low loss and consequently a relatively long propagation distance. For TM polarization, highly confined modes guided in the low-index gap region between the graphene and the high-index GaAs and the normalized modal area is as small as 0.0018 (λ2/4) at 3 THz. In addition to enabling the building of high-density integration of the proposed structure are examined by analyzing crosstalk in a directional coupler composed of two GHPWs. This structure also exhibits ultra-low crosstalk when a center-to-center separation between adjacent GHPWs is 32μm, which shows great promise for constructing various terahertz integrated devices.