We demonstrated a structure with periodic cylinders arranged bilaterally and a thin dielectric layer covered inside that supports bound modes of surface plasmon polaritons at terahertz frequencies. This structure can confine the surface plasmon polaritons in the lateral direction, and at the same time reduce the field expansion into space. We examined and explored the characteristics of several different structures using scanning near-field terahertz microscopy. The proposed designs pave a novel way to terahertz waveguiding and may have important applications in the development of flexible, wideband and compact photonic circuits operating at terahertz frequencies.