In common plasmonic structures, absorption and radiation losses are often mutually restricted and can seriously influence the device performance. The current study presents a compound structure composed of multilayer grating stripes and multilayer shallow trenches. A small depth was adopted for the trench configuration to exclude the extra bend loss. These two sections supported Fabry-Perot resonance and cavity modes, respectively, with hybrid modes formed through intercoupling. In addition, the total loss for the entire framework was clearly reduced due to the introduction of the trench geometry, indicating that both absorption and radiation losses were successfully taken into consideration in the compound structure. Significantly, such a low loss realized by the hybridization of surface plasmon polariton modes has rarely been seen before. Moreover, the debatable relationship between the total and partial quality factors was described for the first time based on a hybrid mode analysis to establish a new approach to investigate the different resonance modes. In the detailed calculation process, the relative electric field intensity was first adopted to stipulate the effective areas for the various modes, which is more reasonable than using the common definition that is based on a unit structure. The multilayer trench grating exhibited a relatively low loss without weakening energy localization, which is significant in the design of plasmonic devices.