A renormalized perturbation method is developed for quantum maps of periodically kicked rotor models to study the tunneling effect in the nearly integrable regime. Integrable Hamiltonians closely approximating the nonintegrable quantum map are systematically generated by the Baker-Hausdorff-Campbell (BHC) expansion for symmetrized quantum maps. The procedure results in an effective integrable renormalization, and the unrenormalized residual part is treated as the perturbation. If a sufficiently high-order BHC expansion is used as the base of perturbation theory, the lowest order perturbation well reproduces tunneling characteristics of the quasibound eigenstates, including the transition from the instanton tunneling to a noninstanton one. This approach enables a comprehensive understanding of the purely quantum mechanisms of tunneling in the nearly integrable regime. In particular, the staircase structure of tunneling probability dependence on quantum number can be clearly explained as the successive transition among multiquanta excitation processes. The transition matrix elements of the residual interaction have resonantly enhanced invariant components that are not removed by the renormalization. Eigenmodes coupled via these invariant components form noninstanton (NI) tunneling channels of two types contributing to the two regions of each step of the staircase structure: one type of channel is inside the separatrix, and the other goes across the separatrix. The amplitude of NI tunneling across the separatrix is insensitive to the Planck constant but shows an essentially singular dependence upon the nonintegrablity parameter. Its relation to the Melnikov integral, which characterizes the scale of classical chaos emerging close to the saddle on the potential top, is discussed.