For Si wire waveguides, we designed a highly efficient fiber coupling structure consisting of a Si inverted taper waveguide and a CMOS-compatible thin SiN waveguide with an SiO2 spacer inserted between them. By using a small SiN waveguide with a 310 nm-square core, the optical field can be expanded to correspond to a fiber with a 4.0-μm mode field diameter. A coupled waveguide system with the SiN waveguide and Si taper waveguide can provide low-loss and low-polarization-dependent mode conversion. Both losses in fiber-SiN waveguide coupling and SiN-Si waveguide mode conversion are no more than 1 dB in a wide wavelength bandwidth from 1.36 μm to 1.65 μm. Through a detailed analysis of the effective refractive indices in the coupled waveguide system, we can understand mode conversion accurately and also derive guidelines for reducing the polarization dependence and for shortening device length.