Inversely tapered spot size converter (SSC) is widely used to connect silicon waveguide with fiber in silicon photonics. However, the tapered structure may cause polarization rotation and further generate interference fluctuation in the transmission spectrum even of a straight waveguide. We analyzed the light propagation in a straight waveguide with SSC at the both ends with coupling matrix and transmission matrix methods. The analysis results matched with the phenomena we observed in the transmission spectrum. Combining the analysis with the measurement results, we calculated the polarization rotation efficiency of the SSC in different samples and analyzed the origin of the polarization rotation effect. Finally, we discussed the influence of the effect to the DP-QPSK signal and proposed several methods to release the impact.