We show that polarization modulated and polarization multiplexed transmission may be significantly impaired by the polarization scattering induced by intra-channel cross-phase modulation and four-wave mixing. In polarization multiplexed transmission, channel interleaving may be used to mitigate the effect when two-pulse collisions are dominant.