We investigate, experimentally and theoretically, polarization rotation effects in dilute photonic crystals with transverse permittivity inhomogeneity perpendicular to the traveling direction of waves. A capsize, namely a drastic change of polarization to the perpendicular direction is observed in a one-dimensional photonic crystal in the frequency range 10 ÷ 140 GHz. To gain more insights into the rotational mechanism, we have developed a theoretical model of dilute photonic crystal, based on Maxwell's equations with a spatially dependent two dimensional inhomogeneous dielectric permittivity. We show that the polarization's rotation can be explained by an optical splitting parameter appearing naturally in Maxwell's equations for magnetic or electric fields components. This parameter is an optical analogous of Rashba like spin-orbit interaction parameter present in quantum waves, introduces a correction to the band structure of the two-dimensional Bloch states, creates the dynamical phase shift between the waves propagating in the orthogonal directions and finally leads to capsizing of the initial polarization. Excellent agreement between theory and experiment is found.