We present the results of numerical simulations of the modal properties of Photonic Band Gap Fibers (PBGFs) in which a structural distortion of the silica ring surrounding the air core is gradually introduced. We demonstrate that surface modes supported within such fibers are very sensitive to structural distortions, and that any asymmetric change in the structure can break their degeneracy resulting in associated changes in the anticrossing behavior of the orthogonally polarized core modes, and the development of polarization dependent transmission properties. Our results provide insight into recent experimental observations of wavelength dependent PDL and birefringence in PBGFs.