Experiments demonstrate a dramatic decrease in polarization-instability threshold as an optical pulse is tuned near the short-wavelength edge of the photonic bandgap formed by a fiber Bragg grating. These enhanced nonlinear interactions and birefringent effects are modeled with coupled-mode numerical simulations. Nonlinearities are shown to increase much more rapidly than the effective birefringence as the pulse wavelength approaches the bandgap edge.