Accurate wavelength assignment is important for Fourier domain polarization-sensitive optical coherence tomography. Incorrect wavelength mapping between the orthogonal horizontal (H) and vertical (V) polarization channels leads to broadening the axial point spread function and generating polarization artifacts. To solve the problem, we propose an automatic spectral calibration method by seeking the optimal calibration coefficient between wavenumber kH and kV. The method first performs a rough calibration to get the relationship between the wavelength λ and the pixel number x of the CCD for each channel. And then a precise calibration is taken to bring both polarization interferograms in the same k range through the optimal calibration coefficient. The optimal coefficient is automatically obtained by evaluating the cross-correlation of A-line signals. Simulations and experiments are implemented to demonstrate the performance of the proposed method. The results show that, compared to the peaks method, the proposed method is suitable in both Gaussian and non-Gaussian spectrums with a higher calibration accuracy.