Silicon photonic spatial heterodyne Fourier transform spectrometers (SH-FTSs) are attractive with chip-scale monolithic arrays of imbalanced Mach-Zehnder interferometers; however, there exist optical path difference (OPD) errors from the inevitable fabrication imperfection, which will severely distort the retrieved spectra. In this Letter, we propose that a predictive model can be created for rapid and accurate spectral recovery based on the conditional generative adversarial network (cGAN) featuring strong input-on-output supervision, instead of both complicated physical OPD modification and time-consuming iterative spectral calculation. As a demonstration, cGAN spectral prediction was performed for our previously presented dual-polarized SH-FTS with large OPD errors [Opt. Lett.44, 2923 (2019)OPLEDP0146-959210.1364/OL.44.002923]. Due to the strong noise-resistant capability, the cGAN-predicted spectra can stay reliable, even though the signal-to-noise ratio of acquired interferograms dramatically drops from 1000 to 100, implying a lower limit of detection.