We devised a novel two-step reference scheme that can greatly suppress the additive and convolutional noises in heterodyne nonlinear spectroscopy. To optimally remove additive noise, we fully utilized the spectral correlation in multi-channel reference data through a linear combination and regression algorithm. Using our pump-probe 2D IR spectrometer, we demonstrated that our scheme can improve the signal-to-noise ratio by 10-30 times and reach the noise floor of the signal detector. The new algorithm is guaranteed to reduce noise, enables the use of unmatched reference detectors, and does not introduce baseline shift or signal distortion. This scheme is applicable to many heterodyne spectroscopic techniques.