We demonstrate an active carrier-envelope phase (CEP) stabilization scheme for optical waveforms generated by difference-frequency mixing of two spectrally detuned and phase-correlated pulses. By performing ellipsometry with spectrally overlapping parts of two co-propagating near-infrared generation pulse trains, we stabilize their relative timing to 18 as. Consequently, we can lock the CEP of the generated mid-infrared (MIR) pulses with a remaining phase jitter below 30 mrad. To validate this technique, we employ these MIR pulses for high-harmonic generation in a bulk semiconductor. Our compact, low-cost, and inherently drift-free concept could bring long-term CEP stability to the broad class of passively phase-locked OPA and OPCPA systems operating in a wide range of spectral windows, pulse energies, and repetition rates.