The arrival time jitter and drift of the photocathode drive laser has an important impact on the performance of a Free-Electron-Laser (FEL). It adversely affects the beam energy jitter, bunch length jitter and bunch arrival time jitter, which becomes especially important for pump-probe experiments with femtosecond time resolution. To measure both parameters background free and stabilize the drift of the Yb:CaF2 based laser we use a well designed balanced optical cross correlator. In this paper we present our results using this device and focus particularly on the performance of the amplifier. We achieve a laser drift of less than 200 fs during 60 h, a 4.5 fs rms jitter of the amplifier relative to its seeding oscillator and 11 fs rms for the whole laser relative to a reference clock integrated from 2 mHz to 100 Hz.