The phenomenon of frequency and phase synchronization in stochastic systems requires a revision of concepts originally phrased in the context of purely deterministic systems. Various definitions of an instantaneous phase are presented and compared with each other with special attention paid to their robustness with respect to noise. We review the results of an analytic approach describing noise-induced phase synchronization in a thermal two-state system. In this context exact expressions for the mean frequency and the phase diffusivity are obtained that together determine the average length of locking episodes. A recently proposed method to quantify frequency synchronization in noisy potential systems is presented and exemplified by applying it to the periodically driven noisy harmonic oscillator. Since this method is based on a threshold crossing rate pioneered by Rice the related phase velocity is termed the Rice frequency. Finally, we discuss the relation between the phenomenon of stochastic resonance and noise-enhanced phase coherence by applying the developed concepts to the periodically driven bistable Kramers oscillator.