We report on the time-resolved coherent anti-Stokes Raman spectroscopy of phonon dephasing in micro- and nanocrystalline diamond films. The dephasing times T(2) were found to be dependent on the morphology of diamond films (average size of crystals and content of nondiamond carbon phase) and changed from 0.7 to 1.72 ps. The dephasing times were found to be temperature independent in the range 10-295 K. In addition to diamond Raman active phonon mode at 1332 cm(-1), we investigated also the dynamics of a broad Raman peak at 1530 cm(-1) which is present in samples with higher content of nondiamond sp(2) hybridized carbon phase. This peak was found to be homogenously broadened with very fast dephasing (T(2)~50 fs).