Three crystals used in solid-state lasers, namely, yttrium aluminum garnet (YAG), yttrium orthovanadate (YVO(4)), and gadolinium calcium oxoborate (GdCOB), were investigated to determine the influence of dopants on their thermal diffusivity. The thermal diffusivity was measured by thermal wave method with a signal detection based on mirage effect. The YAG crystals were doped with Yb or V, the YVO(4) with Nd or Ca and Tm, and the GdCOB crystals contained Nd or Yb. In all cases, the doping caused a decrease in thermal diffusivity. The analysis of complementary measurements of ultrasound velocity changes caused by dopants leads to the conclusion that impurities create phonon scattering centers. This additional scattering reduces the phonon mean free path and accordingly results in the decrease of the thermal diffusivity of the crystal. The influence of doping on lattice parameters was investigated, additionally.