Light-scattering measurements on rugate coatings made out of mixtures of Si(x)Ta(y)O(z) and Si(x)Hf(y)O(z) were performed. Through successive optimization steps for the substrate roughness and deposition parameters, the overall scattering loss could be reduced by 96% to 3.5 ppm. In order to analyze the relevant scattering mechanisms in such coatings, different theoretical models for scattering from bulk and surface imperfections are compared to measured data. The best accordance between simulated and measured data could be achieved for the theory based on bulk imperfections, while the classical roughness based theory, which is used for conventional multilayer systems, gives reasonable good results.