We report an experimental observation of strong variations of quality factor and mode splitting among whispering-gallery modes with the same radial order and different azimuthal orders in a scattering-limited microdisk resonator. A theoretical analysis based on the statistical properties of the surface roughness reveals that mode splittings for different azimuthal orders are uncorrelated, and variations of mode splitting and quality factor among the same radial mode family are possible. Simulation results agree well with the experimental observations.