Hole mobility characteristics were investigated with surface roughness and silicon-on-insulator (SOI) thickness variations to investigate the influence of surface roughness to mobility. The root mean square roughness varied between 0.16, 0.85 and 10.6 nm in 220, 100 and 40 nm thick SOI samples. Hole mobility was measured and analyzed as a function of effective field and temperature with the variations of surface roughness. The hole mobility, determined by transconductance, greatly decreased with the increase of effective field due to the increased surface roughness scattering in 40 nm thick SOI samples. On the other hand, phonon scattering was a dominant mechanism with the increase of temperature, irrespective of surface roughness and SOI thickness. The induced surface roughness of the devices increases the phonon scattering, thereby reducing the electron and hole mobility. The hole mobility decreases with the roughening of the samples, with the increase of temperature due to increased phonon scattering. Therefore, for enhanced mobility, surface scattering and phonon scattering should be controlled even in atomic scale roughened samples.