In order to be able to make a proper interpretation of mercury porosimetry data, to obtain a structural characterization of a porous solid, a full understanding of the causes of hysteresis in mercury porosimetry is required. Several different theories have previously been proposed, but it is still difficult to make a priori predictions of the level of hysteresis anticipated. In this work, the effect of the degree of smaller scale surface roughness on the hysteresis width has been studied using mean-field density functional simulations and the results obtained confirmed by experiments on silica materials. It has been found that the hysteresis width decreases with increased degree of surface roughness, as characterized experimentally by the surface fractal dimension.