The time-domain Helmholtz-Kirchhoff (H-K) integral for surface scattering is derived for a refractive medium, which can handle shadowing effects. The starting point is the H-K integral in the frequency domain. In the high-frequency limit, the Green's function can be calculated by ray theory, while the normal derivative of the incident pressure from a point source is formulated using the ray geometry and ray-based Green's function. For a corrugated pressure-release surface, a stationary phase approximation can be applied to the H-K integral, reducing the surface integral to a line integral. Finally, a computationally-efficient, time-domain H-K integral is derived using an inverse Fourier transform. A broadband signal scattered from a sinusoidal surface in an upwardly refracting medium is evaluated with and without geometric shadow corrections, and compared to the result from a conventional ray model.