The terahertz region of the electromagnetic spectrum spans the frequency range between the mid-infrared and the millimetre/microwave. This region has not been exploited fully to date owing to the limited number of suitable (in particular, coherent) radiation sources and detectors. Recent demonstrations, using pulsed near-infrared femtosecond laser systems, of the viability of THz medical imaging and spectroscopy have sparked international interest; yet much research still needs to be undertaken to optimize both the power and bandwidth in such THz systems. In this paper, we review how femtosecond near-infrared laser pulses can be converted into broad band THz radiation using semiconductor crystals, and discuss in depth the optimization of one specific generation mechanism based on ultra-fast transport of electrons and holes at a semiconductor surface. We also outline a few of the opportunities for a technology that can address a diverse range of challenges spanning the physical and biological sciences, and note the continuing need for the development of solid state, continuous wave, THz sources which operate at room temperature.