The speed of optical modulation on a terahertz (THz) pulse is mainly dominated by the optical response of the photocarrier. In order to achieve ultrafast THz modulation, the effective method is to reduce the lifetime of the photocarrier by introducing defects that can trap the photocarriers efficiently. In this paper, we reported the ultrafast optical modulation of THz switching in a 10 nm CdTe nanostructure film. After photoexcitation at 800/400 nm, the THz response of the film is extremely fast with a lifetime of ${\sim}{1.3}\;{\rm ps}$∼1.3ps. Further, the ultrafast transient THz transmission shows almost temperature independence down to 100 K. On the other hand, the transient absorption spectroscopy reveals that the lifetime of photocarriers in CdTe nanostructure film lasts as long as several ns. The 1.3 ps THz photoconductivity response is due to the substantial decrease of photocarrier mobility in a CdTe nanostructure, which comes from the increase of the photocarrier scattering between the photocarrier and the surface states of CdTe nanostructural film. Our experimental results provide a new method to design optically driven ultrafast THz response devices, such as THz switch and THz modulator.