Exploring the corresponding relation between structural and physical properties of materials at the atomic scale remains the fundamental problem in science. With the development of the aberration-corrected transmission electron microscopy (AC-TEM) and the ultrafast optical spectroscopy technique, sub-angstrom-scale spatial resolution and femtosecond-scale temporal resolution can be achieved, respectively. However, the attempt to combine both their advantages is still a great challenge. Here, we develop in situ optical spectroscopy with high temporal resolution in AC-TEM by utilizing a self-designed and manufactured TEM specimen holder, which has the capacity of sub-angstrom-scale spatial resolution and femtosecond-scale temporal resolution. The key and unique design of our apparatus is the use of the fiber bundle, which enables the delivery of focused pulse beams into TEM and collection of optical response simultaneously. The generated focused spot has a size less than 2 µm and can be scanned in plane with an area larger than 75 × 75 µm2. Most importantly, the positive group-velocity dispersion caused by glass fiber is compensated by a pair of diffraction gratings, thus resulting in the generation of pulse beams with a pulse width of about 300 fs (@ 3 mW) in TEM. The in situ experiment, observing the atomic structure of CdSe/ZnS quantum dots in AC-TEM and obtaining the photoluminescence lifetime (∼4.3 ns) in the meantime, has been realized. Further ultrafast optical spectroscopy with femtosecond-scale temporal resolution could be performed in TEM by utilizing this apparatus.