We report the observation of an anomalous nonlinear optical response of the prototypical three-dimensional topological insulator bismuth selenide through the process of high-order harmonic generation. We find that the generation efficiency increases as the laser polarization is changed from linear to elliptical, and it becomes maximum for circular polarization. With the aid of a microscopic theory and a detailed analysis of the measured spectra, we reveal that such anomalous enhancement encodes the characteristic topology of the band structure that originates from the interplay of strong spin-orbit coupling and time-reversal symmetry protection. The implications are in ultrafast probing of topological phase transitions, light-field driven dissipationless electronics, and quantum computation.
Keywords: High harmonic generation; Nonlinear optics; Strong-field physics; Topological insulators; Ultrafast spectroscopy.