Nonlinear plasmons are becoming an appealing and intriguing research area due to their remarkable light concentration and manipulation abilities. In this work, the nonlinear absorption (NLA) phenomena of polarized nanoantenna arrays coupled with the low dimensional topological insulator Bi2Se3 are studied at different excitation wavelengths. Our experimental results indicate that a significant enhancement in the linear absorption coefficient is achieved by localized surface plasmon (LSP) resonance, with enhancement factors that are 10- and 8-fold in magnitude for the cases of 800 nm (y polarization) and 970 nm (x polarization), respectively. Moreover, by polarization sensitive studies under 800 nm laser excitation, this new Bi2Se3-Au nanoantenna hybrid-structure exhibits adverse absorption responses of enhancement and suppression compared to pure Bi2Se3 film, providing excellent potential for applications in information converters. In particular, under 800 nm pump light (10 GW cm-2), the transmittance intensity of 450 nm or 1064 nm continuous wave (CW) probe light alters back and forth when the polarization direction changes by 90°. Thus, "ON" and "OFF" modes of this Bi2Se3-Au nanoantenna hybrid structure-based switch are achieved by using 450 nm and 1064 nm light, respectively, with a corresponding modulation depth of 3.4% and 21.9%, which can be applied in versatile photonic devices.