We design and develop a scanning probe microscope (SPM) system based on the qPlus sensor for atomic-scale optical experiments. The microscope operates under ultrahigh vacuum and low temperature (6.2 K). In order to obtain high efficiency of light excitation and collection, two front lenses with high numerical apertures (N.A. = 0.38) driven by compact nano-positioners are directly integrated on the scanner head without degrading its mechanical and thermal stability. The electric noise floor of the background current is 5 fA/Hz1/2, and the maximum vibrational noise of the tip height is below 200 fm/Hz1/2. The drift of the tip-sample spacing is smaller than 0.1 pm/min. Such a rigid scanner head yields small background noise (oscillation amplitude of ∼2 pm without excitation) and high quality factor (Q factor up to 140 000) for the qPlus sensor. Atomic-resolution imaging and inelastic electron tunneling spectroscopy are obtained under the scanning tunneling microscope mode on the Au(111) surface. The hydrogen-bonding structure of two-dimensional (2D) ice on the Au(111) surface is clearly resolved under the atomic force microscope (AFM) mode with a CO-terminated tip. Finally, the electroluminescence spectrum from a plasmonic AFM tip is demonstrated, which paves the way for future photon-assisted SPM experiments.