Hydrogen (H2) fuel cells have been developed as an environmentally benign, low-carbon, and efficient energy option in the current period of promoting low-carbon activities, which offer a compelling means to reduce carbon emissions. However, the presence of carbon monoxide (CO) impurities in H2 may potentially damage the fuel cell's anode. As a result, monitoring of the CO levels in fuel cells has become a significant area of research. In this paper, a novel photoacoustic sensor is developed based on photoacoustic heterodyne technology. The sensor combines a 4.61 μm mid-infrared quantum cascade laser with a low-noise differential photoacoustic cell. This combination enables fast, real-time online detection of CO impurity concentrations in H2. Notably, the sensor requires no wavelength locking to monitor CO online in real-time and produces a single effective signal with a period of only 15 ms. Furthermore, the sensor's performance was thoroughly evaluated in terms of detection sensitivity, linearity, and long-term stability. The minimum detection limit of 11 ppb was obtained at an optimal time constant of 1 s.