We show that models of new particle physics containing massless pseudoscalar fields superweakly coupled to photons can be very efficiently probed with CMB polarization anisotropies. The stochastic pseudoscalar fluctuations generated during inflation provide a mechanism for converting E-mode polarization to B-mode during photon propagation from the surface of last scattering. The efficiency of this conversion process is controlled by the dimensionless ratio H/(2pif(a)), where H is the Hubble scale during inflation, and f(a)-1 is the strength of the pseudoscalar coupling to photons. The current observational limits on the B mode constrain this ratio to be less than 0.07, which in many models of inflation translates to a sensitivity to f(a) exceeding 10(14) GeV, which surpasses other tests.