A polarization-insensitive photonic microwave downconverter is proposed and demonstrated, which is comprised of a polarization beam splitter, two Mach-Zehnder modulators (MZMs), and a balanced photodetector. By biasing the MZMs at the quadrature bias points with opposing modulation slopes, the performance of the proposed photonic microwave downconverter is almost independent of the polarization state of the optical microwave signal for down conversion. An experiment is performed, which shows that the polarization dependent loss of the proposed downconverter is less than 0.06 dB. The downconverter is also evaluated in a radio-over-fiber link. A 20 GHz RF signal with 20 MBaud 16 quadrature amplitude modulation baseband data is successfully downconverted to a 1 GHz IF signal. When the polarization state of the input optical microwave signal is adjusted, the variation of the error vector magnitude of the downconverted signal is less than 0.4%.