We perform measurements of microwave spectra of cesium Rydberg 51S1/2 → 51PJ transitions with the linewidth approaching the Fourier limit. A two-photon scheme excites the ground-state atoms to the Rydberg 51S1/2 state, and a weak microwave photon couples the Rydberg transition of 51S1/2 → 51PJ. The hyperfine structure of 51P1/2 can be clearly resolved with a narrow linewidth microwave spectra by using the method of ion detection. Furthermore, we investigate the Zeeman effect of the 51P1/2,3/2 state. The theoretical calculations reproduce the measurement well. Our experimental measurements provide a reliable technical solution for the investigation of high angular momentum Rydberg states, which is conducive to further realizing the coherent manipulation of Rydberg energy levels and improving the sensitivity of electromagnetic field measurement.