We present the design and commissioning of a resonant microwave cavity as a novel diagnostic for the study of ultracold plasmas. This diagnostic is based on the measurements of the shift in the resonance frequency of the cavity, induced by an ultracold plasma that is created from a laser-cooled gas inside. This method is simultaneously non-destructive, very fast (nanosecond temporal resolution), highly sensitive, and applicable to all ultracold plasmas. To create an ultracold plasma, we implement a compact magneto-optical trap based on a diffraction grating chip inside a 5 GHz resonant microwave cavity. We are able to laser cool and trap (7.25 ± 0.03) × 107 rubidium atoms inside the cavity, which are turned into an ultracold plasma by two-step pulsed (nanosecond or femtosecond) photo-ionization. We present a detailed characterization of the cavity, and we demonstrate how it can be used as a fast and sensitive probe to monitor the evolution of ultracold plasmas non-destructively. The temporal resolution of the diagnostic is determined by measuring the delayed frequency shift following femtosecond photo-ionization. We find a response time of 18 ± 2 ns, which agrees well with the value determined from the cavity quality factor and resonance frequency.