We investigate theoretically how the spectroscopy of an ancillary qubit can probe cavity (circuit) QED ground states containing photons. We consider three classes of systems (Dicke, Tavis-Cummings, and Hopfield-like models), where nontrivial vacua are the result of ultrastrong coupling between N two-level systems and a single-mode bosonic field. An ancillary qubit detuned with respect to the boson frequency is shown to reveal distinct spectral signatures depending on the type of vacua. In particular, the Lamb shift of the ancilla is sensitive to both ground state photon population and correlations. Backaction of the ancilla on the cavity ground state is investigated, taking into account the dissipation via a consistent master equation for the ultrastrong coupling regime. The conditions for high-fidelity measurements are determined.