We consider the Rabi Hamiltonian, which exhibits a quantum phase transition (QPT) despite consisting only of a single-mode cavity field and a two-level atom. We prove QPT by deriving an exact solution in the limit where the atomic transition frequency in the unit of the cavity frequency tends to infinity. The effect of a finite transition frequency is studied by analytically calculating finite-frequency scaling exponents as well as performing a numerically exact diagonalization. Going beyond this equilibrium QPT setting, we prove that the dynamics under slow quenches in the vicinity of the critical point is universal; that is, the dynamics is completely characterized by critical exponents. Our analysis demonstrates that the Kibble-Zurek mechanism can precisely predict the universal scaling of residual energy for a model without spatial degrees of freedom. Moreover, we find that the onset of the universal dynamics can be observed even with a finite transition frequency.