Polariton chemistry has attracted great attention as a potential route to modify chemical structure, properties, and reactivity through strong interactions among molecular electronic, vibrational, or rovibrational degrees of freedom. A rigorous theoretical treatment of molecular polaritons requires the treatment of matter and photon degrees of freedom on equal quantum mechanical footing. In the limit of molecular electronic strong or ultrastrong coupling to one or a few molecules, it is desirable to treat the molecular electronic degrees of freedom using the tools of ab initio quantum chemistry, yielding an approach we refer to as ab initio cavity quantum electrodynamics, where the photon degrees of freedom are treated at the level of cavity quantum electrodynamics. Here, we present an approach called Cavity Quantum Electrodynamics Complete Active Space Configuration Interaction theory to provide ground- and excited-state polaritonic surfaces with a balanced description of strong correlation effects among electronic and photonic degrees of freedom. This method provides a platform for ab initio cavity quantum electrodynamics when both strong electron correlation and strong light-matter coupling are important and is an important step toward computational approaches that yield multiple polaritonic potential energy surfaces and couplings that can be leveraged for ab initio molecular dynamics simulations of polariton chemistry.