In this review, we present the theoretical foundations and first-principles frameworks to describe quantum matter within quantum electrodynamics (QED) in the low-energy regime, with a focus on polaritonic chemistry. By starting from fundamental physical and mathematical principles, we first review in great detail ab initio nonrelativistic QED. The resulting Pauli-Fierz quantum field theory serves as a cornerstone for the development of (in principle exact but in practice) approximate computational methods such as quantum-electrodynamical density functional theory, QED coupled cluster, or cavity Born-Oppenheimer molecular dynamics. These methods treat light and matter on equal footing and, at the same time, have the same level of accuracy and reliability as established methods of computational chemistry and electronic structure theory. After an overview of the key ideas behind those ab initio QED methods, we highlight their benefits for understanding photon-induced changes of chemical properties and reactions. Based on results obtained by ab initio QED methods, we identify open theoretical questions and how a so far missing detailed understanding of polaritonic chemistry can be established. We finally give an outlook on future directions within polaritonic chemistry and first-principles QED.