We propose and demonstrate a new scheme for atom interferometry, using light pulses inside an optical cavity as matter wave beam splitters. The cavity provides power enhancement, spatial filtering, and a precise beam geometry, enabling new techniques such as low power beam splitters (<100 μW), large momentum transfer beam splitters with modest power, or new self-aligned interferometer geometries utilizing the transverse modes of the optical cavity. As a first demonstration, we obtain Ramsey-Raman fringes with >75% contrast and measure the acceleration due to gravity, g, to 60 μg/sqrt[Hz] resolution in a Mach-Zehnder geometry. We use >10(7) cesium atoms in the compact mode volume (600 μm 1/e(2) waist) of the cavity and show trapping of atoms in higher transverse modes. This work paves the way toward compact, high sensitivity, multiaxis interferometry.