We describe curved-mirror Fabry-Perot cavities with embedded silicon nitride membranes, fabricated using a monolithic surface-micromachining process. The presence of the suspended membranes was confirmed by confocal microscopy, and their properties were verified through optical studies and thermomechanical calibration of mechanical/vibrational noise spectra measured at room temperature and atmospheric pressure. The cavities exhibit reflectance-limited finesse (F ∼ 103) and wavelength-scale mode volumes (VM ∼ 10·λ3). The short cavity length (L ∼ 2·λ) results in large optomechanical coupling, which is desirable for numerous applications in sensing and quantum information.