We report on a compact, ultrahigh-vacuum compatible optical assembly to create large-scale, two-dimensional optical lattices for use in experiments with ultracold atoms. The assembly consists of an octagon-shaped spacer made from ultra-low-expansion glass, to which we optically contact four fused silica cavity mirrors, making it highly mechanically and thermally stable. The mirror surfaces are nearly plane-parallel, which allows us to create two perpendicular cavity modes with diameters ∼1m m. Such large mode diameters are desirable to increase the optical lattice homogeneity, but lead to strong angular sensitivities of the coplanarity between the two cavity modes. We demonstrate a procedure to precisely position each mirror substrate that achieves a deviation from coplanarity of d=1(5)µm. Creating large optical lattices at arbitrary visible and near-infrared wavelengths requires significant power enhancements to overcome limitations in the available laser power. The cavity mirrors have a customized low-loss mirror coating that enhances the power at a set of relevant visible and near-infrared wavelengths by up to 3 orders of magnitude..