Post-transition metal cations with a lone pair (ns2np0) electronic configuration such as Pb2+ and Bi3+ are important components of materials for solar-to-energy conversion. As in molecules like NH3, the lone pair is often stereochemically active in crystals, associated with distorted coordination environments of these cations. In the present study, we demonstrate that suppressed lone pair stereochemical activity can be used as a tool to enhance visible light absorption. Based on an orbital interaction model, we predict that a centrosymmetric environment of the cations limits the orbital interactions with anions, deactivates the lone pair, and narrows the band gap. A high-symmetry Bi3+ site is realized by isovalent substitutions with Y3+ by considering its similar ionic radius and absence of a lone pair. The quaternary photocatalyst Bi2YO4X is singled out as a candidate for Bi substitution from a survey of the coordination environments in Y-O compounds. The introduction of Bi3+ to the undistorted Y3+ site in Bi2YO4X results in a narrowed band gap, as predicted theoretically and confirmed experimentally. The orbital interaction controlled by site symmetry engineering offers a pathway for the further development of post-transition metal compounds for optoelectronic applications.