In this paper, we introduce an energy constraint to improve topology-based inverse design. Current methods typically place the constraints solely on the device geometry and require many optimization iterations to converge to a manufacturable solution. In our approach the energy constraint directs the optimization process to solutions that best contain the optical field inside the waveguide core medium, leading to more robust designs with relatively larger minimum feature size. To validate our method, we optimize two components: a mode converter (MC) and a wavelength demultiplexer. In the MC, the energy constraint leads to nearly binarized structures without applying independent binarization stage. In the demultiplexer, it also reduces the appearance of small features. Furthermore, the proposed constraint improves the robustness to fabrication imperfections as shown in demultiplexer design. With energy constraint optimization, the corresponding spectrum shifts under ±10 nm dimensional variations are reduced by 17% to 30%. The proposed constraint is unique in simultaneously taking both geometry and electric field into account, opening the door to new ideas and insights to further improve the computationally intensive topology-based optimization process of nanophotonic devices.