Sub-10 nm nanopores drilled by a focused electron beam in a transmission electron microscope are widely used in solid-state nanopore devices for DNA translocation. However, there still remains much controversy surrounding the drilling mechanism. In order to explain the drilling of nanopores by electrons, we undertook a theoretical consideration of the energy transfer from the fast electrons to the solid through such mechanisms as elastic and inelastic scattering. According to the calculations based on the scattering cross-section, the direct atomic displacement cross-section induced by elastic scattering increases with increasing incident electron energy, while the ionization cross-section and temperature increment decrease. We performed nanopore drilling in a Si3N4 membrane using two different electron energies, 200 and 300 kV, to identify the drilling mechanism. The dependence of the nanopore drilling on the incident electron energy was well matched with the direct atomic displacement.