Therapeutic proton and heavier ion beams generate prompt gamma photons that may escape from the patient. In principle, this allows for real-time, in situ monitoring of the treatment delivery, in particular, the hadron range within the patient, by imaging the emitted prompt gamma rays. Unfortunately, the neutrons simultaneously created with the prompt photons create a background that may obscure the prompt gamma signal. To enhance the accuracy of proton dose verification by prompt gamma imaging, we therefore propose a time-of-flight (TOF) technique to reject this neutron background, involving a shifting time window to account for the propagation of the protons through the patient. Time-resolved Monte Carlo simulations of the generation and transport of prompt gamma photons and neutrons upon irradiation of a PMMA phantom with 100, 150 and 200 MeV protons were performed using Geant4 (version 9.2.p02) and MCNPX (version 2.7.D). The influence of angular collimation and TOF selection on the prompt gamma and neutron longitudinal profiles is studied. Furthermore, the implications of the proton beam microstructure (characterized by the proton bunch width and repetition period) are investigated. The application of a shifting TOF window having a width of ΔTOF(z) = 1.0 ns appears to reduce the neutron background by more than 99%. Subsequent application of an energy threshold does not appear to sharpen the distal falloff of the prompt gamma profile but reduces the tail that is observed beyond the proton range. Investigations of the influence of the beam time structure show that TOF rejection of the neutron background is expected to be effective for typical therapeutic proton cyclotrons.