Two-pulse correlation is employed to investigate the temporal dynamics of both two-photon photoluminescence (2PPL) and four-photon photoluminescence (4PPL) in resonant and nonresonant nanoantennas excited at a wavelength of 800 nm. Both 2PPL and 4PPL data are consistent with the same two-step model already established for 2PPL, implying that the first excitation step in 4PPL is a three-photon sp → sp direct interband transition. Considering energy and parity conservation, we also explain why 4PPL behavior is favored over, for example, three- and five-photon photoluminescence in the power range below the damage threshold of our antennas. Since sizable 4PPL requires larger peak intensities of the local field, we are able to select either 2PPL or 4PPL in the same gold nanoantennas by choosing a suitable laser pulse duration. We thus provide a first consistent model for the understanding of multiphoton photoluminescence generation in gold nanoantennas, opening new perspectives for applications ranging from the characterization of plasmonic resonances to biomedical imaging.