We study the double occupancy in a fermionic Mott insulator at half filling generated via a dynamical periodic modulation of the hopping amplitude. Tuning the modulation amplitude, we describe a crossover in the nature of doublon-holon excitations from a Fermi golden rule regime to damped Rabi oscillations. The decay time of excited states diverges at a critical modulation strength, signaling the transition to a dynamically bound nonequilibrium state of doublon-holon pairs. A setup using a fermionic quantum gas should allow the study of the critical exponents.