We study the nonlinear wave interactions between short pulses of different spatio-temporal extents. Unlike the well-understood mixing of quasi-monochromatic waves, this configuration is highly non-intuitive due to the complex coupling between the spatial and temporal degrees of freedom of the interacting pulses. We illustrate the process intuitively with transitions between different branches of the dispersion curves and interpret it in terms of spectral exchange between the interacting pulses. We verify our interpretation with an example whereby a spectrally-narrow pulse "inherits" the wide spectrum of a pump pulse centered at a different wavelength, using exact numerical simulations, as well as a simplified coupled mode analysis and an asymptotic analytical solution. The latter also provides a simple and intuitive quantitative interpretation. The complex wave mixing process studied here may enable flexible spatio-temporal shaping of short pulses and is the starting point of the study of more complicated systems.