When performing multiphoton fluorescence lifetime imaging in multiple spectral emission channels, an instrument response function must be acquired in each channel if accurate measurements of complex fluorescence decays are to be performed. Although this can be achieved using the reference reconvolution technique, it is difficult to identify suitable fluorophores with a mono-exponential fluorescence decay across a broad emission spectrum. We present a solution to this problem by measuring the IRF using the ultrafast luminescence from gold nanorods. We show that ultrafast gold nanorod luminescence allows the IRF to be directly obtained in multiple spectral channels simultaneously across a wide spectral range. We validate this approach by presenting an analysis of multispectral autofluorescence FLIM data obtained from human skin ex vivo.