We demonstrate a simple interferometric technique to directly measure the complex optical transmittance over a large spectral range using a frequency-comb spectrometer based on a virtually imaged phased array. A Michelson interferometer encodes the phase deviations induced by a sample contained in one of its arms into an interferogram image. When combined with an additional image taken from each arm separately, along with a frequency-calibration image, this allows full reconstruction of the sample's optical transfer function. We demonstrate the technique with a vapor cell containing H13C14N, producing transmittance and phase spectra spanning 2.9 THz (∼23 nm) with ∼1 GHz resolution.