Compositional prior information is used to bridge a gap in the theory between optical coherence tomography (OCT), which provides high-resolution structural images by neglecting spectral variation, and imaging spectroscopy, which provides only spectral information without significant regard to structure. A constraint is proposed in which it is assumed that a sample is composed of N distinct materials with known spectra, allowing the structural and spectral composition of the sample to be determined with a number of measurements on the order of N. We present a forward model for a sample with heterogeneities along the optical axis and show through simulation that the N-species constraint allows unambiguous inversion of Fourier transform interferometric data within the spatial frequency passband of the optical system. We then explore the stability and limitations of this model and extend it to a general 3D heterogeneous sample.