Gabor domain optical coherence microscopy (GD-OCM) is one of many variations of optical coherence tomography (OCT) techniques that aims for invariant high resolution across a 3D field of view by utilizing the ability to dynamically refocus the imaging optics in the sample arm. GD-OCM acquires multiple cross-sectional images at different focus positions of the objective lens, and then fuses them to obtain an invariant high-resolution 3D image of the sample, which comes with the intrinsic drawback of a longer processing time as compared to conventional Fourier domain OCT. Here, we report on an alternative Gabor fusing algorithm, the spectral-fusion technique, which directly processes each acquired spectrum and combines them prior to the Fourier transformation to obtain a depth profile. The implementation of the spectral-fusion algorithm is presented and its performance is compared to that of the prior GD-OCM spatial-fusion approach. The spectral-fusion approach shows twice the speed of the spatial-fusion approach for a spectrum size of less than 2000 point sampling, which is a commonly used spectrum size in OCT imaging, including GD-OCM.