We demonstrate the suppression of inhomogeneous dephasing of cold 87Rb atoms optically trapped inside a hollow-core fiber. The differential light shift (DLS) for the clock transition caused by the trapping beam is reduced by one order of magnitude through the use of a weak compensation laser beam that is spatially mode-matched to the trapping beam. The coherence of the DLS-compensated system is characterized by microwave Ramsey interferometry, which shows Ramsey fringes with a contrast of over 0.6 at a separation time of 10 ms. The dephasing time, measured by Ramsey spectroscopy at different separation times, reaches tens of milliseconds after DLS cancellation, limited by the residual DLS caused by mode mismatching between the two laser beams. This work paves the way for compact and portable fiber-guided atom interferometers.