We present a broadband wavelength-swept laser using a 16-µm-core-diameter Cr4+:YAG crystal fiber as the gain medium. The laser-diode-pumped crystal fiber laser has a threshold of only 102 mW due to the low propagation loss and high heat dissipation efficiency. The laser achieves a sweeping wavelength range of 134 nm, centered around 1425 nm, with a scanning speed of 163 k nm/s. Notably, the cross-polarization-coupled excited state absorption of the signal wavelength constrained the long-wavelength lasing limit. This laser has the potential for swept source optical coherence tomography applications, providing an axial resolution of 11.4 µm.