We present a highly stable polarization-maintained supercontinuum (SC) using a setup solely based on ZBLAN (ZrF4-BaF2-LaF3-AlF3-NaF) fibers. The pumping source consists of a femtosecond master oscillator fiber amplifier based on thulium-doped ZBLAN fibers. It provides multi-watts of output power with the center wavelength of 1920 nm at 1 MHz repetition rate. The SC generated by pumping an elliptical core passive single-mode ZBLAN fiber spans from 350 nm to 4.5 µm and exhibits high stability. We characterized the SC pulse using sum-frequency cross-correlation frequency resolved optical gating.