In this work, we propose and demonstrate a spatiotemporal sweeping fiber bundle for ultra-fast optical diagnoses over a multioctave wavelength span, ranging from ∼400 nm to ∼2000 nm. This all-optical spatiotemporal sweeping is realized by precisely controlling the length increment between individual fibers in the fiber bundle. Here, a 200-ps pixel delay increment specifically enables a pixel readout rate of up to 5 GHz. Depending on different configurations of the fiber bundle, either 1D or 2D spatiotemporal sweeping can be realized. Moreover, the high peak power of the short pulse in each pixel can facilitate the highly sensitive optical detection. To showcase its ultra-broadband operation capability, we here perform ultra-fast optical microscopy at three distinctive wavelengths, which are 710 nm, 1030 nm, and 1600 nm, and achieve tens of MHz line-scan rate and few-micrometers resolution for all three experiments. It is anticipated that this inertia-free spatiotemporal sweeping device with ultra-broad bandwidth, GHz pixel readout rate, and high detection sensitivity is promising for ultra-fast optical diagnosis, particularly when hyperspectral characteristics are desired.