Topological insulator bismuth has attracted considerable attention for the fabrication of room-temperature, wide bandwidth, and high-performance photodetectors due to the gapless edge state and insulating bulk state properties. However, both the photoelectric conversion and carrier transportation of the bismuth films are extremely affected by the surface morphology and grain boundaries to limit optoelectronic properties further. Here, we demonstrate a strategy of femtosecond laser treatment for upgrading the quality of bismuth films. After the treatment with proper laser parameters, the measurement of average surface roughness can be reduced from Ra = 44 nm to 6.9 nm, especially with accompany of the evident grain boundary elimination. Consequently, the photoresponsivity of the bismuth films increases approximately 2 times within an ultra-broad spectrum range from the visible to mid-infrared. This investigation suggests that the femtosecond laser treatment can help to benefit the performance of topological insulator ultra-broadband photodetectors.