Metasurfaces demonstrate excellent capabilities in manipulating the phase, amplitude and polarization of light. Metalens, as a typical kind of metasurface devices, shows great prospect in simplifying imaging systems. However, like diffractive optical elements, intrinsic dispersion of metasurfaces is high. Thus, significant chromatic aberration is present in common metalenses, deteriorating imaging quality under broadband illumination condition and limiting their applications. To tackle this problem, broadband achromatic metalenses have been proposed and demonstrated in the visible and near-infrared wavelength regions so far. However, broadband achromatic metalens working in the mid-wave and long-wave infrared is still rare. In this paper, thanks to the ingenious design of meta-units that provide the required local phase and phase dispersion, several all-silicon broadband achromatic metalenses working in the mid-wave infrared (3-5 µm) or long-wave infrared (8-14 µm) wavelengths are proposed. Numerical simulation results demonstrate that the designed broadband achromatic metalenses can provide a near-constant focal length with small deviations and an average focusing efficiency of about 70% over the whole operation bandwidths. In addition, these metalenses hold near diffraction-limited focusing capability and polarization-independent focusing features. The achromatic metalenses proposed here are beneficial for improving imaging quality under broadband illumination and increasing detection efficiency of mid-wave and long-wave infrared detection systems.