It is important to determine the ideal combination of housing materials, groups of refractive materials, and their optical powers for athermalizing achromatic and apochromatic optical systems. This study proposes a combined design approach that utilizes three or more glass types to resolve thermal aberrations and defocus achromatic and apochromatic optical systems. It selects a suitable glass type using a 3D glass chart and calculates the optical power analytically. Furthermore, a temperature-insensitive optical system with a 450-750 nm band based on refractive materials (CDGM Glass Co., Ltd.) is designed, with the modular transfer function value of the center field of view decreasing by less than 0.024 in the temperature range of -40∘ C to +80∘ C and the secondary spectrum aberration decreasing by over three times and being maintained within 0.08 mm.