Due to the high surface roughness requirements of aluminum alloy mirrors used in the visible light band, there are still great challenges in single point diamond turning of high-surface quality aluminum alloy mirrors. In this paper, a processing method for aluminum alloy mirrors is proposed. Based on single point diamond turning technology, the prediction model of aluminum alloy surface roughness was established. The mapping relationship between the surface roughness of the aluminum alloy mirror and each turning parameter was obtained, and the maximum possible surface quality was achieved. On the basis of the turning results, the method of small tool polishing was used to remove the turning texture generated by the copy effect of the tool arc radius, suppress errors of the medium and high-frequency, and reduce the surface roughness. The single abrasive removal efficiency model was established and mechanical removal in the polishing process was analyzed. Combined with the chemical action in the polishing process, two types of polishing liquid-acidic and neutral, were prepared and analyzed. The optimal polishing parameters were obtained through multiple single-factor experiments. On the basis of this, the surface roughness of the aluminum alloy after turning was optimized. The results show that the value was reduced from 4.811 to 1.482 nm, an increase of 69.2%. This method can effectively improve the machining accuracy of aluminum alloy mirrors and provide an important process guarantee for the application of aluminum alloy materials in visible-light systems.