Due to complications of the off-axis three-mirror anastigmat (TMA) telescope, each optical element in the off-axis TMA telescope is introduced with theoretical eccentricity and tilt. Moreover, the introduction of freeform surfaces and other optical elements with complex surface features generally causes the initial alignment accuracy of the optical path to be low. A large initial alignment error amplifies the sensitivity of the misalignment calculation accuracy to the measurement error of the Zernike coefficient, resulting in difficulty obtaining convergence results for a computer-aided alignment algorithm. Considering the above issues, the alignment sensitivity of each component in the optical path is analyzed in this. The large conditional number of the sensitivity matrix results in poor algorithm robustness. Thus, an adaptive damping factor least-squares algorithm model is proposed and derived to improve the efficiency of the classical least-squares algorithm. A method for piecewise optimization of the damping factor is also deduced. Experiments based on a 0.6 m off-axis TMA telescope verify the effectiveness of the algorithm. Simulation and integration experiments show that the proposed method can reduce the accuracy requirements of the initial alignment and improve the adaptability of Zernike coefficient measurement noise. The alignment procedure is carried out for three iterations, and the average of the five field-of-view wave aberration values is enhanced from 2.1λ (RMS; λ=632.8nm) to 0.09λ (average). The improved algorithm can solve the large initial alignment error of a nonsymmetrical off-axis reflective optical system with a freeform surface as well as the problem of the low success rate of the misalignment value due to low Zernike coefficient measurement accuracy.