We theoretically analyze the robustness to potential distortion of mode-locking in a harmonic cavity nanolaser sustaining oscillation of Hermite-Gaussian modes. Different types of imperfections of the harmonic potential that create the Hermite-Gaussian modes are considered: the non-parabolicity of the potential and the possible random errors in the shape of the potential. The influence of the different laser parameters, including the Henry factors of the gain medium and the saturable absorber, on the robustness of the mode-locked regime is discussed in detail.