Based on the extended Huygens-Fresnel principle and the definition of second-order moments of the Wigner distribution function (WDF), the analytical expressions for the propagation factors (M2-factors) and Strehl ratio SR of the Gaussian Schell-model (GSM) vortex beams and GSM non-vortex beams propagation through non-Kolmogorov atmospheric turbulence are derived, and used to study the influence of non-Kolmogorov atmospheric turbulence on beam quality of the GSM vortex beams. It is shown that the smaller the generalized structure constant and the outer scale of turbulence are, and the bigger the inner scale of turbulence is, the smaller the normalized propagation factor is, the bigger the Strehl ratio is, and the better the beam quality of GSM vortex beams in atmospheric turbulence is. The variation of beam quality with the generalized exponent α is nonmonotonic, when α = 3.11, the beam quality of the GSM vortex beams is the poorest through non-Kolmogorov atmospheric turbulence. GSM vortex beams is less affected by turbulence than GSM non-vortex beams under certain condition, and will be useful in long-distance free-space optical communications.