The low-order harmonic generation induced by a strong laser field produces a bright, ultrashort, supercontinuum radiation ranging from the terahertz to ultraviolet band. By controlling the phase-delay and ellipticity of the bi-chromatic laser fields, the third harmonic generation is experimentally and theoretically investigated for elucidating the mechanism of the low-order harmonics. The third harmonic generation is found to be strongly suppressed in the counter-rotating bi-chromatic laser field due to the selection rule for harmonic emissions. The continuum-continuum transition in the strong field approximation is extended to explain the third harmonic generation as a function of the phase delay and ellipticity of the bi-chromatic laser fields. Compared with the semi-classical photocurrent model, the continuum-continuum transition on the basis of quantum-mechanical treatment achieves better agreement with the experimental observations. Our work indicates that the overlapping in continuum states via different quantum paths of a single electron plays a role in low-order harmonics generation under elliptical bi-chromatic laser fields.