The orientation and ellipticity of terahertz (THz) polarization generated by a two-color strong field not only casts light on underlying mechanisms of laser-matter interaction, but also plays an important role for various applications. We develop the Coulomb-corrected classical trajectory Monte Carlo (CTMC) method to well reproduce the joint measurements, that the THz polarization generated by the linearly polarized 800 nm and circularly polarized 400 nm fields is independent on two-color phase delay. The trajectory analysis shows that the Coulomb potential twists the THz polarization by deflecting the orientation of asymptotic momentum of electron trajectories. Further, the CTMC calculations predict that, the two-color mid-infrared field can effectively accelerate the electron rapidly away from the parent core to relieve the disturbance of Coulomb potential, and simultaneously create large transverse acceleration of trajectories, leading to the circularly polarized THz radiation.