In this work, the condensation characteristics of droplets considering the non-condensable gas with different interaction effects are numerically studied utilizing a multicomponent multiphase thermal lattice Boltzmann (LB) model, with a special focus on the asymmetric nature induced by the interaction effect. The results demonstrate that for isolated-like growth with negligible interactions, the condensation characteristics, that is, the concentration profile, the temperature distribution, and the flow pattern, are typically symmetric in nature. For the growth regime in a pattern, the droplet has to compete with its neighbors for catching vapor, which leads to an overlapping concentration profile (namely the interaction effect). The distribution of the condensation flux on the droplet surface is consequently modified, which contributes to the asymmetric flow pattern and temperature profile. The condensation characteristics for droplet growth in a pattern present an asymmetric nature. Significantly, the asymmetric condensation flux resulting from the interaction effect can induce droplet motion. The results further demonstrate that the interaction strongly depends on the droplet's spatial and size distribution, including two crucial parameters, namely the inter-distance and relative size of droplets. The asymmetric condensation characteristics are consequently dependent on the difference in the interaction intensities on both sides of the droplet. Finally, we demonstrate numerically and theoretically that the evolution of the droplet radius versus time can be suitably described by a power law; the corresponding exponent is kept at a constant of 0.50 for isolated-like growth and is strongly sensitive to the interaction effect for the growth in a pattern.