This article proposes an adaptive robust formation control scheme for the connected and autonomous vehicle (CAV) swarm system by utilizing swarm property, diffeomorphism transformation, and constraint following. The control design is processed by starting from a 2-D dynamics model with (possibly fast) time varying but bounded uncertainty. The uncertainty bounds are unknown. For compact formation, the CAV system is treated as an artificial swarm system, for which the ideal swarm performance is taken as a desired constraint. By this, formation control is converted into a problem of constraint following and then a performance measure β is defined as the control object to evaluate the constraint following error. For collision avoidance, a diffeomorphism transformation on space measure between two vehicles is creatively performed, by which the space measure is positive restricted. For uncertainty handling, an adaptive robust control scheme is proposed to render the β -measure to be uniformly bounded and uniformly ultimately bounded, that is, drive the controlled (CAV) swarm system to follow the desired constraint approximatively. As a result, the system can achieve the ideal swarm performance; thereout, compact formation is realized, regardless of the uncertainty. The main contribution of this article is exploring a 2-D formation control scheme for (CAV) swarm system under the consideration of collision avoidance and time-varying uncertainty.