In mixed systems of trapped ions and cold atoms, the ions and atoms can coexist at different temperatures. This is primarily due to their different trapping and cooling mechanisms. The key questions of how ions can cool collisionally with cold atoms and whether the combined system allows stable coexistence, need to be answered. Here we experimentally demonstrate that rubidium ions cool in contact with magneto-optically trapped rubidium atoms, contrary to the general experimental expectation of ion heating. The cooling process is explained theoretically and substantiated with numerical simulations, which include resonant charge exchange collisions. The mechanism of single collision swap cooling of ions with atoms is discussed. Finally, it is experimentally and numerically demonstrated that the combined ion-atom system is intrinsically stable, which is critical for future cold chemistry experiments with such systems.