The surface morphology of III-V semiconductor nanowires (NWs) protected by an arsenic cap and subsequently evaporated in ultrahigh vacuum is investigated with scanning tunneling microscopy and scanning transmission electron microscopy. We show that the changes of the surface morphology as a function of the NW composition and the nature of the seed particles are intimately related to the formation and reaction of surface point defects. Langmuir evaporation close to the congruent evaporation temperature causes the formation of vacancies which nucleate and form vacancy islands on {110} sidewalls of self-catalyzed InAs NWs. However, for annealing temperatures much smaller than the congruent temperature, a new phenomenon occurs: group III vacancies form and are filled by excess As atoms, leading to surface AsGa antisites. The resulting Ga adatoms nucleate with excess As atoms at the NW edges, producing monoatomic-step islands on the {110} sidewalls of GaAs NWs. Finally, when gold atoms diffuse from the seed particle onto the {110} sidewalls during evaporation of the protective As cap, Langmuir evaporation does not take place, leaving the sidewalls of InAsSb NWs atomically flat.