Recognition of behavioral types can facilitate the evolution of cooperation by enabling altruistic behavior to be directed at other cooperators and withheld from defectors. While much is known about the tendency for recognition to promote cooperation, relatively little is known about whether such a capacity can coevolve with the social behavior it supports. Here we use evolutionary game theory and multi-population dynamics to model the coevolution of social behavior and recognition. We show that conditional harming behavior enables the evolution and stability of social recognition, whereas conditional helping leads to a deterioration of recognition ability. Expanding the model to include a complex game where both helping and harming interactions are possible, we find that conditional harming behavior can stabilize recognition, and thereby lead to the evolution of conditional helping. Our model identifies a novel hypothesis for the evolution of cooperation: conditional harm may have coevolved with recognition first, thereby helping to establish the mechanisms necessary for the evolution of cooperation.