Although autism spectrum disorder (ASD) is the prototypical psychiatric disorder of social impairment, several if not most psychiatric disorders are characterized by prominent impairments in social functioning. A challenge in clinically assessing and describing social impairment is that it has been variably defined and can be difficult to measure. In this article we consider the psychiatric differential diagnosis of social impairment within the DSM-5 framework. We describe the features of social impairment in 13 DSM-5 disorders from a developmental perspective and highlight diagnostic factors that differentiate among the disorders, including the main features of social impairment, verbal communication, nonverbal communication, course of social impairment, social cognition, and key features of accompanying neuropsychiatric symptoms. We conclude by describing an approach for assessing social impairment across the lifespan.