One influential account asserts that the anterior temporal lobe (ATL) is a domain-general hub for semantic memory. Other evidence indicates it is part of a domain-specific social cognition system. Arbitrating these accounts using functional magnetic resonance imaging has previously been difficult because of magnetic susceptibility artifacts in the region. The present study used parameters optimized for imaging the ATL, and had subjects encode facts about unfamiliar people, buildings, and hammers. Using both conjunction and region of interest analyses, person-selective responses were observed in both the left and right ATL. Neither building-selective, hammer-selective nor domain-general responses were observed in the ATLs, although they were observed in other brain regions. These findings were supported by "resting-state" functional connectivity analyses using independent datasets from the same subjects. Person-selective ATL clusters were functionally connected with the brain's wider social cognition network. Rather than serving as a domain-general semantic hub, the ATLs work in unison with the social cognition system to support learning facts about others.