Categorical spatial information is considered more useful for recognizing objects, and coordinate spatial information for guiding actions--for example, during navigation or grasping. In contrast with this assumption, we hypothesized that buildings, unlike other categories of objects, require both categorical and coordinate spatial information in order to be recognized. This hypothesis arose from evidence that right-brain-damaged patients have deficits in both coordinate judgments and recognition of buildings and from the fact that buildings are very useful for guiding navigation in urban environments. To test this hypothesis, we assessed 210 healthy college students while they performed four different tasks that required categorical and coordinate judgments and the recognition of common objects and buildings. Our results showed that both categorical and coordinate spatial representations are necessary to recognize a building, whereas only categorical representations are necessary to recognize an object. We discuss our data in view of a recent neural framework for visuospatial processing, suggesting that recognizing buildings may specifically activate the parieto-medial-temporal pathway.