Many studies have suggested that the intraparietal sulcus (IPS), particularly in the dominant hemisphere, is crucially involved in numerical comparisons. However, this parietal structure has been found to be involved in other tasks that require spatial processing or visuospatial attention as well. fMRI was used to investigate three different magnitude comparisons in an event-related-block design: (a) Which digit is larger in numerical value (e.g., 2 or 5)? (b) Which digit is brighter (e.g., 3 or 3)? (c) Which digit is physically larger (e.g., 3 or 3)? Results indicate a widespread cortical network including a bilateral activation of the intraparietal sulci for all different comparisons. However, by computing contrasts of brain activation between the respective comparison conditions and applying a cortical distance effect as an additional criterion, number-specific activation was revealed in left IPS and right temporal regions. These results indicate that there are both commonalities and differences in the spatial layout of the brain systems for numerical and physical comparisons and that especially the left IPS, while involved in magnitude comparison in general, plays a special role in number comparison.