Perceptual expertise is traditionally associated with enhanced brain activity in response to objects of expertise in category-selective visual cortex, primarily face-selective regions. We reevaluated this view by investigating whether the brain activity associated with expertise in object recognition is limited to category-selective cortex and specifically whether the extent of expertise-related activity manifests automatically or whether it can be top-down modulated. We conducted 2 functional magnetic resonance imaging studies comparing changes in hemodynamic activity associated with car expertise in a conventional 1-back task (Experiment 1) and when the task relevance of cars was explicitly manipulated (Experiment 2). Whole-brain analysis unveiled extensive expertise-related activity throughout the visual cortex, starting as early as V1 and extending into nonvisual areas. However, when the cars were task irrelevant, the expertise-related activity drastically diminished, indeed, becoming similar to the activity elicited by cars in novices. We suggest that expertise entails voluntary top-down engagement of multiple neural networks in addition to stimulus-driven activation associated with perceptual mechanisms.