Repetition of the same stimulus leads to a reduction in neural activity known as repetition suppression (RS). In functional magnetic resonance imaging (fMRI), RS is found for multiple object categories. One proposal is that RS reflects locally based "within-region" changes, such as neural fatigue. Thus, if a given region shows RS across changes in stimulus size or view, then it is inferred to hold size- or view-invariant representations. An alternative hypothesis characterizes RS as a consequence of "top-down" between-region modulation. Differentiating between these accounts is central to the correct interpretation of fMRI RS data. It is also unknown whether the same mechanisms underlie RS to identical stimuli and RS across changes in stimulus size or view. Using fMRI, we investigated RS within a body-sensitive network in human visual cortex comprising the extrastriate body area (EBA) and the fusiform body area (FBA). Both regions showed RS to identical images of the same body that was unaffected by changes in body size or view. Dynamic causal modeling demonstrated that changes in backward, top-down (FBA-to-EBA) effective connectivity play a critical role in RS. Furthermore, only repetition of the identical image showed additional changes in forward connectivity (EBA-to-FBA). These results suggest that RS is driven by changes in top-down modulation, whereas the contribution of "feedforward" changes in connectivity is dependent on the precise nature of the repetition. Our results challenge previous interpretations regarding the underlying nature of neural representations made using fMRI RS paradigms.