Rotation of cages within fish farm leases and the subsequent fallowing of areas of seabed is commonly used to allow recovery of infaunal communities following periods of organic enrichment. To investigate the effect of different background environmental conditions on recovery response, two Atlantic salmon (Salmo salar) fish farm sites in southeast Tasmania were sampled over two commercial fallowing cycles. Despite similar stocking levels and feed input there were significant differences in the way in which sediment at each farm responded to the cessation of fish stocking. Sediments at both farms showed some improvement in the community structure over a three month fallow period, but the community structure only recovered to that present before stocking not to that at the reference sites. The similarity of the impact sites to the reference sites increased from ca. 25% to 31% at one site and 11% to 27% at the other after fallowing. Rate and extent of recovery were affected by farm location, initial impact of the sediments, and length of fallow period. Initial recovery was faster at the more sheltered site than at the more exposed site, possibly reflecting differences in environmental resilience with the more sheltered location better able to assimilate organic inputs. Accordingly general fallowing management protocols may need to be adapted to reflect differences between sites. The findings of this study suggest that the recovery response of benthic communities can be predicted once baseline conditions are understood.