Substantial controversy surrounds the efficacy of biological control insects to reduce densities of Centaurea stoebe, a widespread, aggressive invasive plant in North America. We developed a graphical model to conceptualize the conditions required to explain the current contradictory findings, and then employed a series of manipulations to evaluate C. stoebe responses to herbivores. We manipulated soil nitrogen and competition in a field population and measured attack rates of a foliage and seed feeder (Larinus minutus), two gall flies (Urophora spp.), and a root feeder (Cyphocleonus achates), as well as their effects on the growth and reproduction of C. stoebe. Nitrogen limitation and competing vegetation greatly reduced C. stoebe growth. L. minutus most intensively reduced seed production in low-nitrogen soils, and removal of neighboring vegetation increased Larinus numbers per flower head and the percentage of flowers attacked by 15% and 11%, respectively. Cyphocleonus reduced flower production and aboveground biomass over two years, regardless of resources or competition. Our results, in conjunction with other published studies, demonstrate that positive, neutral, and negative plant growth responses to herbivory can be generated. However, under realistic field conditions and in the presence of multiple herbivores, our work repudiates earlier studies that indicate insect herbivores increase C. stoebe dominance.