Phenotypic plasticity is considered the major means by which plants cope with environmental heterogeneity. Although ubiquitous in nature, actual phenotypic plasticity is far from being maximal. This has been explained by the existence of internal limits to its expression. However, phenotypic plasticity takes place within an ecological context and plants are generally exposed to multifactor environments and to simultaneous interactions with many species. These external, ecological factors may limit phenotypic plasticity or curtail its adaptive value, but seldom have they been considered because limits to plasticity have typically addressed factors internal to the plant. We show that plastic responses to abiotic factors are reduced under situations of conservative resource use in stressful and unpredictable habitats, and that extreme levels in a given abiotic factor can negatively influence plastic responses to another factor. We illustrate how herbivory may limit plant phenotypic plasticity because damaged plants can only rarely attain the optimal phenotype in the challenging environment. Finally, it is examined how phenotypic changes involved in trait-mediated interactions can entail costs for the plant in further interactions with other species in the community. Ecological limits to plasticity must be included in any realistic approach to understand the evolution of plasticity in complex environments and to predict plant responses to global change.