Adaptive plasticity allows organisms to cope with environmental change, thereby increasing the population's long-term fitness. However, individual selection can only compare the fitness of individuals within each generation: if the environment changes more slowly than the generation time (i.e., a coarse-grained environment) a population will not experience selection for plasticity even if it is adaptive in the long-term. How does adaptive plasticity then evolve? One explanation is that, if competing alleles conferring different degrees of plasticity persist across multiple environments, natural selection between genetic lineages could select for adaptive plasticity (lineage selection). We show that adaptive plasticity can evolve even in the absence of such lineage selection. Instead, we propose that adaptive plasticity in coarse-grained environments evolves as a by-product of inefficient short-term natural selection: populations that rapidly evolve their phenotypes in response to selective pressures follow short-term optima, with the result that they have reduced long-term fitness across environments. Conversely, populations that accumulate limited genetic change within each environment evolve long-term adaptive plasticity even when plasticity incurs short-term costs. These results remain qualitatively similar regardless of whether we decrease the efficiency of natural selection by increasing the rate of environmental change or decreasing mutation rate, demonstrating that both factors act via the same mechanism. We demonstrate how this mechanism can be understood through the concept of learning rate. Our work shows how plastic responses that are costly in the short term, yet adaptive in the long term, can evolve as a by-product of inefficient short-term selection, without selection for plasticity at either the individual or lineage level.