Four species of fast-growing pioneer tree species in the genus Cecropia exist in the forests in central Panama. Cecropia insignis is dominant in old-growth forests but absent from nearby secondary forests; C. obtusifolia, and C. peltata are abundant in secondary forests but rare in old-growth forest, and C. longipes is uncommon in both. To determine whether Cecropia habitat associations are a consequence of local dispersal or differences in recruitment success, we grew seedlings of these species in common gardens in large treefall gaps in secondary and old-growth forest. In contrast to the observed adult distribution, only C. insignis grew significantly over 16 months in secondary forests; remaining species were heavily browsed by herbivores. C. insignis also grew and survived best in old-growth forest. Differences in susceptibility to herbivory did not result from an ant defence mutualism; none of the plants were colonised by ants during the experiment. To test whether C. insignis, the species least susceptible to herbivory, trades off investment in growth in favour of defence, we also grew the four Cecropia species in a screened growing house under light conditions comparable to large forest gaps. Contrary to expectation, species growth rates were similar; only C. peltata grew significantly faster than C. insignis. These results suggest that (1) conditions in ~40-year-old secondary forests no longer support the recruitment of Cecropia species, which are canopy dominants there; and (2) among congeners, differences in plant traits with little apparent cost to growth can have large impacts on recruitment by affecting palatability to herbivores.