Information regarding plant damage by insects in the past is essential to explore impacts of climate change on herbivory. We asked whether insect herbivory measured from herbarium specimens reflects the levels of herbivory occurring in nature at the time of herbarium sampling. We compared herbivory measurements between herbarium specimens collected by botany students and ecological samples collected simultaneously by the authors by a method that minimized unconscious biases, and asked herbarium curators to select one of two plant specimens, which differed in leaf damage, for their collections. Both collectors and curators generally preferred specimens with lesser leaf damage, but the strength of this preference varied among persons. In addition, the differences in measured leaf damage between ecological samples and herbarium specimens varied among plant species and increased with the increase in field herbivory. Consequently, leaf damage in herbarium specimens did not correlate with the actual level of herbivory. We conclude that studies of herbarium specimens produce biased information on past levels of herbivory, because leaf damage measured from herbarium specimens not only underestimates field herbivory, but it is not proportional to the level of damage occurring in nature due to multiple factors that cannot be controlled in data analysis.