Herbivory defence chemicals in plants can affect higher trophic levels such as predators and parasitoids, but the impact on pollinators has been overlooked. We show that defensive plant chemicals can damage pollinator fitness when expressed in pollen. Crop lupins (Lupinus species from Europe and South America) accumulate toxic quinolizidine alkaloids in vegetative tissues, conferring resistance to herbivorous pests such as aphids. We identified the alkaloid lupanine and its derivatives in lupin pollen, and then provided this compound at ecologically-relevant concentrations to queenless microcolonies of bumblebees (Bombus terrestris) in their pollen to determine how foraging on these crops may impact bee colony health and fitness. Fewer males were produced by microcolonies provided with lupanine-treated pollen and they were significantly smaller than controls. This impact on males was not linked to preference as workers willingly fed lupanine-treated pollen to larvae, even though it was deleterious to colony health. Agricultural systems comprising large monocultures of crops bred for herbivore resistance can expose generalist pollinators to deleterious levels of plant compounds, and the broader environmental impacts of crop resistance must thus be considered.