The prefrontal cortex (PFC) has a central role in working memory (WM). Resistance to distraction is considered a fundamental feature of WM and PFC neuronal activity. However, although unexpected stimuli often disrupt our work, little is known about the underlying neuronal mechanisms involved. In the present study, we investigated whether irregularly presented distracters disrupt WM task performance and underlying neuronal activity. We recorded single neuron activity in the PFC of 2 monkeys performing WM tasks and investigated effects of auditory and visual distracters on WM performance and neuronal activity. Distracters impaired memory task performance and affected PFC neuronal activity. Distraction that was of the same sensory modality as the memorandum was more likely to impair WM performance and interfere with memory-related neuronal activity than information that was of a different sensory modality. The study also shows that neurons not involved in memory processing in less demanding conditions may become engaged in WM processing in more demanding conditions. The study demonstrates that WM performance and underlying neuronal activity are vulnerable to irregular distracters and suggests that the PFC has mechanisms that help to compensate for disruptive effects of external distracters.