Short-term memory (STM) for serially presented visual items is disrupted by task-irrelevant, to-be-ignored speech. Five experiments investigated the extent to which irrelevant speech is processed semantically by contrasting the following two hypotheses: (1) semantic processing of irrelevant speech is limited and does not interfere with serial STM or (2) irrelevant speech is routinely processed semantically at an interlexical level to allow for the detection of stimuli that are of potential relevance for the individual. We tested these hypotheses by comparing the disruption of serial recall by distractor sentences with semantically expected endings to that of sentences with semantically unexpected endings. Sentences with unexpected endings consistently produced more disruption than sentences with expected endings. Phonologically expected, but semantically unexpected endings had the same effect, demonstrating that a semantic and not a phonological mismatch is responsible for the increased disruption. In all five experiments, the semantic mismatch effect was not reduced after repeated exposure to sentences with unexpected endings. The results suggest that (1) semantic processing occurs at the interlexical level of irrelevant speech, which requires the integration of words into the sentence context and (2) the semantic content of a distractor interferes with the maintenance of serial information in short-term-memory if it is unexpected. (PsycINFO Database Record (c) 2019 APA, all rights reserved).