Phonetic Recalibration in Audiovisual Speech

Excerpt

In 2003, we (Bertelson et al. 2003) reported that phonetic recalibration induced by McGurk-like stimuli can indeed be observed. We termed the phenomenon “recalibration” in analogy with the much better known “spatial recalibration,” as we considered it a readjustment or a fine-tuning of an already existing phonetic representation. In the same year, and in complete independence, Norris (2003) reported a very similar phenomenon they named “perceptual learning in speech.” The basic procedure in both studies was very similar: Listeners were presented with a phonetically ambiguous speech sound and another source of contextual information that disambiguated that sound. In our study, we presented listeners a sound halfway between /b/ and /d/ with as context the video of a synchronized face that articulated /b/ or /d/ (in short, lip-read information), whereas in the study of Norris (2003), an ambiguous /s/-/f/ sound was heard embedded in the context of an f- or s-biasing word (e.g., “witlo-s/f” was an f-biasing context because “witlof” is a word in Dutch meaning “chicory,” but “witlos” is not a Dutch word). Recalibration (or perceptual learning) was subsequently measured in an auditory-only identification test in which participants identified members of a speech continuum. Recalibration manifested itself as a shift in phonetic categorization toward the contextually defined speech environment. Listeners thus increased their report of sounds consistent with the context they had received before, so more /b/ responses after exposure to lip-read /b/ rather than lip-read /d/, and more /f/ responses after exposure to f-biasing words rather than /s/-biasing words. Presumably, this shift reflected an adjustment of the phoneme boundary that had helped listeners to understand speech better in the prevailing input environment.

After these seminal reports, there have been a number of studies that examined phonetic recalibration in more detail (Baart and Vroomen 2010a, 2010b; Cutler et al. 2008; Eisner and McQueen 2005, 2006; Jesse and McQueen 2007; Kraljic et al. 2008a, 2008b; Kraljic and Samuel 2005, 2006, 2007; McQueen et al. 2006a, 2006b; Sjerps and McQueen 2010; Stevens 2007; van Linden and Vroomen et al. 2007, 2008; Vroomen and Baart 2009a, 2009b; Vroomen et al. 2004, 2007). In what follows, we will provide an overview of this literature and, given the topic of this book, we will focus on the audiovisual case.