Abstract
Seeing incongruent visual speech can alter the auditory phonetic percept. In
the McGurk fusion illusion the auditory percept is a single consonant
different from both the acoustic and the visual consonant. In the McGurk
combination illusion the auditory percept contains both consonants. It
remains unclear why some audiovisual stimuli elicit combination illusions. It
is also unexplored how actual consonant combinations integrate
audiovisually. Here we investigate the integration of audiovisually congruent
and incongruent combinations of /aba/, /aga/, /ada/, /abga/, and /abda/. We
found that visual stimuli containing a bilabial component (/aba/, /abga/ and
/abda/) all facilitated perception of both acoustic consonant clusters
regardless of audiovisual congruence. This is surprising because
incongruent visual stimuli usually lead to illusory, hence incorrect,
responses. The effect was most likely caused by the visual bilabial closure
as we found a general increase in bilabial responses. Visual consonant
clusters also produced combination illusions for auditory /aga/ and /ada/
and these responses were similar to the combination illusion induced by
visual /aba/. The velar and alveolar components of visual consonant
clusters did, however, also have an effect on auditory perception as they
influenced perception of auditory /aba/ in inducing novel combination
illusions where subjects perceived /abda/. Acoustic consonant clusters
dubbed onto visual velar or alveolar stimuli created novel illusions. For
example, acoustic dubbed /abga/ dubbed onto visual /aga/ created an
illusion of hearing /agda/ or /adga/. This illusion could be due to the acoustic
/b/ and visual /g/ creating a fusion illusion of hearing /d/ while leaving
perception of acoustic /g/ unaffected. This indicates that opening, closing
and release stages of consonants can integrate differentially. We
hypothesise that this may explain why some audiovisual combinations
produce combination illusions while others produce fusion or visual
dominance illusions.
the McGurk fusion illusion the auditory percept is a single consonant
different from both the acoustic and the visual consonant. In the McGurk
combination illusion the auditory percept contains both consonants. It
remains unclear why some audiovisual stimuli elicit combination illusions. It
is also unexplored how actual consonant combinations integrate
audiovisually. Here we investigate the integration of audiovisually congruent
and incongruent combinations of /aba/, /aga/, /ada/, /abga/, and /abda/. We
found that visual stimuli containing a bilabial component (/aba/, /abga/ and
/abda/) all facilitated perception of both acoustic consonant clusters
regardless of audiovisual congruence. This is surprising because
incongruent visual stimuli usually lead to illusory, hence incorrect,
responses. The effect was most likely caused by the visual bilabial closure
as we found a general increase in bilabial responses. Visual consonant
clusters also produced combination illusions for auditory /aga/ and /ada/
and these responses were similar to the combination illusion induced by
visual /aba/. The velar and alveolar components of visual consonant
clusters did, however, also have an effect on auditory perception as they
influenced perception of auditory /aba/ in inducing novel combination
illusions where subjects perceived /abda/. Acoustic consonant clusters
dubbed onto visual velar or alveolar stimuli created novel illusions. For
example, acoustic dubbed /abga/ dubbed onto visual /aga/ created an
illusion of hearing /agda/ or /adga/. This illusion could be due to the acoustic
/b/ and visual /g/ creating a fusion illusion of hearing /d/ while leaving
perception of acoustic /g/ unaffected. This indicates that opening, closing
and release stages of consonants can integrate differentially. We
hypothesise that this may explain why some audiovisual combinations
produce combination illusions while others produce fusion or visual
dominance illusions.
Original language | English |
---|---|
Publication date | 2018 |
Number of pages | 1 |
Publication status | Published - 2018 |
Event | 19th Annual International Multisensory Research Forum - Toronto, Canada Duration: 14 Jun 2018 → 17 Jun 2018 Conference number: 19 |
Conference
Conference | 19th Annual International Multisensory Research Forum |
---|---|
Number | 19 |
Country/Territory | Canada |
City | Toronto |
Period | 14/06/2018 → 17/06/2018 |