Summary
The processing of voices and faces is known to interact, for example, when recognizing
other persons. However, few studies focus on both directions of this interaction,
including the influence of incongruent visual stimulation on voice perception. In
the present study, we implemented an interference paradigm involving 1152 videos of
faces with either gender-congruent or gender-incongruent voices. Participants were
asked to categorize the gender of either the face or the voice via key press. Task
(face-based vs. voice-based gender categorization task) was manipulated both block-wise
(relatively low executive control demands) and in a mixed block (relatively high executive
control demands due to trial-by-trial task switches). We aimed at testing whether
and how gender-incongruent stimuli negatively affected gender categorization speed
and accuracy. The results indicate significant congruency effects in both directions
– gender-incongruent visual information negatively affected voice categorization time
and errors, and gender-incongruent voices affected visual face categorization. However,
the former effect was stronger, supporting theories postulating visual dominance in
face-voice integration. Congruency effects, which were not significantly reduced over
the course of the experiment, were larger under high executive control demands (task
switches), suggesting the availability of fewer attentional resources for incongruency
resolution. Overall, voices generally appear to be processed in conjunction with facial
information, which yields enhanced processing for more authentic voices, that is,
voices that do not violate face-based expectancies. The data strengthen theories of
face-voice processing emphasizing strong interaction between both processing channels.
Key Words
To read this article in full you will need to make a payment
Purchase one-time access:
Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online accessOne-time access price info
- For academic or personal research use, select 'Academic and Personal'
- For corporate R&D use, select 'Corporate R&D Professionals'
Subscribe:
Subscribe to Journal of VoiceAlready a print subscriber? Claim online access
Already an online subscriber? Sign in
Register: Create an account
Institutional Access: Sign in to ScienceDirect
References
- Modularity of Mind: An Essay on Faculty Psychology.MIT Press, Cambridge, MA1983
- Perception and Communication.Oxford University Press, New York1958
- A unified coding strategy for processing faces and voices.Trends Cogn Sci. 2013; 17: 263-271
- Structure and function in acquired prosopagnosia: lessons from a series of 10 patients with brain damage.J Neuropsychol. 2008; 2: 197-225
- The mismatch negativity: a review of underlying mechanisms.Clin Neurophysiol. 2009; 120: 453-463
- Progressive associative phonagnosia: a neuropsychological analysis.Neuropsychologia. 2010; 48: 1104-1114
- Phonagnosia: a voice homologue to prosopagnosia.Vis Cogn. 2014; 22: 1031-1033
- Phonagnosia: a dissociation between familiar and unfamiliar voices.Cortex. 1988; 24: 195-209
- Understanding face recognition.Br J Psychol. 1986; 77: 305-327
- Integrating voice recognition into models of person perception.J Cogn Psychol. 2012; 24: 409-419
- Intra- and inter-modal repetition priming of familiar faces and voices.Br J Psychol. 1997; 88: 143-156
- Hearing faces and seeing voices: the integration and interaction of face and voice processing.Psychol Belg. 2014; 54: 266-281
- Integrating face and voice in person perception.Trends Cogn Sci. 2007; 11: 535-543
- Human sensory dominance.Percept Psychophys. 1974; 16: 409-412
- Explaining the Colavita visual dominance effect.Prog Brain Res. 2009; 176: 245-258
- Automatic visual bias of perceived auditory location.Psychon Bull Rev. 1998; 5: 482-489
- The role of visual-auditory “compellingness” in the ventriloquism effect: implications for transitivity among the spatial senses.Percept Psychophys. 1981; 30: 557-564
- Visual illusion induced by sound.Cogn Brain Res. 2002; 14: 147-152
- Thinking the voice: neural correlates of voice perception.Trends Cogn Sci. 2004; 8: 129-135
- Understanding voice perception.Br J Psychol. 2011; 102: 711-725
- Hearing lips and seeing voices.Nature. 1976; 264: 746-748
- Integrating speech information across talkers, gender, and sensory modality: female faces and male voices in the McGurk effect.Percept Psychophys. 1991; 50: 524-536
- The perception of emotions by ear and by eye.Cogn Emot. 2000; 14: 28-311
- MEG demonstrates a supra-additive response to facial and vocal emotion in the right superior temporal sulcus.Proc Natl Acad Sci USA. 2009; 106: 20010-20015
- Evidence for the integration of audiovisual emotional information at the perceptual level of processing.Eur J Cogn Psychol. 2004; 16: 769-790
- Perception of facial expressions and voices and of their combination in the human brain.Cortex. 2005; 41: 49-59
- Facial biases on vocal perception and memory.Acta Psychol. 2017; 177: 54-68
- Hearing facial identities: brain correlates of face–voice integration in person identification.Cortex. 2011; 47: 1026-1037
- Learned face–voice pairings facilitate visual search.Psychon Bull Rev. 2015; 22: 429-436
- The neural network sustaining the crossmodal processing of human gender from faces and voices: an fMRI study.Neuroimage. 2011; 54: 1654-1661
- Auditory–visual crossmodal integration in perception of face gender.Curr Biol. 2007; 17: 1680-1685
- An interference effect of voice presentation on face gender discrimination task: evidence from event-related potentials.Int Congr Ser. 2005; 1278: 156-159
- When two become one: temporally dynamic integration of the face and voice.J Exp Soc Psychol. 2011; 47: 259-263
- Discrimination of voice gender in the human auditory cortex.Neuroimage. 2015; 15: 208-214
- McGurk effect in gender identification: vision trumps audition in voice judgments.J Gen Psychol. 2017; 144: 59-68
- Top-down and bottom-up modulation in processing bimodal face/voice stimuli.BMC Neurosci. 2010; 11: 36
- Studies of interference in serial verbal reactions.J Exp Psychol. 1935; 18: 643-662
- Half a century of research on the Stroop effect: an integrative review.Psychol Bull. 1991; 109: 163-203
- Domain generality versus modality specificity: the paradox of statistical learning.Trends Cogn Sci. 2015; 19: 117-125
- Shifting intentional set: exploring the dynamic control of tasks.in: Umiltà C. Moscovitch M. Attention and Performance Series. Attention and Performance 15: Conscious and Nonconscious Information Processing. MIT Press, Cambridge, MA1994: 421-452
- Costs of a predictible switch between simple cognitive tasks.J Exp Psychol Gen. 1995; 124: 207-231
- Control and interference in task switching—a review.Psychol Bull. 2010; 136: 849-874
- Effect of attentional load on audiovisual speech perception: evidence from ERPs.Front Psychol. 2014; 15: 727
- Concurrent working memory load can reduce distraction.Proc Natl Acad Sci USA. 2005; 102: 16524-16529
- When it helps to be misled: facilitative effects of increasing the frequency of conflicting stimuli in a Stroop-like task.Mem Cognit. 1979; 3: 166-174
- Reconfiguration of processing mode prior to task performance.J Exp Psychol Learn Mem Cogn. 1996; 22: 1423-1442
- Earwitness testimony 2: voices, faces and context.Appl Cogn Psychol. 1997; 11: 527-541
- Interference in eyewitness and earwitness recognition.Appl Cogn Psychol. 2011; 25: 112-118
- When the face fits: recognition of celebrities from matching and mismatching faces and voices.Memory. 2014; 22: 284-294
- I recognise you but I can't place you: an investigation of familiar-only experiences during tests of voice and face recognition.Q J Exp Psychol. 1998; 51: 179-195
- Recalling semantic information about newly learned faces and voices.Memory. 2012; 20: 527-534
- It is more difficult to retrieve a familiar person's name and occupation from their voice than from their blurred face.Memory. 2009; 17: 830-839
- The role of working memory in visual selective attention.Science. 2001; 291: 1803-1806
- Hearing facial identities.Q J Exp Psychol. 2007; 60: 1446-1456
- The Stroop phenomenon and its use in the study of perceptual, cognitive, and response processes.Mem Cogn. 1973; 1: 106-120
- The role of audiovisual asynchrony in person recognition.Q J Exp Psychol. 2010; 63: 23-30
- Changes in sensory dominance during childhood: converging evidence from the colavita effect and the sound-induced flash illusion.Child Dev. 2013; 84: 604-616
- Preschool children and adults flexibly shift their preferences for auditory versus visual modalities, but do not exhibit auditory dominance.J Exp Child Psychol. 2012; 112: 338-350
Article info
Publication history
Published online: October 27, 2018
Accepted:
September 25,
2018
Identification
Copyright
© 2018 The Voice Foundation. Published by Elsevier Inc. All rights reserved.
