A method for evaluating audio-visual scene analysis in multi-talker environments

Kasper D. Lund*, Axel Ahrens, Torsten Dau

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingArticle in proceedingsResearchpeer-review

47 Downloads (Pure)

Abstract

In cocktail-party environments, listeners are able to comprehend and localize multiple simultaneous talkers. With current virtual reality (VR) technology and virtual acoustics it has become possible to present an audio-visual cocktail-party in a controlled laboratory environment. A new continuous speech corpus with ten monologues from five female and five male talkers was designed and recorded. Each monologue contained a substantially different topic. Using an egocentric interaction method in VR, subjects were asked to label perceived talkers according to source position and content of speech, while varying the number of simultaneously presented talkers. With an increasing number of talkers, the subjects’ accuracy in performing this task was found to decrease. When more than six talkers were in a scene, the number of talkers was underestimated and the azimuth localization error increased. With this method, a new approach is presented to gauge listeners’ ability to analyze complex audio-visual scenes.
Original languageEnglish
Title of host publicationProceedings of the International Symposium on Auditory and Audiological Research : Auditory Learning in Biological and Artificial Systems
Volume7
PublisherThe Danavox Jubilee Foundation
Publication date2020
Pages357-364
Publication statusPublished - 2020
EventInternational Symposium on Auditory and Audiological Research: Auditory Learning in Biological and Artificial Systems - Nyborg, Denmark
Duration: 21 Aug 201923 Aug 2019
http://isaar.eu

Conference

ConferenceInternational Symposium on Auditory and Audiological Research
CountryDenmark
CityNyborg
Period21/08/201923/08/2019
Internet address
SeriesProceedings of the International Symposium on Audiological and Auditory Research
Volume7
ISSN2596-5522

Keywords

  • Auditory scene analysis
  • Speech perception
  • Virtual Reality

Fingerprint Dive into the research topics of 'A method for evaluating audio-visual scene analysis in multi-talker environments'. Together they form a unique fingerprint.

Cite this