Outcome measures based on classification performance fail to predict the intelligibility of binary-masked speech

Abigail Anne Kressner, Tobias May, Christopher J. Rozell

    Research output: Contribution to journalJournal articleResearchpeer-review

    398 Downloads (Pure)

    Abstract

    To date, the most commonly used outcome measure for assessing ideal binary mask estimation algorithms is based on the difference between the hit rate and the false alarm rate (H-FA). Recently, the error distribution has been shown to substantially affect intelligibility. However, H-FA treats each mask unit independently and does not take into account how errors are distributed. Alternatively, algorithms can be evaluated with the short-time objective intelligibility (STOI) metric using the reconstructed speech. This study investigates the ability of H-FA and STOI to predict intelligibility for binary-masked speech using masks with different error distributions. The results demonstrate the inability of H-FA to predict the behavioral intelligibility and also illustrate the limitations of STOI. Since every estimation algorithm will make errors that are distributed in different ways, performance evaluations should not be made solely on the basis of these metrics.
    Original languageEnglish
    JournalJournal of the Acoustical Society of America
    Volume139
    Issue number6
    Pages (from-to)3033–3036
    ISSN0001-4966
    DOIs
    Publication statusPublished - 2016

    Bibliographical note

    C 2016 Author(s). All article content, except where otherwise noted, is licensed under a Creative Commons Attribution (CC BY) license

    Fingerprint

    Dive into the research topics of 'Outcome measures based on classification performance fail to predict the intelligibility of binary-masked speech'. Together they form a unique fingerprint.

    Cite this