Hidden Neural Networks: A Framework for HMM/NN Hybrids

Søren Kamaric Riis, Anders Stærmose Krogh

    Research output: Chapter in Book/Report/Conference proceedingArticle in proceedingsResearchpeer-review

    474 Downloads (Pure)


    This paper presents a general framework for hybrids of hidden Markov models (HMM) and neural networks (NN). In the new framework called hidden neural networks (HNN) the usual HMM probability parameters are replaced by neural network outputs. To ensure a probabilistic interpretation the HNN is normalized globally as opposed to the local normalization enforced on parameters in standard HMMs. Furthermore, all parameters in the HNN are estimated simultaneously according to the discriminative conditional maximum likelihood (CML) criterion. The HNNs show clear performance gains compared to standard HMMs on TIMIT continuous speech recognition benchmarks. On the task of recognizing five broad phoneme classes an accuracy of 84% is obtained compared to 76% for a standard HMM. Additionally, we report a preliminary result of 69% accuracy on the TIMIT 39 phoneme task
    Original languageEnglish
    Title of host publicationIEEE International Conference on Acoustics, Speech and Signal Processing
    Publication date1997
    ISBN (Print)0-8186-7919-0
    Publication statusPublished - 1997
    Event1997 IEEE International Conference on Acoustics, Speech and Signal Processing - Munich, Germany
    Duration: 21 Apr 199724 Apr 1997


    Conference1997 IEEE International Conference on Acoustics, Speech and Signal Processing
    Internet address

    Bibliographical note

    Copyright: 1997 IEEE. Personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution to servers or lists, or to reuse any copyrighted component of this work in other works must be obtained from the IEEE

    Fingerprint Dive into the research topics of 'Hidden Neural Networks: A Framework for HMM/NN Hybrids'. Together they form a unique fingerprint.

    Cite this