Hidden Neural Networks: A Framework for HMM/NN Hybrids

Søren Kamaric Riis; Anders Stærmose Krogh

doi:10.1109/ICASSP.1997.595481

Hidden Neural Networks: A Framework for HMM/NN Hybrids

Søren Kamaric Riis, Anders Stærmose Krogh

Research output: Chapter in Book/Report/Conference proceeding › Article in proceedings › Research › peer-review

474 Downloads (Pure)

Abstract

This paper presents a general framework for hybrids of hidden Markov models (HMM) and neural networks (NN). In the new framework called hidden neural networks (HNN) the usual HMM probability parameters are replaced by neural network outputs. To ensure a probabilistic interpretation the HNN is normalized globally as opposed to the local normalization enforced on parameters in standard HMMs. Furthermore, all parameters in the HNN are estimated simultaneously according to the discriminative conditional maximum likelihood (CML) criterion. The HNNs show clear performance gains compared to standard HMMs on TIMIT continuous speech recognition benchmarks. On the task of recognizing five broad phoneme classes an accuracy of 84% is obtained compared to 76% for a standard HMM. Additionally, we report a preliminary result of 69% accuracy on the TIMIT 39 phoneme task

Original language	English
Title of host publication	IEEE International Conference on Acoustics, Speech and Signal Processing
Volume	4
Publisher	IEEE
Publication date	1997
Pages	3233-3236
ISBN (Print)	0-8186-7919-0
DOIs	https://doi.org/10.1109/ICASSP.1997.595481
Publication status	Published - 1997
Event	1997 IEEE International Conference on Acoustics, Speech and Signal Processing - Munich, Germany Duration: 21 Apr 1997 → 24 Apr 1997 http://ieeexplore.ieee.org/xpl/mostRecentIssue.jsp?punumber=4635

Conference

Conference	1997 IEEE International Conference on Acoustics, Speech and Signal Processing
Country	Germany
City	Munich
Period	21/04/1997 → 24/04/1997
Internet address	http://ieeexplore.ieee.org/xpl/mostRecentIssue.jsp?punumber=4635

Bibliographical note

Copyright: 1997 IEEE. Personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution to servers or lists, or to reuse any copyrighted component of this work in other works must be obtained from the IEEE

Access to Document

10.1109/ICASSP.1997.595481

RiisFinal published version, 501 KB

Fingerprint Dive into the research topics of 'Hidden Neural Networks: A Framework for HMM/NN Hybrids'. Together they form a unique fingerprint.

Cite this

@inproceedings{786ac908720e4fd580de6f15023c0b05,

title = "Hidden Neural Networks: A Framework for HMM/NN Hybrids",

abstract = "This paper presents a general framework for hybrids of hidden Markov models (HMM) and neural networks (NN). In the new framework called hidden neural networks (HNN) the usual HMM probability parameters are replaced by neural network outputs. To ensure a probabilistic interpretation the HNN is normalized globally as opposed to the local normalization enforced on parameters in standard HMMs. Furthermore, all parameters in the HNN are estimated simultaneously according to the discriminative conditional maximum likelihood (CML) criterion. The HNNs show clear performance gains compared to standard HMMs on TIMIT continuous speech recognition benchmarks. On the task of recognizing five broad phoneme classes an accuracy of 84% is obtained compared to 76% for a standard HMM. Additionally, we report a preliminary result of 69% accuracy on the TIMIT 39 phoneme task",

author = "Riis, {S{\o}ren Kamaric} and Krogh, {Anders St{\ae}rmose}",

note = "Copyright: 1997 IEEE. Personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution to servers or lists, or to reuse any copyrighted component of this work in other works must be obtained from the IEEE; 1997 IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP-97 ; Conference date: 21-04-1997 Through 24-04-1997",

year = "1997",

doi = "10.1109/ICASSP.1997.595481",

language = "English",

isbn = "0-8186-7919-0",

volume = "4",

pages = "3233--3236",

booktitle = "IEEE International Conference on Acoustics, Speech and Signal Processing",

publisher = "IEEE",

address = "United States",

url = "http://ieeexplore.ieee.org/xpl/mostRecentIssue.jsp?punumber=4635",

}

TY - GEN

T1 - Hidden Neural Networks: A Framework for HMM/NN Hybrids

AU - Riis, Søren Kamaric

AU - Krogh, Anders Stærmose

N1 - Copyright: 1997 IEEE. Personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution to servers or lists, or to reuse any copyrighted component of this work in other works must be obtained from the IEEE

PY - 1997

Y1 - 1997

N2 - This paper presents a general framework for hybrids of hidden Markov models (HMM) and neural networks (NN). In the new framework called hidden neural networks (HNN) the usual HMM probability parameters are replaced by neural network outputs. To ensure a probabilistic interpretation the HNN is normalized globally as opposed to the local normalization enforced on parameters in standard HMMs. Furthermore, all parameters in the HNN are estimated simultaneously according to the discriminative conditional maximum likelihood (CML) criterion. The HNNs show clear performance gains compared to standard HMMs on TIMIT continuous speech recognition benchmarks. On the task of recognizing five broad phoneme classes an accuracy of 84% is obtained compared to 76% for a standard HMM. Additionally, we report a preliminary result of 69% accuracy on the TIMIT 39 phoneme task

AB - This paper presents a general framework for hybrids of hidden Markov models (HMM) and neural networks (NN). In the new framework called hidden neural networks (HNN) the usual HMM probability parameters are replaced by neural network outputs. To ensure a probabilistic interpretation the HNN is normalized globally as opposed to the local normalization enforced on parameters in standard HMMs. Furthermore, all parameters in the HNN are estimated simultaneously according to the discriminative conditional maximum likelihood (CML) criterion. The HNNs show clear performance gains compared to standard HMMs on TIMIT continuous speech recognition benchmarks. On the task of recognizing five broad phoneme classes an accuracy of 84% is obtained compared to 76% for a standard HMM. Additionally, we report a preliminary result of 69% accuracy on the TIMIT 39 phoneme task

U2 - 10.1109/ICASSP.1997.595481

DO - 10.1109/ICASSP.1997.595481

M3 - Article in proceedings

SN - 0-8186-7919-0

VL - 4

SP - 3233

EP - 3236

BT - IEEE International Conference on Acoustics, Speech and Signal Processing

PB - IEEE

T2 - 1997 IEEE International Conference on Acoustics, Speech and Signal Processing

Y2 - 21 April 1997 through 24 April 1997

ER -