Hidden neural networks

Anders Stærmose Krogh, Søren Kamaric Riis

    Research output: Contribution to journalJournal articleResearchpeer-review

    Abstract

    A general framework for hybrids of hidden Markov models (HMMs) and neural networks (NNs) called hidden neural networks (HNNs) is described. The article begins by reviewing standard HMMs and estimation by conditional maximum likelihood, which is used by the HNN. In the HNN, the usual HMM probability parameters are replaced by the outputs of state-specific neural networks. As opposed to many other hybrids, the HNN is normalized globally and therefore has a valid probabilistic interpretation. All parameters in the HNN are estimated simultaneously according to the discriminative conditional maximum likelihood criterion. The HNN can be viewed as an undirected probabilistic independence network (a graphical model), where the neural networks provide a compact representation of the clique functions. An evaluation of the HNN on the task of recognizing broad phoneme classes in the TIMIT database shows clear performance gains compared to standard HMMs tested on the same task.
    Original languageEnglish
    JournalNeural Computation
    Volume11
    Issue number2
    Pages (from-to)541-563
    ISSN0899-7667
    DOIs
    Publication statusPublished - 1999

    Cite this

    Krogh, A. S., & Riis, S. K. (1999). Hidden neural networks. Neural Computation, 11(2), 541-563. https://doi.org/10.1162/089976699300016764