Pruning Boltzmann networks and hidden Markov models

Morten With Pedersen, D. Stork

    Research output: Chapter in Book/Report/Conference proceedingArticle in proceedingsResearchpeer-review

    375 Downloads (Pure)

    Abstract

    We present sensitivity-based pruning algorithms for general Boltzmann networks. Central to our methods is the efficient calculation of a second-order approximation to the true weight saliencies in a cross-entropy error. Building upon previous work which shows a formal correspondence between linear Boltzmann chains and hidden Markov models (HMMs), we argue that our method can be applied to HMMs as well. We illustrate pruning on Boltzmann zippers, which are equivalent to two HMMs with cross-connection links. We verify that our second-order approximation preserves the rank ordering of weight saliencies and thus the proper weight is pruned at each pruning step. In all our experiments in small problems, pruning reduces the generalization error; in most cases the pruned networks facilitate interpretation as well
    Original languageEnglish
    Title of host publicationProceedings of the 30'th Asilomar Conference on Signals, Systems, and Computers
    VolumeVolume 1
    PublisherIEEE
    Publication date1996
    Pages258-261
    ISBN (Print)08-18-67646-9
    DOIs
    Publication statusPublished - 1996
    EventAsilomar Conference on Signals, Systems, and Computers - Pacific Grove, CA
    Duration: 1 Jan 1996 → …
    Conference number: 30th

    Conference

    ConferenceAsilomar Conference on Signals, Systems, and Computers
    Number30th
    CityPacific Grove, CA
    Period01/01/1996 → …

    Bibliographical note

    Copyright 1996 IEEE. Personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution to servers or lists, or to reuse any copyrighted component of this work in other works must be obtained from the IEEE.

    Fingerprint Dive into the research topics of 'Pruning Boltzmann networks and hidden Markov models'. Together they form a unique fingerprint.

    Cite this