Optimizing Neural Network Architectures Using Generalization Error Estimators

Jan Larsen

    Research output: Contribution to journalJournal articleResearchpeer-review

    Abstract

    This paper addresses the optimization of neural network architectures. It is suggested to optimize the architecture by selecting the model with minimal estimated averaged generalization error. We consider a least-squares (LS) criterion for estimating neural network models, i.e., the associated model weights are estimated by minimizing the LS criterion. The quality of a particular estimated model is measured by the average generalization error. This is defined as the expected squared prediction error on a novel input-output sample averaged over all possible training sets. An essential part of the suggested architecture optimization scheme is to calculate an estimate of the average generalization error. We suggest using the GEN-estimator which allows for dealing with nonlinear, incomplete models, i.e., models which are not capable of modeling the underlying nonlinear relationship perfectly. In most neural network applications, it is impossible to suggest a perfect model, and consequently the ability to handle incomplete models is urgent. A concise derivation of the GEN-estimator is provided, and its qualities are demonstrated by comparative numerical studies
    Original languageEnglish
    JournalRadiophysics and Quantum Electronics
    Volume37
    Issue number9
    Pages (from-to)729-740
    ISSN0033-8443
    DOIs
    Publication statusPublished - 1994

    Keywords

    • generalization error estimation
    • neural networks

    Fingerprint

    Dive into the research topics of 'Optimizing Neural Network Architectures Using Generalization Error Estimators'. Together they form a unique fingerprint.

    Cite this