Generalization performance of regularized neural network models

Jan Larsen, Lars Kai Hansen

    Research output: Chapter in Book/Report/Conference proceedingArticle in proceedingsResearchpeer-review

    685 Downloads (Pure)

    Abstract

    Architecture optimization is a fundamental problem of neural network modeling. The optimal architecture is defined as the one which minimizes the generalization error. This paper addresses estimation of the generalization performance of regularized, complete neural network models. Regularization normally improves the generalization performance by restricting the model complexity. A formula for the optimal weight decay regularizer is derived. A regularized model may be characterized by an effective number of weights (parameters); however, it is demonstrated that no simple definition is possible. A novel estimator of the average generalization error (called FPER) is suggested and compared to the final prediction error (FPE) and generalized prediction error (GPE) estimators. In addition, comparative numerical studies demonstrate the qualities of the suggested estimator
    Original languageEnglish
    Title of host publicationProceedings of the 4th IEEE Workshop Neural Networks for Signal Processing
    PublisherIEEE
    Publication date1994
    Pages42-51
    ISBN (Print)07-80-32026-3
    DOIs
    Publication statusPublished - 1994
    Event1994 IEEE Workshop on Neural Networks for Signal Processing - Ermoino, Greece
    Duration: 6 Sept 19948 Sept 1994
    Conference number: 4
    https://ieeexplore.ieee.org/xpl/conhome/2959/proceeding
    http://ieeexplore.ieee.org/xpl/mostRecentIssue.jsp?punumber=2959

    Conference

    Conference1994 IEEE Workshop on Neural Networks for Signal Processing
    Number4
    Country/TerritoryGreece
    CityErmoino
    Period06/09/199408/09/1994
    Internet address

    Bibliographical note

    Copyright: 1994 IEEE. Personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution to servers or lists, or to reuse any copyrighted component of this work in other works must be obtained from the IEEE

    Fingerprint

    Dive into the research topics of 'Generalization performance of regularized neural network models'. Together they form a unique fingerprint.

    Cite this