Generalization performance of regularized neural network models

Jan Larsen; Lars Kai Hansen

doi:10.1109/NNSP.1994.366065

Generalization performance of regularized neural network models

Jan Larsen
, Lars Kai Hansen

Research output: Chapter in Book/Report/Conference proceeding › Article in proceedings › Research › peer-review

767 Downloads (Orbit)

Abstract

Architecture optimization is a fundamental problem of neural network modeling. The optimal architecture is defined as the one which minimizes the generalization error. This paper addresses estimation of the generalization performance of regularized, complete neural network models. Regularization normally improves the generalization performance by restricting the model complexity. A formula for the optimal weight decay regularizer is derived. A regularized model may be characterized by an effective number of weights (parameters); however, it is demonstrated that no simple definition is possible. A novel estimator of the average generalization error (called FPER) is suggested and compared to the final prediction error (FPE) and generalized prediction error (GPE) estimators. In addition, comparative numerical studies demonstrate the qualities of the suggested estimator

Original language	English
Title of host publication	Proceedings of the 4th IEEE Workshop Neural Networks for Signal Processing
Publisher	IEEE
Publication date	1994
Pages	42-51
ISBN (Print)	07-80-32026-3
DOIs	https://doi.org/10.1109/NNSP.1994.366065
Publication status	Published - 1994
Event	1994 IEEE Workshop on Neural Networks for Signal Processing - Ermoino, Greece Duration: 6 Sept 1994 → 8 Sept 1994 Conference number: 4 https://ieeexplore.ieee.org/xpl/conhome/2959/proceeding http://ieeexplore.ieee.org/xpl/mostRecentIssue.jsp?punumber=2959

Conference

Conference	1994 IEEE Workshop on Neural Networks for Signal Processing
Number	4
Country/Territory	Greece
City	Ermoino
Period	06/09/1994 → 08/09/1994
Internet address	https://ieeexplore.ieee.org/xpl/conhome/2959/proceeding http://ieeexplore.ieee.org/xpl/mostRecentIssue.jsp?punumber=2959

Bibliographical note

Copyright: 1994 IEEE. Personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution to servers or lists, or to reuse any copyrighted component of this work in other works must be obtained from the IEEE

Access to Document

10.1109/NNSP.1994.366065

JanFinal published version, 554 KB

OpenUrl availability

Full text

Cite this

@inproceedings{74f4c9c848144ac98c984940fd215abe,

title = "Generalization performance of regularized neural network models",

abstract = "Architecture optimization is a fundamental problem of neural network modeling. The optimal architecture is defined as the one which minimizes the generalization error. This paper addresses estimation of the generalization performance of regularized, complete neural network models. Regularization normally improves the generalization performance by restricting the model complexity. A formula for the optimal weight decay regularizer is derived. A regularized model may be characterized by an effective number of weights (parameters); however, it is demonstrated that no simple definition is possible. A novel estimator of the average generalization error (called FPER) is suggested and compared to the final prediction error (FPE) and generalized prediction error (GPE) estimators. In addition, comparative numerical studies demonstrate the qualities of the suggested estimator",

author = "Jan Larsen and Hansen, \{Lars Kai\}",

note = "Copyright: 1994 IEEE. Personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution to servers or lists, or to reuse any copyrighted component of this work in other works must be obtained from the IEEE; 1994 IEEE Workshop on Neural Networks for Signal Processing, NNSP 1994 ; Conference date: 06-09-1994 Through 08-09-1994",

year = "1994",

doi = "10.1109/NNSP.1994.366065",

language = "English",

isbn = "07-80-32026-3",

pages = "42--51",

booktitle = "Proceedings of the 4th IEEE Workshop Neural Networks for Signal Processing",

publisher = "IEEE",

address = "United States",

url = "https://ieeexplore.ieee.org/xpl/conhome/2959/proceeding, http://ieeexplore.ieee.org/xpl/mostRecentIssue.jsp?punumber=2959",

}

TY - GEN

T1 - Generalization performance of regularized neural network models

AU - Larsen, Jan

AU - Hansen, Lars Kai

N1 - Conference code: 4

PY - 1994

Y1 - 1994

N2 - Architecture optimization is a fundamental problem of neural network modeling. The optimal architecture is defined as the one which minimizes the generalization error. This paper addresses estimation of the generalization performance of regularized, complete neural network models. Regularization normally improves the generalization performance by restricting the model complexity. A formula for the optimal weight decay regularizer is derived. A regularized model may be characterized by an effective number of weights (parameters); however, it is demonstrated that no simple definition is possible. A novel estimator of the average generalization error (called FPER) is suggested and compared to the final prediction error (FPE) and generalized prediction error (GPE) estimators. In addition, comparative numerical studies demonstrate the qualities of the suggested estimator

AB - Architecture optimization is a fundamental problem of neural network modeling. The optimal architecture is defined as the one which minimizes the generalization error. This paper addresses estimation of the generalization performance of regularized, complete neural network models. Regularization normally improves the generalization performance by restricting the model complexity. A formula for the optimal weight decay regularizer is derived. A regularized model may be characterized by an effective number of weights (parameters); however, it is demonstrated that no simple definition is possible. A novel estimator of the average generalization error (called FPER) is suggested and compared to the final prediction error (FPE) and generalized prediction error (GPE) estimators. In addition, comparative numerical studies demonstrate the qualities of the suggested estimator

U2 - 10.1109/NNSP.1994.366065

DO - 10.1109/NNSP.1994.366065

M3 - Article in proceedings

SN - 07-80-32026-3

SP - 42

EP - 51

BT - Proceedings of the 4th IEEE Workshop Neural Networks for Signal Processing

PB - IEEE

T2 - 1994 IEEE Workshop on Neural Networks for Signal Processing

Y2 - 6 September 1994 through 8 September 1994

ER -

Generalization performance of regularized neural network models

Abstract

Conference

Bibliographical note

Access to Document

OpenUrl availability

Fingerprint

Cite this