Adaptive Regularization of Neural Classifiers

Lars Nonboe Andersen; Jan Larsen; Lars Kai Hansen; Mads Hintz-Madsen; J Principe; L. Giles; N. Morgan; E Wilson

doi:10.1109/NNSP.1997.622380

Adaptive Regularization of Neural Classifiers

Lars Nonboe Andersen, Jan Larsen, Lars Kai Hansen, Mads Hintz-Madsen, J Principe (Editor), L. Giles (Editor), N. Morgan (Editor), E Wilson (Editor)

Research output: Chapter in Book/Report/Conference proceeding › Article in proceedings › Research › peer-review

505 Downloads (Pure)

Abstract

We present a regularization scheme which iteratively adapts the regularization parameters by minimizing the validation error. It is suggested to use the adaptive regularization scheme in conjunction with optimal brain damage pruning to optimize the architecture and to avoid overfitting. Furthermore, we propose an improved neural classification architecture eliminating an inherent redundancy in the widely used SoftMax classification network. Numerical results demonstrate the viability of the method

Original language	English
Title of host publication	Proceedings of the IEEE Workshop on Neural Networks for Signal Processing VII
Place of Publication	Piscataway, New Jersey
Publisher	IEEE
Publication date	1997
Pages	24-33
ISBN (Print)	0-7803-4256-9
DOIs	https://doi.org/10.1109/NNSP.1997.622380
Publication status	Published - 1997
Event	1997 IEEE Workshop on Neural Networks for Signal Processing VII - Amelia Island, United States Duration: 24 Sept 1997 → 26 Sept 1997 Conference number: 7 https://ieeexplore.ieee.org/xpl/conhome/4900/proceeding

Conference

Conference	1997 IEEE Workshop on Neural Networks for Signal Processing VII
Number	7
Country/Territory	United States
City	Amelia Island
Period	24/09/1997 → 26/09/1997
Internet address	https://ieeexplore.ieee.org/xpl/conhome/4900/proceeding

Bibliographical note

Copyright: 1997 IEEE. Personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution to servers or lists, or to reuse any copyrighted component of this work in other works must be obtained from the IEEE

Access to Document

10.1109/NNSP.1997.622380

AndersenFinal published version, 552 KB

OpenUrl availability

Full text

Cite this

@inproceedings{cac77ac2e1d843668432f0be8afdd835,

title = "Adaptive Regularization of Neural Classifiers",

abstract = "We present a regularization scheme which iteratively adapts the regularization parameters by minimizing the validation error. It is suggested to use the adaptive regularization scheme in conjunction with optimal brain damage pruning to optimize the architecture and to avoid overfitting. Furthermore, we propose an improved neural classification architecture eliminating an inherent redundancy in the widely used SoftMax classification network. Numerical results demonstrate the viability of the method",

author = "Andersen, {Lars Nonboe} and Jan Larsen and Hansen, {Lars Kai} and Mads Hintz-Madsen and J Principe and L. Giles and N. Morgan and E Wilson",

note = "Copyright: 1997 IEEE. Personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution to servers or lists, or to reuse any copyrighted component of this work in other works must be obtained from the IEEE; 1997 IEEE Workshop on Neural Networks for Signal Processing VII, NNSP 1997 ; Conference date: 24-09-1997 Through 26-09-1997",

year = "1997",

doi = "10.1109/NNSP.1997.622380",

language = "English",

isbn = "0-7803-4256-9",

pages = "24--33",

booktitle = "Proceedings of the IEEE Workshop on Neural Networks for Signal Processing VII",

publisher = "IEEE",

address = "United States",

url = "https://ieeexplore.ieee.org/xpl/conhome/4900/proceeding",

}

Andersen, LN, Larsen, J, Hansen, LK, Hintz-Madsen, M, Principe, J (ed.), Giles, L (ed.), Morgan, N (ed.) & Wilson, E (ed.) 1997, Adaptive Regularization of Neural Classifiers. in Proceedings of the IEEE Workshop on Neural Networks for Signal Processing VII. IEEE, Piscataway, New Jersey, pp. 24-33, 1997 IEEE Workshop on Neural Networks for Signal Processing VII, Amelia Island, Florida, United States, 24/09/1997. https://doi.org/10.1109/NNSP.1997.622380

TY - GEN

T1 - Adaptive Regularization of Neural Classifiers

AU - Andersen, Lars Nonboe

AU - Larsen, Jan

AU - Hansen, Lars Kai

AU - Hintz-Madsen, Mads

A2 - Principe, J

A2 - Giles, L.

A2 - Morgan, N.

A2 - Wilson, E

N1 - Conference code: 7

PY - 1997

Y1 - 1997

N2 - We present a regularization scheme which iteratively adapts the regularization parameters by minimizing the validation error. It is suggested to use the adaptive regularization scheme in conjunction with optimal brain damage pruning to optimize the architecture and to avoid overfitting. Furthermore, we propose an improved neural classification architecture eliminating an inherent redundancy in the widely used SoftMax classification network. Numerical results demonstrate the viability of the method

AB - We present a regularization scheme which iteratively adapts the regularization parameters by minimizing the validation error. It is suggested to use the adaptive regularization scheme in conjunction with optimal brain damage pruning to optimize the architecture and to avoid overfitting. Furthermore, we propose an improved neural classification architecture eliminating an inherent redundancy in the widely used SoftMax classification network. Numerical results demonstrate the viability of the method

U2 - 10.1109/NNSP.1997.622380

DO - 10.1109/NNSP.1997.622380

M3 - Article in proceedings

SN - 0-7803-4256-9

SP - 24

EP - 33

BT - Proceedings of the IEEE Workshop on Neural Networks for Signal Processing VII

PB - IEEE

CY - Piscataway, New Jersey

T2 - 1997 IEEE Workshop on Neural Networks for Signal Processing VII

Y2 - 24 September 1997 through 26 September 1997

ER -

Adaptive Regularization of Neural Classifiers

Abstract

Conference

Bibliographical note

Access to Document

OpenUrl availability

Fingerprint

Cite this