Training Recurrent Networks

Morten With Pedersen

doi:10.1109/NNSP.1997.622416

Training Recurrent Networks

Morten With Pedersen

Research output: Chapter in Book/Report/Conference proceeding › Article in proceedings › Research › peer-review

410 Downloads (Orbit)

Abstract

Training recurrent networks is generally believed to be a difficult task. Excessive training times and lack of convergence to an acceptable solution are frequently reported. In this paper we seek to explain the reason for this from a numerical point of view and show how to avoid problems when training. In particular we investigate ill-conditioning, the need for and effect of regularization and illustrate the superiority of second-order methods for training

Original language	English
Title of host publication	IEEE Workshop on Neural Networks for Signal Processing VII
Place of Publication	Piscataway, New Jersey
Publisher	IEEE
Publication date	1997
Pages	355-364
ISBN (Print)	0-7803-4256-9
DOIs	https://doi.org/10.1109/NNSP.1997.622416
Publication status	Published - 1997
Event	1997 IEEE Workshop on Neural Networks for Signal Processing VII - Amelia Island, United States Duration: 24 Sept 1997 → 26 Sept 1997 Conference number: 7 https://ieeexplore.ieee.org/xpl/conhome/4900/proceeding

Conference

Conference	1997 IEEE Workshop on Neural Networks for Signal Processing VII
Number	7
Country/Territory	United States
City	Amelia Island
Period	24/09/1997 → 26/09/1997
Internet address	https://ieeexplore.ieee.org/xpl/conhome/4900/proceeding

Bibliographical note

Copyright: 1997 IEEE. Personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution to servers or lists, or to reuse any copyrighted component of this work in other works must be obtained from the IEEE

Access to Document

10.1109/NNSP.1997.622416

PedersenFinal published version, 660 KB

OpenUrl availability

Full text

Cite this

@inproceedings{ab8d8469fe6a4531ba803fec9d93a93b,

title = "Training Recurrent Networks",

abstract = "Training recurrent networks is generally believed to be a difficult task. Excessive training times and lack of convergence to an acceptable solution are frequently reported. In this paper we seek to explain the reason for this from a numerical point of view and show how to avoid problems when training. In particular we investigate ill-conditioning, the need for and effect of regularization and illustrate the superiority of second-order methods for training",

author = "Pedersen, \{Morten With\}",

note = "Copyright: 1997 IEEE. Personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution to servers or lists, or to reuse any copyrighted component of this work in other works must be obtained from the IEEE; 1997 IEEE Workshop on Neural Networks for Signal Processing VII, NNSP 1997 ; Conference date: 24-09-1997 Through 26-09-1997",

year = "1997",

doi = "10.1109/NNSP.1997.622416",

language = "English",

isbn = "0-7803-4256-9",

pages = "355--364",

booktitle = "IEEE Workshop on Neural Networks for Signal Processing VII",

publisher = "IEEE",

address = "United States",

url = "https://ieeexplore.ieee.org/xpl/conhome/4900/proceeding",

}

TY - GEN

T1 - Training Recurrent Networks

AU - Pedersen, Morten With

N1 - Conference code: 7

PY - 1997

Y1 - 1997

N2 - Training recurrent networks is generally believed to be a difficult task. Excessive training times and lack of convergence to an acceptable solution are frequently reported. In this paper we seek to explain the reason for this from a numerical point of view and show how to avoid problems when training. In particular we investigate ill-conditioning, the need for and effect of regularization and illustrate the superiority of second-order methods for training

AB - Training recurrent networks is generally believed to be a difficult task. Excessive training times and lack of convergence to an acceptable solution are frequently reported. In this paper we seek to explain the reason for this from a numerical point of view and show how to avoid problems when training. In particular we investigate ill-conditioning, the need for and effect of regularization and illustrate the superiority of second-order methods for training

U2 - 10.1109/NNSP.1997.622416

DO - 10.1109/NNSP.1997.622416

M3 - Article in proceedings

SN - 0-7803-4256-9

SP - 355

EP - 364

BT - IEEE Workshop on Neural Networks for Signal Processing VII

PB - IEEE

CY - Piscataway, New Jersey

T2 - 1997 IEEE Workshop on Neural Networks for Signal Processing VII

Y2 - 24 September 1997 through 26 September 1997

ER -

Training Recurrent Networks

Abstract

Conference

Bibliographical note

Access to Document

OpenUrl availability

Fingerprint

Cite this