Non-Linear Back-propagation: Doing Back-Propagation withoutDerivatives of the Activation Function

John Hertz; Anders Stærmose Krogh; Benny Lautrup; Torsten Lehmann

Non-Linear Back-propagation: Doing Back-Propagation withoutDerivatives of the Activation Function

John Hertz, Anders Stærmose Krogh, Benny Lautrup, Torsten Lehmann

University of Copenhagen

Research output: Contribution to journal › Journal article › Research › peer-review

Abstract

The conventional linear back-propagation algorithm is replaced by a non-linear version, which avoids the necessity for calculating the derivative of the activation function. This may be exploited in hardware realizations of neural processors. In this paper we derive the non-linear back-propagation algorithms in the framework of recurrent back-propagation and present some numerical simulations of feed-forward networks on the NetTalk problem. A discussion of implementation in analog VLSI electronics concludes the paper.

Original language	English
Journal	I E E E Transactions on Neural Networks
Volume	8
Issue number	6
Pages (from-to)	1321-1327
ISSN	1045-9227
Publication status	Published - 1997

OpenUrl availability

Full text

Cite this

@article{da1944b201584f36900980e2b48414f0,

title = "Non-Linear Back-propagation: Doing Back-Propagation withoutDerivatives of the Activation Function",

abstract = "The conventional linear back-propagation algorithm is replaced by a non-linear version, which avoids the necessity for calculating the derivative of the activation function. This may be exploited in hardware realizations of neural processors. In this paper we derive the non-linear back-propagation algorithms in the framework of recurrent back-propagation and present some numerical simulations of feed-forward networks on the NetTalk problem. A discussion of implementation in analog VLSI electronics concludes the paper.",

author = "John Hertz and Krogh, \{Anders St{\ae}rmose\} and Benny Lautrup and Torsten Lehmann",

year = "1997",

language = "English",

volume = "8",

pages = "1321--1327",

journal = "I E E E Transactions on Neural Networks",

issn = "1045-9227",

publisher = "IEEE Computational Intelligence Society",

number = "6",

}

TY - JOUR

T1 - Non-Linear Back-propagation: Doing Back-Propagation withoutDerivatives of the Activation Function

AU - Hertz, John

AU - Krogh, Anders Stærmose

AU - Lautrup, Benny

AU - Lehmann, Torsten

PY - 1997

Y1 - 1997

N2 - The conventional linear back-propagation algorithm is replaced by a non-linear version, which avoids the necessity for calculating the derivative of the activation function. This may be exploited in hardware realizations of neural processors. In this paper we derive the non-linear back-propagation algorithms in the framework of recurrent back-propagation and present some numerical simulations of feed-forward networks on the NetTalk problem. A discussion of implementation in analog VLSI electronics concludes the paper.

AB - The conventional linear back-propagation algorithm is replaced by a non-linear version, which avoids the necessity for calculating the derivative of the activation function. This may be exploited in hardware realizations of neural processors. In this paper we derive the non-linear back-propagation algorithms in the framework of recurrent back-propagation and present some numerical simulations of feed-forward networks on the NetTalk problem. A discussion of implementation in analog VLSI electronics concludes the paper.

M3 - Journal article

SN - 1045-9227

VL - 8

SP - 1321

EP - 1327

JO - I E E E Transactions on Neural Networks

JF - I E E E Transactions on Neural Networks

IS - 6

ER -

Non-Linear Back-propagation: Doing Back-Propagation withoutDerivatives of the Activation Function

Abstract

OpenUrl availability

Fingerprint

Cite this