Adaptive entropy-based learning with dynamic artificial neural network

Tiago Pinto*, Hugo Morais, Juan Manuel Corchado

*Corresponding author for this work

Research output: Contribution to journalJournal articleResearchpeer-review

Abstract

Entropy models the added information associated to data uncertainty, proving that stochasticity is not purely random. This paper explores the potential improvement of machine learning methodologies through the incorporation of entropy analysis in the learning process. A multi-layer perceptron is applied to identify patterns in previous forecasting errors achieved by a machine learning methodology. The proposed learning approach is adaptive to the training data through a re-training process that includes only the most recent and relevant data, thus excluding misleading information from the training process. The learnt error patterns are then combined with the original forecasting results in order to improve forecasting accuracy, using the Rényi entropy to determine the amount in which the original forecasted value should be adapted considering the learnt error patterns. The proposed approach is combined with eleven different machine learning methodologies, and applied to the forecasting of electricity market prices using real data from the Iberian electricity market operator – OMIE. Results show that through the identification of patterns in the forecasting error, the proposed methodology is able to improve the learning algorithms’ forecasting accuracy and reduce the variability of their forecasting errors.

Original languageEnglish
JournalNeurocomputing
Volume338
Pages (from-to)432-440
ISSN0925-2312
DOIs
Publication statusPublished - 2019

Keywords

  • Artificial Neural Networks
  • Electricity market prices
  • Entropy
  • Forecasting
  • Information theory
  • Machine learning

Cite this