Bayesian dropout

Tue Herlau*, Mikkel N. Schmidt, Morten Mørup

*Corresponding author for this work

Research output: Contribution to journalConference articleResearchpeer-review

52 Downloads (Pure)


In the past decade, Dropout has emerged as a powerful and simple method for training neural networks preventing co-adaptation by stochastically omitting neurons. Dropout is currently not grounded in explicit modelling assumptions which so far has precluded its adoption in Bayesian modeling. Using Bayesian entropic reasoning we show that dropout can be interpreted as optimal inference under constraints. We demonstrate this on an analytically tractable regression model providing a Bayesian interpretation of its mechanism for regularizing and preventing co-adaptation as well as its connection to other Bayesian techniques, and in our experiments we find that dropout can provide robustness under model misspecification. Our framework roots dropout as a theoretically justified and practical tool for statistical modeling allowing Bayesian practitioners to tap into the benefits of dropout training.

Original languageEnglish
JournalProcedia Computer Science
Issue numberC
Pages (from-to)771-776
Publication statusPublished - 2022
Event3rd International Workshop on Statistical Methods and Artificial Intelligence - Porto, Portugal
Duration: 22 Mar 202225 Mar 2022


Conference3rd International Workshop on Statistical Methods and Artificial Intelligence


  • Bayesian learning
  • Dropout
  • Maximum entropy


Dive into the research topics of 'Bayesian dropout'. Together they form a unique fingerprint.

Cite this