Skip to main navigation Skip to search Skip to main content

Mixtures of Locally Bounded Langevin dynamics for Bayesian Model Averaging

  • Swiss Federal Institute of Technology Zurich

Research output: Contribution to journal › Journal article › Research › peer-review

4 Downloads (Orbit)

Abstract

Properties of probability distributions change when going from low to high dimensions, to the extent that they exhibit counterintuitive behavior. Gaussian distributions intuitively illustrate a well-known effect of moving to higher dimensions, namely that the typical set almost surely does not contain the mean, which is the distribution’s most probable point. This can be problematic in Bayesian Deep Learning, as the samples drawn from the high-dimensional posterior distribution are often used as Monte Carlo samples to estimate the integral of the predictive distribution. Here, the predictive distribution will reflect the behavior of the samples and, therefore, of the typical set. For instance, we cannot expect to sample networks close to the maximum a posteriori estimate after fitting a Gaussian approximation to the posterior using the Laplace method. In this paper, we introduce a method that aims to mitigate this typicality problem in high dimensions by sampling from the posterior with Langevin dynamics on a restricted support enforced by a reflective boundary condition. We demonstrate how this leads to improved posterior estimates by illustrating its capacity for fine-grained out-of-distribution (OOD) ranking on the Morpho-MNIST dataset.
Original languageEnglish
JournalTransactions on Machine Learning Research
Volume5
Number of pages23
ISSN2835-8856
Publication statusPublished - 2026

Fingerprint

Dive into the research topics of 'Mixtures of Locally Bounded Langevin dynamics for Bayesian Model Averaging'. Together they form a unique fingerprint.

Cite this