Implicit Variational Inference for High-Dimensional Posteriors

Research output: Chapter in Book/Report/Conference proceedingArticle in proceedingsResearchpeer-review

1 Downloads (Pure)

Abstract

In variational inference, the benefits of Bayesian models rely on accurately cap-
turing the true posterior distribution. We propose using neural samplers that
specify implicit distributions, which are well-suited for approximating complex
multimodal and correlated posteriors in high-dimensional spaces. Our approach
advances inference using implicit distributions by introducing novel approximate
bounds by locally linearising the neural sampler. This is distinct from existing
methods that rely on additional discriminator networks and unstable adversarial
objectives. Furthermore, we present a new sampler architecture that, for the first
time, enables implicit distributions over tens of millions of latent variables, ad-
dressing computational concerns by using differentiable numerical approximations. Our empirical analysis indicates our method is capable of recovering correlations across layers in large Bayesian neural networks, a property that is crucial for a network’s performance but notoriously challenging to achieve. To the best of our knowledge, no other method has been shown to accomplish this task for such large models. Through experiments on downstream tasks, we demonstrate that our expressive posteriors outperform state-of-the-art uncertainty quantification methods, validating the effectiveness of our training algorithm and the quality of the learned implicit distribution.
Original languageEnglish
Title of host publicationProceedings of the 37th Conference on Neural Information Processing Systems
Volume36
PublisherCurran Associates
Publication date2023
Pages73793-73816
Publication statusPublished - 2023
Event37th Conference on Neural Information Processing Systems - New Orleans Ernest N. Morial Convention Center, New Orleans, United States
Duration: 10 Dec 202316 Dec 2023

Conference

Conference37th Conference on Neural Information Processing Systems
LocationNew Orleans Ernest N. Morial Convention Center
Country/TerritoryUnited States
CityNew Orleans
Period10/12/202316/12/2023

Fingerprint

Dive into the research topics of 'Implicit Variational Inference for High-Dimensional Posteriors'. Together they form a unique fingerprint.

Cite this