Implicit Variational Inference for High-Dimensional Posteriors

In variational inference, the benefits of Bayesian models rely on accurately cap-
turing the true posterior distribution. We propose using neural samplers that
specify implicit distributions, which are well-suited for approximating complex
multimodal and correlated posteriors in high-dimensional spaces. Our approach
advances inference using implicit distributions by introducing novel approximate
bounds by locally linearising the neural sampler. This is distinct from existing
methods that rely on additional discriminator networks and unstable adversarial
objectives. Furthermore, we present a new sampler architecture that, for the first
time, enables implicit distributions over tens of millions of latent variables, ad-
dressing computational concerns by using differentiable numerical approximations. Our empirical analysis indicates our method is capable of recovering correlations across layers in large Bayesian neural networks, a property that is crucial for a network’s performance but notoriously challenging to achieve. To the best of our knowledge, no other method has been shown to accomplish this task for such large models. Through experiments on downstream tasks, we demonstrate that our expressive posteriors outperform state-of-the-art uncertainty quantification methods, validating the effectiveness of our training algorithm and the quality of the learned implicit distribution.