Advances in Deep Generative Models, Approximate Inference, and their Applications

Applications of deep learning and deep generative models are becoming increasingly influential. With these models becoming an integral part of our lives, we must ensure that they are not only accurate, but also robust and safe. However, neural networks have been found to be overconfident both in misclassified examples and on test examples that do not belong to the distribution used to generate the training data. At the same time, deep generative models have been shown to sometimes assign a higher likelihood to out-of-distribution examples. This makes the safe deployment of deep learning models challenging. However, the success of deep generative models in image and text generation has paved the way for the application of these models in different fields and different tasks.

This thesis consists of four different self-contained research articles addressing some of the challenges and opportunities mentioned above. We start by focusing on the problem of deep generative models, sometimes assigning a higher likelihood to out-ofdistribution examples. In this context, we proposed to combine different test statistics for anomaly detection using Fisher’s method. We then present a method that borrow tools from Riemannian geometry to avoid that classic Laplace approximation spread mass on low-probability regions of the true posterior when performing approximate inference for neural networks. In the second part of this dissertation, we focus on two different applications of deep generative models. We start by analyzing how clustering is performed with variational autoencoders and then extend this framework to perform co-clustering, i.e. simultaneously clustering both rows and columns of a data matrix. The last article of this thesis applies diffusion models for modeling long-range rollouts of partial differential equations, highlighting the fact that training a joint model and performing conditioning via reconstruction guidance is a promising way to have a flexible and general way to use the same model to tackle different tasks.