Variational Autoencoders with Riemannian Brownian Motion Priors

Dimitris Kalatzis*, David Eklund, Georgios Arvanitidis, Søren Hauberg

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingArticle in proceedingsResearchpeer-review

Abstract

Variational Autoencoders (VAEs) represent the given data in a low-dimensional latent space, which is generally assumed to be Euclidean. This assumption naturally leads to the common choice of a standard Gaussian prior over continuous latent variables. Recent work has, however, shown that this prior has a detrimental effect on model capacity, leading to subpar performance. We propose that the Euclidean assumption lies at the heart of this failure mode. To counter this, we assume a Riemannian structure over the latent space, which constitutes a more principled geometric view of the latent codes, and replace the standard Gaussian prior with a Riemannian Brownian motion prior. We propose an efficient inference scheme that does not rely on the unknown normalizing factor of this prior. Finally, we demonstrate that this prior significantly increases model capacity using only one additional scalar parameter.

Original languageEnglish
Title of host publicationProceedings of the 37th International Conference on Machine Learning
EditorsHal Daume, Aarti Singh
Volume119
PublisherInternational Machine Learning Society (IMLS)
Publication date2020
Pages5020-5033
ISBN (Electronic)9781713821120
Publication statusPublished - 2020
Event37th International Conference on Machine Learning - Virtual event, Virtual, Online
Duration: 13 Jul 202018 Jul 2020
https://icml.cc/Conferences/2020

Conference

Conference37th International Conference on Machine Learning
LocationVirtual event
CityVirtual, Online
Period13/07/202018/07/2020
Internet address
Series37th International Conference on Machine Learning, ICML 2020
VolumePartF168147-7

Bibliographical note

Funding Information:
SH and DE were supported by a research grant (15334) from VILLUM FONDEN. This project has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement no 757360). We gratefully acknowledge the support of the NVIDIA Corporation with the donation of GPU hardware.

Publisher Copyright:
© 2020 by the Authors.

Fingerprint

Dive into the research topics of 'Variational Autoencoders with Riemannian Brownian Motion Priors'. Together they form a unique fingerprint.

Cite this