Difference-of-Convex optimization for variational kl-corrected inference in dirichlet process mixtures

Research output: Chapter in Book/Report/Conference proceedingArticle in proceedingsResearchpeer-review

Abstract

Variational methods for approximate inference in Bayesian models optimise a lower bound on the marginal likelihood, but the optimization problem often suffers from being nonconvex and high-dimensional. This can be alleviated by working in a collapsed domain where a part of the parameter space is marginalized. We consider the KL-corrected collapsed variational bound and apply it to Dirichlet process mixture models, allowing us to reduce the optimization space considerably. We find that the variational bound exhibits consistent and exploitable structure, allowing the application of difference-of-convex optimization algorithms. We show how this yields an interpretable fixed-point update algorithm in the collapsed setting for the Dirichlet process mixture model. We connect this update formula to classical coordinate ascent updates, illustrating that the proposed improvement surprisingly reduces to the traditional scheme.
Original languageEnglish
Title of host publicationProceedings of 2017 IEEE 27th International Workshop on Machine Learning for Signal Processing (MLSP)
Number of pages6
PublisherIEEE
Publication date2017
Pages1-6
ISBN (Print)978-1-5090-6342-0
ISBN (Electronic)978-1-5090-6341-3
DOIs
Publication statusPublished - 2017
Event2017 IEEE international workshop on machine learning for signal processing - Roppongi, Tokyo, Jamaica
Duration: 25 Sep 201728 Sep 2017
Conference number: 27
http://mlsp2017.conwiz.dk/home.htm

Workshop

Workshop2017 IEEE international workshop on machine learning for signal processing
Number27
LocationRoppongi
CountryJamaica
CityTokyo
Period25/09/201728/09/2017
Internet address

Keywords

  • Difference-of-convex optimization
  • Variational inference
  • Collapsed methods
  • Bayesian nonparametrics

Fingerprint Dive into the research topics of 'Difference-of-Convex optimization for variational kl-corrected inference in dirichlet process mixtures'. Together they form a unique fingerprint.

Cite this