Difference-of-Convex optimization for variational kl-corrected inference in dirichlet process mixtures

Publication: Research - peer-reviewArticle in proceedings – Annual report year: 2017

DOI

View graph of relations

Variational methods for approximate inference in Bayesian models optimise a lower bound on the marginal likelihood, but the optimization problem often suffers from being nonconvex and high-dimensional. This can be alleviated by working in a collapsed domain where a part of the parameter space is marginalized. We consider the KL-corrected collapsed variational bound and apply it to Dirichlet process mixture models, allowing us to reduce the optimization space considerably. We find that the variational bound exhibits consistent and exploitable structure, allowing the application of difference-of-convex optimization algorithms. We show how this yields an interpretable fixed-point update algorithm in the collapsed setting for the Dirichlet process mixture model. We connect this update formula to classical coordinate ascent updates, illustrating that the proposed improvement surprisingly reduces to the traditional scheme.
Original languageEnglish
Title of host publicationProceedings of 2017 IEEE 27th International Workshop on Machine Learning for Signal Processing (MLSP)
Number of pages6
PublisherIEEE
Publication date2017
Pages1-6
ISBN (print)978-1-5090-6342-0
ISBN (electronic)978-1-5090-6341-3
DOIs
StatePublished - 2017
Event2017 IEEE international workshop on machine learning for signal processing - Tokyo, Jamaica

Workshop

Workshop2017 IEEE international workshop on machine learning for signal processing
Number27
LocationRoppongi
CountryJamaica
CityTokyo
Period25/09/201728/09/2017
Internet address
CitationsWeb of Science® Times Cited: No match on DOI

    Keywords

  • Difference-of-convex optimization, Variational inference, Collapsed methods, Bayesian nonparametrics
Download as:
Download as PDF
Select render style:
APAAuthorCBE/CSEHarvardMLAStandardVancouverShortLong
PDF
Download as HTML
Select render style:
APAAuthorCBE/CSEHarvardMLAStandardVancouverShortLong
HTML
Download as Word
Select render style:
APAAuthorCBE/CSEHarvardMLAStandardVancouverShortLong
Word

ID: 140491709