Leave-One-Out Cross-Validation for Bayesian Model Comparison in Large Data

Måns Magnusson, Michael Riis Andersen, Johan Jonasson, Aki Vehtari

Research output: Chapter in Book/Report/Conference proceedingBook chapterResearchpeer-review

8 Downloads (Pure)

Abstract

Recently, new methods for model assessment, based on subsampling and posterior approximations, have been proposed for scaling leave-one-out cross-validation (LOO) to large datasets. Although these methods work well for estimating predictive performance for individual models, they are less powerful in model comparison. We propose an efficient method for estimating differences in predictive performance by combining fast approximate LOO surrogates with exact LOO subsampling using the difference estimator and supply proofs with regards to scaling characteristics. The resulting approach can be orders of magnitude more efficient than previous approaches, as well as being better suited to model comparison.
Original languageEnglish
Title of host publicationProceedings of the 23rd International Conference on Artificial Intelligence and Statistics
Volume108
Publication date2020
Pages341-350
Publication statusPublished - 2020
Event23rd International Conference on Artificial Intelligence and Statistics - Virtual event
Duration: 26 Aug 202028 Aug 2020

Conference

Conference23rd International Conference on Artificial Intelligence and Statistics
LocationVirtual event
Period26/08/202028/08/2020
SeriesProceedings of Machine Learning Research
Volume108
ISSN2640-3498

Fingerprint Dive into the research topics of 'Leave-One-Out Cross-Validation for Bayesian Model Comparison in Large Data'. Together they form a unique fingerprint.

Cite this