Model-agnostic out-of-distribution detection using combined statistical tests

Federico Bergamin*, Pierre-Alexandre Mattei, Jakob D. Havtorn, Hugo Senetaire, Hugo Schmutz, Lars Maaløe, Søren Hauberg, Jes Frellsen

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingArticle in proceedingsResearchpeer-review

32 Downloads (Pure)


We present simple methods for out-of-distribution detection using a trainedgenerative model. These techniques, based on classical statistical tests, aremodel-agnostic in the sense that they can be applied to any differentiablegenerative model. The idea is to combine a classical parametric test (Rao'sscore test) with the recently introduced typicality test. These two teststatistics are both theoretically well-founded and exploit different sources ofinformation based on the likelihood for the typicality test and its gradientfor the score test. We show that combining them using Fisher's method overallleads to a more accurate out-of-distribution test. We also discuss the benefitsof casting out-of-distribution detection as a statistical testing problem,noting in particular that false positive rate control can be valuable forpractical out-of-distribution detection. Despite their simplicity andgenerality, these methods can be competitive with model-specificout-of-distribution detection algorithms without any assumptions on the out-distribution.
Original languageEnglish
Title of host publicationProceedings of the 25th International Conference on Artificial Intelligence and Statistics
PublisherInternational Machine Learning Society (IMLS)
Publication date2022
Publication statusPublished - 2022
Event25th International Conference on Artificial Intelligence and Statistics - Virtual Conference
Duration: 28 Mar 202230 Mar 2022
Conference number: 25


Conference25th International Conference on Artificial Intelligence and Statistics
LocationVirtual Conference
Internet address
SeriesProceedings of Machine Learning Research


Dive into the research topics of 'Model-agnostic out-of-distribution detection using combined statistical tests'. Together they form a unique fingerprint.

Cite this