Model-agnostic out-of-distribution detection using combined statistical tests

Federico Bergamin*, Pierre-Alexandre Mattei, Jakob D. Havtorn, Hugo Senetaire, Hugo Schmutz, Lars Maaløe, Søren Hauberg, Jes Frellsen

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingArticle in proceedingsResearchpeer-review

7 Downloads (Pure)

Abstract

We present simple methods for out-of-distribution detection using a trainedgenerative model. These techniques, based on classical statistical tests, aremodel-agnostic in the sense that they can be applied to any differentiablegenerative model. The idea is to combine a classical parametric test (Rao'sscore test) with the recently introduced typicality test. These two teststatistics are both theoretically well-founded and exploit different sources ofinformation based on the likelihood for the typicality test and its gradientfor the score test. We show that combining them using Fisher's method overallleads to a more accurate out-of-distribution test. We also discuss the benefitsof casting out-of-distribution detection as a statistical testing problem,noting in particular that false positive rate control can be valuable forpractical out-of-distribution detection. Despite their simplicity andgenerality, these methods can be competitive with model-specificout-of-distribution detection algorithms without any assumptions on the out-distribution.
Original languageEnglish
Title of host publicationProceedings of the 25th International Conference on Artificial Intelligence and Statistics
PublisherInternational Machine Learning Society (IMLS)
Publication date2022
Pages10753-10776
Publication statusPublished - 2022
Event25th International Conference on Artificial Intelligence and Statistics - Virtual Conference
Duration: 28 Mar 202230 Mar 2022
Conference number: 25
https://aistats.org/aistats2022/
https://proceedings.mlr.press/v151/

Conference

Conference25th International Conference on Artificial Intelligence and Statistics
Number25
LocationVirtual Conference
Period28/03/202230/03/2022
Internet address
SeriesProceedings of Machine Learning Research
Volume151
ISSN2640-3498

Fingerprint

Dive into the research topics of 'Model-agnostic out-of-distribution detection using combined statistical tests'. Together they form a unique fingerprint.

Cite this