Model-agnostic out-of-distribution detection using combined statistical tests

We present simple methods for out-of-distribution detection using a trainedgenerative model. These techniques, based on classical statistical tests, aremodel-agnostic in the sense that they can be applied to any differentiablegenerative model. The idea is to combine a classical parametric test (Rao'sscore test) with the recently introduced typicality test. These two teststatistics are both theoretically well-founded and exploit different sources ofinformation based on the likelihood for the typicality test and its gradientfor the score test. We show that combining them using Fisher's method overallleads to a more accurate out-of-distribution test. We also discuss the benefitsof casting out-of-distribution detection as a statistical testing problem,noting in particular that false positive rate control can be valuable forpractical out-of-distribution detection. Despite their simplicity andgenerality, these methods can be competitive with model-specificout-of-distribution detection algorithms without any assumptions on the out-distribution.