Aleatoric and Epistemic Uncertainty in Image Segmentation

Image segmentation, the pixelwise classification of objects, is an integral part of the computer vision toolbox and an indispensable method in many applications, such as the analysis of medical images and autonomous driving. The safe deployment of such systems in practice requires that they can express uncertainty about their predictions, to then allow for adjustments in downstream decisions that are taken based on these uncertainties. Although neural networks have advanced the field of computer vision, including image segmentation, tremendously, they pose challenges in uncertainty quantification, such as notorious overconfidence for data far away from anything seen during training. The Bayesian framework distinguishes between two types of uncertainty: Aleatoric uncertainty to represent the irreducible variance in the data that the model should capture and represent, and epistemic uncertainty that quantifies the degree of ignorance or lack of expertise of the model for a new input.

This thesis contributes methodological advances that improve the quantification of both aleatoric and epistemic uncertainty in neural network-based segmentation architectures. In addition, it provides guidelines for selecting uncertain segmentation models and methods for a given problem at hand. The contributions in total mark a step towards better uncertainty quantification in image segmentation and, therefore, to a safer deployment of such systems in practice.