Uncertainty estimation is important for interpreting the trustworthiness of machine learning models in many applications. This is especially critical in the data-driven active learning setting where the goal is to achieve a certain accuracy with minimum labeling effort. In such settings, the model learns to select the most informative unlabeled samples for annotation based on its estimated uncertainty. The highly uncertain predictions are assumed to be more informative for improving model performance. In this paper, we explore uncertainty calibration within an active learning framework for medical image segmentation, an area where labels often are scarce. Various uncertainty estimation methods and acquisition strategies (regions and full images) are investigated. We observe that selecting regions to annotate instead of full images leads to more well-calibrated models. Additionally, we experimentally show that annotating regions can cut 50% of pixels that need to be labeled by humans compared to annotating full images.
|Title of host publication||Proceedings of 2020 International Conference on Machine Learning: Workshop on Uncertainty and Robustness in Deep Learning|
|Number of pages||9|
|Publication status||Published - 2020|
|Event||2020 International Conference on Machine Learning: Workshop on Uncertainty and Robustness in Deep Learning - Virtual event|
Duration: 12 Jul 2020 → 18 Jul 2020
|Workshop||2020 International Conference on Machine Learning|
|Period||12/07/2020 → 18/07/2020|