Can you trust predictive uncertainty under real dataset shifts in digital pathology?

Jeppe Thagaard, Søren Hauberg, Bert van der Vegt, Thomas Ebstrup, Johan D. Hansen, Anders Bjorholm Dahl

Research output: Chapter in Book/Report/Conference proceedingArticle in proceedingsResearchpeer-review

25 Downloads (Pure)

Abstract

Deep learning-based algorithms have shown great promise for assisting pathologists in detecting lymph node metastases when evaluated based on their predictive accuracy. However, for clinical adoption, we need to know what happens when the test set dramatically changes from the training distribution. In such settings, we should estimate the uncertainty of the predictions, so we know when to trust the model (and when not to). Here, we i) investigate current popular methods for improving the calibration of predictive uncertainty, and ii) compare the performance and calibration of the methods under clinically relevant in-distribution dataset shifts. Furthermore, we iii) evaluate their performance on the task of out-of-distribution detection of a different histological cancer type not seen during training. Of the investigated methods, we show that deep ensembles are more robust in respect of both performance and calibration for in-distribution dataset shifts and allows us to better detect incorrect predictions. Our results also demonstrate that current methods for uncertainty quantification are not necessarily able to detect all dataset shifts, and we emphasize the importance of monitoring and controlling the input distribution when deploying deep learning for digital pathology.
Original languageEnglish
Title of host publicationProceedings of 23rd International Conference on Medical Image Computing and Computer Assisted Intervention
PublisherSpringer
Publication date2020
Pages824-833
ISBN (Print)978-3-030-59709-2
DOIs
Publication statusPublished - 2020
Event23rd International Conference on Medical Image Computing and Computer Assisted Intervention - Online Event, Lima, Peru
Duration: 4 Oct 20208 Oct 2020
https://www.miccai2020.org/

Conference

Conference23rd International Conference on Medical Image Computing and Computer Assisted Intervention
LocationOnline Event
CountryPeru
CityLima
Period04/10/202008/10/2020
Internet address
SeriesLecture Notes in Computer Science
Volume12261
ISSN0302-9743

Keywords

  • Deep learning
  • Digital pathology
  • Predictive uncertainty

Fingerprint Dive into the research topics of 'Can you trust predictive uncertainty under real dataset shifts in digital pathology?'. Together they form a unique fingerprint.

Cite this