A Cautionary Analysis of STAPLE Using Direct Inference of Segmentation Truth

Koen Van Leemput; Mert R. Sabuncu

doi:10.1007/978-3-319-10404-1_50

A Cautionary Analysis of STAPLE Using Direct Inference of Segmentation Truth

Koen Van Leemput
, Mert R. Sabuncu

Massachusetts General Hospital/Harvard Medical School

Research output: Chapter in Book/Report/Conference proceeding › Article in proceedings › Research › peer-review

Abstract

In this paper we analyze the properties of the well-known segmentation fusion algorithm STAPLE, using a novel inference technique that analytically marginalizes out all model parameters. We demonstrate both theoretically and empirically that when the number of raters is large, or when consensus regions are included in the model, STAPLE devolves into thresholding the average of the input segmentations. We further show that when the number of raters is small, the STAPLE result may not be the optimal segmentation truth estimate, and its model parameter estimates might not reflect the individual raters’ actual segmentation performance. Our experiments indicate that these intrinsic weaknesses are frequently exacerbated by the presence of undesirable global optima and convergence issues. Together these results cast doubt on the soundness and usefulness of typical STAPLE outcomes.

Original language	English
Title of host publication	Proceedings of the 17th International Conference Medical Image Computing and Computer-Assisted Intervention (MICCAI 2014) : Part I
Publisher	Springer
Publication date	2014
Pages	398-406
ISBN (Print)	978-3-319-10403-4
ISBN (Electronic)	978-3-319-10404-1
DOIs	https://doi.org/10.1007/978-3-319-10404-1_50
Publication status	Published - 2014
Event	17th International Conference on Medical Image Computing and Computer Assisted Intervention - Massachusetts Institute of Technology, Cambridge, MA, Boston, United States Duration: 14 Sept 2014 → 18 Sept 2014 Conference number: 17 http://miccai2014.org/ http://miccai2014.org/cfp.html

Conference

Conference	17th International Conference on Medical Image Computing and Computer Assisted Intervention
Number	17
Location	Massachusetts Institute of Technology, Cambridge, MA
Country/Territory	United States
City	Boston
Period	14/09/2014 → 18/09/2014
Internet address	http://miccai2014.org/ http://miccai2014.org/cfp.html

Series	Lecture Notes in Computer Science
Volume	8673
ISSN	0302-9743

Access to Document

10.1007/978-3-319-10404-1_50

OpenUrl availability

Check availability in DTU Findit

Cite this

@inproceedings{a0b649a3f63445ffbac4e15ad65f58a5,

title = "A Cautionary Analysis of STAPLE Using Direct Inference of Segmentation Truth",

abstract = "In this paper we analyze the properties of the well-known segmentation fusion algorithm STAPLE, using a novel inference technique that analytically marginalizes out all model parameters. We demonstrate both theoretically and empirically that when the number of raters is large, or when consensus regions are included in the model, STAPLE devolves into thresholding the average of the input segmentations. We further show that when the number of raters is small, the STAPLE result may not be the optimal segmentation truth estimate, and its model parameter estimates might not reflect the individual raters{\textquoteright} actual segmentation performance. Our experiments indicate that these intrinsic weaknesses are frequently exacerbated by the presence of undesirable global optima and convergence issues. Together these results cast doubt on the soundness and usefulness of typical STAPLE outcomes.",

author = "\{Van Leemput\}, Koen and Sabuncu, \{Mert R.\}",

year = "2014",

doi = "10.1007/978-3-319-10404-1\_50",

language = "English",

isbn = "978-3-319-10403-4",

series = "Lecture Notes in Computer Science",

publisher = "Springer",

pages = "398--406",

booktitle = "Proceedings of the 17th International Conference Medical Image Computing and Computer-Assisted Intervention (MICCAI 2014)",

note = "17th International Conference on Medical Image Computing and Computer Assisted Intervention, MICCAI 2014 ; Conference date: 14-09-2014 Through 18-09-2014",

url = "http://miccai2014.org/, http://miccai2014.org/cfp.html",

}

Van Leemput, K & Sabuncu, MR 2014, A Cautionary Analysis of STAPLE Using Direct Inference of Segmentation Truth. in Proceedings of the 17th International Conference Medical Image Computing and Computer-Assisted Intervention (MICCAI 2014): Part I. Springer, Lecture Notes in Computer Science, vol. 8673, pp. 398-406, 17th International Conference on Medical Image Computing and Computer Assisted Intervention, Boston, Massachusetts, United States, 14/09/2014. https://doi.org/10.1007/978-3-319-10404-1_50

A Cautionary Analysis of STAPLE Using Direct Inference of Segmentation Truth. / Van Leemput, Koen; Sabuncu, Mert R.
Proceedings of the 17th International Conference Medical Image Computing and Computer-Assisted Intervention (MICCAI 2014): Part I. Springer, 2014. p. 398-406 (Lecture Notes in Computer Science, Vol. 8673).

Research output: Chapter in Book/Report/Conference proceeding › Article in proceedings › Research › peer-review

TY - GEN

T1 - A Cautionary Analysis of STAPLE Using Direct Inference of Segmentation Truth

AU - Van Leemput, Koen

AU - Sabuncu, Mert R.

N1 - Conference code: 17

PY - 2014

Y1 - 2014

N2 - In this paper we analyze the properties of the well-known segmentation fusion algorithm STAPLE, using a novel inference technique that analytically marginalizes out all model parameters. We demonstrate both theoretically and empirically that when the number of raters is large, or when consensus regions are included in the model, STAPLE devolves into thresholding the average of the input segmentations. We further show that when the number of raters is small, the STAPLE result may not be the optimal segmentation truth estimate, and its model parameter estimates might not reflect the individual raters’ actual segmentation performance. Our experiments indicate that these intrinsic weaknesses are frequently exacerbated by the presence of undesirable global optima and convergence issues. Together these results cast doubt on the soundness and usefulness of typical STAPLE outcomes.

AB - In this paper we analyze the properties of the well-known segmentation fusion algorithm STAPLE, using a novel inference technique that analytically marginalizes out all model parameters. We demonstrate both theoretically and empirically that when the number of raters is large, or when consensus regions are included in the model, STAPLE devolves into thresholding the average of the input segmentations. We further show that when the number of raters is small, the STAPLE result may not be the optimal segmentation truth estimate, and its model parameter estimates might not reflect the individual raters’ actual segmentation performance. Our experiments indicate that these intrinsic weaknesses are frequently exacerbated by the presence of undesirable global optima and convergence issues. Together these results cast doubt on the soundness and usefulness of typical STAPLE outcomes.

U2 - 10.1007/978-3-319-10404-1_50

DO - 10.1007/978-3-319-10404-1_50

M3 - Article in proceedings

SN - 978-3-319-10403-4

T3 - Lecture Notes in Computer Science

SP - 398

EP - 406

BT - Proceedings of the 17th International Conference Medical Image Computing and Computer-Assisted Intervention (MICCAI 2014)

PB - Springer

T2 - 17th International Conference on Medical Image Computing and Computer Assisted Intervention

Y2 - 14 September 2014 through 18 September 2014

ER -

A Cautionary Analysis of STAPLE Using Direct Inference of Segmentation Truth

Abstract

Conference

Access to Document

OpenUrl availability

Fingerprint

Cite this