3D Image Segmentation with Explicit Surface-Based Priors

In the past decades, the amount and quality of 3D image data being produced have increased tremendously. This has brought the need for automated tools to analyze these images. In some domains, models based on deep learning have become dominant. However, there are still areas where the application of such models is challenging due to the lack of annotated data and the sheer size of 3D images. This thesis explores the use of explicit surfaces in image segmentation to address this issue. Explicit surfaces in the form of meshes have two concrete benefits: First, they easily allow the incorporation of prior knowledge on shape and topology. This helps alleviate the issues of no annotated data for machine learning models. Second, they move the computational scaling to the size of the mesh instead of the size of the image. This allows mesh-based models to scale to very large segmentation problems, as this thesis will show. The first part of the thesis focuses on fitting meshes to images with minimum cut/maximum flow (min-cut/max-flow). We review the current state of the art for serial and parallel min-cut/max-flow algorithms, develop a new min-cut/max-flow-based parallel algorithm for quadratic pseudo-Boolean optimization (QPBO), and a new method for multi-object segmentation with exclusion constraints based on QPBO. We demonstrate that mesh fitting with min-cut/max-flow can be scaled to large segmentation problems with thousands of interacting objects. Furthermore, for large problems, their solution can be parallelized efficiently, although we show parallel min-cut/max-flow algorithms struggle with speeding up smaller problems. Next, we contribute a method for segmentation of 4D (3D + time) images, which focuses on objects that either split or merge over time. We demonstrate how a simple topology constraint leads to a highly efficient method that fully captures object dynamics. In the last part of the thesis, we investigate the use of explicit surfaces in learning-based segmentation. Using surface fitting, we develop a method that allows the training of neural network segmentation models from only sparse point annotations. We show that this allows training more accurate models from significantly less annotation effort. Finally, we develop a new mesh-based shape model which can learn a latent shape space and then smoothly combine multiple latent representations. We demonstrate that this makes for a highly flexible model which can easily be used for a variety of tasks — for example, efficient image annotation and segmentation post-processing. In conclusion, the contributions in this thesis show meshes to be a powerful tool for image segmentation from both a modeling and computational standpoint. Therefore, meshes have great potential to form the basis of accurate and scalable segmentation models to handle the ever-growing amount of 3D image data.