Neural Form Representation

Over the last couple of years there have been major advances within the fields of image segmentation, object recognition and image generation using deep learning models. Programs such as DALL-E and Midjourney have gained much popularity for their remarkable ability to generate realistically looking images given only a text prompt. However, similar techniques for 3D shapes have not experienced the same success. This difference can largely be attributed to a representation problem. In contrast to the simple and regular grid structure found in images, we do not have a canonical and simple way of representing 3D shapes which is easy to learn. Voxel-based representations are computationally demanding, and since shapes can have different geometry and topology, we do not have a generic mesh in which we can synthesize arbitrary shapes.

Therefore, the main problem is: How do you represent objects of different topology using learning based methods?

In recent years different works on neural implicit surface representations have tried to solve this problem, but numerous limitations remain. Furthermore, current state of the art methods have only focused on the surfaces of objects, but most items also have an internal structure. This inner structure has a great importance in many areas such as mechanics, if one wants to compute, how an object can be deformed/do strength and stress calculations/solve topology optimization problems, or in medical image analysis, if one wants to determine e.g. how the shape and structure of a heart of one patient differs from the heart of another patient.

Consequently, this PhD thesis seeks to address the limitations with regards to surface representations but also tries to advance the research by going beyond a pure surface presentation and finding ways to represent objects that have an interior structure.