Prediction of Patient Demographics using 3D Craniofacial Scans and Multi-view CNNs

Umaer Rashid Hanif, Rasmus Reinhold Paulsen, Eileen B. Leary, Emmanuel Mignot, Poul Jennum, Helge Bjarup Dissing Sørensen

Research output: Chapter in Book/Report/Conference proceedingArticle in proceedingsResearchpeer-review

Abstract

3D data is becoming increasingly popular and accessible for computer vision tasks. A popular format for 3D data is the mesh format, which can depict a 3D surface accurately and cost-effectively by connecting points in the (x, y, z) plane, known as vertices, into triangles that can be combined to approximate geometrical surfaces. However, mesh objects are not suitable for standard deep learning techniques due to their non-euclidean structure. We present an algorithm which predicts the sex, age, and body mass index of a subject based on a 3D scan of their face and neck. This algorithm relies on an automatic pre-processing technique, which renders and captures the 3D scan from eight different angles around the x-axis in the form of 2D images and depth maps. Subsequently, the generated data is used to train three convolutional neural networks, each with a ResNet18 architecture, to learn a mapping between the set of 16 images per subject (eight 2D images and eight depth maps from different angles) and their demographics. For age and body mass index, we achieved a mean absolute error of 7.77 years and 4.04 kg/m2 on the respective test sets, while Pearson correlation coefficients of 0.76 and 0.80 were obtained, respectively. The prediction of sex yielded an accuracy of 93%. The developed framework serves as a proof of concept for prediction of more clinically relevant variables based on 3D craniofacial scans stored in mesh objects.
Original languageEnglish
Title of host publicationProceedings of the 42nd Annual International Conference of the IEEE Engineering in Medicine & Biology Society
PublisherIEEE
Publication date2020
Pages1950-1953
ISBN (Print)978-1-7281-1990-8
DOIs
Publication statusPublished - 2020
Event42nd Annual International Conference of the IEEE Engineering in Medicine & Biology Society - EMBS Virtual Academy, Montreal, Canada
Duration: 20 Jul 202024 Jul 2020

Conference

Conference42nd Annual International Conference of the IEEE Engineering in Medicine & Biology Society
LocationEMBS Virtual Academy
CountryCanada
CityMontreal
Period20/07/202024/07/2020

Cite this

Hanif, U. R., Paulsen, R. R., Leary, E. B., Mignot, E., Jennum, P., & Sørensen, H. B. D. (2020). Prediction of Patient Demographics using 3D Craniofacial Scans and Multi-view CNNs. In Proceedings of the 42nd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (pp. 1950-1953). IEEE. https://doi.org/10.1109/EMBC44109.2020.9176333