Markerless 3D Face Tracking

Christian Walder, Martin Breidt, Heinrich Bulthoff, Bernhard Scholkopf, Cristobal Curio

    Research output: Chapter in Book/Report/Conference proceedingArticle in proceedingsResearchpeer-review

    429 Downloads (Pure)


    We present a novel algorithm for the markerless tracking of deforming surfaces such as faces. We acquire a sequence of 3D scans along with color images at 40Hz. The data is then represented by implicit surface and color functions, using a novel partition-of-unity type method of efficiently combining local regressors using nearest neighbor searches. Both these functions act on the 4D space of 3D plus time, and use temporal information to handle the noise in individual scans. After interactive registration of a template mesh to the first frame, it is then automatically deformed to track the scanned surface, using the variation of both shape and color as features in a dynamic energy minimization problem. Our prototype system yields high-quality animated 3D models in correspondence, at a rate of approximately twenty seconds per timestep. Tracking results for faces and other objects are presented.
    Original languageEnglish
    Title of host publicationPattern Recognition
    Number of pages10
    Publication date2009
    ISBN (Print)978-3-642-03797-9
    Publication statusPublished - 2009
    EventDeutsche Arbeitsgemeinschaft fuer Mustererkennung : DAGM -
    Duration: 1 Jan 2009 → …


    ConferenceDeutsche Arbeitsgemeinschaft fuer Mustererkennung : DAGM
    Period01/01/2009 → …
    SeriesLecture Notes in Computer Science


    Dive into the research topics of 'Markerless 3D Face Tracking'. Together they form a unique fingerprint.

    Cite this