A Variational Analysis of Shape from Specularities using Sparse Data

Looking around in our every day environment, many of the encountered objects are specular to some degree. Actively using this fact when reconstructing objects from image sequences is the scope of the shape from specularities problem. One reason why this problem is important is that standard structure from motion techniques fail when the object surfaces are specular. Here this problem is addressed by estimating surface shape using information from the specular reflections. A specular reflection gives constraints on the surface normal. The approach taken in this paper differs significantly from many earlier shape from specularities methods since the normal data used is sparse. The main contribution of this paper is to give a solid foundation for shape from specularities problems. Estimation of surface shape using reflections is formulated as a variational problem and the surface is represented implicitly using a level set formulation. A functional incorporating all surface constraints is proposed and the corresponding level set motion PDE is explicitly derived. This motion is then proven to minimize the functional. As a part of this functional a variational approach to normal alignment is proposed and analyzed. Also novel methods for implicit surface interpolation to sparse point sets are presented together with a variational analysis. Experiments on both real and synthetic data support the proposed method.