Estimating and Simulating Structure and Motion

Estimation and simulation of structure and motion are key topics in computer vision, computer graphics, and augmented reality. There are many connections between these terms. Structure and motion are naturally connected, and knowledge of either implies
knowledge about the other. When dealing with images, estimation and simulation are, in some aspects, the opposites of each other. When we estimate, we extract information from images, and when we simulate the imaging process, we create images given information about the appearance of the scene. In recent years, the interplay between these four terms has grown larger. An example of this is in augmented reality applications where simulation and estimation must work together. Accurate estimation of the surrounding environment is necessary to create realistic, digitally overlaid content and immersive experiences. In this thesis, we present work that lies in the span of estimation, simulation, structure, and motion. Our goal is to explore synergies between simulation and estimation concerning structure and motion. We investigate how we can use simulations to make better estimations and how we can use estimations to create realistic simulations. The work is split into three areas. Estimating motion to interpolate frames in video sequences, estimating geometry to create illusory motion of physical objects, and simulating the imaging process to estimate geometry. Our contributions demonstrate that we can use simulations to estimate geometry better and that by estimating structure and motion, we can simulate novel camera viewpoints in time and space that can be used to create both video frame interpolations and immersive augmented reality experiences.