A Simulation System for Scene Synthesis in Virtual Reality

Jingyu Liu*, Claire Mantel, Florian Schweiger, Søren Forchhammer

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingArticle in proceedingsResearchpeer-review

116 Downloads (Pure)


Real-world scene synthesis can be realized through view synthesis or 3D reconstruction methods. While industrial and commercial demands emerge for real-world scene synthesis in virtual reality (VR) using head-mounted displays (HMDs), the methods in the literature generally do not target specific display devices. To meet the rising demands, we propose a simulation system to evaluate scene synthesis methods in VR. Our system aims at providing the full pipeline of scene capturing, processing, rendering, and evaluation. The capturing module provides various input dataset formulations. The processing and rendering module integrates three representative scene synthesis methods with a voluntary performance-aid option. Finally, the evaluation module supports traditional metrics as well as perception-based metrics. An experiment demonstrates the use of our system for identifying the best capturing strategy among the three degrees of foveation tested. As can be expected, the FovVideoVDP metric (based on a model of the human visual system) finds the highest degree of foveation giving best results. The three other quality metrics from the evaluation module (which use features to measure similarity) confirm that result. The synthetic scenes in the experiment can run in VR with an average latency of 5.9 ms for the two selected scenarios across the tested methods on Nvidia GTX 2080 ti.
Original languageEnglish
Title of host publicationInternational Conference on Virtual Reality and Mixed Reality
Publication date2021
ISBN (Print)978-3-030-90738-9
Publication statusPublished - 2021
Event18th EuroXR International Conference - Virtual event, Milano, Italy
Duration: 24 Nov 202126 Nov 2021


Conference18th EuroXR International Conference
LocationVirtual event
SeriesLecture Notes in Computer Science


  • View synthesis
  • 3D reconstruction
  • Virtual reality


Dive into the research topics of 'A Simulation System for Scene Synthesis in Virtual Reality'. Together they form a unique fingerprint.

Cite this