A Simulation System for Scene Synthesis in Virtual Reality

Research output: Chapter in Book/Report/Conference proceedingArticle in proceedingsResearchpeer-review

26 Downloads (Pure)

Abstract

Real-world scene synthesis can be realized through view synthesis or 3D reconstruction methods. While industrial and commercial demands emerge for real-world scene synthesis in virtual reality (VR) using head-mounted displays (HMDs), the methods in the literature generally do not target specific display devices. To meet the rising demands, we propose a simulation system to evaluate scene synthesis methods in VR. Our system aims at providing the full pipeline of scene capturing, processing, rendering, and evaluation. The capturing module provides various input dataset formulations. The processing and rendering module integrates three representative scene synthesis methods with a voluntary performance-aid option. Finally, the evaluation module supports traditional metrics as well as perception-based metrics. An experiment demonstrates the use of our system for identifying the best capturing strategy among the three degrees of foveation tested. As can be expected, the FovVideoVDP metric (based on a model of the human visual system) finds the highest degree of foveation giving best results. The three other quality metrics from the evaluation module (which use features to measure similarity) confirm that result. The synthetic scenes in the experiment can run in VR with an average latency of 5.9 ms for the two selected scenarios across the tested methods on Nvidia GTX 2080 ti.
Original languageEnglish
Title of host publicationProceedings of EuroXR 2021
Number of pages18
PublisherIEEE
Publication statusAccepted/In press - 2021
Event18th EuroXR International Conference - Virtual event, Milano, Italy
Duration: 24 Nov 202126 Nov 2021

Conference

Conference18th EuroXR International Conference
LocationVirtual event
Country/TerritoryItaly
CityMilano
Period24/11/202126/11/2021

Keywords

  • View synthesis
  • 3D reconstruction
  • Virtual reality

Fingerprint

Dive into the research topics of 'A Simulation System for Scene Synthesis in Virtual Reality'. Together they form a unique fingerprint.

Cite this