Perceptual Evaluation of 360 Audiovisual Quality and Machine Learning Predictions

Randy Frans Fela, Nick Zacharov, Søren Forchhammer

Research output: Chapter in Book/Report/Conference proceedingArticle in proceedingsResearchpeer-review

33 Downloads (Pure)

Abstract

In an earlier study, we gathered perceptual evaluations of the audio, video, and audiovisual quality for 360 audiovisual content. This paper investigates perceived audiovisual quality prediction based on objective quality metrics and subjective scores of 360 video and spatial audio content. Thirteen objective video quality metrics and three objective audio quality metrics were evaluated for five stimuli for each coding parameter. Four regression-based machine learning models were trained and tested here, i.e., multiple linear regression, decision tree, random forest, and support vector machine. Each model was constructed using a combination of audio and video quality metrics and two cross-validation methods (k-Fold and Leave-One-Out) were investigated and produced 312 predictive models. The results indicate that the model based on the evaluation of VMAF and AMBIQUAL is better than other combinations of audio-video quality metric. In this study, support vector machine provides higher performance using k-Fold (PCC = 0.909, SROCC = 0.914, and RMSE = 0.416). These results can provide insights for the design of multimedia quality metrics and the development of predictive models for audiovisual omnidirectional media.
Original languageEnglish
Title of host publicationProceedings of 2021 IEEE International Workshop on Multimedia Signal Processing
Number of pages6
PublisherIEEE
ISBN (Print)978-1-6654-3288-7
Publication statusAccepted/In press - 2022
Event2021 IEEE 23rd International Workshop on Multimedia Signal Processing - Hybrid event, Tampere, Finland
Duration: 6 Oct 20218 Oct 2021
Conference number: 23
https://attend.ieee.org/mmsp-2021/

Conference

Conference2021 IEEE 23rd International Workshop on Multimedia Signal Processing
Number23
LocationHybrid event
Country/TerritoryFinland
CityTampere
Period06/10/202108/10/2021
Internet address

Keywords

  • Perceptual evaluation
  • 360 video
  • Spatial audio
  • Machine learning
  • Multimedia quality metrics
  • Higher order ambisonsics

Fingerprint

Dive into the research topics of 'Perceptual Evaluation of 360 Audiovisual Quality and Machine Learning Predictions'. Together they form a unique fingerprint.

Cite this