How to evaluate objective video quality metrics reliably

Jari Korhonen, Nino Burini, Junyong You, Ehsan Nadernejad

Research output: Chapter in Book/Report/Conference proceedingArticle in proceedingsResearchpeer-review

Abstract

The typical procedure for evaluating the performance of different objective quality metrics and indices involves comparisons between subjective quality ratings and the quality indices obtained using the objective metrics in question on the known video sequences. Several correlation indicators can be employed to assess how well the subjective ratings can be predicted from the objective values. In this paper, we give an overview of the potential sources for uncertainties and inaccuracies in such studies, related both to the method of comparison, possible inaccuracies in the subjective data, as well as processing of subjective data. We also suggest some general guidelines for researchers to make comparison studies of objective video quality metrics more reliable and useful for the practitioners in the field.
Original languageEnglish
Title of host publication2012 Fourth International Workshop on Quality of Multimedia Experience (QoMEX)
PublisherIEEE
Publication date2012
Pages57-62
ISBN (Print)978-1-4673-0724-6
ISBN (Electronic)978-1-4673-0725-3
DOIs
Publication statusPublished - 2012
Event4th International Workshop onQuality of Multimedia Experience (QoMEX) - Melbourne, Australia
Duration: 5 Jul 20127 Jul 2012
http://www.qomex2012.org/

Conference

Conference4th International Workshop onQuality of Multimedia Experience (QoMEX)
CountryAustralia
CityMelbourne
Period05/07/201207/07/2012
Internet address

Keywords

  • Video quality assessment
  • Objective metrics
  • Reliability

Cite this

Korhonen, J., Burini, N., You, J., & Nadernejad, E. (2012). How to evaluate objective video quality metrics reliably. In 2012 Fourth International Workshop on Quality of Multimedia Experience (QoMEX) (pp. 57-62 ). IEEE. https://doi.org/10.1109/QoMEX.2012.6263839