Reducing the duration of paired-comparison experiments for visual quality assessment

Claire Mantel; Søren Forchhammer; Jari Korhonen

doi:10.1109/QoMEX.2013.6603209

Reducing the duration of paired-comparison experiments for visual quality assessment

Claire Mantel, Søren Forchhammer, Jari Korhonen

Research output: Chapter in Book/Report/Conference proceeding › Article in proceedings › Research › peer-review

Abstract

The subjective methodology of paired comparison is currently recognized as the most precise methodology. However, it is not widely used because it implies a larger number of assessments than other methodologies and therefore a longer test duration. In this paper we investigate how sorting algorithms can be used to decrease the duration of such experiments by selecting only a fraction of all the possible comparisons. Three methods based on the insertion sort and the binary tree algorithms are applied to paired-comparison data. Performance analysis shows interesting trade-offs between the number of comparisons and the accuracy of the results.

Original language	English
Title of host publication	2013 Fifth International Workshop on Quality of Multimedia Experience (QoMEX)
Publisher	IEEE
Publication date	2013
Pages	50-51
ISBN (Print)	978-1-4799-0738-0
DOIs	https://doi.org/10.1109/QoMEX.2013.6603209
Publication status	Published - 2013
Event	5th International Workshop on Quality of Multimedia Experience (QoMEX 2013) - Klagenfurt am Wörthersee, Austria Duration: 3 Jul 2013 → 5 Jul 2013 http://www.qomex2013.org/

Workshop

Workshop	5th International Workshop on Quality of Multimedia Experience (QoMEX 2013)
Country/Territory	Austria
City	Klagenfurt am Wörthersee
Period	03/07/2013 → 05/07/2013
Internet address	http://www.qomex2013.org/

Access to Document

10.1109/QoMEX.2013.6603209

OpenUrl availability

Full text

Cite this

@inproceedings{8a51dd6a8a154b05aebc91b3cafceb1a,

title = "Reducing the duration of paired-comparison experiments for visual quality assessment",

abstract = "The subjective methodology of paired comparison is currently recognized as the most precise methodology. However, it is not widely used because it implies a larger number of assessments than other methodologies and therefore a longer test duration. In this paper we investigate how sorting algorithms can be used to decrease the duration of such experiments by selecting only a fraction of all the possible comparisons. Three methods based on the insertion sort and the binary tree algorithms are applied to paired-comparison data. Performance analysis shows interesting trade-offs between the number of comparisons and the accuracy of the results.",

author = "Claire Mantel and S{\o}ren Forchhammer and Jari Korhonen",

year = "2013",

doi = "10.1109/QoMEX.2013.6603209",

language = "English",

isbn = "978-1-4799-0738-0",

pages = "50--51",

booktitle = "2013 Fifth International Workshop on Quality of Multimedia Experience (QoMEX)",

publisher = "IEEE",

address = "United States",

note = "5th International Workshop on Quality of Multimedia Experience (QoMEX 2013) ; Conference date: 03-07-2013 Through 05-07-2013",

url = "http://www.qomex2013.org/",

}

Mantel, C , Forchhammer, S & Korhonen, J 2013, Reducing the duration of paired-comparison experiments for visual quality assessment. in 2013 Fifth International Workshop on Quality of Multimedia Experience (QoMEX). IEEE, pp. 50-51, 5th International Workshop on Quality of Multimedia Experience (QoMEX 2013), Klagenfurt am Wörthersee, Austria, 03/07/2013. https://doi.org/10.1109/QoMEX.2013.6603209

TY - GEN

T1 - Reducing the duration of paired-comparison experiments for visual quality assessment

AU - Mantel, Claire

AU - Forchhammer, Søren

AU - Korhonen, Jari

PY - 2013

Y1 - 2013

N2 - The subjective methodology of paired comparison is currently recognized as the most precise methodology. However, it is not widely used because it implies a larger number of assessments than other methodologies and therefore a longer test duration. In this paper we investigate how sorting algorithms can be used to decrease the duration of such experiments by selecting only a fraction of all the possible comparisons. Three methods based on the insertion sort and the binary tree algorithms are applied to paired-comparison data. Performance analysis shows interesting trade-offs between the number of comparisons and the accuracy of the results.

AB - The subjective methodology of paired comparison is currently recognized as the most precise methodology. However, it is not widely used because it implies a larger number of assessments than other methodologies and therefore a longer test duration. In this paper we investigate how sorting algorithms can be used to decrease the duration of such experiments by selecting only a fraction of all the possible comparisons. Three methods based on the insertion sort and the binary tree algorithms are applied to paired-comparison data. Performance analysis shows interesting trade-offs between the number of comparisons and the accuracy of the results.

U2 - 10.1109/QoMEX.2013.6603209

DO - 10.1109/QoMEX.2013.6603209

M3 - Article in proceedings

SN - 978-1-4799-0738-0

SP - 50

EP - 51

BT - 2013 Fifth International Workshop on Quality of Multimedia Experience (QoMEX)

PB - IEEE

T2 - 5th International Workshop on Quality of Multimedia Experience (QoMEX 2013)

Y2 - 3 July 2013 through 5 July 2013

ER -

Reducing the duration of paired-comparison experiments for visual quality assessment

Abstract

Workshop

Access to Document

OpenUrl availability

Fingerprint

Cite this