IEICE Transactions on Communications
Online ISSN : 1745-1345
Print ISSN : 0916-8516
Regular Section
Performance Comparisons of Subjective Quality Assessment Methods for Video
Toshiko TOMINAGAMasataka MASUDAJun OKAMOTOAkira TAKAHASHITakanori HAYASHI
Author information
JOURNAL RESTRICTED ACCESS

2014 Volume E97.B Issue 1 Pages 66-75

Details
Abstract

Many subjective assessment methods for video quality are provided by ITU-T and ITU-R recommendations, but the differences among these methods have not been sufficiently studied. We compare five subjective assessment methods using four quantitative performance indices for both HD and QVGA resolution video. We compare the Double-Stimulus Continuous Quality-Scale (DSCQS), Double-Stimulus Impairment Scale (DSIS), Absolute Category Rating method (ACR), and ACR with Hidden Reference (ACR-HR) as common subjective assessment methods for HD and QVGA resolution videos. Furthermore, we added ACR with an 11-grade scale (ACR11) for the HD test and Subjective Assessment of Multimedia Video Quality (SAMVIQ) for the QVGA test for quality scale variations. The performance indices are correlation coefficients, rank correlation coefficients, statistical reliability, and assessment time. For statistical reliability, we propose a performance index for comparing different quality scale tests. The results of the performance comparison showed that the correlation coefficients and rank correlation coefficients of the mean opinion scores between pairs of methods were high for both HD and QVGA tests. As for statistical reliability provided by the proposed index, DSIS of HD and ACR of QVGA outperformed the other methods. Moreover, ACR, ACR-HR, and ACR11 were the most efficient subjective quality assessment methods from the viewpoint of assessment time.

Content from these authors
© 2014 The Institute of Electronics, Information and Communication Engineers
Previous article Next article
feedback
Top