Skip to main content
SpringerLink
Log in

Understanding and estimating quality of experience in WebRTC applications

  • Published:
Computing Aims and scope Submit manuscript

Abstract

WebRTC comprises a set of technologies and standards that provide real-time communication with web browsers, simplifying the embedding of voice and video communication in web applications and mobile devices. The perceived quality of WebRTC communication can be measured using quality of experience (QoE) indicators. QoE is defined as the degree of delight or annoyance of the user with an application or service. This paper is focused on the QoE assessment of WebRTC-based applications and its contribution is threefold. First, an analysis of how WebRTC topologies affect the quality perceived by users is provided. Second, a group of Key Performance Indicators for estimating the QoE of WebRTC users is proposed. Finally, a systematic survey of the literature on QoE assessment in the WebRTC arena is presented.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

Notes

  1. https://jitsi.org/.

  2. https://janus.conf.meetecho.com/.

  3. http://www.medooze.com/.

  4. http://lynckia.com/licode/.

  5. http://www.kurento.org/.

  6. https://www.scopus.com/.

  7. http://academic.research.microsoft.com/.

  8. http://www.sciencedirect.com/.

  9. http://ieeexplore.ieee.org/.

  10. http://dl.acm.org/.

  11. Our search was last updated in March 2018.

References

  1. Alvestrand H, Holmer S (2012) A Google congestion control for real-time communication on the World Wide Web. Tech. rep, IETF

  2. Ammar D, De Moor K, Xie M, Fiedler M, Heegaard P (2016) Video QoE killer and performance statistics in WebRTC-based video communication. In: Communications and Electronics (ICCE), 2016 IEEE Sixth International Conference on, IEEE, pp 429–436

  3. Bandung Y, Subekti LB, Tanjung D, Chrysostomou C (2017) QoS analysis for WebRTC videoconference on bandwidth-limited network. In: 2017 20th International symposium on wireless personal multimedia communications (WPMC), pp 547–553

  4. Bevan N (1999) Quality in use: meeting user needs for quality. J Syst Softw 49(1):89–96

    Article  Google Scholar 

  5. Boubendir A, Bertin E, Simoni N (2016) On-demand, dynamic and at-the-edge VNF deployment model application to Web Real-Time Communications. In: Network and service management (CNSM), 2016 12th international conference on, IEEE, pp 318–323

  6. Brunnström K, Beker SA, De Moor K, Dooms A, Egger S, Garcia MN, Hossfeld T, Jumisko-Pyykkö S, Keimel C, Larabi MC, et al (2013) Qualinet white paper on definitions of quality of experience

  7. Carlucci G, De Cicco L, Holmer S, Mascolo S (2016) Analysis and design of the Google congestion control for web real-time communication (WebRTC). In: Proceedings of the 7th international conference on multimedia systems, ACM, pp 13:1–13:12

  8. Carullo G, Tambasco M, Di Mauro M, Longo M (2016) A performance evaluation of WebRTC over LTE. In: Wireless on-demand network systems and services (WONS), 2016 12th annual conference on, IEEE, pp 1–6

  9. Chandler DM, Hemami SS (2007) VSNR: a wavelet-based visual signal-to-noise ratio for natural images. IEEE Trans Image Process 16(9):2284–2298

    Article  MathSciNet  Google Scholar 

  10. Chen Y, Wu K, Zhang Q (2015) From QoS to QoE: a tutorial on video quality assessment. IEEE Commun Surv Tutor 17(2):1126–1165

    Article  Google Scholar 

  11. Chikkerur S, Sundaram V, Reisslein M, Karam LJ (2011) Objective video quality assessment methods: a classification, review, and performance comparison. IEEE Trans Broadcast 57(2):165–182

    Article  Google Scholar 

  12. Chodorek RR, Chodorek A, Rzym G, Wajda K (2017) A comparison of QoS parameters of WebRTC videoconference with conference bridge placed in private and public cloud. In: Enabling technologies: infrastructure for collaborative enterprises (WETICE), 2017 IEEE 26th international conference on, IEEE, pp 86–91

  13. Chong HM, Matthews HS (2004) Comparative analysis of traditional telephone and voice-over-internet protocol (voip) systems. In: Electronics and the environment, 2004. Conference record. 2004 IEEE international symposium on, IEEE, pp 106–111

  14. Cisco VNI (2017) Forecast and Methodology, 2016–2021. White Paper

  15. Edan NM, Al-Sherbaz A, Turner S (2017) WebNSM: A novel scalable WebRTC signalling mechanism for many-to-many video conferencing. In: Collaboration and internet computing (CIC), 2017 IEEE 3rd international conference on, IEEE, pp 27–33

  16. Egger S, Schatz R, Scherer S (2010) It takes two to tango—assessing the impact of delay on conversational interactivity on perceived speech quality. In: 11th Annual conference of the international speech communication association (ISCA), pp 1321–1324

  17. García B, Gortázar F, López-Fernández L, Gallego M (2017a) WebRTC testing: challenges and practical solutions. IEEE Commun Stand Mag 1(2):36–42

    Article  Google Scholar 

  18. García B, López-Fernández L, Gallego M, Gortázar F (2017b) Kurento: the Swiss army knife of WebRTC media servers. IEEE Commun Stand Mag 1(2):44–51

    Article  Google Scholar 

  19. Garvin DA (1984) What does “product quality” really mean? MIT Sloan Manag Rev 26(1):25–43

    Google Scholar 

  20. Grigorik I (2013) High performance browser networking: what every web developer should know about networking and web performance. O’Reilly Media, Inc

  21. Handley M, Perkins C, Jacobson V (2006) RFC 4566. Session description protocol. Tech. rep., IETF, SDP

  22. Hekstra AP, Beerends JG, Ledermann D, De Caluwe F, Kohler S, Koenen R, Rihs S, Ehrsam M, Schlauss D (2002) PVQM: a perceptual video quality measure. Signal Process: Image Commun 17(10):781–798

    Google Scholar 

  23. Herrero R (2017) Integrating HEC with circuit breakers and multipath RTP to improve RTC media quality. Telecommun Syst 64(1):211–221

    Article  Google Scholar 

  24. Hoßfeld T, Schatz R, Varela M, Timmerer C (2012) Challenges of QoE management for cloud applications. IEEE Commun Mag 50(4):28–36

    Article  Google Scholar 

  25. Husić JB, Baraković S, Veispahić A (2017) What factors influence the quality of experience for WebRTC video calls? In: Information and communication technology, electronics and microelectronics (MIPRO), 2017 40th international convention on, IEEE, pp 428–433

  26. Huynh-Thu Q, Ghanbari M (2008) Scope of validity of PSNR in image/video quality assessment. Electron Lett 44(13):800–801

    Article  Google Scholar 

  27. ISO (2005) ISO 9000: Quality management systems—fundamentals and vocabulary. International Organization for Standardization

  28. ITU-R (2007) Recommendation BT.1788. Methodology for the subjective assessment of video quality in multimedia applications

  29. ITU-T (2001) Recommendation G.1010. End-user multimedia QoS categories

  30. ITU-T (2003) Recommendation G.114. Transmission systems and media digital systems and networks

  31. ITU-T (2006) Recommendation P.10. Vocabulary for performance and quality of service

  32. ITU-T (2008a) Recommendation E.800. Definitions of terms related to quality of service

  33. ITU-T (2008b) Recommendation J.247. Objective perceptual multimedia video quality measurement in the presence of a full reference

  34. ITU-T (2011) Recommendation P.863. Perceptual objective listening quality assessment

  35. ITU-T (2014) Recommendation E.807: definitions, associated measurement methods and guidance targets of user-centric parameters for call handling in cellular mobile voice service

  36. ITU-T (2016a) Recommendation P.10/G.100. Vocabulary for performance and quality of service (Amendment 5)

  37. ITU-T (2016b) Recommendation. P.800.2. Mean opinion score interpretation and reporting

  38. Ivov E, Rescorla E, Uberti J (2013) Trickle ICE: incremental provisioning of candidates for the interactive connectivity establishment (ICE) protocol. Tech. rep, IETF

  39. Jin J, Nahrstedt K (2004) QoS specification languages for distributed multimedia applications: a survey and taxonomy. IEEE Multimed 11(3):74–87

    Article  Google Scholar 

  40. Johansson I, Sarker Z (2017) Self-clocked rate adaptation for multimedia. Tech. rep, IETF

  41. Khan M (2017) WebRTCPedia! the Encyclopedia! https://www.webrtc-experiment.com/webrtcpedia/, [Online; accessed 11 June 2018]

  42. Kilinc C, Andersson K (2014) A congestion avoidance mechanism for WebRTC interactive video sessions in LTE networks. Wirel Pers Commun 77(4):2417–2443

    Article  Google Scholar 

  43. Kitchenham B (2004) Procedures for performing systematic reviews. Keele UK Keele Univ 33(2004):1–26

    Google Scholar 

  44. Kitchenham B, Pfleeger SL (1996) Software quality: the elusive target [special issues section]. IEEE Softw 13(1):12–21

    Article  Google Scholar 

  45. Komperda O, Melvin H, Pota P (2016) A black box analysis of WebRTC mouth-to-ear delays. Communications pp 11–16

  46. Loreto S, Romano SP (2014) Real-time communication with WebRTC: peer-to-peer in the browser. O’Reilly Media, Inc

  47. Matthews P, Mahy R, Rosenberg J (2010) RFC 5766. Traversal using relays around NAT (TURN): relay extensions to session traversal utilities for NAT (STUN). Tech. rep., IETF

  48. Muñoz-Gea JP, Aparicio-Pardo R, Wehbe H, Simon G, Nuaymi L (2014) Optimization framework for uplink video transmission in HetNets. In: Proceedings of workshop on mobile video delivery, ACM

  49. Pinson MH, Wolf S (2004) A new standardized method for objectively measuring video quality. IEEE Trans Broadcast 50(3):312–322

    Article  Google Scholar 

  50. Rix AW, Hollier MP, Hekstra AP, Beerends JG (2002) Perceptual evaluation of speech quality (PESQ) the new ITU standard for end-to-end speech quality assessment Part I-Time-delay compensation. J Audio Eng Soc 50(10):755–764

    Google Scholar 

  51. Rosenberg J (2010) RFC 5245. Interactive connectivity establishment (ICE): a methodology for network address translator (NAT) traversal for offer/answer protocols. Tech. rep., IETF

  52. Rosenberg J, Schulzrinne H (2002) RFC 3264. An offer/answer model with SDP. Tech. rep, IETF

  53. Rosenberg J, Weinberger J, Huitema C, Mahy R (2003) RFC 3489. STUN, simple traversal of user datagram protocol (UDP) through network address translators (NATs). Tech. rep., IETF

  54. Rosenberg J, Mahy R, Matthews P, Wing D (2008) RFC 5389. Session traversal utilities for NAT (STUN)Tech. rep., IETF

  55. Sale S, Rebbeck T (2014) Operators need to engage with WebRTC and the opportunities it presents. Analysis Mason Report

  56. Santos-González I, Rivero-García A, Molina-Gil J, Caballero-Gil P (2017) Implementation and analysis of real-time streaming protocols. Sensors 17(4):1–17

    Article  Google Scholar 

  57. Takahashi A, Hands D, Barriac V (2008) Standardization activities in the ITU for a QoE assessment of IPTV. IEEE Commun Mag 46(2):78–84

    Article  Google Scholar 

  58. Timmerer C, Ebrahimi T, Pereira F (2015) Toward a new assessment of quality. Computer 48(3):108–110

    Article  Google Scholar 

  59. Tsiaras C, Rösch M, Stiller B (2015) VoIP-based calibration of the DQX Model. In: IFIP networking conference (IFIP networking), 2015, IEEE, pp 1–9

  60. Vucic D, Skorin-Kapov L (2015) The impact of mobile device factors on QoE for multi-party video conferencing via WebRTC. In: Telecommunications (ConTEL), 2015 13th international conference on, IEEE, pp 1–8

  61. Wang Z, Bovik AC, Sheikh HR, Simoncelli EP (2004) Image quality assessment: from error visibility to structural similarity. IEEE Trans Image Process 13(4):600–612

    Article  Google Scholar 

  62. Westerlund M, Wenger S (2015) RFC 5117. RTP topologies. Tech. rep, IETF

  63. Xiao F (2000) DCT-based video quality evaluation. Final Project for EE392J

  64. Zhang L, Amin SO, Westphal C (2017) VR video conferencing over named data networks. In: Proceedings of the workshop on virtual reality and augmented reality network, ACM, pp 7–12

  65. Zhao T, Liu Q, Chen CW (2017) QoE in video transmission: a user experience-driven strategy. IEEE Commun Surv Tutor 19(1):285–302

    Article  Google Scholar 

  66. Zhu X, Pan R, Ramalho MA, Mena S, Ganzhorn C, Jones PE (2015) NADA: A unified congestion control scheme for real-time media. Tech. rep, IETF

Download references

Acknowledgements

This work has been supported by the European Commission under project ElasTest (H2020-ICT-10-2016, GA-731535); by the Regional Government of Madrid (CM) under project Cloud4BigData (S2013/ICE-2894) cofunded by FSE & FEDER; and the Spanish Government under project LERNIM (RTC-2016-4674-7) cofunded by the Ministry of Economy and Competitiveness, FEDER & AEI.

Author information

Authors and Affiliations

  1. Universidad Rey Juan Carlos, Madrid, Spain

    Boni García, Micael Gallego & Francisco Gortázar

  2. Consiglio Nazionale delle Ricerche, Rome, Italy

    Antonia Bertolino

Authors
  1. Boni García
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Micael Gallego
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Francisco Gortázar
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Antonia Bertolino
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Boni García.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

García, B., Gallego, M., Gortázar, F. et al. Understanding and estimating quality of experience in WebRTC applications. Computing 101, 1585–1607 (2019). https://doi.org/10.1007/s00607-018-0669-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00607-018-0669-7

Keywords

Mathematics Subject Classification

Search

Navigation