Skip to main content
SpringerLink
Log in

Binaural Evaluation of Sound Quality and Quality of Experience

  • Chapter
  • First Online:
The Technology of Binaural Understanding

Part of the book series: Modern Acoustics and Signal Processing ((MASP))

Abstract

The chapter outlines the concepts of Sound Quality  and Quality of Experience (QoE).  Building on these, it describes a conceptual model of sound quality perception and experience during active listening in a spatial-audio context. The presented model of sound quality perception considers both bottom-up (signal-driven) as well as top-down (hypothesis-driven) perceptual functional processes. Different studies by the authors and from the literature are discussed in light of their suitability to help develop implementations of the conceptual model. As a key prerequisite, the underlying perceptual ground-truth data required for model training and validation are discussed, as well as means for deriving these from respective listening tests. Both feature-based and more holistic modeling approaches are analyzed. Overall, open research questions are summarized, deriving trajectories for future work on spatial-audio Sound Quality and Quality of Experience modeling.

Hagen Wierstorf is now with audEERING GmbH.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 189.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Hardcover Book
USD 249.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    The chapter is a synthesis and extension of the current authors’ work presented in Raake and Blauert (2013), Raake and Egger (2014), Raake et al. (2014b), Raake (2016) and Raake and Wierstorf (2016).

  2. 2.

    https://medium.com/netflix-techblog/dynamic-optimizer-a-perceptual-video-encoding-optimization-framework-e19f1e3a277f [last accessed: August 30, 2019].

  3. 3.

    The modeling concepts presented are related to the authors’ work in the Two!Ears project. Evaluating sound quality for spatial-audio systems has been one of Two!Ears ’ two proof-of-concept applications (Raake and Blauert 2013; Raake and Wierstorf 2016; Wierstorf et al. 2018). The Two!Ears-system architecture is open and modular. All documentation, code, data as well as descriptions for hardware implementation are accessible open-source under www.twoears.eu [last accessed: February 18, 2020].

  4. 4.

    The representations available in memory in abstracted form, and used at the different perceptual and evaluation stages.

  5. 5.

    Often referred to as subjective, a somewhat misleading term avoided here.

  6. 6.

    Often referred to as objective, erroneously implying that instrumental measurements bear objectivity, which they only do in case that they can be generalized.

  7. 7.

    Note that this nomenclature is misleading in at least two ways. First, the ACR scale ideally should be interpreted as an ordinal and not as an interval scale. This means that calculating averages may be inappropriate. Second, any average of ratings may be called “MOS”, that is, not only using the 5-point ACR scale.

  8. 8.

    Note that findings that result in proportionality of relevant perceptual factors depend heavily on the specific test conditions used. Compare Zieliński et al. (2008) for a discussion of biases in listening tests.

  9. 9.

    It may be argued that the specific down-mix may have affected the liking already, depending on the piece and its original recording. It is difficult, though, to address this topic in a different way.

  10. 10.

    www.twoears.eu [last accessed, August 30, 2019].

  11. 11.

    Incorporating a head-and-torso-simulator (Kemar) with a motorized neck to enable horizontal-plane panning, mounted on a carriage for lateral motion. See http://docs.twoears.eu/en/latest/ [last accessed February 22, 2020].

  12. 12.

    See http://docs.twoears.eu/en/1.5/afe/ [last accessed: February 22, 2020].

  13. 13.

    Crowd-sourcing tests involving dedicated crowd-workers are distinguished from data collection in the field with a more arbitrary and hence real-life sample of users, and with a less guided, more natural usage behavior.

  14. 14.

    https://github.com/stephencwelch/Perceptual-Coding-In-Python [last accessed: August 30, 2019].

  15. 15.

    http://docs.twoears.eu/en/1.5/afe/ [last accessed: August 30, 2019].

  16. 16.

    Søndergaard et al. (2011) and Søndergaard and Majdak (2013), http://amtoolbox.sourceforge.net/ [last accessed: August 31, 2019].

References

  • Amelang, M., D.G.S. Bartussek, and D. Hagemann. 2006. Differentielle Psychologie und Persönlichkeitsforschung (Differential Psychology and Personality Research). Stuttgart: W. Kohlhammer Verlag.

    Google Scholar 

  • Baddeley, A. 1997. Human Memory—Theory and Practice. East Sussex, UK: Taylor & Francis, Psychology Press.

    Google Scholar 

  • Baddeley, A. 2003. Working memory: Looking back and looking forward. Nature Reviews Neuroscience 4: 829–839. https://doi.org/10.1038/nrn1201.

  • Bech, S., and N. Zacharov. 2006. Perceptual Audio Evaluation. Chichester, UK: Wiley.

    Google Scholar 

  • Beerends, J.G., C. Schmidmer, J. Berger, M. Obermann, R. Ullmann, J. Pomy, and M. Keyhl. 2013. Perceptual Objective Listening Quality Assessment (POLQA), The third generation ITU-T standard for end-to-end speech quality measurement. Part II—Perceptual model. Journal of the Audio Engineering Society 61 (6): 385–402. http://www.aes.org/e-lib/browse.cfm?elib=16829. Accessed 9 Oct 2019.

  • Benoit, A., P. LeCallet, P. Campisi, and R. Cousseau. 2008. Quality assessment of stereoscopic images. In IEEE International Conference Image Processing (ICIP) 1231–1234.

    Google Scholar 

  • Bentham, J. 1789. An Introduction to the Principle of Morals and Legislations. Oxford, UK: Blackwell (Reprint 1948).

    Google Scholar 

  • Biberger, T., and S.D. Ewert. 2016. Envelope and intensity based prediction of psychoacoustic masking and speech intelligibility. The Journal of the Acoustical Society of America 140 (2): 1023–1038. https://doi.org/10.1121/1.4960574.

  • Biberger, T., J.-H. Fleßner, R. Huber, and S.D. Ewert. 2018. An objective audio quality measure based on power and envelope power cues. Journal of the Audio Engineering Society 66 (7/8), 578–593. http://www.aes.org/e-lib/browse.cfm?elib=19707. Accessed 23 Sept 2019.

  • Blauert, J. 1997. Spatial Hearing: The Psychophysics of Human Sound Localization. Cambridge, MA, USA: The MIT Press.

    Google Scholar 

  • Blauert, J. 2013. Conceptual aspects regarding the qualification of spaces for aural performances. Acta Acustica united with Acustica 99: 1–13. https://doi.org/10.3813/AAA.918582.

  • Blauert, J. 2020. A virtual testbed for binaural agents. In The Technology of Binaural Understanding, eds. J. Blauert, and J. Braasch, 491–510. Cham, Switzerland: Springer and ASA Press.

    Google Scholar 

  • Blauert, J., J. Braasch, J. Buchholz, H.S. Colburn, U. Jekosch, A. Kohlrausch, J. Mourjopoulos, V. Pulkki, and A. Raake. 2009. Aural assessment by means of binaural algorithms – the AabbA project. In Proceedings of the 2nd International Symposium Auditory and Audiological Research–ISAAR’09, 113–124.

    Google Scholar 

  • Blauert, J., and G. Brown. 2020. Reflexive and reflective auditory feedback. In The Technology of Binaural Understanding, eds. J. Blauert, and J. Braasch, 3–31, Cham, Switzerland: Springer and ASA Press. This volume.

    Google Scholar 

  • Blauert, J., and U. Jekosch. 2012. A layer model of sound quality. Journal of the Audio Engineering Society 60 (1/2): 4–12. http://www.aes.org/e-lib/browse.cfm?elib=16160. Accessed 19 Sept 2019.

  • Blauert, J., D. Kolossa, K. Obermayer, and K. Adiloglu. 2013. Further challenges—and the road ahead. In The Technology of Binaural Listening, ed. J. Blauert. Berlin: Springer and ASA Press. https://doi.org/10.1007/978-3-642-37762-4_18.

  • Bowman, D.A., and R.P. McMahan. 2007. Virtual reality: How much immersion is enough? Computer 40 (7): 36–43.

    Google Scholar 

  • Braasch, J. 2020. Binaural modeling from an evolving habitat perspective. In The Technology of Binaural Understanding, eds. J. Blauert, and J. Braasch, 251–286, Cham, Switzerland: Springer and ASA Press.

    Google Scholar 

  • Bradley, J.S., and G.A. Soulodre. 1995. Objective measures of listener envelopment. Journal of the Acoustical Society of America 98 (5): 2590–2597.

    Article  ADS  Google Scholar 

  • Brandenburg, K., F. Klein, A. Neidhardt, U. Sloma, and S. Werner. 2020. Creating auditory illusions with binaural technology. In The Technology of Binaural Understanding, eds. J. Blauert, and J. Braasch, 623–663, Cham, Switzerland: Springer and ASA Press.

    Google Scholar 

  • Bregman, A.S. 1990. Auditory Scene Analysis. Cambridge, USA: The MIT Press.

    Book  Google Scholar 

  • Brown, G., R. Decorsière, D. Kolossa, N. Ma, T. May, C. Schymura, and I. Trowitzsch. 2014. D3.1: TWO!EARS Software Architecture, Two!Ears FET-Open Project. https://doi.org/10.5281/zenodo.2595254.

  • Brüggen, M. 2001a. Coloration and binaural decoloration in natural environments. Acta Acustica united with Acustica 87: 400–406.

    Google Scholar 

  • Brüggen, M. 2001b. Sound coloration due to reflections and its auditory and instrumental compensation. PhD thesis, Ruhr-Universität Bochum.

    Google Scholar 

  • Carroll, J.D. 1972. Individual preferences and multidimensional scaling. In Multidimensional Scaling: Theory and Applications in the Behavioral Sciences, vol. I, ed. R.N. Shepard, A.K. Romney, and S.B. Nerlove, 105–155.

    Google Scholar 

  • Choisel, S., and F. Wickelmaier. 2007. Evaluation of multichannel reproduced sound: Scaling auditory attributes underlying listener preference. The Journal of the Acoustical Society of America 121 (1): 388–400. https://doi.org/10.1121/1.2385043.

    Article  ADS  Google Scholar 

  • Cohen l’Hyver, B. 2017. Modulation de mouvements de tête pour l’analyse multimodale d’un environnement inconnu (modulation of head movements for the multimodal analysis of an unknown environment). PhD thesis, Université Pierre et Marie Curie, Ecole Doctorale SMAER, Sciences Mécaniques, Acoustique, Electronique et Robotique de Paris, France.

    Google Scholar 

  • Cohen-L’Hyver, B., S. Argentieri, and B. Gas. 2020. Audition as a trigger of head movements. In The Technology of Binaural Understanding, eds. J. Blauert, and J. Braasch, 697–731, Cham, Switzerland: Springer and ASA Press.

    Google Scholar 

  • Coltheart, M. 1980. Iconic memory and visible persistence. Perception & Psychophysics 27 (3): 183–228. https://doi.org/10.3758/BF03204258.

    Article  Google Scholar 

  • Cowan, N. 1984. On short and long auditory stores. Psychol. Bulletin 96 (2): 341–370. https://doi.org/10.1037/0033-2909.96.2.341.

    Article  Google Scholar 

  • Dobrian, F., A. Awan, D. Joseph, A. Ganjam, J. Zhan, V. Sekar, I. Stioca, and H. Zhang. 2013. Understanding the impact of video quality on user engagement. Communications of the ACM 56 (3): 91–99. https://doi.org/10.1145/2043164.2018478.

    Article  Google Scholar 

  • Emiya, V., E. Vincent, N. Harlander, and V. Hohmann. 2011. Subjective and objective quality assessment of audio source separation. IEEE Transactions on Audio, Speech, and Language Processing 19 (7): 2046–2057. https://doi.org/10.1109/TASL.2011.2109381.

    Article  Google Scholar 

  • Engelke, U., D.P. Darcy, G.H. Mulliken, S. Bosse, M.G. Martini, S. Arndt, J.-N. Antons, K.Y. Chan, N. Ramzan, and K. Brunnström. 2017. Psychophysiology-based qoe assessment: A survey. IEEE Journal of Selected Topics in Signal Processing 11 (1): 6–21. https://doi.org/10.1109/JSTSP.2016.2609843.

    Article  ADS  Google Scholar 

  • Francombe, J., T. Brookes, and R. Mason. 2017a. Evaluation of spatial audio reproduction methods (part 1): Elicitation of perceptual differences. Journal of the Audio Engineering Society 65 (3): 198–211. https://doi.org/10.17743/jaes.2016.0070.

  • Francombe, J., T. Brookes, R. Mason, and J. Woodcock. 2017b. Evaluation of spatial audio reproduction methods (part 2): Analysis of listener preference. Journal of the Audio Engineering Society 65 (3): 212–225. https://doi.org/10.17743/jaes.2016.0071.

  • Garcia, M.-N., R. Schleicher, and A. Raake. 2011. Impairment-factor-based audiovisual quality model for iptv: Influence of video resolution, degradation type, and content type. EURASIP Journal on Image and Video Processing 2011 (1): 1–14. https://doi.org/10.1155/2011/629284.

    Article  Google Scholar 

  • Geerts, D., K.D. Moor, I. Ketyko, A. Jacobs, J.V. den Bergh, W. Joseph, L. Martens, and L.D. Marez. 2010. Linking an integrated framework with appropriate methods for measuring QoE. In Proceedings of the International Workshop on Quality of Multimedia Experience (QoMEX). https://doi.org/10.1109/QOMEX.2010.5516292.

  • Göring, S., J. Skowronek, and A. Raake. 2018. DeViQ - A deep no reference video quality model. In Proceedings Human Vision and Electronic Imaging (HVEI) 1–6: https://doi.org/10.2352/ISSN.2470-1173.2018.14.HVEI-518.

  • Griesinger, D. 1998. General overview of spatial impression, envelopment, localization, and externalization. In Audio Engineering Society Conference: 15th International Conference: Audio, Acoustics & Small Spaces, Audio Engineering Society. http://www.aes.org/e-lib/browse.cfm?elib=8095. Accessed 17 Sept 2019.

  • Harlander, N., R. Huber, and S.D. Ewert. 2014. Sound quality assessment using auditory models. Journal of the Audio Engineering Society 62 (5): 324–336. https://doi.org/10.17743/jaes.2014.0020.

  • Härmä, A., M. Park, and A. Kohlrausch. 2014. Data-driven modeling of the spatial sound experience. In Audio Engineering Society Convention 136. http://www.aes.org/e-lib/browse.cfm?elib=17172. Accessed 18 Sept 2019.

  • Hartmann, W.M., and A. Wittenberg. 1996. On the externalization of sound images. Journal of the Acoustical Society of America 99 (6): 3678–3688.

    Article  ADS  Google Scholar 

  • Hassenzahl, M. 2001. The effect of perceived hedonic quality on product appealingness. International Journal of Human-Computer Interaction 13 (4): 481–499. https://doi.org/10.1207/S15327590IJHC1304_07.

    Article  Google Scholar 

  • Hossfeld, T., C. Keimel, M. Hirth, B. Gardlo, J. Habigt, K. Diepold, and P. Tran-Gia. 2014. Best practices for QoE crowdtesting: QoE assessment with crowdsourcing. IEEE Transactions on Multimedia 16 (2): 541–558. https://doi.org/10.1109/TMM.2013.2291663.

    Article  Google Scholar 

  • Houtgast, T., and H.J.M. Steeneken. 1985. A review of the mtf concept in room acoustics and its use for estimating speech intelligibility in auditoria. The Journal of the Acoustical Society of America 77 (3): 1069–1077. https://doi.org/10.1121/1.392224.

    Article  ADS  Google Scholar 

  • ISO 9000:2000. 2000. Quality Management Systems: Fundamentals and Vocabular, International Organization for Standardization.

    Google Scholar 

  • ITU–R BS. 1116-1. 1997. Methods for the Subjective Assessment of Small Impairments in Audio Systems Including Multichannel Sound Systems. Geneva, CH: International Telecommunication Union.

    Google Scholar 

  • ITU–R BS. 1283-1. 2003. A Guide to ITU-R Recommendations for Subjective Assessment of Sound Quality. Geneva, CH: International Telecommunication Union.

    Google Scholar 

  • ITU–R BS. 1534-3. 2015. Method for the Subjective Assessment of Intermediate Quality Level of Coding Systems. Geneva, CH: International Telecommunication Union.

    Google Scholar 

  • ITU–T Rec. P.10/G.100. 2017. Vocabulary for Performance and Quality of Service. Geneva, CH: International Telecommunication Union.

    Google Scholar 

  • ITU–T Rec. P.800. 1996. Methods for Subjective Determination of Transmission Quality. Geneva, CH: International Telecommunication Union.

    Google Scholar 

  • ITU–T Rec. P.862. 2001. Perceptual Evaluation of Speech Quality (PESQ), International Telecommunication Union.

    Google Scholar 

  • ITU–T Rec. P.863. 2011. Perceptual Objective Listening Quality Assessment (POLQA), International Telecommunication Union.

    Google Scholar 

  • Jekosch, U. 2004. Basic concepts and terms of “quality”, reconsidered in the context of product sound quality. Acta Acustica united with Acustica 90 (6): 999–1006.

    Google Scholar 

  • Jekosch, U. 2005a. Assigning meaning to sounds: Semiotics in the context of product-sound design. In Communication Acoustics, ed. J. Blauert. Berlin: Springer. https://doi.org/10.1007/3-540-27437-5_8.

  • Jekosch, U. 2005b. Voice and Speech Quality Perception—Assessment and Evaluation. D-Berlin: Springer.

    Google Scholar 

  • Kahneman, D. 1999. Objective happiness. In Well-Being: The Foundations of Hedonic Psychology, ed. D. Kahneman, E. Diener, and N. Schwarz, 3–25. New York: Russell Sage Foundation.

    Google Scholar 

  • Kahneman, D. 2003. Experienced utility and objective happiness: A moment-based approach. In The Psychology of Economic Decisions, ed. I. Brocas, and J.D. Carrillo, 187–208. Oxford: Oxford University Press.

    Google Scholar 

  • Kahneman, D. 2011. Thinking, Fast and Slow. New York, NY: Farrar, Straus and Giroux.

    Google Scholar 

  • Kim, C., R. Mason, and T. Brookes. 2013. Head movements made by listeners in experimental and real-life listening activities. Journal of the Audio Engineering Society 61 (6): 425–438. http://www.aes.org/e-lib/browse.cfm?elib=16833. Accessed 18 Sept 2019.

  • Kuttruff, H. 2016. Room Acoustics. Boca Raton: CRC Press.

    Book  Google Scholar 

  • Lawless, H.T., and H. Heymann. 2010. Sensory Evaluation of Food: Principles and Practices, vol. 5999. Berlin: Springer.

    Book  Google Scholar 

  • Lebreton, P., A. Raake, M. Barkowsky, and P.L. Callet. 2013. Perceptual preference of S3D over 2D for HDTV in dependence of video quality and depth. In IVMSP Workshop: 3D Image/Video Technologies and Applications, 10–12 June, 1–4. Korea, Seoul.

    Google Scholar 

  • Lepa, S., E. Ungeheuer, H.-J. Maempel, and S. Weinzierl. 2013. When the medium is the message: An experimental exploration of medium effects on the emotional expressivity of music dating from different forms of spatialization. In Proceedings of the 8th Conference of the Media Psychology Division of Deutsche Gesellschaft für Psychologie (DGPs).

    Google Scholar 

  • Lepa, S., S. Weinzierl, H.-J. Maempel, and E. Ungeheuer. 2014. Emotional impact of different forms of spatialization in everyday mediatized music listening: Placebo or technology effects? In Audio Engineering Society Convention 136, Audio Engineering Society. http://www.aes.org/e-lib/browse.cfm?elib=17171. Accessed 18 Sept 2019.

  • Letowski, T. 1989. Sound quality assessment: Concepts and criteria. In Audio Engineering Society Convention 87, 18–21 Oct, New York, USA. http://www.aes.org/e-lib/browse.cfm?elib=5869. Accessed 18 Sept 2019.

  • Li, J., M. Barkowsky, and P. LeCallet. 2012. Analysis and improvement of a paired comparison method in the application of 3DTV subjective experiment. In IEEE International Conference Image Processing (ICIP), 30 Sept–03 Oct, Orlando, Florida, USA.

    Google Scholar 

  • Liebetrau, J., T. Sporer, S. Kämpf, and S. Schneider. 2010. Standardization of PEAQ-MC: Extension of ITU-R BS.1387-1 to multichannel audio. In Audio Engineering Society, 40th International Conference: Spatial Audio, 8–10 Oct, Tokyo, Japan. http://www.aes.org/e-lib/browse.cfm?elib=15571. Accessed 23 Sept 2019.

  • Lindau, A., V. Erbes, S. Lepa, H.-J. Maempel, F. Brinkman, and S. Weinzierl. 2014. A spatial audio quality inventory (SAQI). Acta Acustica united with Acustica 100 (5): 984–994. https://doi.org/10.3813/AAA.918778.

    Article  Google Scholar 

  • Lokki, T., J. Pätynen, A. Kuusinen, H. Vertanen, and S. Tervo. 2011. Concert hall acoustics assessment with individually elicited attributes. The Journal of the Acoustical Society of America 130 (2): 835–849. https://doi.org/10.1121/1.3607422.

    Article  ADS  Google Scholar 

  • Martens, H., and M. Martens. 2001. Multivariate Analysis of Quality. Chichester: Wiley.

    MATH  Google Scholar 

  • Massaro, D.W. 1975. Backward recognition masking. The Journal of the Acoustical Society of America 58 (5): 1059–1065. https://doi.org/10.1121/1.380765.

    Article  ADS  Google Scholar 

  • Mattila, V. 2001. Perceptual Analysis of Speech Quality in Mobile Communications, vol. 340. Doctoral Dissertation, Tampere University of Technology, FIN–Tampere.

    Google Scholar 

  • Mattila, V. 2002. Ideal point modelling of speech quality in mobile communications based on multidimensional scaling. Audio Engineering Society Convention, vol. 112. http://www.aes.org/e-lib/browse.cfm?elib=11433. Accessed 23 Sept 2019.

  • Mausfeld, R. 2003. Conjoint representations and the mental capacity for multiple simultaneous perspectives. In Looking into Pictures: An Interdisciplinary Approach to Pictorial Space, ed. H. Hecht, R. Schwartz, and M. Atherton, 17–60. Cambridge: MIT Press.

    Google Scholar 

  • Moor, K.D. 2012. Are engineers from mars and users from venus? Bridging the gaps in quality of experience research: Reflections on and experiences from an interdisciplinary journey. PhD thesis, Universiteit Gent.

    Google Scholar 

  • Moore, B.C., and B.R. Glasberg. 2007. Modeling binaural loudness. The Journal of the Acoustical Society of America 121 (3): 1604–1612. https://doi.org/10.1121/1.2431331.

    Article  ADS  Google Scholar 

  • Moore, B.C.J., and C.-T. Tan. 2004. Development and validation of a method for predicting the perceived naturalness of sounds subjected to spectral distortion. Journal of the Audio Engineering Society 52 (9): 900–914. http://www.aes.org/e-lib/browse.cfm?elib=13018. Accessed 23 Sept 2019.

  • Mourjopoulos, J. 2020. Aesthetics aspects regarding recorded binaural sounds. In The Technology of Binaural Understanding, eds. J. Blauert, and J. Braasch, 455–490, Cham, Switzerland: Springer and ASA Press.

    Google Scholar 

  • Neisser, U. 1978. Perceiving, anticipating and imagining. Minnesota Studies in the Philosophy of Science 9: 89–106.

    Google Scholar 

  • Neisser, U. 1994. Multiple systems: A new approach to cognitive theory. European Journal of Cognitive Psychology 6 (3): 225–241. https://doi.org/10.1080/09541449408520146.

    Article  Google Scholar 

  • Oldoni, D., B. De Coensel, M. Boes, M. Rademaker, B. De Baets, T. Van Renterghem, and D. Botteldooren. 2013. A computational model of auditory attention for use in soundscape research. The Journal of the Acoustical Society of America 134 (1): 852–861. https://doi.org/10.1121/1.4807798.

    Article  ADS  Google Scholar 

  • Olko, M., D. Dembeck, Y.-H. Wu, A. Genovese, and A. Roginska. 2017. Identification of perceived sound quality attributes of 360-degree audiovisual recordings in VR – Using a free verbalization method. In Audio Engineering Society Convention 143, 18–21 Oct, New York, USA. Audio Engineering Society. http://www.aes.org/e-lib/browse.cfm?elib=19227. Accessed 23 Sept 2019.

  • Parasuraman, A., V. Zeithaml, and L. Berry. 1985. A conceptual model of service quality and its implications for future research. Journal of Marketing 49 (Fall 1985): 41–50. https://doi.org/10.2307/1251430.

  • Piaget, J. 1962. The Child’s Conception of the World (La représentation du monde chez l’enfant). London: Routledge & Kegan. Translated from the 1926 original.

    Google Scholar 

  • Pike, C., and H. Stenzel. 2017. Direct and indirect listening test methods – A discussion based on audio-visual spatial coherence experiments. In Audio Engineering Society Convention 143, Audio Engineering Society. http://www.aes.org/e-lib/browse.cfm?elib=19226. Accessed 23 Sept 2019.

  • Qualinet. 2012. White Paper on Definitions of Quality of Experience, COST Action IC 1003, ed. Möller, S., P. Le Callet, and A. Perkis, Lausanne, CH

    Google Scholar 

  • Raake, A. 2006. Speech Quality of VoIP–Assessment and Prediction. Chichester, West Sussex, UK: Wiley.

    Google Scholar 

  • Raake, A. 2016. Views on sound quality. In Proceedings 22nd International Congress on Acoustics (ICA), 5–9 Sept, 1–10, Buenos Aires, Argentina.

    Google Scholar 

  • Raake, A., and J. Blauert. 2013. Comprehensive modeling of the formation process of sound-quality. In Proceedings of the IEEE International Conference Quality of Multimedia Experience (QoMEX), 3–5 July, Klagenfurt, Austria. https://doi.org/10.1109/QoMEX.2013.6603214.

  • Raake, A., J. Blauert, J. Braasch, G. Brown, P. Danes, T. Dau, B. Gas, S. Argentieri, A. Kohlrausch, D. Kolossa, N. Le Goeff, T. May, K. Obermayer, C. Schymura, T. Walther, H. Wierstorf, F. Winter, and S. Spors. 2014a. Two!ears – Integral interactive model of auditory perception and experience. In 40th German Annual Conference on Acoustics (DAGA), 10–13 March, Oldenburg, Germany.

    Google Scholar 

  • Raake, A., H. Wierstorf, and J. Blauert. 2014b. A case for Two!Ears in audio quality assessment. Forum Acusticum, 7–12 Sept., Krakow, Poland.

    Google Scholar 

  • Raake, A., and S. Egger. 2014. Quality and quality of experience. In Quality of Experience. Advanced Concepts, Applications and Methods, ed. S. Möller, and A. Raake. Berlin: Springer. Chap. 2. https://doi.org/10.1007/978-3-319-02681-7_2.

  • Raake, A., C. Schlegel, K. Hoeldtke, M. Geier, and J. Ahrens. 2010. Listening and conversational quality of spatial audio conferencing. In 40th International Conference on Spatial Audio: Sense the Sound of Space, Audio Engineering Society. http://www.aes.org/e-lib/browse.cfm?elib=15567. Accessed 23 Sept 2019.

  • Raake, A., and H. Wierstorf. 2016. Assessment of audio quality and experience using binaural-hearing models. In Proceedings 22nd International Congress on Acoustics (ICA), 5–9 Sept., 1–10. Buenos Aires, Argentina.

    Google Scholar 

  • Reardon, G., A. Genovese, G. Zalles, P. Flanagan, and A. Roginska. 2018. Evaluation of binaural renderers: Multidimensional sound quality assessment. In 2018 International Conference on Audio for Virtual and Augmented Reality, Audio Engineering Society. http://www.aes.org/e-lib/browse.cfm?elib=19694. Accessed 23 Sept 2019.

  • Rees-Jones, J., and D.T. Murphy. 2018. The impact of multichannel game audio on the quality and enjoyment of player experience. In Emotion in Video Game Soundtracking, 143–163. Berlin: Springer. https://doi.org/10.1007/978-3-319-72272-6_11.

  • Reeves, C.A., and D.A. Bednar. 1994. Defining quality: Alternatives and implications. Academy of Management Review 19 (3): 419–445. https://doi.org/10.2307/258934.

  • Reiter, U., K. Brunnström, K. De Moor, M.-C. Larabi, M. Pereira, A. Pinheiro, J. You, and A. Zgank. 2014. Factors influencing quality of experience. In Quality of Experience. Advanced Concepts, Applications and Methods, ed. S. Möller, and A. Raake. Berlin: Springer. Chap. 4. https://doi.org/10.1007/978-3-319-02681-7_4.

  • Richards, D.L. 1973. Telecommunication by Speech. London, UK: Butterworths.

    Google Scholar 

  • Richards, D.L. 1973. Telecommunication by Speech. London, UK: Butterworths.

    Google Scholar 

  • Rummukainen, O., T. Robotham, S.J. Schlecht, A. Plinge, J. Herre, and E.A. Habels. 2018. Audio quality evaluation in virtual reality: Multiple stimulus ranking with behavior tracking. In 2018 AES International Conference on Audio for Virtual and Augmented Reality, Audio Engineering Society. http://www.aes.org/e-lib/browse.cfm?elib=19678. Accessed 23 Sept 2019.

  • Rummukainen, O., S. Schlecht, A. Plinge, and E.A. Habets. 2017. Evaluation of binaural reproduction systems from behavioral patterns in a six-degrees-of-freedom wayfinding task. In 2017 Ninth International Conference on Quality of Multimedia Experience (QoMEX), IEEE, 1–3. https://doi.org/10.1109/QoMEX.2017.7965680.

  • Rumsey, F. 2002. Spatial quality evaluation for reproduced sound: Terminology, meaning, and a scene-based paradigm. Journal of the Audio Engineering Society 50 (9): 651–666. http://www.aes.org/e-lib/browse.cfm?elib=11067. Accessed 23 Sept 2019.

  • Rumsey, F., S. Zieliński, P. Jackson, M. Dewhirst, R. Conetta, S. George, S Bech, and D. Meares. 2008. QESTRAL (part 1): Quality evaluation of spatial transmission and reproduction using an artificial listener. In Audio Engineering Society Convention 125, 3–5 Oct, San Francisco, USA. http://www.aes.org/e-lib/browse.cfm?elib=14746. Accessed 23 Sept 2019.

  • Rumsey, F., S. Zieliński, R. Kassier, and S. Bech. 2005. On the relative importance of spatial and timbral fidelities in judgements of degraded multichannel audio quality. Journal of the Acoustical Society of America 118 (2): 968–976. https://doi.org/10.1121/1.1945368.

  • Schoeffler, M., and J. Herre. 2013. About the impact of audio quality on overall listening experience. In Proceedings of the Sound and Music Computing Conference (SMC), 30 July–3 Aug., Stockholm, Sweden, 53–58.

    Google Scholar 

  • Schoeffler, M., and J. Herre. 2016. The relationship between basic audio quality and overall listening experience. Journal of the Acoustical Society of America 140 (3): 2101–2112. https://doi.org/10.1121/1.4963078.

  • Schoeffler, M., A. Silzle, and J. Herre. 2017. Evaluation of spatial/3d audio: Basic audio quality versus quality of experience. IEEE Journal of Selected Topics in Signal Processing 11 (1): 75–88. https://doi.org/10.1109/JSTSP.2016.2639325.

  • Schoenenberg, K. 2016. The quality of mediated-conversations under transmission delay. PhD thesis, Technische Universität Berlin. https://doi.org/10.14279/depositonce-4990.

  • Schoenenberg, K., A. Raake, and J. Koeppe. 2014. Why are you so slow?—misattribution of transmission delay to attributes of the conversation partner at the far-end. International Journal of Human-Computer Studies 72 (5): 477–487. https://doi.org/10.1016/j.ijhcs.2014.02.004.

  • Schymura, C., and D. Kolossa. 2020. Blackboard systems for cognitive audition. In The Technology of Binaural Understanding, eds. J. Blauert, and J Braasch, 91–111, Cham, Switzerland: Springer and ASA Press. Chap. 4.

    Google Scholar 

  • Seo, J.-H., S.B. Chon, K.-M. Sung, and I. Choi. 2013. Perceptual objective quality evaluation method for high- quality multichannel audio codecs. Journal of the Audio Engineering Society 61 (7/8): 535–545. http://www.aes.org/e-lib/browse.cfm?elib=16869. Accessed 23 Sept 2019.

  • Singla, A., S. Fremerey, W. Robitza, and A. Raake. 2017. Measuring and comparing qoe and simulator sickness of omnidirectional videos in different head mounted displays. In Proceedings of the International Conference on Quality of Multimedia Experience (QoMEX), Erfurt, Germany. IEEE, 1–6. https://doi.org/10.1109/QoMEX.2017.7965658.

  • Skowronek, J., L. Nagel, C. Hold, H. Wierstorf, and A. Raake. 2017. Towards the development of preference models accounting for the impact of music production techniques. In 43rd German Annual Conference on Acoustics (DAGA), 856–860.

    Google Scholar 

  • Skowronek, J., and A. Raake. 2015. Assessment of cognitive load, speech communication quality and quality of experience for spatial and non-spatial audio conferencing calls. Speech Communication 66: 154–175. https://doi.org/10.1016/j.specom.2014.10.003.

  • Søndergaard, P., J. Culling, T. Dau, N. Le Goff, M. Jepsen, P. Majdak, and H. Wierstorf. 2011. Towards a binaural modelling toolbox. In Proceedings of the Forum Acusticum, European Acoustics Association (EAA), 27 June–01 July, Aalborg, Denmark, 2081–2086.

    Google Scholar 

  • Søndergaard, P., and P. Majdak. 2013. The auditory-modeling toolbox. In The Technology of Binaural Listening, ed. J. Blauert. Berlin: Springer and ASA Press. Chap. 2. https://doi.org/10.1007/978-3-642-37762-4_2.

  • Sotujo, S., J. Thiemann, A. Kohlrausch, and S. Van de Paar. 2020. Auditory gestalt rules and their application. In The Technology of Binaural Understanding, eds. J. Blauert, and J. Braasch, 33–59, Cham, Switzerland: Springer and ASA Press.

    Google Scholar 

  • Spille, C., S.D. Ewert, B. Kollmeier, and B. Meyer. 2018. Predicting speech intelligibility with deep neural networks. Computer Speech & Language 48: 51–66. https://doi.org/10.1016/j.csl.2017.10.004.

  • Spors, S., M. Geier, and H. Wierstorf. 2017. Towards open science in acoustics: Foundations and best practices. In Proceedings of the 43. Jahrestagung f. Akustik (43th Annual Meeting German Society Acoustics, DAGA), 6–9 March, Kiel, Germany, 218–221.

    Google Scholar 

  • Spors, S., H. Wierstorf, A. Raake, F. Melchior, M. Frank, and F. Zotter. 2013. Spatial sound with loudspeakers and its perception: A review of the current state. Proceedings of the IEEE 101 (9): 1920–1938. https://doi.org/10.1109/JPROC.2013.2264784.

  • Strohmeier, D., S. Jumisko-Pyykkö, and K. Kunze. 2010. Open profiling of quality: A mixed method approach to understanding multimodal quality perception. Advances in Multimedia 2010 (Article ID 658980): 28. https://doi.org/10.1155/2010/658980.

  • Thiede, T., W. Treurniet, R. Bitto, C. Schmidmer, T. Sporer, J. Beerends, C. Colomes, M. Keyhl, G. Stoll, K. Brandenburg, and B. Feiten. 2000. PEAQ—the ITU standard for objective measurement of perceived audio quality. Journal of the Audio Engineering Society 48: 3–29. http://www.aes.org/e-lib/browse.cfm?elib=12078. Accessed 23 Sept. 2019.

  • Uhrig, S., S. Arndt, S. Möller, and J. Voigt-Antons. 2017. Perceptual references for independent dimensions of speech quality as measured by electro-encephalography. Quality and User Experience 2 (1): 1–10. https://doi.org/10.1007/s41233-017-0011-8.

  • Uhrig, S., G. Mittag, S. Möller, and J.-N. Voigt-Antons. 2018. Neural correlates of speech quality dimensions analyzed using electroencephalography (EEG). Journal of Neural Engineering.

    Google Scholar 

  • van Ee, R., J.J.A. van Boxtel, A.L. Parker, and D. Alais. 2009. Multisensory congruency as a mechanism for attentional control over perceptual selection. Journal of Neuroscience 29 (37): 11641–11649. https://doi.org/10.1523/JNEUROSCI.0873-09.2009.

  • Wältermann, M. 2005. Bestimmung relevanter Qualitätsdimensionen bei der Sprachübertragung in modernen Telekommunikationsnetzen. Diploma thesis (unpublished), Institut für Kommunikationsakustik, Ruhr-Universität, D-Bochum.

    Google Scholar 

  • Wältermann, M. 2013. Dimension-Based Quality Modeling of Transmitted Speech. Berlin: Springer Science & Business Media.

    Google Scholar 

  • Wältermann, M., A. Raake, and S. Möller. 2010. Quality dimensions of narrowband and wideband speech transmission. Acta Acustica united with Acustica 96 (6): 1090–1103. https://doi.org/10.3813/AAA.918370.

  • Weiss, B., D. Guse, S. Möller, A. Raake, A. Borowiak, and U. Reiter. 2014. Temporal development of quality of experience. In Quality of Experience. Advanced Concepts, Applications and Methods, ed. S. Möller, and A. Raake, 133–147. Berlin: Springer. Chap. 10. https://doi.org/10.1007/978-3-319-02681-7_10.

  • Werner, S., F. Klein, T. Mayenfels, and K. Brandenburg. 2016. A summary on acoustic room divergence and its effect on externalization of auditory events. In 2016 Eighth International Conference on Quality of Multimedia Experience (QoMEX), IEEE, 1–6. https://doi.org/10.1109/QoMEX.2016.7498973.

  • Wickelmaier, F., N. Umbach, K. Sering, and S. Choisel. 2009. Comparing three methods for sound quality evaluation with respect to speed and accuracy. In Audio Engineering Society Convention 126, Audio Engineering Society.

    Google Scholar 

  • Wierstorf, H. 2014. Perceptual assessment of sound field synthesis. PhD thesis, TU Berlin. https://doi.org/10.14279/depositonce-4310.

  • Wierstorf, H., M. Geier, A. Raake, and S. Spors. 2013. Perception of focused sources in wave field synthesis. Journal of the Audio Engineering Society 61 (1/2): 5–16. http://www.aes.org/e-lib/browse.cfm?elib=16663. Accessed 23 Sept. 2019.

  • Wierstorf, H., C. Hohnerlein, S. Spors, and A. Raake. 2014. Coloration in wave field synthesis. In AES 55th International Conference: Spatial Audio, 27–29 August, Helsinki, Finland, Audio Engineering Society, 1–8. http://www.aes.org/e-lib/browse.cfm?elib=17381. Accessed 23 Sept. 2019.

  • Wierstorf, H., C. Hold, and A. Raake. 2018. Listener preference for wave field synthesis, stereophony, and different mixes in popular music. Journal of the Audio Engineering Society 66 (5): 385–396. https://doi.org/10.17743/jaes.2018.0019.

  • Wierstorf, H., A. Raake, and S. Spors. 2017a. Assessing localization accuracy in sound field synthesis. Journal of the Acoustical Society of America. 141 (2): 1111–1119. https://doi.org/10.1121/1.4976061.

    Article  ADS  Google Scholar 

  • Wierstorf, H., F. Winter, and S. Spors. 2017b. Open science in the Two!Ears project - Experiences and best practices. In 173rd Meeting of the Acoustical Society of America and the 8th Forum Acusticum. Boston, MA: Acoustical Society of America.

    Google Scholar 

  • Wilson, A., and B. Fazenda. 2016. Relationship between hedonic preference and audio quality in tests of music production quality. In Proceedings of the IEEE 8th International Conference Quality of Multimedia Experience (QoMEX), 1–6. https://doi.org/10.1109/QoMEX.2016.7498937.

  • Winter, F., H. Wierstorf, A. Raake, and S. Spors. 2017. The two!ears database. In 142nd Convention of the Audio Engineering Society, Berlin, Germany, eBrief 330. http://www.aes.org/e-lib/browse.cfm?elib=18705. Accessed 23 Sept. 2019.

  • Woodcock, J., J. Francombe, R. Hughes, R. Mason, W.J. Davies, and T.J. Cox. 2018. A quantitative evaluation of media device orchestration for immersive spatial audio reproduction. In 2018 AES International Conference on Spatial Reproduction - Aesthetics and Science, Audio Engineering Society. http://www.aes.org/e-lib/browse.cfm?elib=19606. Accessed 23 Sept. 2019.

  • Zacharov, N. (ed.). 2019. Sensory Evaluation of Sound. Boca Raton, FL: CRC Press.

    Google Scholar 

  • Zacharov, N., T. Pedersen, C. Pike. 2016a. A common lexicon for spatial sound quality assessment-latest developments. In 2016 Eighth International Conference on Quality of Multimedia Experience (QoMEX), IEEE, 1–6. https://doi.org/10.1109/QoMEX.2016.7498967.

  • Zacharov, N., C. Pike, F. Melchior, and T. Worch. 2016b. Next generation audio system assessement using the multiple stimulus ideal profile method. In Proceedings of the IEEE QoMEX 2016, IEEE, 1–6. https://doi.org/10.1109/QoMEX.2016.7498966.

  • Zahorik, P., D.S. Brungart, and A.W. Bronkhorst. 2005. Auditory distance perception in humans: A summary of past and present research. ACTA Acustica united with Acustica 91 (3): 409–420.

    Google Scholar 

  • Zieliński, S., F. Rumsey, and S. Bech. 2008. On some biases encountered in modern audio quality listening tests – A review. Journal of the Audio Engineering Society 56 (6): 427–451. http://www.aes.org/e-lib/browse.cfm?elib=14393. Accessed 23 Sept. 2019.

Download references

Acknowledgements

This research has partly been supported by EU-FET grant Two!Ears, ICT-618075. The authors are grateful to Chris Hold, Marie-Neige Garcia, Werner Robitza, Sebastian Egger, Sebastian Möller, John Mourjopoulos, Sascha Spors, Karlheinz Brandenburg, Janina Fels, and Patrick Danès for fruitful discussions and conceptual contributions. Two external reviewer have provided useful comments and advice for improving this chapter.

Author information

Authors and Affiliations

  1. Audiovisual Technology Group, Institute for Media Technology, Ilmenau University of Technology (TU Ilmenau), Ilmenau, Germany

    Alexander Raake & Hagen Wierstorf

Authors
  1. Alexander Raake
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hagen Wierstorf
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Alexander Raake .

Editor information

Editors and Affiliations

  1. Institut für Kommunikationsakustik, Ruhr-Universität Bochum, Bochum, Nordrhein-Westfalen, Germany

    Jens Blauert

  2. School of Architecture, Rensselaer Polytechnic Institute, Troy, NY, USA

    Jonas Braasch

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Raake, A., Wierstorf, H. (2020). Binaural Evaluation of Sound Quality and Quality of Experience. In: Blauert, J., Braasch, J. (eds) The Technology of Binaural Understanding. Modern Acoustics and Signal Processing. Springer, Cham. https://doi.org/10.1007/978-3-030-00386-9_14

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-00386-9_14

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-00385-2

  • Online ISBN: 978-3-030-00386-9

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation