Skip to main content
SpringerLink
Log in

Usage and Application of Heatmap Visualizations on Usability User Testing: A Systematic Literature Review

  • Conference paper
  • First Online:
Design, User Experience, and Usability (HCII 2023)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 14032))

Included in the following conference series:

  • 838 Accesses

Abstract

Nowadays, high-quality user experience (UX) has become a key competitive factor for product development. User experience methods and techniques investigate how people feel about a system. However, we must take into account that traditional evaluation methods, whether interviews or surveys are based on self-reported data, which are often exposed to social desire. Therefore, by obtaining additional information from the user's interaction with the system being evaluated, besides interviews or surveys, we build a more robust user experience. In this research, we present the outcomes of a Systematic Literature Review (SLR) aimed at uncovering case studies, difficulties, issues, and opportunities related to the use of heatmaps in usability testing. The SLR was carried out using the protocol proposed by Kitchenham and Charters. The research was carried out on August 23, 2022, and a total of 371 articles were retrieved, from which 22 were selected. The results show that a formal process reported in the literature has not been identified that indicates the tasks that an evaluation team should carry out to complement the results of the heatmap application with user tests for a better analysis of usability problems and design errors. On the other hand, it can be seen that the tools necessary to carry out this analysis only obtain information from the user but do not obtain metrics from it or vice versa. According to the results obtained, we can conclude that heatmaps are useful for specialists who perform usability tests with users because the user's interaction with the system to be evaluated can be graphically visualized.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 109.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 139.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Paz, F.: Método para la evaluación de usabilidad de sitios web transaccionales basado en el proceso de inspección heurística. 275 (2017)

    Google Scholar 

  2. Nielsen, J.: Usability engineering. Academic Press, Boston (1993)

    Book  MATH  Google Scholar 

  3. Georges, V., Courtemanche, F., Senecal, S., Baccino, T., Fredette, M., Leger, P.-M.: UX Heatmaps: Mapping User Experience on Visual Interfaces. In: Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems, pp. 4850–4860. ACM, San Jose California USA (2016). https://doi.org/10.1145/2858036.2858271

  4. Kitchenham, B., Charters, S.: Guidelines for performing Systematic Literature Reviews in Software Engineering (2007)

    Google Scholar 

  5. Petticrew, M., Roberts, H.: Systematic Reviews in the Social Sciences. 354 (2006)

    Google Scholar 

  6. Maslov, I., Nikou, S.: Usability and UX of learning management systems: an eye- tracking approach. In: 2020 IEEE International Conference on Engineering, Technology and Innovation (ICE/ITMC). pp. 1–9. IEEE, Cardiff, United Kingdom (2020). https://doi.org/10.1109/ICE/ITMC49519.2020.9198333

  7. International Organization for Standardization: ISO 9241–210: Ergonomics of human-system interaction – Part 210: Human-Centred Design for Interactive Systems, Geneva (2010)

    Google Scholar 

  8. Rubin, J., Chisnell, D.: Handbook of usability testing: how to plan, design, and conduct effective tests. Wiley Pub, Indianapolis, IN (2008)

    Google Scholar 

  9. Stuart, S. (ed.): Eye Tracking: Background, Methods, and Applications. Springer US, New York, NY (2022). https://doi.org/10.1007/978-1-0716-2391-6

  10. Chen, C., Härdle, W., Unwin, A.: Handbook of Data Visualization. Springer Berlin Heidelberg, Berlin, Heidelberg (2008). https://doi.org/10.1007/978-3-540-33037-0

  11. Nielsen Norman Group: UX Metrics and ROI. Nielsen Norman Group (2016)

    Google Scholar 

  12. Sunhem, W., Pasupa, K.: A scenario-based analysis of front-facing camera eye tracker for UX-UI survey on mobile banking app. In: 2020 12th International Conference on Knowledge and Smart Technology (KST), Pattaya, Chonburi, Thailand, pp. 80–85. IEEE (2020). https://doi.org/10.1109/KST48564.2020.9059376

  13. Menges, R., Kramer, S., Hill, S., Nisslmueller, M., Kumar, C., Staab, S.: A visualization tool for eye tracking data analysis in the web. In: ACM Symposium on Eye Tracking Research and Applications, Stuttgart Germany, pp. 1–5. ACM (2020). https://doi.org/10.1145/3379156.3391831

  14. Souza, K.E.S. de, et al.: An evaluation framework for user experience using eye tracking, mouse tracking, keyboard input, and artificial intelligence: a case study. Int. J. Hum.–Comput. Interact. 38, 646–660 (2022). https://doi.org/10.1080/10447318.2021.1960092

  15. Dimitrakopoulou, D., Faliagka, E., Rigou, M.: An eye-tracking study of web search interaction design patterns. In: Proceedings of the 20th Pan-Hellenic Conference on Informatics, Patras Greece, pp. 1–6. ACM (2016). https://doi.org/10.1145/3003733.3003795

  16. Batliner, M., Hess, S., Ehrlich-Adám, C., Lohmeyer, Q., Meboldt, M.: Automated areas of interest analysis for usability studies of tangible screen-based user interfaces using mobile eye tracking. AIEDAM 34, 505–514 (2020). https://doi.org/10.1017/S0890060420000372

  17. Aviz, I.L., Souza, K.E., Ribeiro, E., de Mello Junior, H., Seruffo, M.C. da R.: Comparative study of user experience evaluation techniques based on mouse and gaze tracking. In: Proceedings of the 25th Brazillian Symposium on Multimedia and the Web, pp. 53–56. ACM, Rio de Janeiro Brazil (2019). https://doi.org/10.1145/3323503.3360623

  18. Menges, R., Tamimi, H., Kumar, C., Walber, T., Schaefer, C., Staab, S.: Enhanced representation of web pages for usability analysis with eye tracking. In: Proceedings of the 2018 ACM Symposium on Eye Tracking Research & Applications, pp. 1–9. ACM, Warsaw Poland (2018). https://doi.org/10.1145/3204493.3214308

  19. Wang, J., Antonenko, P., Celepkolu, M., Jimenez, Y., Fieldman, E., Fieldman, A.: Exploring relationships between eye tracking and traditional usability testing data. Int. J. Hum.-Comput. Interact. 35, 483–494 (2019). https://doi.org/10.1080/10447318.2018.1464776

    Article  Google Scholar 

  20. Drewes, H.: Eye Gaze Tracking for Human Computer Interaction. 167 (2010)

    Google Scholar 

  21. Dogan, K.M., Suzuki, H., Gunpinar, E.: Eye tracking for screening design parameters in adjective-based design of yacht hull. Ocean Eng. 166, 262–277 (2018). https://doi.org/10.1016/j.oceaneng.2018.08.026

    Article  Google Scholar 

  22. Bagnara, S., Tartaglia, R., Albolino, S., Alexander, T., Fujita, Y. (eds.): IEA 2018. AISC, vol. 822. Springer, Cham (2019). https://doi.org/10.1007/978-3-319-96077-7

    Book  Google Scholar 

  23. Chynał, P., Szymański, J.M.: Remote usability testing using eyetracking. In: Campos, P., Graham, N., Jorge, J., Nunes, N., Palanque, P., Winckler, M. (eds.) INTERACT 2011. LNCS, vol. 6946, pp. 356–361. Springer, Heidelberg (2011). https://doi.org/10.1007/978-3-642-23774-4_29

    Chapter  Google Scholar 

  24. Suxe Ramírez, M.A.: Sistema de rastreo ocular para la experiencia del usuario en los contenidos de la página web en la empresa grupo CELLCH – Huaraz; 2017 (2019. https://hdl.handle.net/20.500.13032/11961)

  25. Meiselwitz, G. (ed.): Social Computing and Social Media: Design, User Experience and Impact, HCII 2022. Springer, Cham (2022). https://doi.org/10.1007/978-3-031-05061-9

  26. Bodrunova, S.S., Yakunin, A.V.: U-index: an eye-tracking-tested checklist on webpage aesthetics for university web spaces in Russia and the USA. In: Marcus, A., Wang, W. (eds.) DUXU 2017. LNCS, vol. 10288, pp. 219–233. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-58634-2_17

    Chapter  Google Scholar 

  27. Lago, B.L., Pereira, V.C., Cardoso, J.M.: Usability and eye tracking: Analysis of the home pages of international universities: Do they clearly show the objetives of the institution? In: 2018 13th Iberian Conference on Information Systems and Technologies (CISTI), Caceres, pp. 1–5. IEEE (2018). https://doi.org/10.23919/CISTI.2018.8399462

  28. Da Silva Franco, R.Y., et al.: UXmood - a tool to investigate the user experience (UX) based on multimodal sentiment analysis and information visualization (InfoVis). In: 2019 23rd International Conference Information Visualisation (IV), Paris, France, pp. 175–180. IEEE (2019). https://doi.org/10.1109/IV.2019.00038

  29. Bhatt, S., Agrali, A., McCarthy, K., Suri, R., Ayaz, H.: Web usability testing with concurrent fNIRS and eye tracking. In: Neuroergonomics, pp. 181–186. Elsevier (2019). https://doi.org/10.1016/B978-0-12-811926-6.00030-0

  30. Papoutsaki, A., Sangkloy, P., Laskey, J., Daskalova, N., Huang, J., Hays, J.: WebGazer: Scalable Webcam Eye Tracking Using User Interactions 7 (2016)

    Google Scholar 

Download references

Acknowledgment

We express our gratitude to the "HCI, Design, User Experience, Accessibility & Innovation Technologies (HCI DUXAIT)" research group from Pontificia Universidad Católica del Perú (PUCP) for their support throughout the entire research process.

Author information

Authors and Affiliations

  1. Pontificia Universidad Católica del Perú, Av. Universitaria, 1801, Lima, Peru

    Fabricio Davila, Freddy Paz & Arturo Moquillaza

Authors
  1. Fabricio Davila
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Freddy Paz
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Arturo Moquillaza
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Fabricio Davila .

Editor information

Editors and Affiliations

  1. Aaron Marcus and Associates, Berkeley, CA, USA

    Aaron Marcus

  2. World Usability Day and Bubble Mountain Consulting, Newton Center, MA, USA

    Elizabeth Rosenzweig

  3. Southern University of Science and Technology – SUSTech, Shenzhen, China

    Marcelo M. Soares

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Davila, F., Paz, F., Moquillaza, A. (2023). Usage and Application of Heatmap Visualizations on Usability User Testing: A Systematic Literature Review. In: Marcus, A., Rosenzweig, E., Soares, M.M. (eds) Design, User Experience, and Usability. HCII 2023. Lecture Notes in Computer Science, vol 14032. Springer, Cham. https://doi.org/10.1007/978-3-031-35702-2_1

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-35702-2_1

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-35701-5

  • Online ISBN: 978-3-031-35702-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation