Skip to main content
SpringerLink
Log in

How Do Programming Students Read and Act upon Compiler Error Messages?

  • Conference paper
  • First Online:
Augmented Cognition (HCII 2023)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 14019))

Included in the following conference series:

Abstract

The research on understanding how programmers read and comprehend error messages is still limited. Through an eye-tracking study, we hope to add to what is known about how student programmers read and act upon error messages. In this paper, we elaborate on the methodology that we plan to use in the hopes of eliciting feedback from the research community that can help us improve upon our research design. We propose the study to enable us to collect data about whether student programmers read error messages, how they parse the code to find the source of the error, and how students with different ability and confidence levels vary in the way they process the errors. We will be examining where the participants fixate and how they read through the code to find the error. We are also interested in comparing the scan patterns of students who are proficient versus less proficient as well as those with high and low self-efficacy. We hope that the findings will contribute to the discourse about how student programmers read and parse code for errors, as well as to the discourse about the design of better error messages.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 69.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 89.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Barik, T., et al.: Do developers read compiler error messages?. In: 2017 IEEE/ACM 39th International Conference on Software Engineering (ICSE), pp. 575–585. IEEE (2017)

    Google Scholar 

  2. Becker, B.A., et al.: Compiler error messages considered unhelpful: the landscape of text-based programming error message research. In: Proceedings of the Working Group Reports on Innovation and Technology in Computer Science Education, pp. 177–210 (2019)

    Google Scholar 

  3. Becker, B.A., Murray, C., Tao, T., Song, C., McCartney, R., Sanders, K.: Fix the first, ignore the rest: dealing with multiple compiler error messages. In: Proceedings of the 49th ACM Technical Symposium on Computer Science Education, pp. 634–639 (2018)

    Google Scholar 

  4. Brechmann, A., Sharif, B., Siegmund, J., Weimer, W., Endres, M.: Foundations for a New Perspective on Programming. Report from Dagstuhl Seminar 22402 (to appear)

    Google Scholar 

  5. Denny, P., Luxton-Reilly, A., Carpenter, D.: Enhancing syntax error messages appears ineffectual. In: Proceedings of the ITiCSE ‘14: Proceedings of the 2014 Conference on Innovation & Technology in Computer Science Education, pp. 273–278 (2014)

    Google Scholar 

  6. Dong, T., Khandwala, K.: The IMPACT of “Cosmetic” changes on the usability of error messages. In: Extended Abstracts of the 2019 Chi Conference on Human Factors in Computing Systems, pp. 1–6 (2019)

    Google Scholar 

  7. Du Boulay, B., Matthew, I.: Fatal error in pass zero: how not to confuse novices. Behav. Inf. Technol. 3(2), 109–118 (1984)

    Article  Google Scholar 

  8. Dy, T., Rodrigo, M.M.: A detector for non-literal Java errors. In: Proceedings of the 10th Koli Calling International Conference on Computing Education Research, pp. 118–122 (2010)

    Google Scholar 

  9. Eraslan, S., Yesilada, Y., Harper, S.: Identifying patterns in eyetracking scanpaths in terms of visual elements of web pages. In: Casteleyn, S., Rossi, G., Winckler, M. (eds.) ICWE 2014. LNCS, vol. 8541, pp. 163–180. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-08245-5_10

    Chapter  Google Scholar 

  10. Eraslan, S., Yesilada, Y., Harper, S.: Scanpath Trend Analysis on Web Pages: Clustering Eye Tracking Scanpaths. ACM Trans. Web 10(4), 35 (2016). Article 20. https://doi.org/10.1145/2970818

  11. Eraslan, S., Yesilada, Y., Harper, S.: Engineering web- based interactive systems: trend analysis in eye tracking scanpaths with a tolerance. In: Proceedings of the ACM SIGCHI Symposium on Engineering Interactive Computing Systems, pp. 3–8 (2017)

    Google Scholar 

  12. Feitelson, D.G.: Considerations and pitfalls in controlled experiments on code comprehension. In: 2021 IEEE/ACM 29th International Conference on Program Comprehension (ICPC), pp. 106–117. IEEE (2021)

    Google Scholar 

  13. Goldberg, J.H., Helfman, J.I.: Scanpath clustering and aggregation. In: Proceedings of the 2010 Symposium on Eye Tracking Research and Applications, pp. 18–27 (2010)

    Google Scholar 

  14. Kohn, T.: The error behind the message: finding the cause of error messages in python. In: Proceedings of the 50th ACM Technical Symposium on Computer Science Education, pp. 524–530 (2019)

    Google Scholar 

  15. Marceau, G., Fisler, K., Krishnamurthi, S.: Measuring the effectiveness of error messages designed for novice programmers. In: Proceedings of the 42nd ACM Technical Symposium on Computer science Education, pp. 499–504 (2011)

    Google Scholar 

  16. Marceau, G., Fisler, K., Krishnamurthi, S.: Mind your language: on novices’ interactions with error messages. In: Proceedings of the 10th SIGPLAN Symposium on New Ideas, New Paradigms, and Reflections on Programming and Software, pp. 3–18 (2011)

    Google Scholar 

  17. Nienaltowski, M.H., Pedroni, M., Meyer, B.: Compiler error messages: what can help novices?. In: Proceedings of the 39th SIGCSE Technical Symposium on Computer Science Education, pp. 168–172 (2008)

    Google Scholar 

  18. Prather, J., et al.: On novices’ interaction with compiler error messages: a human factors approach. In: Proceedings of the 2017 ACM Conference on International Computing Education Research, pp. 74–82 (2017)

    Google Scholar 

  19. Sharafi, Z., Sharif, B., Guéhéneuc, Y.-G., Begel, A., Bednarik, R., Crosby, M.: A practical guide on conducting eye tracking studies in software engineering. Empir. Softw. Eng. 25(5), 3128–3174 (2020). https://doi.org/10.1007/s10664-020-09829-4

    Article  Google Scholar 

  20. Sharif, B., Falcone, M., Maletic, J.I.: An eye-tracking study on the role of scan time in finding source code defects. In: Proceedings of the Symposium on Eye Tracking Research and Applications, pp. 381–384 (2012)

    Google Scholar 

  21. Tablatin, C.L.S., Rodrigo, M.M.T.: Identifying code reading strategies in debugging using STA with a tolerance algorithm. APSIPA Trans. Sig. Inf. Process. 11(1) (2022)

    Google Scholar 

  22. Tsai, M.-J., Wang, C.-Y., Hsu, P.-F.: Developing the computer programming self-efficacy scale for computer literacy education. J. Educ. Comput. Res. 56(8), 1345–1360 (2019)

    Google Scholar 

  23. Villamor, M.M., Rodrigo, M.M.T.: Predicting pair success in a pair programming eye tracking experiment using cross-recurrence quantification analysis. APSIPA Trans. Sig. Inf. Process. 11(1) (2022)

    Google Scholar 

  24. Wyrich, M., Bogner, J., Wagner, S.: 40 years of designing code comprehension experiments: a systematic mapping study. arXiv preprint arXiv:2206.11102 (2022)

Download references

Acknowledgment

The authors would like to acknowledge the following Dagstuhl Workshops:

• The Human Factors Impact of Programming Error Messages (22052)

• Foundations for a New Perspective of Understanding Programming (22402).

Author information

Authors and Affiliations

  1. Ateneo de Manila University, Quezon City, Metro Manila, Philippines

    Maria Mercedes T. Rodrigo & Christine Lourrine S. Tablatin

  2. Pangasinan State University, Urdaneta City, Pangasinan, Philippines

    Christine Lourrine S. Tablatin

Authors
  1. Maria Mercedes T. Rodrigo
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Christine Lourrine S. Tablatin
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Maria Mercedes T. Rodrigo .

Editor information

Editors and Affiliations

  1. Soar Technology Inc., Orlando, FL, USA

    Dylan D. Schmorrow

  2. Katmai Government Services, Orlando, FL, USA

    Cali M. Fidopiastis

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Rodrigo, M.M.T., Tablatin, C.L.S. (2023). How Do Programming Students Read and Act upon Compiler Error Messages?. In: Schmorrow, D.D., Fidopiastis, C.M. (eds) Augmented Cognition. HCII 2023. Lecture Notes in Computer Science(), vol 14019. Springer, Cham. https://doi.org/10.1007/978-3-031-35017-7_11

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-35017-7_11

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-35016-0

  • Online ISBN: 978-3-031-35017-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation