skip to main content
research-article

A Review of Worked Examples in Programming Activities

Published:29 December 2022Publication History
Skip Abstract Section

Abstract

This article reviews literature on worked examples in the context of programming activities. We focus on two types of examples, namely, code-tracing and code-generation, because there is sufficient research on these to warrant a review. We synthesize key results according to themes that emerged from the review. This synthesis aims to provide practical guidance for educators and shed light on future research opportunities. While there is established work in some areas (e.g., dynamic code-tracing examples in the form of program visualization tools, utility of subgoals in code-generation examples, and incomplete examples in the form of Parsons puzzles), there are also gaps. Thus, the article concludes with directions for future work on examples in computer science education.

REFERENCES

  1. [1] Al-Barakati Nouf M. and Al-Aama Arwa Y.. 2009. The effect of visualizing roles of variables on student performance in an introductory programming course. In Proceedings of the 14th Annual ACM SIGCSE Conference on Innovation and Technology in Computer Science Education. ACM, New York, NY, 228232. Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. [2] Anderson John R.. 1993. Rules of the Mind. Psychology Press. Google ScholarGoogle ScholarCross RefCross Ref
  3. [3] Atkinson Robert K., Derry Sharon J., Renkl Alexander, and Wortham Donald. 2000. Learning from examples: Instructional principles from the worked examples research. Rev. Edu. Res. 70, 2 (2000), 181214. Google ScholarGoogle ScholarCross RefCross Ref
  4. [4] Awasekar Dipali D.. 2013. Effect of program visualization to teach computer programming in a resource constrained classroom. In Proceedings of the IEEE 5th International Conference on Technology for Education. IEEE, 93100. Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. [5] Banerjee Gargi, Murthy Sahana, and Iyer Sridhar. 2015. Effect of active learning using program visualization in technology-constrained college classrooms. Res. Pract. Technol. Enhanced Learn. 10, 1 (2015), 125. Google ScholarGoogle ScholarCross RefCross Ref
  6. [6] Bayman Piraye and Mayer Richard E.. 1988. Using conceptual models to teach BASIC computer programming. J. Edu. Psychol. 80, 3 (1988), 291. Google ScholarGoogle ScholarCross RefCross Ref
  7. [7] Beege Maik, Schneider Sascha, Nebel Steve, Zimm Justus, Windisch Sarah, and Rey Günter D.. 2021. Learning programming from erroneous worked-examples. Which type of error is beneficial for learning? Learn. Instruct. 75 (2021), 101497. Google ScholarGoogle ScholarCross RefCross Ref
  8. [8] Ben-Ari Mordechai, Bednarik Roman, Levy Ronit B. B., Ebel Gil, Moreno Andrés, Myller Niko, and Sutinen Erkki. 2011. A decade of research and development on program animation: The Jeliot experience. J. Visual Lang. Comput. 22, 5 (2011), 375384. Google ScholarGoogle ScholarCross RefCross Ref
  9. [9] Bisra Kiran, Liu Qing, Nesbit John C., Salimi Farimah, and Winne Philip H.. 2018. Inducing Self-Explanation: A Meta-Analysis. Edu. Psychol. Rev. 30, 3 (2018), 703725. Google ScholarGoogle ScholarCross RefCross Ref
  10. [10] Boulay Benedict Du. 1986. Some difficulties of learning to program. J. Edu. Comput. Res. 2, 1 (1986), 5773. . arXiv:https://doi.org/10.2190/3LFX-9RRF-67T8-UVK9Google ScholarGoogle ScholarCross RefCross Ref
  11. [11] Brusilovsky Peter, Malmi Lauri, Hosseini Roya, Guerra Julio, Sirkiä Teemu, and Pollari-Malmi Kerttu. 2018. An integrated practice system for learning programming in Python: Design and evaluation. Res. Pract. Technol. Enhanced Learn. 13, 1 (2018), 140. Google ScholarGoogle ScholarCross RefCross Ref
  12. [12] Catrambone Richard. 1998. The subgoal learning model: Creating better examples so that students can solve novel problems. J. Exper. Psychol.: Gen. 127, 4 (1998), 355. Google ScholarGoogle ScholarCross RefCross Ref
  13. [13] Cetin Ibrahim. 2020. Teaching loops concept through visualization construction. Inform. Edu. Int. J. 19, 4 (2020), 589609. Google ScholarGoogle ScholarCross RefCross Ref
  14. [14] Chang Kuo-En, Chiao Bea-Chu, Chen Sei-Wang, and Hsiao Rong-Shue. 2000. A programming learning system for beginners-A completion strategy approach. IEEE Trans. Edu. 43, 2 (2000), 211220. Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. [15] Chen Xingliang, Mitrovic Antonija, and Mathews Moffat. 2016. Do erroneous examples improve learning in addition to problem solving and worked examples? In Proceedings of the Intelligent Tutoring Systems Conference, Micarelli Alessandro, Stamper John, and Panourgia Kitty (Eds.). Springer International Publishing, Cham, 1322. Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. [16] Chi Michelene T. H., Bassok Miriam, Lewis Matthew W., Reimann Peter, and Glaser Robert. 1989. Self-explanations: How students study and use examples in learning to solve problems. Cogn. Sci. 13, 2 (1989), 145182. Google ScholarGoogle ScholarCross RefCross Ref
  17. [17] Chi Michelene T. H.. 2009. Active-constructive-interactive: A conceptual framework for differentiating learning activities. Top. Cogn. Sci. 1, 1 (2009), 73105. Google ScholarGoogle ScholarCross RefCross Ref
  18. [18] Conati Cristina and Vanlehn Kurt. 2000. Toward computer-based support of meta-cognitive skills: A computational framework to coach self-explanation. Int. J. Artific. Intell. Edu. 11 (2000), 398415.Google ScholarGoogle Scholar
  19. [19] Cooper Graham and Sweller John. 1987. Effects of schema acquisition and rule automation on mathematical problem-solving transfer. J. Edu. Psychol. 79, 4 (1987), 347.Google ScholarGoogle ScholarCross RefCross Ref
  20. [20] Cunningham Kathryn, Blanchard Sarah, Ericson Barbara, and Guzdial Mark. 2017. Using tracing and sketching to solve programming problems: Replicating and extending an analysis of what students draw. In Proceedings of the ACM Conference on International Computing Education Research (ICER’17). ACM, New York, NY, 164172. Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. [21] Cunningham Kathryn, Ke Shannon, Guzdial Mark, and Ericson Barbara. 2019. Novice rationales for sketching and tracing, and how they try to avoid it. In Proceedings of the ACM Conference on Innovation and Technology in Computer Science Education (ITiCSE’19). ACM, New York, NY, 3743. Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. [22] Davidovic Aleksandar, Warren Jim, and Trichina Elena. 2003. Learning benefits of structural example-based adaptive tutoring systems. IEEE Trans. Edu. 46, 2 (2003), 241251. Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. [23] Du Yuemeng, Luxton-Reilly Andrew, and Denny Paul. 2020. A review of research on parsons problems. In Proceedings of the 22nd Australasian Computing Education Conference (ACE’20). ACM, New York, NY, 195202. Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. [24] Durkin Kelley and Rittle-Johnson Bethany. 2012. The effectiveness of using incorrect examples to support learning about decimal magnitude. Learn. Instruct. 22, 3 (2012), 206214. Google ScholarGoogle ScholarCross RefCross Ref
  25. [25] Ebel Gil and Ben-Ari Mordechai. 2006. Affective effects of program visualization. In Proceedings of the 2nd International Workshop on Computing Education Research (ICER’06). ACM, New York, NY, 15. Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. [26] Ericson Barbara J., Foley James D., and Rick Jochen. 2018. Evaluating the efficiency and effectiveness of adaptive parsons problems. In Proceedings of the ACM Conference on International Computing Education Research (ICER’18). ACM, New York, NY, 6068. Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. [27] Ericson Barbara J., Margulieux Lauren E., and Rick Jochen. 2017. Solving parsons problems versus fixing and writing code. In Proceedings of the 17th Koli Calling International Conference on Computing Education Research (KoliCalling’17). ACM, New York, NY, 2029. Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. [28] Fabic Geela V. F., Mitrovic Antonija, and Neshatian Kourosh. 2019. Evaluation of parsons problems with menu-based self-explanation prompts in a mobile python tutor. Int. J. Artific. Intell. Edu. 29, 4 (2019), 507535. Google ScholarGoogle ScholarCross RefCross Ref
  29. [29] Fitzgerald Sue, Simon Beth, and Thomas Lynda. 2005. Strategies that students use to trace code: An analysis based in grounded theory. In Proceedings of the 1st International Workshop on Computing Education Research (ICER’05). ACM, New York, NY, 6980. Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. [30] Gadgil Soniya, Nokes-Malach Timothy J., and Chi Michelene T. H.. 2012. Effectiveness of holistic mental model confrontation in driving conceptual change. Learn. Instruct. 22, 1 (2012), 4761. Google ScholarGoogle ScholarCross RefCross Ref
  31. [31] Gentner Dedre. 1983. Structure-mapping: A theoretical framework for analogy. Cogn. Sci. 7, 2 (1983), 155170. Google ScholarGoogle ScholarCross RefCross Ref
  32. [32] Giacaman Nasser. 2012. Teaching by example: Using analogies and live coding demonstrations to teach parallel computing concepts to undergraduate students. In Proceedings of the IEEE 26th International Parallel and Distributed Processing Symposium Workshops and PhD Forum. IEEE, 12951298. Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. [33] Gick Mary L. and Holyoak Keith J.. 1980. Analogical problem solving. Cogn. Psychol. 12, 3 (1980), 306355. Google ScholarGoogle ScholarCross RefCross Ref
  34. [34] Griffin Jean M.. 2019. Designing intentional bugs for learning. In Proceedings of the 1st UK and Ireland Computing Education Research Conference. ACM, New York, NY, Article 5, 7 pages. Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. [35] Große Cornelia S. and Renkl Alexander. 2007. Finding and fixing errors in worked examples: Can this foster learning outcomes? Learn. Instruct. 17, 6 (2007), 612634. Google ScholarGoogle ScholarCross RefCross Ref
  36. [36] Hamouda Sally, Edwards Stephen H., Elmongui Hicham G., Ernst Jeremy V., and Shaffer Clifford A.. 2018. RecurTutor: An interactive tutorial for learning recursion. ACM Trans. Comput. Edu. 19, 1, Article 1 (Nov. 2018), 25 pages. Google ScholarGoogle ScholarDigital LibraryDigital Library
  37. [37] Hao Qiang, IV David H. Smith, Iriumi Naitra, Tsikerdekis Michail, and Ko Amy J.. 2019. A systematic investigation of replications in computing education research. ACM Trans. Comput. Edu. 19, 4, Article 42 (Aug2019), 18 pages. Google ScholarGoogle ScholarDigital LibraryDigital Library
  38. [38] Harms Kyle J., Chen Jason, and Kelleher Caitlin L.. 2016. Distractors in parsons problems decrease learning efficiency for young novice programmers. In Proceedings of the ACM Conference on International Computing Education Research (ICER’16). ACM, New York, NY, 241250. Google ScholarGoogle ScholarDigital LibraryDigital Library
  39. [39] Harsley Rachel, Green Nick, Alizadeh Mehrdad, Acharya Sabita, Fossati Davide, Eugenio Barbara Di, and AlZoubi Omar. 2016. Incorporating analogies and worked out examples as pedagogical strategies in a computer science tutoring system. In Proceedings of the 47th ACM Technical Symposium on Computing Science Education (SIGCSE’16). ACM, New York, NY, 675680. Google ScholarGoogle ScholarDigital LibraryDigital Library
  40. [40] Haynes Carl C. and Ericson Barbara J.. 2021. Problem-solving efficiency and cognitive load for adaptive parsons problems vs. writing the equivalent code. In Proceedings of the CHI Conference on Human Factors in Computing Systems (CHI’21). ACM, New York, NY, Article 60, 15 pages. Google ScholarGoogle ScholarDigital LibraryDigital Library
  41. [41] Helminen Juha, Ihantola Petri, Karavirta Ville, and Alaoutinen Satu. 2013. How do students solve parsons programming problems?–Execution-based vs. line-based feedback. In Proceedings of the Learning and Teaching in Computing and Engineering. IEEE, 5561. Google ScholarGoogle ScholarDigital LibraryDigital Library
  42. [42] Hertz Matthew and Jump Maria. 2013. Trace-based teaching in early programming courses. In Proceedings of the 44th ACM Technical Symposium on Computer Science Education (SIGCSE’13). ACM, New York, NY, 561566. Google ScholarGoogle ScholarDigital LibraryDigital Library
  43. [43] Hoffswell Jane, Satyanarayan Arvind, and Heer Jeffrey. 2018. Augmenting code with in situ visualizations to aid program understanding. In Proceedings of the CHI Conference on Human Factors in Computing Systems (CHI’18). ACM, New York, NY, 112. Google ScholarGoogle ScholarDigital LibraryDigital Library
  44. [44] Hosseini Roya, Akhuseyinoglu Kamil, Brusilovsky Peter, Malmi Lauri, Pollari-Malmi Kerttu, Schunn Christian, and Sirkiä Teemu. 2020. Improving engagement in program construction examples for learning python programming. Int. J. Artific. Intell. Edu. 30, 2 (2020), 299336. Google ScholarGoogle ScholarCross RefCross Ref
  45. [45] Hosseini Roya, Akhuseyinoglu Kamil, Petersen Andrew, Schunn Christian D., and Brusilovsky Peter. 2018. PCEX: Interactive program construction examples for learning programming. In Proceedings of the 18th Koli Calling International Conference on Computing Education Research (KoliCalling’18). ACM, New York, NY, Article 5, 9 pages. Google ScholarGoogle ScholarDigital LibraryDigital Library
  46. [46] Hosseini Roya, Sirkiä Teemu, Guerra Julio, Brusilovsky Peter, and Malmi Lauri. 2016. Animated examples as practice content in a Java programming course. In Proceedings of the 47th ACM Technical Symposium on Computing Science Education (SIGCSE’16). ACM, New York, NY, 540545. Google ScholarGoogle ScholarDigital LibraryDigital Library
  47. [47] Huang Ruanqianqian, Ferdowsi Kasra, Selvaraj Ana, Raj Adalbert Gerald Soosai, and Lerner Sorin. 2022. Investigating the impact of using a live programming environment in a CS1 course. In Proceedings of the 53rd ACM Technical Symposium on Computer Science Education (SIGCSE’22). ACM, New York, NY, 495501. Google ScholarGoogle ScholarDigital LibraryDigital Library
  48. [48] Hwang Derek, Agarwal Vardhan, Lyu Yuzi, Rana Divyam, Susarla Satya Ganesh, and Raj Adalbert Gerald Soosai. 2021. A qualitative analysis of lecture videos and student feedback on static code examples and live coding: A case study. In Proceedings of the Australasian Computing Education Conference (ACE’21). ACM, New York, NY, 147157. Google ScholarGoogle ScholarDigital LibraryDigital Library
  49. [49] Ichinco Michelle, Harms Kyle J., and Kelleher Caitlin. 2017. Towards understanding successful novice example use in blocks-based programming. J. Visual Lang. Sent. Syst. 3 (2017), 101118. Google ScholarGoogle ScholarCross RefCross Ref
  50. [50] Ichinco Michelle, Hnin Wint Yee, and Kelleher Caitlin L.. 2017. Suggesting API usage to novice programmers with the example guru. In Proceedings of the CHI Conference on Human Factors in Computing Systems (CHI’17). ACM, New York, NY, 11051117. Google ScholarGoogle ScholarDigital LibraryDigital Library
  51. [51] Jarosz Andrew F. and Wiley Jennifer. 2014. What are the odds? A practical guide to computing and reporting Bayes factors. J. Problem Solv. 7, 1 (2014), 2. Google ScholarGoogle ScholarCross RefCross Ref
  52. [52] Jennings Jay and Muldner Kasia. 2021. Investigating students’ reasoning in a code-tracing tutor. In Proceedings of the 22nd International Conference on Artificial Intelligence in Education (AIED’21). Springer-Verlag, Berlin, 203214. Google ScholarGoogle ScholarDigital LibraryDigital Library
  53. [53] Jennings Jay and Muldner Kasia. 2021. When does scaffolding provide too much assistance? A code-tracing tutor investigation. Int. J. Artific. Intell. Edu. 31, 4 (2021), 784819. Google ScholarGoogle ScholarCross RefCross Ref
  54. [54] Joentausta Johanna and Hellas Arto. 2018. Subgoal labeled worked examples in K-3 education. In Proceedings of the 49th ACM Technical Symposium on Computer Science Education (SIGCSE’18). ACM, New York, NY, 616621. Google ScholarGoogle ScholarDigital LibraryDigital Library
  55. [55] Kaila Erkki, Rajala Teemu, Laakso Mikko-Jussi, and Salakoski Tapio. 2010. Effects of course-long use of a program visualization tool. In Proceedings of the 12th Australasian Conference on Computing Education, Vol. 103. ACM, 97106.Google ScholarGoogle Scholar
  56. [56] Kalyuga Slava, Ayres Paul, Chandler Paul, and Sweller John. 2003. The expertise reversal effect. Edu. Psychol. 38, 1 (2003), 2331. . arXiv:https://doi.org/10.1207/S15326985EP3801_4Google ScholarGoogle ScholarCross RefCross Ref
  57. [57] Kapur Manu. 2014. Productive failure in learning math. Cogn. Sci. 38, 5 (2014), 10081022. Google ScholarGoogle ScholarCross RefCross Ref
  58. [58] Kumar Amruth N.. 2014. An evaluation of self-explanation in a programming tutor. In Proceedings of the International Conference on Intelligent Tutoring Systems. Springer International Publishing, 248253. Google ScholarGoogle ScholarDigital LibraryDigital Library
  59. [59] Kumar Amruth N.. 2019. Helping students solve parsons puzzles better. In Proceedings of the ACM Conference on Innovation and Technology in Computer Science Education (ITiCSE’19). ACM, New York, NY, 6570. Google ScholarGoogle ScholarDigital LibraryDigital Library
  60. [60] Lee Bridjet and Muldner Kasia. 2020. Instructional video design: Investigating the impact of monologue-and dialogue-style presentations. In Proceedings of the CHI Conference on Human Factors in Computing Systems (CHI’20). ACM, New York, NY, 112. Google ScholarGoogle ScholarDigital LibraryDigital Library
  61. [61] Lishinski Alex, Yadav Aman, Good Jon, and Enbody Richard. 2016. Learning to program: Gender differences and interactive effects of students’ motivation, goals, and self-efficacy on performance. In Proceedings of the ACM Conference on International Computing Education Research (ICER’16). ACM, New York, NY, 211220. Google ScholarGoogle ScholarDigital LibraryDigital Library
  62. [62] Lister Raymond, Adams Elizabeth S., Fitzgerald Sue, Fone William, Hamer John, Lindholm Morten, McCartney Robert, Moström Jan Erik, Sanders Kate, Seppälä Otto, Simon Beth, and Thomas Lynda. 2004. A multi-national study of reading and tracing skills in novice programmers. In Working Group Reports from ITiCSE on Innovation and Technology in Computer Science Education (ITiCSE-WGR’04). ACM, New York, NY, 119150. Google ScholarGoogle ScholarDigital LibraryDigital Library
  63. [63] Loboda Tomasz D. and Brusilovsky Peter. 2010. User-adaptive explanatory program visualization: Evaluation and insights from eye movements. User Model. User-Adapt. Interact. 20, 3 (2010), 191226. Google ScholarGoogle ScholarDigital LibraryDigital Library
  64. [64] Margulieux Lauren E. and Catrambone Richard. 2016. Improving problem solving with subgoal labels in expository text and worked examples. Learn. Instruct. 42 (2016), 5871. Google ScholarGoogle ScholarCross RefCross Ref
  65. [65] Margulieux Lauren E. and Catrambone Richard. 2017. Using learners’ self-explanations of subgoals to guide initial problem solving in app inventor. In Proceedings of the ACM Conference on International Computing Education Research (ICER’17). ACM, New York, NY, 2129. Google ScholarGoogle ScholarDigital LibraryDigital Library
  66. [66] Margulieux Lauren E. and Catrambone Richard. 2019. Finding the best types of guidance for constructing self-explanations of subgoals in programming. J. Learn. Sci. 28, 1 (2019), 108151. Google ScholarGoogle ScholarCross RefCross Ref
  67. [67] Margulieux Lauren E., Catrambone Richard, and Guzdial Mark. 2016. Employing subgoals in computer programming education. Comput. Sci. Edu. 26, 1 (2016), 4467. Google ScholarGoogle ScholarCross RefCross Ref
  68. [68] Margulieux Lauren E., Catrambone Richard, and Schaeffer Laura M.. 2018. Varying effects of subgoal labeled expository text in programming, chemistry, and statistics. Instruct. Sci. 46, 5 (2018), 707722. Google ScholarGoogle ScholarCross RefCross Ref
  69. [69] Margulieux Lauren E., Morrison Briana B., and Decker Adrienne. 2020. Reducing withdrawal and failure rates in introductory programming with subgoal labeled worked examples. Int. J. STEM Edu. 7, 1 (2020), 116. Google ScholarGoogle ScholarCross RefCross Ref
  70. [70] Margulieux Lauren E., Morrison Briana B., Franke Baker, and Ramilison Harivololona. 2020. Effect of implementing subgoals in Code.org’s intro to programming unit in computer science principles. ACM Trans. Comput. Edu. 20, 4, Article 26 (Oct.2020), 24 pages. Google ScholarGoogle ScholarDigital LibraryDigital Library
  71. [71] Margulieux Lauren E., Morrison Briana B., Guzdial Mark, and Catrambone Richard. 2016. Training learners to self-explain: Designing instructions and examples to improve problem solving. In Proceedings of the International Conference of the Learning Sciences (ICLS’16), Vol. 2. International Society of the Learning Sciences, 98105. Retrieved from https://www.isls.org/icls/2016/docs/ICLS2016_Volume_1_30June2016.pdf.Google ScholarGoogle Scholar
  72. [72] Mayer Richard E.. 1975. Different problem-solving competencies established in learning computer programming with and without meaningful models. J. Edu. Psychol. 67, 6 (1975), 725. Google ScholarGoogle ScholarCross RefCross Ref
  73. [73] Mayer Richard E.. 1976. Some conditions of meaningful learning for computer programming: Advance organizers and subject control of frame order. J. Edu. Psychol. 68, 2 (1976), 143. Google ScholarGoogle ScholarCross RefCross Ref
  74. [74] Mayer Richard E.. 1989. Models for understanding. Rev. Edu. Res. 59, 1 (1989), 4364. Google ScholarGoogle ScholarCross RefCross Ref
  75. [75] Mayer Richard E. and Bromage Bruce K.. 1980. Difference recall protocols for technical texts due to advance organizers. J. Edu. Psychol. 72, 2 (1980), 209. Google ScholarGoogle ScholarCross RefCross Ref
  76. [76] Mladenović Monika, Žanko Žana, and Cuvic Marin A.. 2021. The impact of using program visualization techniques on learning basic programming concepts at the K-12 level. Comput. Appl. Eng. Edu. 29, 1 (2021), 145159. Google ScholarGoogle ScholarCross RefCross Ref
  77. [77] Moreno Andrés, Sutinen Erkki, and Joy Mike. 2014. Defining and evaluating conflictive animations for programming education: The case of Jeliot ConAn. In Proceedings of the 45th ACM Technical Symposium on Computer Science Education (SIGCSE’14). ACM, New York, NY, 629634. Google ScholarGoogle ScholarDigital LibraryDigital Library
  78. [78] Morrison Briana B.. 2017. Dual modality code explanations for novices: Unexpected results. In Proceedings of the ACM Conference on International Computing Education Research (ICER’17). ACM, New York, NY, 226235. Google ScholarGoogle ScholarDigital LibraryDigital Library
  79. [79] Morrison Briana B., Margulieux Lauren E., and Decker Adrienne. 2020. The curious case of loops. Comput. Sci. Edu. 30, 2 (2020), 127154. Google ScholarGoogle ScholarCross RefCross Ref
  80. [80] Morrison Briana B., Margulieux Lauren E., Ericson Barbara, and Guzdial Mark. 2016. Subgoals help students solve Parsons problems. In Proceedings of the 47th ACM Technical Symposium on Computing Science Education (SIGCSE’16). ACM, New York, NY, 4247. Google ScholarGoogle ScholarDigital LibraryDigital Library
  81. [81] Morrison Briana B., Margulieux Lauren E., and Guzdial Mark. 2015. Subgoals, context, and worked examples in learning computing problem solving. In Proceedings of the 11th Annual International Conference on Computing Education Research (ICER’15). ACM, New York, NY, 2129. Google ScholarGoogle ScholarDigital LibraryDigital Library
  82. [82] Mousavi Seyedreza, Low Renae, and Sweller John. 1995. Reducing cognitive load by mixing auditory and visual presentation modes. J. Edu. Psychol. 87, 2 (1995), 319334. Google ScholarGoogle ScholarCross RefCross Ref
  83. [83] Muldner Kasia, Burleson Winslow, and Chi Michelene. 2014. Learning from self-explaining emergent phenomena. In Proceedings of the International Conference of the Learning Sciences (ICLS’14), Vol. 2. International Society of the Learning Sciences, 847854. Google ScholarGoogle Scholar
  84. [84] Muldner Kasia and Conati Cristina. 2010. Scaffolding meta-cognitive skills for effective analogical problem solving via tailored example selection. Int. J. Artific. Intell. Edu. 20, 2 (2010), 99136. Google ScholarGoogle ScholarCross RefCross Ref
  85. [85] Najar Amir Shareghi, Mitrovic Antonija, and McLaren Bruce M.. 2016. Learning with intelligent tutors and worked examples: selecting learning activities adaptively leads to better learning outcomes than a fixed curriculum. User Model. User-Adapt. Interact. 26, 5 (Dec.2016), 459491. Google ScholarGoogle ScholarDigital LibraryDigital Library
  86. [86] Nelson Greg L., Xie Benjamin, and Ko Amy J.. 2017. Comprehension first: Evaluating a novel pedagogy and tutoring system for program tracing in CS1. In Proceedings of the ACM Conference on International Computing Education Research (ICER’17). ACM, New York, NY, 4251. Google ScholarGoogle ScholarDigital LibraryDigital Library
  87. [87] Nevalainen Seppo and Sajaniemi Jorma. 2006. An experiment on short-term effects of animated versus static visualization of operations on program perception. In Proceedings of the 2nd International Workshop on Computing Education Research (ICER’06). ACM, New York, NY, 716. Google ScholarGoogle ScholarDigital LibraryDigital Library
  88. [88] Parsons Dale and Haden Patricia. 2006. Parson’s programming puzzles: A fun and effective learning tool for first programming courses. In Proceedings of the 8th Australasian Conference on Computing Education (ACE’06). Australian Computer Society, AUS, 157163. Google ScholarGoogle Scholar
  89. [89] Patitsas Elizabeth, Craig Michelle, and Easterbrook Steve. 2013. Comparing and contrasting different algorithms leads to increased student learning. In Proceedings of the 9th Annual International ACM Conference on International Computing Education Research (ICER’13). ACM, New York, NY, 145152. Google ScholarGoogle ScholarDigital LibraryDigital Library
  90. [90] Pérez-Schofield José Baltasar García, García-Rivera Matías, Ortin Francisco, and Lado María J.. 2019. Learning memory management with C-Sim: A C-based visual tool. Comput. Appl. Eng. Edu. 27, 5 (2019), 12171235. Google ScholarGoogle ScholarCross RefCross Ref
  91. [91] Perkins D. N., Hancock Chris, Hobbs Renee, Martin Fay, and Simmons Rebecca. 1986. Conditions of learning in novice programmers. J. Edu. Comput. Res. 2, 1 (1986), 3755. Google ScholarGoogle ScholarCross RefCross Ref
  92. [92] Price Thomas W., Williams Joseph Jay, Solyst Jaemarie, and Marwan Samiha. 2020. Engaging students with instructor solutions in online programming homework. In Proceedings of the CHI Conference on Human Factors in Computing Systems. ACM, New York, NY, 17. Google ScholarGoogle ScholarDigital LibraryDigital Library
  93. [93] Qian Yizhou and Lehman James. 2017. Students’ misconceptions and other difficulties in introductory programming: A literature review. ACM Trans. Comput. Edu. 18, 1, Article 1 (Oct.2017), 24 pages. Google ScholarGoogle ScholarDigital LibraryDigital Library
  94. [94] Quilici Jill L. and Mayer R.. 1996. Role of examples in how students learn to categorize statistics word problems. J. Edu. Psychol. 88, 1 (1996), 144161. Google ScholarGoogle ScholarCross RefCross Ref
  95. [95] Raj Adalbert Gerald Soosai, Gu Pan, Zhang Eda, R. Arokia Xavier Annie, Williams Jim, Halverson Richard, and Patel Jignesh M.. 2020. Live-coding vs static code examples: Which is better with respect to student learning and cognitive load? In Proceedings of the 22nd Australasian Computing Education Conference (ACE’20). ACM, New York, NY, 152159. Google ScholarGoogle ScholarDigital LibraryDigital Library
  96. [96] Raj Adalbert Gerald Soosai, Patel Jignesh M., Halverson Richard, and Halverson Erica Rosenfeld. 2018. Role of live-coding in learning introductory programming. In Proceedings of the 18th Koli Calling International Conference on Computing Education Research (KoliCalling’18). ACM, New York, NY, Article 13, 8 pages. Google ScholarGoogle ScholarDigital LibraryDigital Library
  97. [97] Rajala Teemu, Kaila Erkki, Holvitie Johannes, Haavisto Riku, Laakso Mikko-Jussi, and Salakoski Tapio. 2011. Comparing the collaborative and independent viewing of program visualizations. In Proceedings of the Frontiers in Education Conference (FIE’11). IEEE Computer Society, F3G–1-1–F3G–7. Google ScholarGoogle ScholarDigital LibraryDigital Library
  98. [98] Reed Stephen K., Dempster Alexandra, and Ettinger Michael. 1985. Usefulness of analogous solutions for solving algebra word problems. J. Exper. Psychol.: Learn., Mem. Cogn. 11, 1 (Jan.1985), 106125. Google ScholarGoogle ScholarCross RefCross Ref
  99. [99] Renkl Alexander. 1997. Learning from worked-out examples: A study on individual differences. Cogn. Sci. 21, 1 (1997), 129. Google ScholarGoogle ScholarCross RefCross Ref
  100. [100] Renkl Alexander. 2014. Toward an instructionally oriented theory of example-based learning. Cogn. Sci. 38 (2014), 137. Google ScholarGoogle ScholarCross RefCross Ref
  101. [101] Rittle-Johnson Bethany, Loehr Abbey M., and Durkin Kelley. 2017. Promoting self-explanation to improve mathematics learning: A meta-analysis and instructional design principles. ZDM Math. Edu. 49 (2017), 599611. Google ScholarGoogle ScholarCross RefCross Ref
  102. [102] Rittle-Johnson Bethany, Star Jon R., and Durkin Kelley. 2012. Developing procedural flexibility: Are novices prepared to learn from comparing procedures? Brit. J. Edu. Psychol. 82, Pt 3 (2012), 436–55.Google ScholarGoogle Scholar
  103. [103] Robins A., Rountree J., and Rountree N.. 2003. Learning and teaching programming: A review and discussion. Comput. Sci. Edu. 13, 2 (2003), 137172.Google ScholarGoogle ScholarCross RefCross Ref
  104. [104] Ross Brian H.. 1987. This is like that: The use of earlier problems and the separation of similarity effects. J. Exper. Psychol.: Learn., Mem. Cogn. 13, 4 (Oct.1987), 629639. Google ScholarGoogle ScholarCross RefCross Ref
  105. [105] Rubin Marc J.. 2013. The effectiveness of live-coding to teach introductory programming. In Proceedings of the 44th ACM Technical Symposium on Computer Science Education (SIGCSE’13). ACM, New York, NY, 651656. Google ScholarGoogle ScholarDigital LibraryDigital Library
  106. [106] Sajaniemi Jorma, Kuittinen Marja, and Tikansalo Taina. 2008. A study of the development of students’ visualizations of program state during an elementary object-oriented programming course. J. Edu. Res. Comput. 7, 4, Article 3 (Jan.2008), 31 pages. Google ScholarGoogle ScholarDigital LibraryDigital Library
  107. [107] Schunk Dale, Hanson Antoinette R., and Cox Paula D.. 1987. Peer-model attributes and children’s achievement behaviors. J. Edu. Psychol. 79, 1 (1987), 5461. Google ScholarGoogle ScholarCross RefCross Ref
  108. [108] Selvaraj Ana, Zhang Eda, Porter Leo, and Raj Adalbert Gerald Soosai. 2021. Live coding: A review of the literature. In Proceedings of the 26th ACM Conference on Innovation and Technology in Computer Science Education. ACM, New York, NY, 164170. Google ScholarGoogle ScholarDigital LibraryDigital Library
  109. [109] Najar Amir Shareghi and Mitrovic Antonija. 2013. Examples and tutored problems: How can self-explanation make a difference to learning? In Artificial Intelligence in Education, Lane H. Chad, Yacef Kalina, Mostow Jack, and Pavlik Philip (Eds.). Springer, Berlin, 339348. Google ScholarGoogle Scholar
  110. [110] Shi Nianfeng, Min Zhiyu, and Zhang Ping. 2017. Effects of visualizing roles of variables with animation and IDE in novice program construction. Telemat. Informat. 34, 5 (2017), 743754. Google ScholarGoogle ScholarDigital LibraryDigital Library
  111. [111] Sirkiä Teemu and Sorva Juha. 2015. How do students use program visualizations within an interactive ebook? In Proceedings of the 11th Annual International Conference on International Computing Education Research (ICER’15). ACM, New York, NY, 179188. Google ScholarGoogle ScholarDigital LibraryDigital Library
  112. [112] Soloway Elliot and Ehrlich Kate. 1984. Empirical studies of programming knowledge. IEEE Trans. Softw. Eng. SE-10, 5 (1984), 595609. Google ScholarGoogle ScholarDigital LibraryDigital Library
  113. [113] Sorva Juha. 2013. Notional machines and introductory programming education. ACM Trans. Comput. Edu. 31, 2, Article 8 (July2013), 31 pages. Google ScholarGoogle ScholarDigital LibraryDigital Library
  114. [114] Sorva Juha, Karavirta Ville, and Malmi Lauri. 2013. A review of generic program visualization systems for introductory programming education. ACM Trans. Comput. Edu. 13, 4, Article 15 (Nov.2013), 64 pages. Google ScholarGoogle ScholarDigital LibraryDigital Library
  115. [115] Stephenson Ben. 2019. Coding demonstration videos for CS1. In Proceedings of the 50th ACM Technical Symposium on Computer Science Education (SIGCSE’19). ACM, New York, NY, 105111. Google ScholarGoogle ScholarDigital LibraryDigital Library
  116. [116] Sweller John. 2011. Cognitive load theory, in Psychology of Learning and Motivation, Vol. 55. Academic Press, 3776. Google ScholarGoogle ScholarCross RefCross Ref
  117. [117] Sweller John, Ayres Paul, and Kalyuga Slava. 2011. The split-attention effect. In Cognitive Load Theory. Springer, New York, NY, 111128. Google ScholarGoogle ScholarCross RefCross Ref
  118. [118] Sweller John and Cooper Graham A.. 1985. The use of worked examples as a substitute for problem solving in learning algebra. Cogn. Instruct. 2, 1 (1985), 5989.Google ScholarGoogle ScholarCross RefCross Ref
  119. [119] Trafton John G. and Reiser Brian J.. 1993. Studying examples and solving problems: Contributions to skill acquisition. In Proceedings of the 15th Annual Conference of the Cognitive Science Society. Lawrence Erlbaum Associates, 10171022.Google ScholarGoogle Scholar
  120. [120] Urquiza-Fuentes Jaime and Velázquez-Iturbide J. Ángel. 2013. Toward the effective use of educational program animations: The roles of student’s engagement and topic complexity. Comput. Edu. 67 (2013), 178192. Google ScholarGoogle ScholarCross RefCross Ref
  121. [121] Vainio Vesa and Sajaniemi Jorma. 2007. Factors in novice programmers’ poor tracing skills. In Proceedings of the 12th Annual SIGCSE Conference on Innovation and Technology in Computer Science Education (ITiCSE’07). ACM, New York, NY, 236240. Google ScholarGoogle ScholarDigital LibraryDigital Library
  122. [122] Gog Tamara van, Kester Liesbeth, and Paas Fred. 2011. Effects of worked examples, example-problem, and problem-example pairs on novices’ learning. Contemp. Edu. Psychol. 36, 3 (2011), 212218. Google ScholarGoogle ScholarCross RefCross Ref
  123. [123] Gog Tamara van and Rummel Nikol. 2010. Example-Based Learning: Integrating Cognitive and Social-Cognitive Research Perspectives. Edu. Psychol. Rev. 22, 2 (June2010), 155174. Google ScholarGoogle ScholarCross RefCross Ref
  124. [124] Gog Tamara van, Rummel Nikol, and Renkl Alexander. 2019. Learning how to solve problems by studying examples. In The Cambridge Handbook of Cognition and Education, Dunlosky John and Rawson Katherine A.Editors (Eds.). Cambridge University Press, 183208.Google ScholarGoogle Scholar
  125. [125] Merriënboer Jeroen J. G. van, Schuurman Jan Gerrit, Croock Marcel B. M. de, and Paas Fred. 2002. Redirecting learners attention during training: Effects on cognitive load, transfer test performance and training efficiency. Learn. Instruct. 12, 1 (2002), 1137. Google ScholarGoogle ScholarCross RefCross Ref
  126. [126] Merriënboer Jeroen J. G. Van. 1990. Strategies for programming instruction in high school program: Program completion vs. program generation. J. Edu. Comput. Res. 6, 3 (1990), 265285. Google ScholarGoogle ScholarCross RefCross Ref
  127. [127] Merriënboer Jeroen J. G. van and Croock Marcel B. M. de. 1992. Strategies for computer-based programming instruction: Program completion vs. program generation. J. Edu. Comput. Res. 8, 3 (1992), 365394. Google ScholarGoogle ScholarCross RefCross Ref
  128. [128] VanLehn Kurt. 1996. Cognitive skill acquisition. Annu. Rev. Psychol. 47, 1 (1996), 513539. . arXiv:https://doi.org/10.1146/annurev.psych.47.1.513PMID: 15012487.Google ScholarGoogle ScholarCross RefCross Ref
  129. [129] VanLehn Kurt. 1998. Analogy events: How examples are used during problem solving. Cogn. Sci. 22, 3 (1998), 347388.Google ScholarGoogle ScholarCross RefCross Ref
  130. [130] VanLehn Kurt. 1999. Rule-learning events in the acquisition of a complex skill: An evaluation of cascade. J. Learn. Sci. 8, 1 (1999), 71125. Google ScholarGoogle ScholarCross RefCross Ref
  131. [131] Wang Peng, Bednarik Roman, and Moreno Andrés. 2012. During automatic program animation explanations after animations have greater impact than before animations. In Proceedings of the 12th Koli Calling International Conference on Computing Education Research (KoliCalling’12). ACM, New York, NY, 100109. Google ScholarGoogle ScholarDigital LibraryDigital Library
  132. [132] Wang Wengran, Kwatra Archit, Skripchuk James, Gomes Neeloy, Milliken Alexandra, Martens Chris, Barnes Tiffany, and Price Thomas. 2021. Novices’ learning barriers when using code examples in open-ended programming. In Proceedings of the 26th ACM Conference on Innovation and Technology in Computer Science Education V. 1. ACM, New York, NY, 394400. Google ScholarGoogle ScholarDigital LibraryDigital Library
  133. [133] Wang Wengran, Rao Yudong, Zhi Rui, Marwan Samiha, Gao Ge, and Price Thomas W.. 2020. Step Tutor: Supporting students through step-by-step example-based feedback. In Proceedings of the ACM Conference on Innovation and Technology in Computer Science Education (ITiCSE’20). ACM, New York, NY, 391397. Google ScholarGoogle ScholarDigital LibraryDigital Library
  134. [134] Weber Gerhard. 1996. Individual selection of examples in an intelligent learning environment. Int. J. Artific. Intell. Edu. 7, 1 (1996), 331.Google ScholarGoogle ScholarDigital LibraryDigital Library
  135. [135] Weinman Nathaniel, Fox Armando, and Hearst Marti A.. 2021. Improving instruction of programming patterns with faded Parsons problems. In Proceedings of the CHI Conference on Human Factors in Computing Systems (CHI’21). ACM, New York, NY, Article 53, 4 pages. Google ScholarGoogle ScholarDigital LibraryDigital Library
  136. [136] Wylie Ruth and Chi Michelene T. H.. 2014. The self-explanation principle in multimedia learning. In The Cambridge Handbook of Multimedia Learning (2nd ed.), Mayer Richard E. (Ed.). Cambridge University Press, 413432. Google ScholarGoogle ScholarCross RefCross Ref
  137. [137] Xie Benjamin, Loksa Dastyni, Nelson Greg L., Davidson Matthew J., Dong Dongsheng, Kwik Harrison, Tan Alex Hui, Hwa Leanne, Li Min, and Ko Amy J.. 2019. A theory of instruction for introductory programming skills. Comput. Sci. Edu. 29, 2–3 (2019), 205253. Google ScholarGoogle ScholarCross RefCross Ref
  138. [138] Xie Benjamin, Nelson Greg L., and Ko Amy J.. 2018. An explicit strategy to scaffold novice program tracing. In Proceedings of the 49th ACM Technical Symposium on Computer Science Education (SIGCSE’18). ACM, New York, NY, 344349. Google ScholarGoogle ScholarDigital LibraryDigital Library
  139. [139] Yang Jeong, Lee Young, and Chang Kai-Hsiung. 2018. Evaluations of JaguarCode: A web-based object-oriented programming environment with static and dynamic visualization. J. Syst. Softw. 145 (2018), 147163. Google ScholarGoogle ScholarCross RefCross Ref
  140. [140] Zavgorodniaia Albina, Hellas Arto, Seppälä Otto, and Sorva Juha. 2020. Should explanations of program code use audio, text, or both? A replication study. In Proceedings of the 20th Koli Calling International Conference on Computing Education Research (KoliCalling’20). ACM, New York, NY, Article 5, 10 pages. Google ScholarGoogle ScholarDigital LibraryDigital Library
  141. [141] Zavgorodniaia Albina, Tilanterä Artturi, Korhonen Ari, Seppälä Otto, Hellas Arto, and Sorva Juha. 2021. Algorithm visualization and the elusive modality effect. In Proceedings of the 17th ACM Conference on International Computing Education Research (ICER’21). ACM, New York, NY, 368378. Google ScholarGoogle ScholarDigital LibraryDigital Library
  142. [142] Zhi Rui, Chi Min, Barnes Tiffany, and Price Thomas W.. 2019. Evaluating the effectiveness of Parsons problems for block-based programming. In Proceedings of the ACM Conference on International Computing Education Research (ICER’19). ACM, New York, NY, 5159. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. A Review of Worked Examples in Programming Activities

    Recommendations

    Comments

    Login options

    Check if you have access through your login credentials or your institution to get full access on this article.

    Sign in

    Full Access

    • Published in

      cover image ACM Transactions on Computing Education
      ACM Transactions on Computing Education  Volume 23, Issue 1
      March 2023
      396 pages
      EISSN:1946-6226
      DOI:10.1145/3578368
      • Editor:
      • Amy J. Ko
      Issue’s Table of Contents

      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      • Published: 29 December 2022
      • Online AM: 2 September 2022
      • Accepted: 4 August 2022
      • Revised: 20 May 2022
      • Received: 31 July 2021
      Published in toce Volume 23, Issue 1

      Permissions

      Request permissions about this article.

      Request Permissions

      Check for updates

      Qualifiers

      • research-article
      • Refereed

    PDF Format

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Full Text

    View this article in Full Text.

    View Full Text

    HTML Format

    View this article in HTML Format .

    View HTML Format