Abstract
The main learning activity provided by intelligent tutoring systems is problem solving, although several recent projects investigated the effectiveness of combining problem solving with worked examples. Previous research has shown that learning from examples is an effective learning strategy, especially for novice learners. A worked example provides step-by-step explanations of how a problem is solved. Many studies have compared learning from examples to unsupported problem solving, and suggested presenting worked examples to students in the initial stages of learning, followed by problem solving once students have acquired enough knowledge. This paper presents a study in which we compare a fixed sequence of alternating worked examples and tutored problem solving with a strategy that adapts learning tasks to students’ needs. The adaptive strategy determines the type of the task (a worked example, a faded example or a problem to be solved) based on how much assistance the student received on the previous problem. The results show that students in the adaptive condition learnt significantly more than their peers who were presented with a fixed sequence of worked examples and problem solving. Novices from the adaptive condition learnt faster than novices from the control group, while the advanced students from the adaptive condition learnt more than their peers from the control group.
Similar content being viewed by others
Notes
Cognitive efficiency \(=\) Performance / Mental Effort Rating.
Please note that P is scaled to the range [1,9].
Normalised learning gain \(=\) (post test—pre test) / (max score—pre test).
References
Aleven, V., Koedinger, K.R.: An effective metacognitive strategy: learning by doing and explaining with a computer-based Cognitive Tutor. Cognit. Sci. 26(2), 147–179 (2002)
Anthony, L., Mankoff, J., Mitchell, T., Gross, M.: Developing Handwriting-based Intelligent Tutors to Enhance Mathematics Learning. Doctoral Dissertation, Carnegie Mellon University (2008)
Atkinson, R.K., Derry, S.J., Renkl, A., Wortham, D.: Learning from examples: instructional principles from the worked examples research. Rev. Educ. Res. 70(2), 181–214 (2000)
Brown, A.L., Kane, M.J.: Preschool children can learn to transfer: learning to learn and learning from example. Cognit. Psychol. 20(4), 493–523 (1988)
Brusilovsky, P.: Adaptive navigation support in educational hypermedia: the role of student knowledge level and the case for meta-adaptation. Br. J. Educ. Technol. 34(4), 487–497 (2003)
Burow, R., Weber, G.: Example explanation in learning environments. In: Frasson, C., Gauthier, G., Lesgoed, A. (eds.) Proceedings of the 3rd International Conference Intelligent Tutoring Systems, Montreal, Canada, pp. 457–465. Springer, Berlin (1996)
Chandler, P., Sweller, J.: Cognitive load theory and the format of instruction. Cognit. Instr. 8(4), 293–332 (1991)
Chi, M., De Leeuw, N., Chiu, M.H., LaVancher, C.: Eliciting self-explanations improves understanding. Cognit. Sci. 18(3), 439–477 (1994)
Chi, M.T.H., Bassok, M., Lewis, M.W., Reimann, P., Glaser, R.: Self-explanations: how students study and use examples in learning to solve problems. Cognit. Sci. 13(2), 145–182 (1989)
Chi, M.T.H., VanLehn, K.A.: The content of physics self-explanations. J. Learn. Sci. 1(1), 69–105 (1991)
Clark, R.C., Nguyen, F., Sweller, J.: Efficiency in learning: evidence-based guidelines to manage cognitive load. Jossey-Bass, San Francisco (2006)
Corbett, A., Kauffman, L., MacLaren, B., Wagner, A., Jones, E.: A cognitive tutor for genetics problem solving: learning gains and student modeling. J Educ Comput Res. 42, 219–239 (2010)
Corbett, A., MacLaren, B., Wagner, A., Kauffman, L., Mitchell, A., d Baker, R.S.: Differential Impact of Learning Activities Designed to Support Robust Learning in the Genetics Cognitive Tutor. In: Lane, H.C., Yacef, K., Mostow, J. (Eds.) Proceedings of 16th Artificial Intelligence in Education, Memphis, pp. 319–328 (2013)
Davidovic, A., Warren, J., Trichina, E.: Learning benefits of structural example based adaptive tutoring systems. IEEE Trans Educ. 46(2), 241–251 (2003)
Deeks, A.: Web Based Assignments in Structural Analysis. Nanyang Technological University, Deeks, Centre for Educational Development (2000)
Hattie, J.: Visible Learning: A Synthesis of over 800 Meta-Analyses Relating to Achievement. Routledge, New York (2009)
Hausmann, R., Nokes, T., VanLehn, K., Gershman, S.: The design of self-explanation prompts: the fit hypothesis. In: Proceedings of 31st Annual Conference of the Cognitive Science Society, pp. 2626–2631 (2009)
Hilbert, T.S., Renkl, A.: Learning how to use a computer-based concept-mapping tool: self-explaining examples helps. Comput. Hum. Behav. 25(2), 267–274 (2009)
Hsiao, I.H., Bakalov, F., Brusilovsky, P., König-Ries, B.: Progressor: social navigation support through open social student modeling. New Rev. Hypermed. Multimed. 19(2), 112–131 (2013)
Kalyuga, S.: Expertise reversal effect and its implications for learner-tailored instruction. Educ. Psychol. Rev. 19, 509–539 (2007)
Kalyuga, S., Chandler, P., Tuovinen, J., Sweller, J.: When problem solving is superior to studying worked examples. Educ. Psychol. 93(3), 579–588 (2001)
Kalyuga, S., Sweller, J.: Rapid dynamic assessment of expertise to improve the efficiency of adaptive e-learning. Educ. Technol. Res. Dev. 53(3), 83–93 (2005)
Karampiperis, P., Sampson, D.G.: Adaptive Instructional Planning Using Ontologies. In: Proceedings of 4th IEEE International Conference Advanced Learning Technologies Joensuu, Vol. 4, pp. 126-1 (2004)
Kim, R.S., Weitz, R., Heffernan, N.T., Krach, N.: Tutored Problem Solving vs . “ Pure ” Worked Examples. In: Proceedings of 31st Annual Conference of the Cognitive Science Society, pp. 3121–3126. Cognitive Science Society, Austin, TX (2007)
Koedinger, K., Aleven, V.: Exploring the assistance dilemma in experiments with cognitive tutors. Educ. Psychol. Rev. 19(3), 239–264 (2007)
Kostons, D., van Gog, T., Paas, F.: Self-assessment and task selection in learner-controlled instruction: differences between effective and ineffective learners. Comput. Educ. 54(4), 932–940 (2010)
Kyun, S., Kalyuga, S., Sweller, J.: The effect of worked examples when learning to write essays in English literature. J. Exp. Educ 81(3), 385–408
Martin, B., Mitrovic, A.: Using learning curves to mine student models. In: Ardissono, L., Brna, P., Mitrovic, A. (eds.) Proceedings of 10th International Conference User Modeling, pp. 79–88. Springer, Edinburgh (2005)
Masthoff, J.: Design and evaluation of a navigation agent with a mixed locus of control. In: Cerri, S.A., Gouarderes, G., Paraguacu, F. (eds.) Proceedings of International Conference Intelligent Tutoring Systems, pp. 982–991. Springer, Berlin (2002)
McLaren, B.M., van Gog, T., Ganoe, C., Karabinos, M., Yaron, D.: The efficiency of worked examples compared to erroneous examples, tutored problem solving, and problem solving in classroom experiments. Comput. Hum. Behav. 55, 87–99 (2016)
McLaren, B., Isotani, S.: When Is It Best to Learn with All Worked Examples? In: Biswas, G., Bull, S., Kay, J., Mitrovic, A. (eds.) Proceedings of 15th International Conference Artificial Intelligence in Education, pp. 222–229. Springer, Berlin (2011)
McLaren, B.M., Lim, S.-J., Koedinger, K.R.: When and how often should worked examples be given to students? New results and a summary of the current state of research. In: Proceedings of Conference Cognitive Science Society, pp. 2176–2181. Cognitive Science Society, Austin, TX (2008)
Mitrovic, A.: An intelligent SQL tutor on the web. Int. J. Artif. Intell. Educ. 13(2–4), 173–197 (2003)
Mitrovic, A., Martin, B.: Evaluating the effectiveness of feedback in SQL-Tutor. In: Kinshuk, Jesshope, C., Okamoto, T. (eds.) Proceedings International Workshop on Advanced Learning Technologies, pp. 143–144. Palmerston North (2000)
Mitrovic, A., Martin, B.: Scaffolding and fading problem selection in SQL-Tutor. In: Hoppe, U., Verdejo, F., Kay, J. (eds.) Proceedings of 11th International Conference Artificial Intelligence in Education, pp. 479–481. Australia, Sydney (2003)
Mitrovic, A., Weerasinghe, A.: Revisiting Ill-Definedness and the Consequences for ITSs. In: Dimitrova, V., Mizoguchi, R. (eds.) Proceedings of 14th International Conference on Artificial Intelligence in Education, pp. 375–382. IOS Press, Brighton (2009)
Najar, A.S., Mitrovic, A.: Do novices and advanced students benefit differently from worked examples and ITS? In: Wong, L.H., Liu, C.-C., Hirashima, T., Sumedi, P., Lukman, M. (eds.) Proceedings of International Conference ICCE, pp. 20–29. Bali (2013a)
Najar, A.S., Mitrovic, A.: In: Yacef, K., et al. (eds.) Examples and Tutored Problems: How can Self-Explanation Make a Difference to Learning? Proceedings of 16th International Conference AIED, pp. 339–348. Springer, Heidelberg (2013b)
Najar, A.S., Mitrovic, A.: Examples and tutored problems: is alternating examples and problems the best instructional strategy? Res. Pract. Technol. Enhanc. Learn. 9(3), 439–459 (2014)
Najar, A.S., Mitrovic, A., McLaren, B.M.: Adaptive support versus alternating worked examples and tutored problems: which leads to better learning? In: Dimitrova, V., et al. (eds.) User Modeling, Adaptation, and Personalization, pp. 171–182. Springer International Publishing, Aalborg (2014)
Najar, A.S., Mitrovic, A., McLaren, B.M.: Examples and tutored problems: adaptive support using assistance scores. In: Proceedings of 24th International Joint Conference on Artificial Intelligence Buenos Aires, pp. 4317-4323. AAAI Press (2015)
Paas, F.G.W.C., Van Merrienboer, J.J.G.: The efficiency of instructional conditions: an approach to combine mental effort and performance measures. Hum. Factors 35, 737–743 (1993)
Peachey, D., McCalla, G.: Using planning techniques in intelligent tutoring systems. Int. J. Man-Mach. Stud. 24, 77–98 (1986)
Roediger, H.L., Karpicke, J.D.: Test-enhanced learning taking memory tests improves long-term retention. Psychol. Sci. 17(3), 249–255 (2006)
Rourke, A., Sweller, J.: The worked-example effect using ill-defined problems: learning to recognise designers’ styles. Learn. Instr. 19(2), 185–199 (2009)
Salden, R., Aleven, V., Renkl, A., Schwonke, R.: Worked examples and tutored problem solving: redundant or synergistic forms of support? Topics Cognit. Sci. 1(1), 203–213 (2009)
Salden, R., Koedinger, K., Renkl, A., Aleven, V., McLaren, B.: Accounting for beneficial effects of worked examples in tutored problem solving. Educ. Psychol. Rev. 22(4), 379–392 (2010)
Santos, O.C.: (Ed.) Educational Recommender Systems and Technologies: Practices and Challenges: Practices and Challenges. IGI Global, Hershey (2011)
Schwonke, R., Renkl, A., Krieg, C., Wittwer, J., Aleven, V., Salden, R.: The worked-example effect: not an artefact of lousy control conditions. Comput. Hum. Behav. 25, 258–266 (2009)
Schworm, S., Renkl, A.: Computer-supported example-based learning: when instructional explanations reduce self-explanations. Comput. Educ. 46, 426–445 (2006)
Sosnovsky, S., Brusilovsky, P.: Evaluation of topic-based adaptation and student modeling in QuizGuide. User Model. User-Adapt. Interact. 25(4), 371–424 (2015)
Sweller, J.: The worked example effect and human cognition. Learn. Instr. 16(2), 165–169 (2006)
Sweller, J., Ayres, P., Kalyuga, S.: Cognitive Load Theory. Springer, New York (2011)
Sweller, J., Cooper, G.A.: The use of worked examples as a substitute for problem solving in learning algebra. Cognit. Instr. 2, 59–89 (1985)
Trafton, J. G., Reiser, B. J.: The Contributions of Studying Examples and Solving Problems to Skill Acquisition. In: Proceedings of 15th Annual Conference of the Cognitive Science Society, ACM Press, pp. 1017–1022 (1993)
Van Gog, T.: Effects of identical example-problem and problem-example pairs on learning. Comput. Educ. 57(2), 1775–1779 (2011)
Van Gog, T., Kester, L., Paas, F.: Effects of worked examples, example-problem, and problem-example pairs on novices’ learning. Contemp. Educ. Psychol. 36(3), 212–218 (2011)
Van Gog, T., Paas, F.: Instructional efficiency: revisiting the original construct in educational research. Educ. Psychol. 43(1), 16–26 (2008)
Van Gog, T., Rummel, N.: Example-based learning: integrating cognitive and social-cognitive research perspectives. Educ. Psychol. Rev. 22(2), 155–174 (2010)
Vassileva, J.: Reactive instructional planning to support interacting teaching strategies. In: Proceedings of 7th Conference Artificial Intelligence in Education, pp. 334–342 (1995)
Webb, N.M.: Peer interaction and learning in small groups. Educ. Res. 13(1), 21–39 (1989)
Weber, G.: Examples and remindings in a case-based help system. In: European Workshop on Advances in Case-Based Reasoning, pp. 165–177. Springer, Berlin (1994)
Weber, G., Brusilovsky, P.: ELM-ART-an interactive and intelligent web-based electronic textbook. Int. J. Artif. Intell. Educ. 26(1), 72–81 (2016)
Weerasinghe, A., Mitrovic, A.: Facilitating deep learning through self-explanation in an open-ended domain. Knowl.-Based Intell. Eng. Syst. 10(1), 3–19 (2006)
Weerasinghe, A., Mitrovic, A., Martin, B.: Towards individualized dialogue support for ill-defined domains. Int. J. Artifici. Intell. Educ. 19(4), 357–379 (2009)
Weerasinghe, A., Mitrovic, A.: Facilitating adaptive tutorial dialogues in EER-tutor. In: International Conference on Artificial Intelligence in Education, pp. 630–631. Springer, Berlin (2011)
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Najar, A.S., Mitrovic, A. & McLaren, B.M. Learning with intelligent tutors and worked examples: selecting learning activities adaptively leads to better learning outcomes than a fixed curriculum. User Model User-Adap Inter 26, 459–491 (2016). https://doi.org/10.1007/s11257-016-9181-y
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11257-016-9181-y