Abstract
Research Methods Tutor (RMT) is a dialogue-based intelligent tutoring system for use in conjunction with undergraduate psychology research methods courses. RMT includes five topics that correspond to the curriculum of introductory research methods courses: ethics, variables, reliability, validity, and experimental design. We evaluated the effectiveness of the RMT system in the classroom using a nonequivalent control group design. Students in three classes (n = 83) used RMT, and students in two classes (n = 53) did not use RMT. Results indicated that the use of RMT yielded strong learning gains of 0.75 standard deviations above classroom instruction alone. Further, the dialogue-based tutoring condition of the system resulted in higher gains than did the textbook-style condition (CAI version) of the system. Future directions for RMT include the addition of new topics and tutoring elements.
Article PDF
Similar content being viewed by others
References
Anderson, J. R., Corbett, A. T., Koedinger, K. R., & Pelletier, R. (1995). Cognitive tutors: Lessons learned. Journal of the Learning Sciences, 4, 167–207.
Bloom, B. S. (1956). Taxonomy of educational objectives: The classification of educational goals. Essex, U.K.: Longman.
Bloom, B. S. (1984). The 2 sigma problem: The search for methods of group instruction as effective as one-to-one tutoring. Educational Researcher, 13, 4–16.
Chi, M. T. H., Siler, S. A., Jeong, H., Yamauchi, T., & Hausmann, R. G. (2001). Learning from human tutoring. Cognitive Science, 25, 471–533.
Clark, R. C., & Mayer, R. E. (2003). e-Learning and the science of instruction: Proven guidelines for consumers and designers of multimedia learning. San Francisco: Pfeiffer.
Corbett, A. T., & Anderson, J. R. (1991, April). Feedback control and learning to program with the CMU LISP tutor. Paper presented at the annual meeting of the American Educational Research Association, Chicago.
Graesser, A. C., Jackson, G. T., Mathews, E. C., Mitchell, H. H., Olney, A., Ventura, M., et al. (2003). Why/AutoTutor: A test of learning gains from a physics tutor with natural language dialog. In R. Alterman & D. Kirsh (Eds.), Proceedings of the 25th Annual Meeting of the Cognitive Science Society (pp. 474–479). Mahwah, NJ: Erlbaum.
Graesser, A. C., Person, N. K., & Magliano, J. P. (1995). Collaborative dialogue patterns in naturalistic one-to-one tutoring. Applied Cognitive Psychology, 9, 495–522.
Koedinger, K. R., Anderson, J. R., Hadley, W. H., & Mark, M. A. (1997). Intelligent tutoring goes to school in the big city. International Journal of Artificial Intelligence in Education, 8, 30–43.
Landauer, T. K., Foltz, P. W., & Laham, D. (1998). An introduction to latent semantic analysis. Discourse Processes, 25, 259–284.
Lane, H. C., & VanLehn, K. (2005). Teaching the tacit knowledge of programming to novices with natural language tutoring. Computer Science Education, 15, 183–201.
Lesgold, A., Lajoie, S., Bunzo, M., & Eggan, G. (1992). Sherlock: A coached practice environment for an electronics troubleshooting job. In J. H. Larkin & R. W. Chabay (Eds.), Computer assisted instruction and intelligent tutoring systems: Shared goals and complementary approaches (pp. 201–238). Hillsdale, NJ: Erlbaum.
Moreno, R. (2004). Animated pedagogical agents in educational technology. Educational Technology, 44, 23–30.
National Reading Panel (2000). Teaching children to read: An evidence-based assessment of the scientific research literature on reading and its implications for reading instruction (NIH Publication No. 00-4754). Washington, DC: National Institute of Child Health & Human Development.
Salvucci, D. D., & Anderson, J. R. (1998). Tracing eye movement protocols with cognitive process models. In Proceedings of the Twentieth Annual Conference of the Cognitive Science Society (pp. 923–928). Hillsdale, NJ: Erlbaum.
VanLehn, K., Graesser, A. C., Jackson, G. T., Jordan, P., Olney, A., & Rosé, C. P. (2007). When are tutorial dialogues more effective than reading? Cognitive Science, 31, 3–62.
Wiemer-Hastings, P., Graesser, A. C., Harter, D., & the Tutoring Research Group (1998). The foundations and architecture of Auto-Tutor. In Proceedings of the 4th International Conference on Intelligent Tutoring Systems (pp. 334–343). Berlin: Springer.
Wood, D., & Middleton, D. (1975). A study of assisted problem-solving. British Journal of Psychology, 66, 181–191.
Wood, D., Wood, H., & Middleton, D. (1978). An experimental evaluation of four face-to-face teaching strategies. International Journal of Behavioral Development, 1, 131–147.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Arnott, E., Hastings, P. & Allbritton, D. Research Methods Tutor: Evaluation of a dialogue-based tutoring system in the classroom. Behavior Research Methods 40, 694–698 (2008). https://doi.org/10.3758/BRM.40.3.694
Received:
Accepted:
Issue Date:
DOI: https://doi.org/10.3758/BRM.40.3.694