Abstract
Asking questions in a context relevant manner is a critical behavior for intelligent tutoring systems; however even within a single pedagogy there may be numerous valid strategies. This paper explores the use of supervised ranking models to rank candidate questions in the context of tutorial dialogues. By training models on individual and aggregate judgments from experienced tutors, we learn to reproduce individual and average preferences in questioning. Analysis of our models’ performance across different tutors highlights differences in individual teaching preferences and illustrates the impact of surface form, semantic and pragmatic features for modeling variations in tutoring styles. This work has implications for dialogue system design and provides a natural starting point towards creating tunable and customizable tutorial dialogue interactions.
This work was supported by grants from the NSF (DRL-0733322, DRL-0733323), the IES (R3053070434) and the DARPA GALE program (HR0011-06-C-0022).
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Ai, H., Litman, D.: Assessing dialog system user simulation evaluation measures using human judges. In: Proc. of ACL 2008: HLT, Columbus, Ohio, USA (June 2008)
Beck, I.L., McKeown, M.G., Worthy, J., Sandora, C.A., Kucan, L.: Questioning the author: A year-long classroom implementation to engage students with text. The Elementary School Journal 96(4), 387–416 (1996)
Becker, L., Ward, W., van Vuuren, S., Palmer, M.: DISCUSS: A dialogue move taxonomy layered over semantic representations. In: Proc. IWCS 2011, Oxford, England, January 12-14 (2011)
Boyer, K., Ha, E., Wallis, M., Phillips, R., Vouk, M., Lester, J.: Discovering tutorial dialogue strategies with hidden markov models. In: Proc. of AIED 2009, Brighton, U.K, pp. 141–148 (2009)
Boyer, K., Lahti, W., Phillips, R., Wallis, M.D., Vouk, M.A., Lester, J.C.: An empirically derived question taxonomy for task-oriented tutorial dialogue. In: Proc. of the 2nd WS on Question Generation, Brighton, U.K., pp. 9–16 (2009)
Buckley, M., Wolska, M.: A classification of dialogue actions in tutorial dialogue. In: Proc. of COLING 2008, pp. 73–80. ACL (2008)
Bunt, H.C.: The DIT++ taxonomy for functional dialogue markup. In: Proc. EDAML 2009 (2009)
Chi, M., Jordan, P., VanLehn, K., Hall, M.: Reinforcement learning-based feature selection for developing pedagogically effective tutorial dialogue tactics. In: Proc. EDM 2008, pp. 258–265 (2008)
Chi, M., VanLehn, K., Litman, D.: Do Micro-Level Tutorial Decisions Matter: Applying Reinforcement Learning to Induce Pedagogical Tutorial Tactics. In: Aleven, V., Kay, J., Mostow, J. (eds.) ITS 2010. LNCS, vol. 6094, pp. 224–234. Springer, Heidelberg (2010)
Chi, M., Siler, S., Jeong, H., Yamauchi, T., Hausman, R.: Learning from tutoring. Cognitive Science 25, 471–533 (2001)
Core, M.G., Allen, J.: Coding dialogs with the DAMSL annotation scheme. In: AAAI Fall Symposium, pp. 28–35 (1997)
Graesser, A., Person, N.: Question asking during tutoring. American Educational Research Journal 31, 104–137 (1994)
Jackson, G., Person, N., Graesser, A.: Adaptive tutorial dialogue in autotutor. In: Proc. of WS on Dialog-based Intelligent Tutoring Systems (2004)
Joachims, T.: Making large-scale svm learning practical. In: Advances in Kernel Methods - Support Vector Learning. MIT Press (1999)
Kendall, M.: A new measure of rank correlation. Biometrika 30(1-2), 81–89 (1938)
Kim, J., Glass, M., Freedman, R., Evens, M.: Learning the use of discourse markers in tutorial dialogue learning the use of discourse markers in tutorial dialogue. In: Proc. of the Cognitive Sciences Society (2000)
Litman, D., Forbes-Riley, K.: Correlations between dialogue acts and learning in spoken tutoring dialogue. Natural Language Engineering 12(2), 161–176 (2006)
Mann, W., Thompson, S.: Rhetorical structure theory: Description and construction of text structures. In: Proc. of the 3rd Int’l WS on Text Generation (1986)
McCallum, A.K.: MALLET: A Machine Learning for Language Toolkit (2002), http://mallet.cs.umass.edu
Nielsen, R.D., Buckingham, J., Knoll, G., Marsh, B., Palen, L.: A taxonomy of questions for question generation. In: Proc. of the WS on the Question Generation Shared Task and Evaluation Challenge (September 2008)
Pilkington, R.: Analysing educational discourse: The discount scheme. Tech. Rep. 99/2, Computer Based Learning Unit, University of Leeds (1999)
Rose, C., Bhembe, D., Siler, S., Srivastava, R., VanLehn, K.: The role of why questions in effective human tutoring. In: Proc. of AIED 2003 (2003)
Tsovaltzi, D., Karagjosova, E.: A view on dialogue move taxonomies for tutorial dialogues. In: Proc. of SIGDIAL 2004, pp. 35–38. ACL (2004)
Tsovaltzi, D., Matheson, C.: Formalising hinting in tutorial dialogues. In: EDILOG: 6th WS on the Semantics and Pragmatics of Dialogue, pp. 185–192 (2001)
Ward, W., Cole, R., Bolaños, D., Buchenroth-Martin, C., Svirsky, E., van Vuuren, S., Weston, T., Zheng, J., Becker, L.: My Science Tutor: A conversational multi-media virtual tutor for elementary school science. ACM Transactions on Speech and Language Processing (TSLP) 7(4) (August 2011)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2012 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Becker, L., Palmer, M., van Vuuren, S., Ward, W. (2012). Learning to Tutor Like a Tutor: Ranking Questions in Context. In: Cerri, S.A., Clancey, W.J., Papadourakis, G., Panourgia, K. (eds) Intelligent Tutoring Systems. ITS 2012. Lecture Notes in Computer Science, vol 7315. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-30950-2_48
Download citation
DOI: https://doi.org/10.1007/978-3-642-30950-2_48
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-30949-6
Online ISBN: 978-3-642-30950-2
eBook Packages: Computer ScienceComputer Science (R0)