Vineet Kumar
ABSTRACT
Intelligent personal assistants (IPAs) and interactive question answering (IQA) systems frequently encounter incomplete follow-up questions. The incomplete follow-up questions only make sense when seen in conjunction with the conversation context: the previous question and answer. Thus, IQA and IPA systems need to utilize the conversation context in order to handle the incomplete follow-up questions and generate an appropriate response. In this work, we present a retrieval based sequence to sequence learning system that can generate the complete (or intended) question for an incomplete follow-up question (given the conversation context). We can train our system using only a small labeled dataset (with only a few thousand conversations), by decomposing the original problem into two simpler and independent problems. The first problem focuses solely on selecting the candidate complete questions from a library of question templates (built offline using the small labeled conversations dataset). In the second problem, we re-rank the selected candidate questions using a neural language model (trained on millions of unlabelled questions independently). Our system can achieve a BLEU score of 42.91, as compared to 29.11 using an existing generation based approach. We further demonstrate the utility of our system as a plug-in module to an existing QA pipeline. Our system when added as a plug-in module, enables Siri to achieve an improvement of 131.57% in answering incomplete follow-up questions.
- Martın Abadi, Ashish Agarwal, Paul Barham, Eugene Brevdo, Zhifeng Chen, Craig Citro, Greg S. Corrado, Andy Davis, Jeffrey Dean, Matthieu Devin, and others. 2016. Tensorflow: Large-scale machine learning on heterogeneous distributed systems. arXiv preprint arXiv:1603.04467 (2016).Google Scholar
- Layla El Asri, Jing He, and Kaheer Suleman. 2016. A Sequence-to-Sequence Model for User Simulation in Spoken Dialogue Systems. CoRR Vol. abs/1607.00070 (2016).Google Scholar
- Amos Azaria, Jayant Krishnamurthy, and Tom M. Mitchell. 2016. Instructable Intelligent Personal Agent. In AAAI.Google Scholar
- Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2014. Neural Machine Translation by Jointly Learning to Align and Translate. CoRR Vol. abs/1409.0473 (2014).Google Scholar
- Steven Bird. 2006. NLTK: the natural language toolkit. In Proceedings of the COLING/ACL on Interactive presentation sessions. Association for Computational Linguistics, 69--72. Google ScholarDigital Library
- Marco De Boni and Suresh Manandhar 2005. Implementing clarification dialogues in open domain question answering. Natural Language Engineering Vol. 11 (2005), 343--361. Google ScholarDigital Library
- Jaime G. Carbonell. 1983. Discourse pragmatics and ellipsis resolution in task-oriented natural language interfaces Proceedings of the 21st annual meeting on Association for Computational Linguistics. Association for Computational Linguistics, 164--168.Google Scholar
- Yun-Nung Chen, Dilek Hakkani-Tür, Gokhan Tur, Jianfeng Gao, and Li Deng 2016. End-to-end memory networks with knowledge carryover for multi-turn spoken language understanding Proceedings of Interspeech.Google Scholar
- Junyoung Chung, Caglar Gülçehre, Kyunghyun Cho, and Yoshua Bengio. 2015. Gated feedback recurrent neural networks. CoRR, abs/1502.02367 (2015).Google Scholar
- Jeffrey L. Elman. 1990. Finding Structure in Time. Cognitive Science Vol. 14 (1990), 179--211. Google ScholarCross Ref
- Emad Elwany. 2014. Enhancing Cortana User Experience Using Machine Learning.Google Scholar
- Oren Etzioni, Michael Cafarella, Doug Downey, Stanley Kok, Ana-Maria Popescu, Tal Shaked, Stephen Soderland, Daniel S. Weld, and Alexander Yates. 2004. Web-scale information extraction in knowitall: (preliminary results) Proceedings of the 13th international conference on World Wide Web. ACM, 100--110. Google ScholarDigital Library
- Junichi Fukumoto. 2006. Answering questions of Information Access Dialogue (IAD) task using ellipsis handling of follow-up questions. In Proceedings of the Interactive Question Answering Workshop at HLT-NAACL 2006. Association for Computational Linguistics, 41--48. Google ScholarCross Ref
- Junichi Fukumoto, Tatsuhiro Niwa, Makoto Itoigawa, and Megumi Matsuda 2004. RitsQA: List answer detection and context task with ellipses handling Working notes of the Fourth NTCIR Workshop Meeting, Okyo, Japan. 310--314.Google Scholar
- Olivier Galibert, Gabriel Illouz, and Sophie Rosset. 2005. Ritel: an open-domain, human-computer dialog system INTERSPEECH.Google Scholar
- Sadid A. Hasan, Bo Liu, Joey Liu, Ashequl Qadir, Kathy Lee, Vivek Datla, Aaditya Prakash, and Oladimeji Farri. 2016. Neural Clinical Paraphrase Generation with Attention.Google Scholar
- Sepp Hochreiter. 1998. The vanishing gradient problem during learning recurrent neural nets and problem solutions. International Journal of Uncertainty, Fuzziness and Knowledge-Based Systems, Vol. 6, 02 (1998), 107--116. Google ScholarDigital Library
- Sepp Hochreiter and Jürgen Schmidhuber 1997. Long short-term memory. Neural computation, Vol. 9, 8 (1997), 1735--1780. Google ScholarDigital Library
- Kentaro Inui, Akiko Yamashita, and Yuji Matsumoto. 2003. Dialogue Management for Language-based Information Seeking.Google Scholar
- Xu Jia, Efstratios Gavves, Basura Fernando, and Tinne Tuytelaars. 2015. Guiding the Long-Short Term Memory Model for Image Caption Generation ICCV.Google Scholar
- Anjuli Kannan, Karol Kurach, Sujith Ravi, Tobias Kaufmann, Andrew Tomkins, Balint Miklos, Greg Corrado, László Lukács, Marina Ganea, Peter Young, and others 2016. Smart Reply: Automated Response Suggestion for Email Proceedings of the ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD), Vol. Vol. 36. 495--503.Google ScholarDigital Library
- Andrej Karpathy and Li Fei-Fei. 2015. Deep visual-semantic alignments for generating image descriptions CVPR.Google Scholar
- Diederik Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014).Google Scholar
- Natalia Konstantinova and Constantin Orasan. 2012. Interactive question answering. (2012).Google Scholar
- Vineet Kumar and Sachindra Joshi. 2016. Non-sentential Question Resolution using Sequence to Sequence Learning COLING.Google Scholar
- Gakuto Kurata, Bing Xiang, Bowen Zhou, and Mo Yu. 2016. Leveraging Sentence-level Information with Encoder LSTM for Semantic Slot Filling EMNLP.Google Scholar
- Steven Loria. 2014. TextBlob: simplified text processing. Secondary TextBlob: Simplified Text Processing (2014).Google Scholar
- Megumi Matsuda and Jun-ichi Fukumoto. 2005. Answering Questions of IAD Task using Reference Resolution of Follow-up Questions. NTCIR. Citeseer.Google Scholar
- Yoichi Matsuyama, Arjun Bhardwaj, Ran Zhao, Oscar Romeo, Sushma Akoju, and Justine Cassell. 2016. Socially-Aware Animated Intelligent Personal Assistant Agent SIGDIAL Conference.Google Scholar
- Jason Merchant. 2001. The syntax of silence: Sluicing, islands, and the theory of ellipsis. Oxford University Press on Demand.Google Scholar
- Tomas Mikolov, Anoop Deoras, Daniel Povey, Lukás Burget, and Jan Cernocký. 2011. Strategies for training large scale neural network language models ASRU.Google Scholar
- Tomas Mikolov, Martin Karafiát, Lukás Burget, Jan Cernocký, and Sanjeev Khudanpur. 2010. Recurrent neural network based language model. In INTERSPEECH.Google Scholar
- Karen L. Myers, Pauline M. Berry, Jim Blythe, Ken Conley, Melinda T. Gervasio, Deborah L. McGuinness, David N. Morley, Avi Pfeffer, Martha E. Pollack, and Milind Tambe. 2007. An Intelligent Personal Assistant for Task and Time Management. AI Magazine Vol. 28 (2007), 47--61.Google ScholarDigital Library
- Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. BLEU: a method for automatic evaluation of machine translation Proceedings of the 40th annual meeting on association for computational linguistics. Association for Computational Linguistics, 311--318.Google Scholar
- Razvan Pascanu, Tomas Mikolov, and Yoshua Bengio. 2013. On the difficulty of training recurrent neural networks. ICML (3) Vol. 28 (2013), 1310--1318.Google ScholarDigital Library
- Aaditya Prakash, Sadid A. Hasan, Kathy Lee, Vivek Datla, Ashequl Qadir, Joey Liu, and Oladimeji Farri. 2016. Neural Paraphrase Generation with Stacked Residual LSTM Networks COLING.Google Scholar
- Silvia Quarteroni and Suresh Manandhar. 2009. Designing an interactive open-domain question answering system. Natural Language Engineering Vol. 15, 01 (2009), 73--95. Google ScholarDigital Library
- Dinesh Raghu, Sathish Indurthi, Jitendra Ajmera, and Sachindra Joshi. 2015. A Statistical Approach for Non-Sentential Utterance Resolution for Interactive QA System 16th Annual Meeting of the Special Interest Group on Discourse and Dialogue. 335.Google Scholar
- Norbert Reithinger, Simon Bergweiler, Ralf Engel, Gerd Herzog, Norbert Pfleger, Massimo Romanelli, and Daniel Sonntag. 2005. A look under the hood: design and development of the first SmartWeb system demonstrator ICMI.Google Scholar
- Boris Van Schooten and Rieks Op Den Akker. 2010. Vidiam: Corpus-based Development of a Dialogue Manager for Multimodal Question Answering.Google Scholar
- Louis Shao, Stephan Gouws, Denny Britz, Anna Goldie, Brian Strope, and Ray Kurzweil. 2017. Generating Long and Diverse Responses with Neural Conversation Models.Google Scholar
- Yangyang Shi, Kaisheng Yao, Hu Chen, Yi-Cheng Pan, Mei-Yuh Hwang, and Baolin Peng. 2015. Contextual spoken language understanding using recurrent neural networks ICASSP.Google Scholar
- Max Sklar. 2016. Marsbot: Building a Personal Assistant. In RecSys.Google ScholarDigital Library
- Noah A. Smith, Michael Heilman, and Rebecca Hwa. 2008. Question generation as a competitive undergraduate course project Proceedings of the NSF Workshop on the Question Generation Shared Task and Evaluation Challenge.Google Scholar
- Jinsong Su, Zhixing Tan, Deyi Xiong, and Yang Liu. 2016. Lattice-Based Recurrent Neural Network Encoders for Neural Machine Translation. CoRR Vol. abs/1609.07730 (2016).Google Scholar
- Ilya Sutskever, Oriol Vinyals, and Quoc V Le. 2014. Sequence to sequence learning with neural networks Advances in neural information processing systems. 3104--3112.Google Scholar
- Gökhan Tür, Ye-Yi Wang, and Dilek Z. Hakkani-Tür. 2014. Understanding Spoken Language. In Computing Handbook, 3rd ed.Google Scholar
- Boris W. van Schooten, Rieks op den Akker, Sophie Rosset, Olivier Galibert, Aurélien Max, and Gabriel Illouz. 2009. Follow-up question handling in the IMIX and Ritel systems: A comparative study. Natural Language Engineering Vol. 15 (2009), 97--118. Google ScholarDigital Library
- Haifeng Wang. 2016. Duer: Intelligent Personal Assistant. In CIKM.Google ScholarDigital Library
- Jason D. Williams, Pascal Poupart, and Steve Young. 2005. Factored Partially Observable Markov Decision Processes for Dialogue Management.Google Scholar
- Kelvin Xu, Jimmy Ba, Jamie Ryan Kiros, Kyunghyun Cho, Aaron C. Courville, Ruslan Salakhutdinov, Richard S. Zemel, and Yoshua Bengio. 2015. Show, Attend and Tell: Neural Image Caption Generation with Visual Attention ICML.Google Scholar
- Rui Yan, Yiping Song, and Hua Wu. 2016. Learning to Respond with Deep Neural Networks for Retrieval-Based Human-Computer Conversation System. In SIGIR. Google ScholarDigital Library
- Wojciech Zaremba, Ilya Sutskever, and Oriol Vinyals. 2014. Recurrent Neural Network Regularization. CoRR Vol. abs/1409.2329 (2014).Google Scholar
- Barret Zoph, Deniz Yuret, Jonathan May, and Kevin Knight. 2016. Transfer Learning for Low-Resource Neural Machine Translation EMNLP.Google Scholar
Index Terms
- Incomplete Follow-up Question Resolution using Retrieval based Sequence to Sequence Learning
Recommendations
Conversational Query Understanding Using Sequence to Sequence Modeling
WWW '18: Proceedings of the 2018 World Wide Web ConferenceUnderstanding conversations is crucial to enabling conversational search in technologies such as chatbots, digital assistants, and smart home devices that are becoming increasingly popular. Conventional search engines are powerful at answering open ...
Personification of the Amazon Alexa: BFF or a Mindless Companion
CHIIR '18: Proceedings of the 2018 Conference on Human Information Interaction & RetrievalThe conversational nature of intelligent personal assistants (IPAs) has the potential to trigger personification tendencies in users, which in turn can translate into consumer loyalty and satisfaction. We conducted a study of Amazon Alexa usage and ...
Exploring Humor as a Repair Strategy During Communication Breakdowns with Voice Assistants
CUI '23: Proceedings of the 5th International Conference on Conversational User InterfacesVoice assistants are becoming increasingly useful and support realistic conversations, yet communication breakdowns occur. We investigate the use of humor as a repair strategy in an experiment where the voice assistant makes a mistake and then utilizes ...
Comments