skip to main content
research-article

Intrance: Designing an Interactive Enhancement System for the Development of QA Chatbots

Published:11 November 2022Publication History
Skip Abstract Section

Abstract

User input is essential for the successful development of question-and-answer (QA) chatbots. Therefore, interactive development systems are emerging that allow developers to involve test-users in the QA chatbot development process. Although the feasibility and effectiveness of this approach have been demonstrated, there is a lack of knowledge on how to design interactive QA chatbot development systems to increase test-user engagement as well as data quality in this time-consuming and tedious task. To address this research gap, we propose an interactive system design based on the interactivity effects model. We instantiate the proposed design and introduce Intrance, an interactive enhancement system for QA chatbots. Subsequently, we show in two online experiments that the proposed design significantly increases subjective and objective engagement of test-users and has a positive effect on data quality, conceptualized as data completeness and data accuracy. We discuss design implications for an implementation of the proposed design in commercial chatbot development systems.

References

  1. Daniel Adiwardana, Minh-Thang Luong, David R. So, Jamie Hall, Noah Fiedel, Romal Thoppilan, Zi Yang, Apoorv Kulshreshtha, Gaurav Nemade, Yifeng Lu, and Quoc V. Le. 2020. Towards a Human-like Open-Domain Chatbot. https://arxiv.org/pdf/2001.09977Google ScholarGoogle Scholar
  2. Bghiel Afrae, Ben Ahmed Mohamed, and A. Anouar Boudhir. 2020. A Question answering System with a sequence to sequence grammatical correction. In Proceedings of the 3rd International Conference on Networking, Information Systems & Security, Mohamed Ben Ahmed and Anouar Abdelhakim Boudhir (Eds.). Association for Computing Machinery, [S.l.], 1--6. https://doi.org/10.1145/3386723.3387894Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. Chiara Alzetta, Giovanni Adorni, Ilknur Celik, Frosina Koceva, and ILARIA TORRE. 2018. Toward a User-Adapted Question/Answering Educational Approach. In UMAP'18 adjunct, July 8--11, 2018, Singapore, Tanja Mitrovic, Jie Zhang, Li Chen, and David Chin (Eds.). ACM, New York, NY, USA, 173--177. https://doi.org/10.1145/3213586.3226214Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. Amazon. 2020a. How we taught Alexa to correct her own defects. https://www.amazon.science/blog/how-we-taught-alexa-to-correct-her-own-defectsGoogle ScholarGoogle Scholar
  5. Amazon. 2020b. New Alexa features: Interactive teaching by customers. https://www.amazon.science/blog/new-alexa-features-interactive-teaching-by-customersGoogle ScholarGoogle Scholar
  6. Saleema Amershi, Maya Cakmak, William Bradley Knox, and Todd Kulesza. 2014. Power to the people: The role of humans in interactive machine learning. AI MAGAZINE, Vol. 35, 4 (2014), 105--120. https://doi.org/10.1609/aimag.v35i4.2513Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. Bushra Anjum. 2019. A conversation with Danish Contractor. Ubiquity, Vol. 2019, September (2019), 1--5. https://doi.org/10.1145/3363787Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. Zahra Ashktorab, Mohit Jain, Q. Vera Liao, and Justin D. Weisz. 2019. Resilient Chatbots: Repair Strategy Preferences for Conversational Breakdowns. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems - CHI '19. ACM Press, New York, USA, 1--12. https://doi.org/10.1145/3290605.3300484Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. Ram G. Athreya, Axel-Cyrille Ngonga Ngomo, and Ricardo Usbeck. 2018. Enhancing Community Interactions with Data-Driven Chatbots--The DBpedia Chatbot. In Companion Proceedings of the The Web Conference 2018 (ACM Digital Library), Pierre-Antoine Champin, Fabien Gandon, Mounia Lalmas, and Panagiotis G. Ipeirotis (Eds.). International World Wide Web Conferences Steering Committee, Republic and Canton of Geneva, 143--146. https://doi.org/10.1145/3184558.3186964Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. Carlo Batini, Cinzia Cappiello, Chiara Francalanci, and Andrea Maurino. 2009. Methodologies for data quality assessment and improvement. Comput. Surveys, Vol. 41, 3 (2009), 1--52. https://doi.org/10.1145/1541880.1541883Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. Pranav Bhagat, Sachin Kumar Prajapati, and Aaditeshwar Seth. 2020. Initial Lessons from Building an IVR-based Automated Question-Answering System. In Proceedings of the 2020 International Conference on Information and Communication Technologies and Development : Escuela Superior Politécnica del Litoral (ESPOL), Guayaquil, Ecuador, June 17--20, 2020 (ICPS: ACM international conference proceeding series), Carleen Maitland, Monica Villavicencio Cabezas, and Monica Villavicencio Cabezas (Eds.). The Association for Computing Machinery, New York, New York, 1--5. https://doi.org/10.1145/3392561.3397581Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. A. Bozzon. 2018. Enterprise Crowd Computing for Human Aided Chatbots. In 2018 IEEE/ACM 1st International Workshop on Software Engineering for Cognitive Services (SE4COG). 29--30. https://doi.org/10.1145/3195555.3195566Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. Petter Bae Brandtzaeg and Asbjørn Følstad. 2018. Chatbots: Changing User Needs and Motivations. Interactions, Vol. 25, 5 (2018), 38--43. https://doi.org/10.1145/3236669Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. A.Colin Cameron and Pravin K. Trivedi. 1990. Regression-based tests for overdispersion in the Poisson model. Journal of Econometrics, Vol. 46, 3 (1990), 347--364.Google ScholarGoogle ScholarCross RefCross Ref
  15. A. Colin Cameron and Pravin K. Trivedi. 2013. Regression analysis of count data. Vol. 53. Cambridge University Press.Google ScholarGoogle Scholar
  16. Nuno Ramos Carvalho and Luís Soares Barbosa. 2019. Deep Learning Powered Question-Answering Framework for Organizations Digital Transformation. In Proceedings of the 12th International Conference on Theory and Practice of Electronic Governance (ICPS), Soumaya Ben Dhaou, Lemuria Carter, and Mark A. Gregory (Eds.). The Association for Computing Machinery, New York New York, 76--79. https://doi.org/10.1145/3326365.3326375Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. Yoonseo Choi, Toni-Jan Keith Monserrat, Jeongeon Park, Hyungyu Shin, Nyoungwoo Lee, and Juho Kim. 2020. ProtoChat: Supporting the Conversation Design Process with Crowd Feedback. In Conference Companion Publication of the 2020 on Computer Supported Cooperative Work and Social Computing (CSCW '20 Companion). Association for Computing Machinery, New York, NY, USA, 19--23. https://doi.org/10.1145/3406865.3418568Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. Leigh Clark, Cosmin Munteanu, Vincent Wade, Benjamin R. Cowan, Nadia Pantidi, Orla Cooney, Philip Doyle, Diego Garaialde, Justin Edwards, Brendan Spillane, Emer Gilmartin, and Christine Murad. 2019. What Makes a Good Conversation?. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems - CHI '19. ACM Press, New York, USA, 1--12. https://doi.org/10.1145/3290605.3300705Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. J. Cohen. 1992. A power primer. Psychological bulletin, Vol. 112, 1 (1992), 155--159.Google ScholarGoogle Scholar
  20. G. Daniel, J. Cabot, L. Deruelle, and M. Derras. 2020. Xatkit: A Multimodal Low-Code Chatbot Development Framework. IEEE Access, Vol. 8 (2020), 15332--15346. https://doi.org/10.1109/ACCESS.2020.2966919Google ScholarGoogle ScholarCross RefCross Ref
  21. Arthur R. T. de Lacerda and Carla S. R. Aguiar. 2019. FLOSS FAQ chatbot project reuse. In Proceedings of the 15th International Symposium on Open Collaboration, Björn Lundell, Jonas Gamalielsson, Lorraine Morgan, and Gregorio Robles (Eds.). The Association for Computing Machinery, New York, New York, 1--8. https://doi.org/10.1145/3306446.3340823Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. Kevin Doherty and Gavin Doherty. 2019. Engagement in HCI: Conception, Theory and Measurement. Comput. Surveys, Vol. 51, 5 (2019), 1--39. https://doi.org/10.1145/3234149Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. John J. Dudley and Per Ola Kristensson. 2018. A Review of User Interface Design for Interactive Machine Learning. ACM Trans. Interact. Intell. Syst., Vol. 8, 2 (2018), 1--37.Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. Jasper Feine, Ulrich Gnewuch, Stefan Morana, and Alexander Maedche. 2019a. A Taxonomy of Social Cues for Conversational Agents. International Journal of Human-Computer Studies, Vol. 132 (2019), 138--161. https://doi.org/10.1016/j.ijhcs.2019.07.009Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. Jasper Feine, Ulrich Gnewuch, Stefan Morana, and Alexander Maedche. 2020a. Gender Bias in Chatbot Design. In Chatbot Research and Design: Third International Workshop, CONVERSATIONS 2019, Amsterdam, The Netherlands, November 19--20, 2019, Revised Selected Papers. Vol. 11970. Springer.Google ScholarGoogle Scholar
  26. Jasper Feine, Stefan Morana, and Alexander Maedche. 2019b. Designing a Chatbot Social Cue Configuration System. In Proceedings of the 40th International Conference on Information Systems (ICIS). AISel, Munich.Google ScholarGoogle Scholar
  27. Jasper Feine, Stefan Morana, and Alexander Maedche. 2019c. Leveraging Machine-Executable Descriptive Knowledge in Design Science Research -- The Case of Designing Socially-Adaptive Chatbots. In Extending the Boundaries of Design Science Theory and Practice, Bengisu Tulu, Soussan Djamasbi, and Gondy Leroy (Eds.). Springer International Publishing, Cham, 76--91.Google ScholarGoogle Scholar
  28. Jasper Feine, Stefan Morana, and Alexander Maedche. 2020b. Designing Interactive Chatbot Development Systems. In Proceedings of the 41st International Conference on Information Systems (ICIS) 2020. AISel, India.Google ScholarGoogle Scholar
  29. Thomas Franke, Christiane Attig, and Daniel Wessel. 2019. A Personal Resource for Technology Interaction: Development and Validation of the Affinity for Technology Interaction (ATI) Scale. International Journal of Human-Computer Interaction, Vol. 35, 6 (2019), 456--467.Google ScholarGoogle ScholarCross RefCross Ref
  30. David Frohlich. 1993. The history and future of direct manipulation. Behaviour & Information Technology, Vol. 12, 6 (1993), 315--329. https://doi.org/10.1080/01449299308924396Google ScholarGoogle ScholarCross RefCross Ref
  31. Ulrich Gnewuch, Stefan Morana, Marc Adam, and Alexander Maedche. 2018. “The Chatbot is typing $łdots$” -- The Role of Typing Indicators in Human-Chatbot Interaction. In Proceedings of the Sixteenth Annual Pre-ICIS Workshop on HCI Research in MIS, San Francisco, CA, December 13, 2019.Google ScholarGoogle Scholar
  32. Gabriela Goldschmidt. 2016. Linkographic Evidence for Concurrent Divergent and Convergent Thinking in Creative Design. Creativity Research Journal, Vol. 28, 2 (2016), 115--122. https://doi.org/10.1080/10400419.2016.1162497Google ScholarGoogle ScholarCross RefCross Ref
  33. Jonathan Grudin and Richard Jacques. 2019. Chatbots, Humbots, and the Quest for Artificial General Intelligence. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (CHI '19). ACM, New York, NY, USA, 209:1--209:11. https://doi.org/10.1145/3290605.3300439Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. Joseph F. Hair, William C. Black, Barry J. Babin, and Rolph E. Anderson. 2010. Multivariate data analysis: A GLobal Perspective. Vol. 5. Pearson Prentice Hall, New Jersey.Google ScholarGoogle Scholar
  35. Braden Hancock, Antoine Bordes, Pierre-Emmanuel Mazaré, and Jason Weston. 2019. Learning from Dialogue after Deployment: Feed Yourself, Chatbot! In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. ACL, 3667--3684.Google ScholarGoogle Scholar
  36. Lynette Hirschman and Robert Gaizauskas. 2001. Natural language question answering: the view from here. Natural Language Engineering, Vol. 7, 4 (2001), 275. https://doi.org/10.1017/S1351324901002807Google ScholarGoogle ScholarDigital LibraryDigital Library
  37. Ting-Hao Huang, Joseph Chee Chang, and Jeffrey P. Bigham. 2018. Evorus: A Crowd-powered Conversational Assistant Built to Automate Itself Over Time. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems - CHI '19. ACM Press, 1--13. https://doi.org/10.1145/3173574.3173869Google ScholarGoogle ScholarDigital LibraryDigital Library
  38. Patrik Jonell, Mattias Bystedt, Fethiye Irmak Dogan, Per Fallgren, Jonas Ivarsson, Marketa Slukova, José Lopes Ulme Wennberg, Johan Boye, and Gabriel Skantze. 2018. Fantom: A Crowdsourced Social Chatbot using an Evolving Dialog Graph. In 1st Proceedings of Alexa Prize. https://m.media-amazon.com/images/G/01/mobile-apps/dex/alexa/alexaprize/assets/pdf/2018/Fantom.pdfGoogle ScholarGoogle Scholar
  39. Hyang-Sook Kim and S. Shyam Sundar. 2011. Using interface cues in online health community boards to change impressions and encourage user contribution. In Conference proceedings of the 29th Annual CHI Conference on Human Factors in Computing Systems. ACM, New York, NY, 599. https://doi.org/10.1145/1978942.1979028Google ScholarGoogle ScholarDigital LibraryDigital Library
  40. Ben Krause, Marco Damonte, Mihai Dobre, Daniel Duma, Joachim Fainberg, Federico Fancellu, Emmanuel Kahembwe, Jianpeng Cheng, and Bonnie Webber. 2017. Edina: Building an open domain socialbot with self-dialogues. In 1st Proceedings of Alexa Prize.Google ScholarGoogle Scholar
  41. Ashish Kulkarni, Kartik Mehta, Shweta Garg, Vidit Bansal, Nikhil Rasiwasia, and Srinivasan Sengamedu. 2019. ProductQnA: Answering User Questions on E-Commerce Product Pages. In The Web Conference 2019, Ling Liu, Ryen White, Sihem Amer-Yahia, and Mohammad Madian (Eds.). International World Wide Web Conferences Steering Committee, [Geneva, Switzerland], 354--360. https://doi.org/10.1145/3308560.3316597Google ScholarGoogle ScholarDigital LibraryDigital Library
  42. Felix Kwizera, Isaac Markus, Purity Mugambi, and Abdigani Diriye. 2017. Use of Twitter Data Toward the Development of an English and Swahili Question Answering Agent for the Kenyan Customer Service Market. In ICTD Pakistan 2017 (ICPS: ACM international conference proceeding series), Umar Saif (Ed.). The Association for Computing Machinery, New York, New York, 1--5. https://doi.org/10.1145/3136560.3136587Google ScholarGoogle ScholarDigital LibraryDigital Library
  43. Mounia Lalmas, Heather O'Brien, and Elad Yom-Tov. 2014. Measuring User Engagement. Synthesis Lectures on Information Concepts, Retrieval, and Services, Vol. 6, 4 (2014), 1--132. https://doi.org/10.2200/S00605ED1V01Y201410ICR038Google ScholarGoogle ScholarCross RefCross Ref
  44. Nancy K. Lankton, D. Harrison McKnight, and John Tripp. 2015. Technology, Humanness, and Trust: Rethinking Trust in Technology. Journal of the Association for Information Systems, Vol. 16, 10 (2015). http://aisel.aisnet.org/jais/vol16/iss10/1Google ScholarGoogle ScholarCross RefCross Ref
  45. Walter S. Lasecki, Jaime Teevan, and Ece Kamar. 2014. Information extraction and manipulation threats in crowd-powered systems. In Proceedings of the ACM conference on Computer supported cooperative work & social computing. ACM. https://doi.org/10.1145/2531602.253173Google ScholarGoogle ScholarCross RefCross Ref
  46. James M. LeBreton and Jenell L. Senter. 2008. Answers to 20 questions about interrater reliability and interrater agreement. Organizational research methods, Vol. 11, 4 (2008), 815--852. https://doi.org/10.1177/1094428106296642Google ScholarGoogle Scholar
  47. Ji Young Lee and S. Shyam Sundar. 2013. To tweet or to retweet? That is the question for health professionals on twitter. Health communication, Vol. 28, 5 (2013), 509--524. https://doi.org/10.1080/10410236.2012.700391Google ScholarGoogle Scholar
  48. Jiwei Li, Alexander H. Miller, Sumit Chopra, Marc'Aurelio Ranzato, and Jason Weston. 2016. Dialogue Learning With Human-In-The-Loop. In Proceedings of the International Conference on Learning Representations (ICLR). https://arxiv.org/pdf/1611.09823Google ScholarGoogle Scholar
  49. Toby Jia-Jun Li, Jingya Chen, Haijun Xia, Tom M. Mitchell, and Brad A. Myers. 2020. Multi-Modal Repairs of Conversational Breakdowns in Task-Oriented Dialogs. In Proceedings of the 33rd Annual ACM Symposium on User Interface Software and Technology (UIST '20). Association for Computing Machinery, New York, NY, USA, 1094--1107. https://doi.org/10.1145/3379337.3415820Google ScholarGoogle ScholarDigital LibraryDigital Library
  50. Q. Vera Liao, Werner Geyer, Muhammed Mas-ud Hussain, Praveen Chandar, Matthew Davis, Yasaman Khazaeni, Marco Patricio Crasso, Dakuo Wang, Michael Muller, and N. Sadat Shami. 2018. All Work and No Play?. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems - CHI '18. ACM Press, 1--13. https://doi.org/10.1145/3173574.3173577Google ScholarGoogle ScholarDigital LibraryDigital Library
  51. Lue Lin, Luis D'Haro, and Rafael Banchs. 2016. A Web-based Platform for Collection of Human-Chatbot Interactions. In HAI'16, October 4--7, 2016, Biopolis, Singapore. 363--366. https://doi.org/10.1145/2974804.2980500Google ScholarGoogle ScholarDigital LibraryDigital Library
  52. Bang Liu, Haojie Wei, Di Niu, Haolan Chen, and Yancheng He. 2020a. Asking Questions the Human Way: Scalable Question-Answer Generation from Text Corpus. In Proceedings of The Web Conference 2020, Huang (Ed.). Association for Computing Machinery, [S.l.], 2032--2043. https://doi.org/10.1145/3366423.3380270Google ScholarGoogle ScholarDigital LibraryDigital Library
  53. Xiao-Yang Liu, Yimeng Zhang, Yukang Liao, and Ling Jiang. 2020b. Dynamic Updating of the Knowledge Base for a Large-Scale Question Answering System. ACM Transactions on Asian and Low-Resource Language Information Processing, Vol. 19, 3 (2020), 1--13. https://doi.org/10.1145/3377708Google ScholarGoogle ScholarDigital LibraryDigital Library
  54. Yuping Liu. 2003. Developing a Scale to Measure the Interactivity of Websites. Journal of Advertising Research, Vol. 43 (2003), 207--216. https://doi.org/10.1017/S0021849903030204Google ScholarGoogle ScholarCross RefCross Ref
  55. Ewa Luger and Abigail Sellen. 2016. Like Having a Really Bad PAtextquotedbl: The Gulf Between User Expectation and Experience of Conversational Agents. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI '16). ACM, New York, NY, USA, 5286--5297. https://doi.org/10.1145/2858036.2858288Google ScholarGoogle ScholarDigital LibraryDigital Library
  56. Roman Lukyanenko, Jeffrey Parsons, Yolanda F. Wiersma, and Mahed Maddah. 2019. Expecting the Unexpected: Effects of Data Collection Design Choices on the Quality of Crowdsourced User-Generated Content. MIS Quarterly, Vol. 43, 2 (2019), 623--647. https://doi.org/10.25300/MISQ/2019/14439Google ScholarGoogle ScholarCross RefCross Ref
  57. Microsoft. 2021a. Test your knowledge base in QnA Maker. https://docs.microsoft.com/de-de/azure/cognitive-services/qnamaker/how-to/test-knowledge-base?tabs=v1Google ScholarGoogle Scholar
  58. Microsoft. 2021b. What is QnA Maker? https://docs.microsoft.com/en-us/azure/cognitive-services/qnamaker/overview/overviewGoogle ScholarGoogle Scholar
  59. Adam S. Miner, Liliana Laranjo, and A. Baki Kocaballi. 2020. Chatbots in the fight against the COVID-19 pandemic. NPJ digital medicine, Vol. 3 (2020), 65. https://doi.org/10.1038/s41746-020-0280-0Google ScholarGoogle Scholar
  60. Christine Murad and Cosmin Munteanu. 2020. Designing Voice Interfaces: Back to the (Curriculum) Basics. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. ACM, 1--12. https://doi.org/10.1145/3313831.3376522Google ScholarGoogle ScholarDigital LibraryDigital Library
  61. Clifford Nass and Youngme Moon. 2000. Machines and Mindlessness: Social Responses to Computers. Journal of Social Issues, Vol. 56, 1 (2000), 81--103. https://doi.org/10.1111/0022--4537.00153Google ScholarGoogle ScholarCross RefCross Ref
  62. Gina Neff and Peter Nagy. 2016. Automation, algorithms, and politics| talking to Bots: Symbiotic agency and the case of Tay. International Journal of Communication, Vol. 10 (2016), 17.Google ScholarGoogle Scholar
  63. Heather L. O'Brien, Paul Cairns, and Mark Hall. 2018. A practical approach to measuring user engagement with the refined user engagement scale (UES) and new UES short form. International Journal of Human-Computer Studies, Vol. 112 (2018), 28--39. https://doi.org/10.1016/j.ijhcs.2018.01.004Google ScholarGoogle ScholarCross RefCross Ref
  64. Ankur Pandey, Inshita Mutreja, Saru Brar, and Pushpendra Singh. 2020. Exploring Automated Q&A Support System for Maternal and Child Health in Rural India. In Proceedings of the 3rd ACM SIGCAS Conference on Computing and Sustainable Societies. 349--350. https://doi.org/10.1145/3378393.3402281Google ScholarGoogle ScholarDigital LibraryDigital Library
  65. Silvia Quarteroni. 2008. Personalized, Interactive Question Answering on the Web. In Coling 2008: Proceedings of the Workshop on Knowledge and Reasoning for Answering Questions (KRAQ '08). Association for Computational Linguistics, USA, 33--40.Google ScholarGoogle Scholar
  66. Silvia Quarteroni and S. Manandhar. 2009. Designing an interactive open-domain question answering system. Natural Language Engineering, Vol. 15, 1 (2009), 73--95. https://doi.org/10.1017/S1351324908004919Google ScholarGoogle ScholarDigital LibraryDigital Library
  67. Sheizf Rafaeli. 1988. From new media to communication. Sage annual review of communication research: Advancing communication science, Vol. 16 (1988), 110--134.Google ScholarGoogle Scholar
  68. Lila Rao and Kweku-Muata Osei-Bryson. 2007. Towards defining dimensions of knowledge systems quality. Expert Systems with Applications, Vol. 33, 2 (2007), 368--378. https://doi.org/10.1016/j.eswa.2006.05.003Google ScholarGoogle ScholarDigital LibraryDigital Library
  69. Rasa. 2021. Interactive Learning. https://legacy-docs.rasa.com/docs/core/interactive_learning/Google ScholarGoogle Scholar
  70. Thomas C. Redman. 1996. Data quality for the information age. Artech House, Boston, Mass.Google ScholarGoogle ScholarDigital LibraryDigital Library
  71. Elayne Ruane, Théo Faure, Ross Smith, Dan Bean, Julie Carson-Berndsen, and Anthony Ventresque. 2018. BoTest. In Proceedings of the 23rd International Conference on Intelligent User Interfaces Companion - IUI'18, Unknown (Ed.). ACM Press, New York, New York, USA, 1--2. https://doi.org/10.1145/3180308.3180373Google ScholarGoogle ScholarDigital LibraryDigital Library
  72. Nithya Sambasivan, Shivani Kapania, Hannah Highfill, Diana Akrong, Praveen Paritosh, and Lora M. Aroyo. 2021. “Everyone Wants to Do the Model Work, Not the Data Work”: Data Cascades in High-Stakes AI. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. Association for Computing Machinery, New York, NY, USA.Google ScholarGoogle Scholar
  73. Isabella Seeber, Gert-Jan de Vreede, Ronald Maier, and Barbara Weber. 2017. Beyond Brainstorming: Exploring Convergence in Teams. Journal of Management Information Systems, Vol. 34 (2017), 939--969. https://doi.org/10.1080/07421222.2017.1393303Google ScholarGoogle ScholarCross RefCross Ref
  74. Ben Shneiderman. 1997. Direct manipulation for comprehensible, predictable and controllable user interfaces. In Proceedings of the international conference on Intelligent user interfaces. ACM, 33--39. https://doi.org/10.1145/238218.238281Google ScholarGoogle ScholarDigital LibraryDigital Library
  75. Jonathan Steuer. 1992. Defining virtual reality: Dimensions determining telepresence. Journal of Communication, Vol. 42, 4 (1992), 73--93. https://doi.org/10.1111/j.1460--2466.1992.tb00812.xGoogle ScholarGoogle ScholarCross RefCross Ref
  76. S. Shyam Sundar. 2012. Social psychology of interactivity in human-website interaction. In Oxford handbook of internet psychology. Oxford University Press.Google ScholarGoogle Scholar
  77. S. Shyam Sundar, Saraswathi Bellur, Jeeyun Oh, Haiyan Jia, and Hyang-Sook Kim. 2016. Theoretical Importance of Contingency in Human-Computer Interaction: Effects of Message Interactivity on User Engagement. Communication Research, Vol. 43, 5 (2016), 595--625.Google ScholarGoogle ScholarCross RefCross Ref
  78. S. Shyam Sundar, Haiyan Jia, T. Franklin Waddell, and Yan Huang. 2015. Toward a Theory of Interactive Media Effects (TIME). In The handbook of the psychology of communication technology, S. Shyam Sundar (Ed.). Wiley Blackwell, Chichester, West Sussex, UK and Malden, MA, 47--86. https://doi.org/10.1002/9781118426456.ch3Google ScholarGoogle Scholar
  79. S. Shyam Sundar, Sriram Kalyanaraman, and Justin Brown. 2003. Explicating Web Site Interactivity: Impression Formation Effects in Political Campaign Sites. Communication Research, Vol. 30, 1 (2003), 30--59. https://doi.org/10.1177/0093650202239025Google ScholarGoogle ScholarCross RefCross Ref
  80. Anuj Tewari and John Canny. 2014. What did spot hide?. In CHI 2014, one of a CHInd, Matt Jones, Philippe Palanque, Albrecht Schmidt, and Tovi Grossman (Eds.). Assoc. for Computing Machinery, New York, NY, 1807--1816. https://doi.org/10.1145/2556288.2557205Google ScholarGoogle ScholarDigital LibraryDigital Library
  81. Yair Wand and Richard Y. Wang. 1996. Anchoring data quality dimensions in ontological foundations. Commun. ACM, Vol. 39, 11 (1996), 86--95. https://doi.org/10.1145/240455.240479Google ScholarGoogle ScholarDigital LibraryDigital Library
  82. Jason Weston. 2016. Dialog-Based Language Learning. In Proceedings of the 30th International Conference on Neural Information Processing Systems (NIPS'16). Curran Associates Inc, Red Hook, NY, USA, 829--837.Google ScholarGoogle ScholarDigital LibraryDigital Library
  83. William E. Winkler. 1990. String Comparator Metrics and Enhanced Decision Rules in the Fellegi-Sunter Model of Record Linkage. In Proceedings of the Section on Survey Research Methods. 354--359.Google ScholarGoogle Scholar
  84. Anbang Xu, Zhe Liu, Yufan Guo, Vibha Sinha, and Rama Akkiraju. 2017. A New Chatbot for Customer Service on Social Media. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems - CHI '17, Gloria Mark, Susan Fussell, Cliff Lampe, m.c schraefel, Juan Pablo Hourcade, Caroline Appert, and Daniel Wigdor (Eds.). ACM Press, New York, New York, USA, 3506--3510. https://doi.org/10.1145/3025453.3025496Google ScholarGoogle ScholarDigital LibraryDigital Library
  85. Jianfei Yu, Minghui Qiu, Jing Jiang, Jun Huang, Shuangyong Song, Wei Chu, and Haiqing Chen. 2018. Modelling Domain Relationships for Transfer Learning on Retrieval-based Question Answering Systems in E-commerce. In WSDM'19, Yi Chang, Chengxiang Zhai, Yan Liu, and Yoelle Maarek (Eds.). ACM Association for Computing Machinery, New York, NY, USA, 682--690. https://doi.org/10.1145/3159652.3159685Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Intrance: Designing an Interactive Enhancement System for the Development of QA Chatbots

      Recommendations

      Comments

      Login options

      Check if you have access through your login credentials or your institution to get full access on this article.

      Sign in

      Full Access

      • Published in

        cover image Proceedings of the ACM on Human-Computer Interaction
        Proceedings of the ACM on Human-Computer Interaction  Volume 6, Issue CSCW2
        CSCW
        November 2022
        8205 pages
        EISSN:2573-0142
        DOI:10.1145/3571154
        Issue’s Table of Contents

        Copyright © 2022 ACM

        Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

        Publisher

        Association for Computing Machinery

        New York, NY, United States

        Publication History

        • Published: 11 November 2022
        Published in pacmhci Volume 6, Issue CSCW2

        Permissions

        Request permissions about this article.

        Request Permissions

        Check for updates

        Qualifiers

        • research-article
      • Article Metrics

        • Downloads (Last 12 months)110
        • Downloads (Last 6 weeks)12

        Other Metrics

      PDF Format

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader