Questions and Answers: Important Steps to Let AI Chatbots Answer Questions in the Museum

Abstract

In this paper, we describe our work within the research project “CHIM - Chatbot in the Museum”. CHIM is an AI-based chatbot prototype that enables conversational interaction using text and speech input: visitors can ask questions about certain artworks and receive answers in multimodal formats (text, audio, image, video). The application will be tested in the Städel Museum, Frankfurt/Main, Germany. To develop a proper Natural Language Understanding module, we adapted an existing categorization approach, gathered visitor questions, and structured them into twelve distinct content types. The preliminary results suggest that our approach to subdivide the previously overloaded content type meaning into further categories was successful, leading to a more balanced distribution of the data. We further describe the Natural Language Processing mechanisms employed here; these follow a multi-tiered approach using techniques like Rasa, BERT, and cosine-similarity to generate answers with different degrees of effort. Future steps are the implementation of dialog management, the refinement of the NLP strategies by integrating additional answers for selected exhibits, and the implementation of the final layout and interaction design. We are planning to test and evaluate the CHIM prototype on site in the Städel Museum in late 2021.

We want to thank the Städel Museum Frankfurt for their support. This research is part of the CHIM project of the research initiative “KMU-innovativ: Mensch-Technik-Interaktion", which is funded by the Federal Ministry of Education and Research (BMBF) of the Federal Republic of Germany under funding number 16SV8331.