AI-Driven Support for People with Speech & Language Difficulties

1 INTRODUCTION

Reaching an understanding in everyday communicative interactions hinges on the successful reception, understanding, acceptance, and reciprocity between the speaker and the listener [23, 29]. When the speaker or listener encounters difficulty achieving these elements, the ability to convey or understand thoughts and ideas may be hindered, leading to a communication breakdown [23]. Given the complex nature of human interaction, communication breakdowns are observed in events involving people with “fully developed speech and language functions” [23, p.1]. However, individuals with speech and language difficulties experience a significantly higher rate of communication breakdowns per communication unit [16].

Speech and language difficulties comprise a wide range of conditions, including difficulty in producing speech sounds correctly (e.g., stuttering), the inability to express or comprehend language, difficulties in social and cognitive communication, as well as feeding and swallowing difficulties [1]. These difficulties can make traditional communication methods challenging, causing frustration and isolation for both the affected individual and their interlocutors [17]. For instance, studies indicate that numerous individuals with speech and language difficulties experience depression, social isolation, and a lower quality of life [18]. It can also negatively impact a person’s employment status [20] and their ability to receive proper healthcare [9]. The impact of speech and language development is also noticeable among young children. Currently, there exists a significant academic and socio-emotional disparity between children with speech and language difficulties and their matched peers [4].

Early identification and access to Speech Language Pathologists (SLPs) is crucial to improving speech, but the shortage of qualified professionals and the limited time available to provide ability-based interventions pose significant challenges to addressing these needs comprehensively [30]. For example, within the U.S. public school system, despite SLP vacancies being among the most prevalent types of vacancies, they do not receive the necessary attention or consideration[21]. This shortage of SLPs is creating significant challenges for individuals with speech and language concerns to access timely and comprehensive therapeutic interventions, hindering their ability to overcome communication barriers.

To address challenges related to the effectiveness of interventions and accessibility, many researchers have proposed AI-based automated speech therapy tools for individuals with speech and language difficulties [7, 9, 25, 27, 28]. In the past, CHI has been a platform for hosting numerous papers around this very topic [5, 6, 7, 10, 22, 31]. With the recent surge in popularity of generative AI, there is a growing recognition not just among researchers but also among care providers to leverage AI-powered capabilities like speech recognition, alongside quantitative data such as facial expressions and intonations for automating some aspects of care, such as analyzing learning patterns, identifying speech difficulties, and dynamically adjusting instructional strategies [8, 15]. Not only can AI-driven technology alleviate SLPs’ burden on delivering and executing interventions, but its scalability can also help reach out to broader populations who have been underserved [19, 24, 26].

At the same time, the distribution of these benefits may vary among stakeholders. For example, bias and fairness issues, potentially embedded in AI systems through training data [3, 34], can create disparities in intervention effectiveness among different demographic groups [14]. Another concern is that these systems may not be as empathetic [13] or unable to provide emotional support to individuals going through speech and language therapy in the same way direct communication with a human would. Thus, there is a need to adopt and advance human centered AI design methodologies in the development of reliable, safe, and trustworthy AI based technologies.

2 GOALS OF THE SIG

The goal of this SIG is to bring together interested researchers to start a conversation around hard questions in designing and evaluating AI-driven support for mediating communication breakdowns with individuals who have speech and language difficulties. By providing a dedicated space for the interdisciplinary exchange of ideas, methodologies, and findings at the intersection of AI, healthcare, education, accessibility, and HCI, this SIG seeks to connect people from different areas of CHI who are working in this domain and provide a forum for them to plan ways to collaboratively advance the development and implementation of AI-driven interventions to support individuals with speech and language difficulties. A few of our goals and action items for the SIG are as follows:

3 SIG TOPICS AND THEIR RELEVANCE TO THE HCI COMMUNITY

From an HCI standpoint, it is crucial to design and assess AI systems that prioritize the experiences and needs of all stakeholders to ensure a human-centered approach throughout the development lifecycle. For example, recognizing that children often perceive technologies as "creepy" when they involve deception, exhibit unpredictability, or reduce their sense of control [36] is important as frustration induced by poorly designed AI systems could lead children to swiftly abandon these technologies. While actively incorporating feedback from stakeholders throughout the development process can help mitigate initial design challenges, developing prototypes for AI user experiences poses additional complexities due to the probabilistic nature of AI’s outputs [11, 35]. For example, evaluating design concepts through conventional low or medium-fidelity prototyping methods, such as Wizard of Oz prototyping, can pose challenges due to uncertainty surrounding AI’s capabilities and AI’s output complexity [35]. Moreover, both the intended and unintended consequences of AI’s outputs such as recognition errors and algorithmic bias, need to be understood which can prevent teams from engaging in rapid iterative prototyping in the first place[35]. For example, speech recognition bias is evident in automated speech recognition systems, which show less proficiency for Black adult speakers of African American English (AAE) compared to white speakers of General American English (GAE) [14]. These disparities are often more pronounced in children, who may have limited exposure to speakers outside their community and consequently display more features of their dialect [32, 33]. While the HCI and design communities have proposed numerous guidelines and recommendations for designing AI user experiences [2, 12, 35], these recommendations often provide guidance at a high-level and lack actionable specifics about how to prototype likely interaction scenarios and failures [11]. At the same time, it is important design and build on general advances in AI ethics, fairness, explainability as a foundation for creating responsible and safe AI systems. Taking these into consideration, the list of potential discussion topics are:

4 EXPECTED OUTCOMES

During the SIG itself and in related asynchronous discussions, we will actively encourage the formation of partnerships, including research collaborations or larger efforts. Specifically, participants will be encouraged to engage in discussions with those they have not yet had the opportunity to meet or work with. The SIG could serve as a meeting point for researchers, educators, and practitioners from multiple backgrounds, encompassing HCI, NLP, accessibility, speech-language pathology, AI ethics, and beyond, to come together. Post-SIG, we will create a LinkedIn group to continue coordinating on these issues. All attendees will be welcome to sustain discussions and maintain their presence on the LinkedIn group channel. Any guidelines or findings from this SIG will be written up for publication in Interactions or Communications of the ACM. Finally, we aspire to continue discussions about the developed research agendas and SIG themes in the future by organizing bi-monthly conversations with invited speakers, which the SIG organizers will facilitate.

ACKNOWLEDGMENTS

This SIG is supported under the AI Research Institutes program by National Science Foundation and the Institute of Education Sciences, U.S. Department of Education, through Award 2229873 - AI Institute for Transforming Education for Children with Speech and Language Processing Challenges or National AI Institute for Exceptional Education. Any opinions, findings and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation, the Institute of Education Sciences, or the U.S. Department of Education.