Exploring the Use of Wearable Sensors and Natural Language Processing Technology to Improve Patient-Clinician Communication: Protocol for a Feasibility Study

Background Effective communication is the bedrock of quality health care, but it continues to be a major problem for patients, family caregivers, health care providers, and organizations. Although progress related to communication skills training for health care providers has been made, clinical practice and research gaps persist, particularly regarding how to best monitor, measure, and evaluate the implementation of communication skills in the actual clinical setting and provide timely feedback about communication effectiveness and quality. Objective Our interdisciplinary team of investigators aims to develop, and pilot test, a novel sensing system and associated natural language processing algorithms (CommSense) that can (1) be used on mobile devices, such as smartwatches; (2) reliably capture patient-clinician interactions in a clinical setting; and (3) process these communications to extract key markers of communication effectiveness and quality. The long-term goal of this research is to use CommSense in a variety of health care contexts to provide real-time feedback to end users to improve communication and patient health outcomes. Methods This is a 1-year pilot study. During Phase I (Aim 1), we will identify feasible metrics of communication to extract from conversations using CommSense. To achieve this, clinical investigators will conduct a thorough review of the recent health care communication and palliative care literature to develop an evidence-based “ideal and optimal” list of communication metrics. This list will be discussed collaboratively within the study team and consensus will be reached regarding the included items. In Phase II (Aim 2), we will develop the CommSense software by sharing the “ideal and optimal” list of communication metrics with engineering investigators to gauge technical feasibility. CommSense will build upon prior work using an existing Android smartwatch platform (SWear) and will include sensing modules that can collect (1) physiological metrics via embedded sensors to measure markers of stress (eg, heart rate variability), (2) gesture data via embedded accelerometer and gyroscope sensors, and (3) voice and ultimately textual features via the embedded microphone. In Phase III (Aim 3), we will pilot test the ability of CommSense to accurately extract identified communication metrics using simulated clinical scenarios with nurse and physician participants. Results Development of the CommSense platform began in November 2021, with participant recruitment expected to begin in summer 2022. We anticipate that preliminary results will be available in fall 2022. Conclusions CommSense is poised to make a valuable contribution to communication science, ubiquitous computing technologies, and natural language processing. We are particularly eager to explore the ability of CommSense to support effective virtual and remote health care interactions and reduce disparities related to patient-clinician communication in the context of serious illness. International Registered Report Identifier (IRRID) PRR1-10.2196/37975

REVIEWER 1: Critique: Unclear how communication metrics will be implemented in clinical practice; if these metrics are based on existing NLP systems/methods more detail is needed regarding the accuracy of such systems/methods. Response: The CommSense application will not need to be built from scratch, but instead will involve designing new software and associated algorithms on top of an existing Android smart watch platform (SWear) developed and tested by co-investigators Barnes and Boukhechba (SEAS). Prior work with SWear has demonstrated acceptance of the technology, accuracy of the underlying NLP technology, and the ability to successfully use the platform across multiple contexts and study samples. Additional details have been added to the 'Preliminary Work' section under 'Project Team.' It is also important to note that although CommSense builds upon known and validated NLP systems and methods, it is a key goal of this pilot to explore how accurately CommSense can identify quality communication metrics that can be used to evaluate and assess real-time patient-provider interactions. Please see the "Impact" section of the proposal for a detailed example of how CommSense communication metrics could be implemented in clinical practice.
Critique: Scale of data collection is unclear. Concern related to participant sample size. Response: 10 participants is an appropriate sample size for a 1-year pilot study in which the goal is to establish proof of concept (versus testing for statistical significance) 1,2 . It is also important to emphasize that each participant will generate multiple and numerous data points. For example, each participant will pilot test CommSense with 2 scripted conversations, therefore 10 participants will generate 20 conversations. For each conversation (n=20), we will gather multiple data points regarding paralinguistic and linguistic markers, as well as body language and physiological markers (see Figure 1 and clarifying details added to Specific Aim 3, Data Collection). Lastly, establishing Ground Truth (Aim 3) for this study will be a fairly labor-intensive process and a sample size of 10 participants (and 20 conversations) is realistic given the scale, scope, and budget of this pilot project.
REVIEWER 2: Critique: Concern whether biosensors will detect meaningful changes in likely healthy nursing and medical student participants. Response: While we do have the capability and plan to passively collect basic physiological data using the commercial Android smartwatch on which CommSense is deployed, it is important to reiterate that physiological/non-verbal data are not the primary focus of our study. This has been clarified within Specific Aim 2, Data Collection; Figure 1 and Table 1. Instead, the primary focus for our pilot study is to accurately collect and analyze linguistic markers that can be utilized to evaluate the quality of patient-provider conversations. Any non-verbal data gathered through biosensors will not be used to extrapolate information related to the participant's overall health but could provide preliminary insights regarding non-verbal aspects of communication that could help inform next steps, contextualize our results, or inform additional questions, such as, do physiological data differ between experts/novices, such as senior versus junior faculty or between student versus experienced clinicians.

Critique:
The CommSense application has been validated in the SWear format with socially anxious individuals. Will this be able to be extrapolated to the medical professionals you hope to study? Response: The SWear platform and associated algorithms provide the foundation from which we will build the CommSense application. SWear has been validated in multiple studies, including measuring wellbeing for elderly, analyzing social anxiety from audio data, analyzing the effect of route context on driving behaviors, and measuring fluid consumption of kidney patients, and many others. A key advantage of the SWear platform is that its core functionality can be applied across multiple contexts and samples -e.g., socially anxious individuals or individuals engaging in medically related conversations. We have added clarifying information and references within the section 'Preliminary Work' that describes prior feasibility and acceptability of the SWear platform and how it can be applied in this study population.
Critique: Do you think patients will allow recording of their conversations for studying and examination when you are beyond the pilot program? Response: Based on prior related research that has involved audio-recording of highly sensitive patient-provider conversations (such as Lee Ellington's work recording conversations between hospice patients/families and hospice care nurses 3-5 or James Tulsky's work recording conversations between providers and patients with serious illness 6,7 ), as well as a large body of literature that demonstrates both patients' and providers' desire for improved communication (see Background and Significance), we hypothesize CommSense will be acceptable beyond the pilot program. To help validate this, at the end of the interaction participants will complete a brief Qualtrics survey to assess acceptability of using CommSense, suggestions for future iterations, preferences regarding data sharing, and to rate their self-perceived communication performance (see Specific Aim 3, Data Collection).