Patient and clinician perspectives on a patient‐facing dashboard that visualizes patient reported outcomes in rheumatoid arthritis

Abstract Background Poor patient‐clinician communication around patient‐reported outcomes (PROs) is a barrier to the effective management of rheumatoid arthritis (RA). We aimed to develop an RA ‘dashboard’ that could facilitate conversations about PROs and that would be acceptable to a wide range of patients, including English and Spanish speakers and patients with adequate or limited health literacy. Methods A diverse group of RA patients along with clinicians from two academic rheumatology clinics joined separate focus groups. We solicited feedback and made iterative changes to mock‐ups of an RA dashboard that visualized PROs using a human‐centred design process. We used the thematic analysis method to identify and characterize themes from the focus groups and used these insights to refine the dashboard. Results We conducted six focus groups involving 25 RA patients and three groups with 11 clinicians. Patients and clinicians agreed that the dashboard could enhance communication about PROs and RA disease activity and could promote patient self‐management. Patients varied in their (a) comprehension, (b) preferences for the display and features of the dashboard, and (c) desired uses for the dashboard. Clinicians expressed significant concerns about the logistics of using the dashboard in clinical practice. Conclusion Using principles of human‐centred design, we created an RA dashboard that was well‐accepted among patients and clinicians. The ability to customize the data display is important for tailoring the dashboard to patients with diverse needs and preferences. Special attention should be given to feasibility concerns voiced by clinicians.


| INTRODUC TI ON
The effective use of patient-reported outcomes (PROs) data is anticipated to play a critical role in improving health-care delivery, patient experiences with care, and outcomes. In rheumatoid arthritis (RA), a complex chronic condition characterized by joint pain and inflammation, validated PROs have been used over the past several decades to assess levels of RA disease activity and functional status. Routine assessment of PROs is now recommended by American College of Rheumatology (ACR) guidelines, and quality measures to encourage the regular collection of RA PROs have been endorsed by the National Quality Forum. 1 Treatment algorithms for RA rely on PROs to guide clinical decisions around use of disease-modifying drugs (DMARDs). 2 RA is thus an important example of how an application to visualize PROs could be used during face-to-face visits to influence care, providing key data for clinical decision making, quality measurement, practice improvement, and research.
In recent years, scientists and clinicians have become more interested in developing patient-facing dashboards for many chronic conditions, some of which have shown great promise in empowering patients and facilitating self-management. In diabetes, for example, some mobile applications have demonstrated clinical improvement in glucose control. 3 PRO-specific dashboards have also been developed to track outcomes relating to prostate cancer, multiple sclerosis, various surgical procedures, among others. [4][5][6][7][8] In their pilot study, Register, and Rheum4U from the University of Calgary. 3,[9][10][11] These platforms aggregated disease activity measures and PROs, but were designed to support clinician decision-making, rather than focus on enhancing a patient's understanding of their own disease. 12 Building on preliminary work by our group to design a first prototype, 13 the objective of this study was to develop an RA 'dashboard' that could facilitate conversations about PROs between patients and clinicians. One key requirement was that the dashboard be acceptable to a wide range of patients, including English and Spanish speakers and patients with adequate or limited health literacy. To this end, we performed focus groups with a diverse group of patients and clinicians, eliciting information to help us tailor this digital tool to address the needs and preferences of patients as well as the health-care team.

| Dashboard prototype development
We tested various dashboard prototypes during patient and clinician focus groups to arrive at an optimal dashboard design. We used the principles of human-centred design, which aims to integrate the needs of people, the possibilities of technology, and the requirements for success, and is used widely in the development of health-IT tools. 14, 15 We utilized a modified version of A/B testing (ie, comparing two versions of the dashboard with only one major difference side by side) with patients where they provided verbal feedback on prototypes ranging from rough sketches on paper to interactive mock-ups on a tablet computer.
The first prototype was previously developed by our investigative team using principles of human-centred design. 13 In brief, we conducted patient and clinician semi-structured interviews, small focus groups of English speakers, and heuristic evaluations in rheumatology clinics to generate the first prototype ( Figure 1, Prototype A). The first prototype included visualizations of three PROs: (a) a measure of RA disease activity; we chose to display a validated instrument to measure RA disease activity that is used routinely in our clinic, the Clinical Disease Activity Index (CDAI).
The CDAI is a composite score that includes a tender joint count, swollen joint count, patient global assessment score, and physician global assessment score that has been endorsed by the American College of Rheumatology, 16 17 ), and (c) a pain score that is collected during routine clinical care on all patients. These were each displayed in series on a graph with level or score for the PRO on the y-axis and time on the x-axis. The five most-current time points were visible to begin with, and scroll buttons allow users to see historical scores.
Relevant medications used specifically for treating RA were also shown as solid bars (representing time spent receiving the drug) underneath the PRO measures. Key laboratory results, including a measure of inflammation (CRP), liver function (ALT), and kidney function (Creatinine) in relation to normal ranges, were also displayed. Prototype A was translated into Spanish by a native speaker on the research team (GS).
The second set of prototypes (Prototype B, see Figure 2A,B) were developed in response to the early patient focus groups (groups 1, 2, 3), and refined and tested for acceptability in the later focus groups (groups 4, 5, 6). In general, the second prototypes were partly customizable by making each section of the dashboard expandible/ collapsible so that each PRO (or laboratory) could be displayed one at a time, based on a patient's preferences. Additional key changes from the first prototype included (a) adding a homunculus (a cartoon humanoid creature with red dots in the extremities representing areas of joint inflammation) to correspond to swollen joints counts, with historical data available by scrolling to the left; (b) changing the orientation of the disease activity visualization so that 'better' was in the 'up' direction; (c) removing references to 'ideal ranges'; and (d) adding 'pop-up' explanatory text to provide more details on a PRO or test result.

| Focus group participants
Human-centred design prioritizes the iterative and in-depth engagement with potential users of the product under design in order to ensure that the product is sensitive to the user population's hopes, needs, capacities and preferences. 18    At the beginning of each focus group, participants were assured that their responses would be aggregated for presentation and that standard steps would be taken to ensure confidentiality of the data.

| Focus groups
Verbal consent was obtained. Facilitators encouraged all individuals to participate; invited participants to voice diverse experiences and ideas rather than seek to find consensus in the group; and explicitly solicited participants' concerns about the dashboard and its use in clinic settings. The study team provided paper-as well as iPad-and laptop-based prototypes. Participants were encouraged to write and draw on the paper versions. All focus groups were audio recorded.
Research staff collected annotated copies of the prototype at the end of each focus group. After each focus group, the study team shared notes and discussed emergent themes.
All focus group guides were developed in English by an expert in qualitative research (SBG) and authors who had participated in qualitative research activities with RA patients during an earlier phase of the study 13 : though patients were not directly involved in guide development, the patient guides were informed by preliminary data the team had collected from patients in prior semi-structured interviews and focus groups. Native speakers on the research team then translated (GS) and back-translated (CK) the guide for the Spanish focus groups.
Consistent with human-centred design principles, focus group guides were designed to encourage patients to think concretely about their individual experiences (past, present and future) when evaluating the dashboard rather than discussing the dashboards abstractly or for a 'typical user'. Patients first discussed their experiences in the RA clinic, focusing on their expectations and needs during appointments. The RA dashboard was then introduced, and patients described their understanding of the dashboard and shared feedback on whether the dashboard adequately represented factors important to them and their disease. Patients were asked to discuss how the dashboard might be used for managing their disease, and how it might change their future appointments or self-care. They were also given an opportunity to share thoughts on different formats of the dashboard.
Cognizant that some participants might feel pressure to agree with group sentiment or to embrace the dashboard prototype in front of its designers, we sought to capture patients' private views as well. We developed an anonymous, 2-item survey administered at the end of the focus groups, that asked patients to indicate whether they would like to use a dashboard themselves at their next visit and what dashboard format they would prefer (eg, in-clinic computer screen; home-based computer screen; mobile device; paper). The survey required less than a minute to complete. Results were descriptively tabulated and summarized in the text below.
The focus group guide for clinicians was similar in format to the patient guide and focused on how the dashboard could enhance communication with patients. Guide designers solicited the feedback of clinicians on the research team in order to assess and improve the relevance, clarity and appropriateness of the guide questions. Clinician focus groups began by discussing clinician goals and challenges for RA face-to-face visits, followed by the introduction of the dashboards. Clinicians gave feedback on the dashboard, commented on potential application in their practice, and suggested improvements.

| Focus group analysis
All audio recordings were professionally transcribed and, for the Spanish focus groups, translated. An author with expertise in qualitative methods (SBG) conducted iterative inductive and deductive thematic analysis 20,21 during and after data collection to identify themes relevant to patient or clinician perspectives on the dashboard and its use. Deductively, for example, we investigated expressions of interest in and understanding of the dashboards in each group, as these were phenomena the team had a priori sought to study. Inductively, we explored what factors participants cited when describing why they would or would not be interested in using the dashboard. We used analytic memo-ing to document preliminary themes within and across focus groups; to explore relationships between themes (eg, interest and understanding); and to integrate the interpretations of other team members who observed and took notes on the focus groups. Other members of the team (LHL, JL, GS) reviewed the data and preliminary analyses and collectively evaluated, refined and affirmed the proposed themes. Members of the team who attended the focus groups (SBG, JL, DR, CK) reviewed the data and preliminary analyses and collectively refined and affirmed these themes. 20,21 This study was approved by our institution's Institutional Review Board. The data that support the findings of this study are available from the corresponding author upon reasonable request.

| Patient focus groups
Six-patient focus groups were conducted, four in English and two in Spanish, involving a total of 25 RA patients. Characteristics for the patients are shown in Table 1. Eighty percentage of patients were female; average age was 59.9. There were 15 English-speaking and 10 Spanish-speaking patients. Sixteen patients had adequate health literacy and nine patients had limited health literacy (three groups were majority adequate health literacy; three groups were majority limited health literacy). For nearly all groups, patient attendance represented approximately 50% of the patients recruited for the event.
We identified four main themes across focus groups: (a) acceptability and interest in the dashboard, (b) comprehension of the data displayed in the dashboard, (c) preferences for dashboard display and features, and (d) suggestions for potential uses. Illustrative quotes are presented in Table 2.

| High levels of acceptability and interest in dashboard use
Rheumatoid arthritis patients in the focus groups were generally positive about the dashboard and desired to use it for RA management. Patients described it as 'helpful', 'useful' and 'interesting'. Some patients saw it as a repackaging of information with which they were familiar; others felt it would give them novel information about themselves and their disease. Spanish-speaking and limited health literacy participants were more likely to consider this information new or more accessible to them than what they were used to. This support and interest emerged across all patient focus groups (Table 2A).
In the anonymous summary survey, 21 of 24 patients expressed that they would like to see their own data represented in a dashboard at a future appointment. The remaining three patients responded 'maybe'. Most patients indicated that they would be interested in using a dashboard similar to one or both prototypes. Very few patients indicated that they would not use the dashboard unless it were substantially modified. Many patients, especially adequate health literacy patients but also some limited health literacy patients, expressed preference for the presentation of more detailed information.

| Variation in the comprehension of the dashboard
Many patients found the dashboard difficult to understand at first glance. This initial confusion was more common among limited health literacy patients but not exclusive to them. A few patients were able to understand the dashboard nearly immediately; others after a couple minutes of study or talking through the content. Several patients, particularly in the limited health literacy and Spanish-language groups, did not notice or understand the longitudinal nature of the data from left to right nor the temporal connection between the different graphic elements, particularly the timing

| Diverse views on dashboard display and features
Patients were divided in terms of preferred dashboard style, with some patients preferring Prototype A (which was more numerical and displayed a large amount of detailed information) and some preferring Prototype B (which had a simpler design with more graphics; see Table 2C). A few patients (both adequate and limited

| Diverse potential uses
Patients expressed the desire to use the dashboard to further not just their own understanding of their disease process, but also to This aspect of use was particularly important to Spanish-speaking patients, who described constrained communication opportunities with clinicians. Many of these patients felt that the dashboard would help minimize the number of questions that they would need to ask their clinicians. Spanish-speaking patients did not anticipate challenges in using the dashboard with an interpreter.

| Clinician focus groups
Eleven clinicians participated in three focus groups. Eight women and three men differed in their level of training and included three faculty rheumatologists, six rheumatology clinical fellows, one nurse practitioner and one medical assistant. Most participating clinicians practiced at the university hospital, although one group included clinicians from the county hospital as well.

| High levels of interest in dashboard use
Most clinicians were supportive of the concepts presented in the dashboard (

| Scepticism about utility of the dashboard
Despite interest in the content and potential uses of the dashboard, a few clinicians expressed significant scepticism that they would use the dashboard during face-to-face visits with their own patients (

| Interest in enhancing the dashboard to support more capabilities
Similar to the patients, clinicians were also very interested in customizing the dashboard to their own needs and recommended that it . 'I think it would be most helpful when we're counseling around continuing medications. The patient that comes in wanting to stop medications, if you had a way visually to say well, actually, I really do think this has been helping you. I know we're not necessarily in a perfect scenario and your pain level isn't zero, but look what we've been able to achieve. I think that's a helpful visual tool'. 'It's nice because it compiles a few things in the same place' B. Interest in using the dashboard as a tool primarily for clinicians 'Often the fellows are having to inherit a patient who's been cared for by many different people. And especially for patients who have a long-standing disease who are complicated and who have been on a lot of medications, sometimes things were stopped for unclear reasons. … So, having that historical medication information that's accurate with when it stopped, why it was stopped, etc, would…potentially improve safety for patients in treatment'. 'The homunculus makes a really big difference because it does help us as providers also see, gee, where did they hurt last time and are they better, any worse'. 'I think these details that the other people are discussing kind of point to the importance of probably having a different dashboard for the physicians as opposed to the patients, because I think for us to be able to use this in a way that would help us in making a treatment decision for example, we do want to see more granular detail. Which would include the trends in the labs, and also the dosing of the medications. Whereas, if you add all of that detail into a dashboard for the patients, it's probably going to be overwhelming; and it might just not be interpretable because they're coming from such a different background'. 'I like the idea of being able to click through a few labs. Although, I would need to see more than these three labs for my own assessment of their status. And I definitely don't want to be looking at labs in two different places in the same visit'. 'I think also it would help me in engaging my threshold to change a medication as well. You kind of do this as you're pre-charting and…looking back at the last three to four visits over the last year… So being able to kind of pull this up automatically [on the dashboard], actually would I think be helpful'. it's a little bit easier for patients to understand rather than CDAI, as a gauge of like how well their RA is being managed'. 'All of these [elements] depend heavily on the subjective pain sense of the patient, which I don't think is that uncommon that it may be quite different than our RA-focused specific disease assessment. Having at least one visual that was maybe more based on the physician's assessment would be [helpful]'.
(Continues) be designed to present more detailed information (Table 3D)

| Concerns about the feasibility of building and using the dashboard
Perhaps the most serious concerns expressed by clinicians were around the availability and accuracy of the EHR data to be imported into the dashboard (Table 3E). All of the clinicians worried about the accuracy of the medication list in the EHR and the availability of laboratories, especially those performed outside of the institution.
Most clinicians were also concerned about how using the dashboard would fit into their existing workflows and affect their time management (Table 3F). They worried it would raise a large number of questions from patients that would lengthen the visit. Some mentioned that having medical assistants review the dashboard with patients while checking vital signs before the clinical visit began physicians might be a way to save time. focus groups also generated innovative ideas for expanded capabilities in a dashboard, such as a journaling function to help track self-management strategies. End-user participation is critical in designing a tool that will have a high likelihood of being adopted by its intended audience. 28 This study has several strengths. First, the iterative, human centred-design process engaged end-users at every stage of development. We included a diverse group of patients, including

| D ISCUSS I ON
Spanish-speaking patients. Native Spanish speakers facilitated and analysed data from the Spanish-language groups. We included adequate and limited health literacy patients in the focus groups, which builds confidence in our findings of utility and acceptability of the tool. Facilitators specifically designed the focus groups to solicit patient and clinician concerns about the tool or its use in clinic. As a result, we are more confident that we have a realistic view of stakeholder perspectives on the implementation of the dashboard.
There are some limitations to this study as well. First, as is the case for all small studies, the generalizability of these findings is unknown. We included clinicians and diverse patients from multiple health systems to improve the potential applicability of these findings to other settings, but more research will be needed to confirm our findings outside of this institution. Second, we cannot be sure from the focus group data whether the RA dashboard will, once implemented, significantly change patients' health behaviours, such as improving medication adherence, exercise or other self-management strategies, although some patients predicted it might. Our hypothesis is that by improving patient-clinician communication around RA disease activity, use of the dashboard will promote shared decision-making, which would ultimately improve patient knowledge and engagement, and perhaps medication adherence and patient outcomes. 29 Further studies are needed to evaluate patient activation once the RA dashboard has been piloted.
Work is currently underway to implement the RA dashboard into our local EHR. The dashboard will be launched from inside the patient's chart in the EHR during the face-to-face visit with the clinician, and data elements will be automatically populated from real-time EHR data. We plan for the dashboard to be curated-that is, it will be displayed on the screen in the examination room and its elements can be explained by the clinician; it can serve as a jumping-off point for a conversation between the patient and the clinician in order to support shared decision-making.
Informed by the findings of this study, we have designed the RA dashboard to be partially customizable to patient and clinician preferences. Specifically, for patients, we have made each section of the dashboard expandable and collapsible so that only one section can be 'expanded' at a given time. Additional ideas for future iterations of the dashboard include making it possible for patients to contribute additional information or annotations to the visualizations, such as information about exercise, mood, or sleep; to change the time range of the graphs to the most relevant period for an individual patient; or to edit the 'goal' green zone for each PRO based on their personalized targets. Explanatory 'pop-ups' in each section of the dashboard could also be tailored to the patient's preferences. Training videos or tutorials may need to be developed in order to enhance patient understanding of the RA dashboard.
We also anticipate that the RA dashboard could be made accessible to patients through the online patient portal. On the clinician's side, we can make it possible to add other 'widgets' to the dashboard, including other useful information such as X-ray results, osteoporosis screening results, or relevant vaccination information. Future studies should focus on how different types of patients or clinicians engage with the dashboard and on evaluating the impact the RA dashboard will have on patient care, particularly on disease outcomes, medication adherence and patient self-management.
In sum, using principles of human-centred design, we created a RA dashboard that is well-accepted among RA patients, although clinicians remained sceptical about its implementation. The ability to customize data display preferences is important in tailoring the dashboard to patients with diverse needs and preferences. Special attention should be given to feasibility concerns voiced by clinicians.

ACK N OWLED G EM ENTS
We would like to acknowledge the contributions of Celia Kaplan DrPH (Philip Lee Institute for Health Policy Studies, UCSF) to the conduct of the Spanish-language focus groups.

CO N FLI C T S O F I NTE R E S T S
The authors report no conflict of interests.

DATA AVA I L A B I L I T Y S TAT E M E N T
The data that support the findings of this study are available from the corresponding author upon reasonable request.