Patients’ Use of Mobile Health for Self-management of Knee Osteoarthritis: Results of a 6-Week Pilot Study

Background In a previous study, a prototype mobile health (mHealth) app was co-designed with patients, family physicians, and researchers to enhance self-management and optimize conservative management for patients with mild to moderate knee osteoarthritis (OA). Objective This study aims to evaluate the overall usability, quality, and effectiveness of the mHealth app prototype for aiding knee OA self-management from the perspectives of patients with OA and health care providers (HCPs). Methods Using methods triangulation of qualitative and quantitative data, we conducted a pilot evaluation of an mHealth app prototype that was codeveloped with patients and HCPs. We recruited adult patients aged ≥20 years with early knee OA (n=18) who experienced knee pain on most days of the month at any time in the past and HCPs (n=7) to participate. In the qualitative assessment, patient and HCP perspectives were elicited on the likeability and usefulness of app features and functionalities and the perceived impact of the app on patient-HCP communication. The quantitative assessment involved evaluating the app using usability, quality, and effectiveness metrics. Patient baseline assessments included a semistructured interview and survey to gather demographics and assess the quality of life (European Quality-of-Life 5-Dimension 5-Level Questionnaire [EQ-5D-5L]) and patient activation (patient activation measure [PAM]). Following the 6-week usability trial period, a follow-up survey assessed patients’ perceptions of app usability and quality and longitudinal changes in quality of life and patient activation. Semistructured interviews and surveys were also conducted with HCPs (n=7) at baseline to evaluate the usability and quality of the app prototype. Results Interviews with patients and HCPs revealed overall positive impressions of the app prototype features and functionalities related to likeability and usefulness. Between the baseline and follow-up patient assessments, the mean EQ-5D-5L scores improved from 0.77 to 0.67 (P=.04), and PAM scores increased from 80.4 to 87.9 (P=.01). Following the 6-week evaluation, patients reported a mean System Usability Scale (SUS) score of 57.8, indicating marginal acceptability according to SUS cutoffs. The mean number of goals set during the usability period was 2.47 (SD 3.08), and the mean number of activities completed for knee OA self-management during the study period was 22.2 (SD 17.8). Spearman rank correlation (rs) calculations revealed that the follow-up PAM scores were weakly correlated (rs=−0.32) with the number of goals achieved and the number (rs=0.19) of activities performed during the 6-week usability period. HCPs reported a mean SUS score of 39.1, indicating unacceptable usability. Conclusions This evidence-based and patient-centered app prototype represents a potential use of mHealth for improving outcomes and enhancing conservative care by promoting patient activation and patient-HCP communication regarding OA management. However, future iterations of the app prototype are required to address the limitations related to usability and quality.


Introduction
Background Osteoarthritis (OA) is the most common form of arthritis and a leading cause of disability worldwide [1]. Current management strategies for knee OA largely target symptom control with the primary aim of reducing pain, improving function, and delaying the need for surgical intervention [2]. There are currently no disease-modifying therapies for OA, and existing pharmacological treatments for targeting symptoms are largely ineffective [3,4]. Nonpharmacological interventions such as exercise, weight management, and patient education have been shown to be effective in reducing OA-related pain and disability [5,6].

OA Self-management
The latest clinical guidelines for OA recommend self-management, including health education and goal setting, as the core treatment alongside nonpharmacological and pharmacological treatments [7,8] when used as a supplement to medical care [9]. In the context of chronic disease management, goal setting and action planning have been found to improve self-efficacy, encourage behavioral change, and improve health outcomes [10][11][12]. Educational interventions, when delivered alongside exercise and weight loss programs, have also been shown to lead to better treatment adherence, reductions in pain, better self-management, and improved quality of life [13]. OA self-management tools can have an important impact on improving the delivery of interventional programs targeting patient education [14] and behavioral modification [15] and can also influence the effectiveness of patient-health care provider (HCP) communication and shared decision-making [9,16]. As such, acceptance and adoption of mobile health (mHealth) strategies for self-management can help improve health outcomes, reduce costs to the health care system, and encourage patients to take a more active role in improving their health. mHealth Apps mHealth apps have shown promise in supporting patient self-management of health conditions, especially for chronic diseases that require long-term care and maintenance [9]. However, barriers to the adoption and use of mHealth technologies have been identified. First, the lack of user-centered designs integrating patients' needs and preferences intercepts the adoption of and sustained engagement with such mHealth technologies [9,17]. Second, there has been a lack of formal mHealth evaluations to date [18,19], raising concerns about the safety and effectiveness of mHealth technologies [20,21]. Our understanding of the effectiveness of mHealth apps in supporting self-management and their impact on patient-reported outcomes in adults with knee OA remains in its early stages. Thus, there is both a need to fill the gap in the availability of mHealth apps for knee OA management and provide a framework to evaluate mHealth apps targeting OA self-management driven by the priorities and feedback of target users.

Objectives
In our previous work, we explored the perspectives of end users on an mHealth app for knee OA self-management, leading to the co-design of an app prototype aimed at facilitating self-management and improving patient-physician communication [22,23]. The resulting app reflected a consensus of patient and HCP priority functional requirements, achieved through co-design. In this study, the overarching objective is to evaluate the app from both patient and HCP perspectives using qualitative and quantitative assessments. The primary aims of the 2-part patient evaluation are to assess the app by its (1) overall usability and quality, (2) ability to improve patient self-management behavior (goal setting and activity completion), and (3) effectiveness in improving quality of life, patient activation, and patient-HCP communication. The primary aim of the 1-time HCP evaluation is to assess the overall usability, quality, and perceived impact on patient-HCP communication.

Overview
We used a combination of qualitative and quantitative methods and triangulated data for increased rigor to evaluate the app features and functionalities from patient and HCP perspectives. Details on the main features and functionalities included in the app, which were determined in preceding co-design sessions, are summarized in Textbox 1. The app was assessed at baseline by patients and HCPs using semistructured interviews and surveys, followed by a 6-week usability period and a final follow-up survey evaluation for patients only. This study was reviewed and approved by the University of Calgary research ethics board (REB16-1372). Privacy and confidentiality of data were maintained throughout all phases of the study. All personal identifying information was stripped from interview documents and recruitment materials, and all the data were deidentified and stored in a password-protected electronic file on secure servers at the University of Calgary. Textbox 1. Functionalities associated with each feature of the final mobile health app prototype.

Symptom tracking (dashboard)
• Symptoms are tracked on the following dimensions: pain, stiffness, and functional limitation.
• Data on symptoms gathered from the patients were evaluated according to a threshold-based approach based on the Western Ontario and McMaster Universities Osteoarthritis Index criteria.

Goals
• Patients prospectively identify goals from a set list of categories (exercise and activity, pain reduction, and weight loss).
• Goals are further delineated into customized activities using the specific, measurable, attainable, relevant, and time-based goal-setting schema.
• Goals and activities can be tracked over a limited or continuous period.

Activities
• Activity is tracked on the following 2 dimensions: (1) activities related to goal setting (eg, linked to favorite activity, such as golf or biking) and (2) exercise (evidence-based recommendations from a physiotherapist or other health care provider) • Activity categories provided were aerobic activity, aquatic activity, muscle strengthening, and others.

Red flags
• Red flags can be identified by patients in a journal-like manner, including any experienced symptom or activity-related difficulties that users may wish to discuss with their health care providers.
• Red flag categories provided were infection, trauma (eg, fracture), persistent inflammation, warmth, swelling or persistent pain, low mood, activity avoidance, or others.

•
The summary of evidence-based self-management resources for knee osteoarthritis self-management was defined by 3 main headings: (1) information for self-management, (2) exercise therapies, and (3) guidance for goal setting.

Recruitment
Patients were recruited in partnership with the Arthritis Alliance of Canada and the Alberta Strategy for Patient-Oriented Research Support Unit (16/18, 89%) to capture a range of patient perspectives. We also recruited those who were included in preceding co-design sessions and expressed interest in participating (2/18, 11%). Adults aged ≥20 years with early knee OA and who experienced knee pain on most days of the month at any time in the past were eligible to participate. Patient participants completed a baseline (week 0) semistructured interview and survey evaluation, followed by a 6-week usability period and final (week 6) follow-up survey evaluation.

Qualitative Evaluation
Patients were provided with a weblink to the app 1 week before the launch of the baseline evaluation to allow them sufficient time to familiarize themselves with the main app features (dashboard and symptom tracking, goals, activities, red flags, and resources). At baseline, research team members (JM and ST) from the Patient and Community Engagement Research (PaCER) facilitated semistructured talk-out-loud interviews with patient participants, which served as an orientation to the app features and functionalities. The interview guide for the qualitative evaluation can be found in Multimedia Appendix 1. Interviews were conducted via telephone or in person depending on the patients' preferences and were audio recorded for subsequent validation. The main objective of the baseline interview was to elicit patients' initial impressions of the app related to user experience. Patient perspectives on the 5 main app features were elicited on the following parameters: (1) usefulness, (2) ease of use, (3) contribution to OA knowledge, (4) self-management potential, and (5) perceptions of the impact on patient-HCP communication. With the goal of summarizing the patient experience when using different app features [24], PaCER researchers subsequently used descriptive analysis to generate a compilation of descriptive statements that reflected the scope of responses for each of the 5 main app features. The descriptive statements were summarized using the following headings: likeability, usefulness, areas lacking and suggestions, and usability.

Quantitative Evaluation
Patient-reported measures related to app evaluation were collected using surveys administered at baseline and follow-up. The baseline survey collected information on patient demographics, history of OA symptoms and risk factors, quality of life, self-management behavior, and patient activation.
This baseline assessment was followed by a 6-week pilot trial in which patient metadata, including number of goals set and number of activities, were captured from direct inputs into the app and frequency of daily use. The follow-up survey collected information on perceived app usability and quality and reassessed postpilot quality of life and patient activation.

Recruitment
HCPs (n=7) were recruited through our primary care research partners, Enhancing Alberta Primary Care Research Networks and Accelerating Change Transformation Team (ACTT), using purposive sampling [25]. Enhancing Alberta Primary Care Research Networks and ACTT invited HCPs who represented the early majority, defined as the first sizable group of providers to adopt an innovation after it has been established by early adopters [26]. Participating HCPs included those who partook in the preceding co-design sessions (5/7, 71%) and those recruited via existing relationships with practices and primary care networks (2/7, 29%).

Qualitative and Quantitative Evaluations
The HCP assessment involved a 1-time evaluation comprising a semistructured interview and survey (see Multimedia Appendix 2 for the interview guide). During the interview component, 1 researcher from ACTT explored the app with the HCP and, using the think-aloud method [27,28], elicited their views on the clinical utility of the app, with emphasis on the perceived impact on patient-HCP communication. Interviews took place over the phone at a time of the HCP's choosing, were approximately 45 minutes in length, and were audio recorded for subsequent analysis. The audio recordings were not transcribed but used later to validate the notes taken during the call. At the time of the interview, both the researcher and the HCP had access to the app to navigate its functions together.
HCPs also completed a questionnaire to measure their perceptions of the app's usability and quality.

Evaluation Measures
Surveys were administered to patients at baseline and follow-up to evaluate changes in patient-reported outcomes over the course of the 6-week pilot trial. At the baseline patient evaluation, data were captured on demographics, knee symptoms, OA risk factors, quality of life, and patient activation. Quality of life was assessed using the European Quality-of-Life 5-Dimension 5-Level Questionnaire (EQ-5D-5L), a preference-based measure for describing and evaluating health, covering 5 dimensions (mobility, self-care, usual activities, pain or discomfort, and anxiety or depression) scored on a 5-point Likert scale [29]. The 10-item Patient Activation Measure (PAM-10) was administered to patients at week 0 and week 6 to provide a measure of patient activation, defined as a person's knowledge, skills, and confidence related to managing their own health [30]. The PAM-10 was designed to minimize the response burden from its 13-item successor, the PAM-13, and has been reported to have comparable levels of consistency and reliability with the PAM-13 [31]. The PAM provides a measure of a patient's engagement in self-management of their disease and thus enables change in patient activation to be monitored between baseline and follow-up evaluations.
At the patient follow-up evaluation, the EQ-5D-5L and PAM tools were re-administered along with additional measures assessing perceived app usability and quality. App usability was measured using the System Usability Scale (SUS), a well-established psychometric tool used worldwide with high levels of reported validity and reliability [32]. App quality was assessed using the App Chronic Disease Checklist (ACDC), an expert opinion-based checklist developed to evaluate the usability of chronic disease apps for monitoring, self-management, and behavioral change [33]. Finally, 2 sections (app subjective quality and app-specific quality) from the Mobile App Rating Scale (MARS) were used to provide an objective rating of perceived app quality [34], with each question measured on a 5-point Likert scale. However, it should be noted that not all sections from the ACDC and MARS tools were included in the assessment, as they included dimensions beyond the scope of the app prototype features (eg, gamification).
During the 1-time evaluation with HCPs, the SUS and MARS measures were used to assess perceived app usability and quality, respectively, from the provider perspective. Further details on each survey instrument used in the pilot testing evaluation are included in Tables 1 and 2. Table 1. Description of measures used for qualitative evaluation of the app prototype throughout the 6-week pilot trial.

Follow-up Baseline
Semistructured talk-out-loud interview; see Multimedia Appendices 1 and 2 for patient and HCP qualitative interview guides, respectively, and Multimedia Appendices 3 and 4 for qualitative reports from qualitative interviews for patients and HCPs, respectively ✓ ✓ Likeability, usefulness, areas lacking or suggestions, and usability Semistructured talk-out-loud interview; see Multimedia Appendices 1 and 2 for patient and HCP qualitative interview guides, respectively, and Multimedia Appendices 3 and 4 for qualitative reports from qualitative interviews for patients and HCPs, respectively ✓ ✓ Enhancing patient-HCP communication Semistructured talk-out-loud interview; see Multimedia Appendices 1 and 2 for patient and HCP qualitative interview guides, respectively, and Multimedia Appendices 3 and 4 for qualitative reports from qualitative interviews for patients and HCPs, respectively Semistructured talk-out-loud interview; see Multimedia Appendices 1 and 2 for patient and HCP qualitative interview guides, respectively, and Multimedia Appendices 3 and 4 for qualitative reports from qualitative interviews for patients and HCPs, respectively ✓ Improving OA self-management a HCP: health care provider.
low scores correspond to higher activities, pain or discomfort, and anx-HRQoL, and high scores correspond iety or depression). The health state of to lower HRQoL; MIDs f , which is the each participant is determined (series minimum important change in EQ-5D-of 5 numbers corresponding to individ-5L, scores are determined for specific ual selections for each dimension) and patient populations and used to inter-then converted to an index score using pret EQ-5D-5L scores (MID for degenerative knee population=0.20

Data Analysis
Descriptive statistics were computed for instruments measured only once at the baseline and follow-up assessments, including demographic characteristics, app usability (SUS), and app quality (MARS and ACDC), reporting the mean and SD or frequency and percentage where appropriate. Because of the small sample size, normality assumptions were not met; thus, nonparametric methods were used for all subsequent analyses. Wilcoxon matched-pairs signed-rank tests were performed to compare outcomes measured at both baseline and follow-up (PAM and EQ-5D-5L), including P values. In addition, app user metadata collected during the 6-week app usability period were exported, including the frequency of goal setting and activity completion. Spearman rank correlations (r s ) were used to evaluate correlations between patient activation (as measured by the PAM), number of goals set, and number of activities completed for knee OA management, with correlations <0.20 considered negligible [37].
For the descriptive thematic analysis, PaCER researchers generated descriptive statements for each app feature that reflected the scope of the responses. Key messages were then extracted and summarized using inductive coding [38] (Multimedia Appendix 3). The notes taken during the HCP interviews were organized according to participants, questions, and app features. Using the audio recordings, the notes were then expanded upon and validated to ensure that they accurately represented the perspectives of participating HCPs. Once all interviews were completed, primary care researchers from ACTT compared similarities and differences in HCP responses and summarized the findings into descriptive categories by each app feature.

Qualitative Component
The main messages of the qualitative report generated by the research team members from PaCER (JM and ST) are summarized in Table 3. The corresponding detailed summary of the descriptive statements can be found in Multimedia Appendix 3. In terms of overall impressions, participants responded positively to the app, noting that it was simple to use and provided a complete picture of how things were going with their OA, along with a visual record to help them keep track of their self-management activities. Patients thought the app would motivate them to create goals, which in turn would encourage them to complete their activities. In addition, patients were keen on the resources feature, which included recommended exercises and other self-management information. Patients identified the dashboard as the most effective feature for improving communication with their HCPs, providing an overall picture of their knee OA and highlighting issues to discuss with their HCPs, such as pain in relation to activity levels.
However, participants expressed some hesitancy around the potential receptiveness of HCPs to the app, suggesting that this may limit the communication potential of the app. There was wide agreement among patients that the app would likely improve their knowledge of OA, enabling them to pay more attention to their symptoms and identify their limits. Patients emphasized that the resource feature would be most helpful and most frequently used, although those who reported having a greater understanding of OA were less confident that it would advance their knowledge of OA. Finally, most patients were hopeful that the app would help them self-manage their knee OA, and they were tentative to make predictions until they had used the app.
In the last phase of the qualitative evaluation, patients provided specific input on how a subsequent iteration of the app may be improved. Patients suggested that simplifying the data input function, making the app more personalized to individual goals, and enhancing the depth of content would be valuable enhancements (Table 3). Suggestions about making the app more user-friendly included more detailed instructions for each feature (including clickable information icons), expanding data input options (eg, allowing the addition of ≥1 activity or red flag per entry, a larger list of activity inputs, and retrospective data entry), and ensuring consistent terminology.
Recommendations from patients for improving the personalization of the app included the ability to add personal notes for activities and red flags. Feedback on enhancing the app content included adding a function to track pain medications, building in a reminder feature for activities, including more options and guidance on exercises for knee OA, and linking to more existing OA information tools (eg, My Health Alberta).

Demographic Characteristics
A total of 18 patients participated in the pilot trial, of whom 17 (94%) provided complete data at baseline and follow-up. The mean age of study participants was 62.2 (SD 6.9) years; 61% (11/18) of participants were female, and 83% (15/18) of participants had completed postsecondary education (Table 4).

Knee OA Symptoms and Risk Factors
All participants reported experiencing pain, aching, or discomfort in at least one knee within the past 12 months and reported experiencing pain, stiffness, or discomfort in either or both knees on an average of 22 (SD 11.5) days per month. Of the patients diagnosed with OA by a physician (18/18, 100%), 88% (16/18) received their diagnosis at least 1 year before participating in the study. All participants reported engaging in physical activity at least once a week, 94% (17/18) of whom reported stopping or changing the type of physical activity because of knee pain.

Knee OA Self-management
Approximately 78% (14/18) of participants indicated that they had performed exercises or activities to improve their knee OA symptoms at any point in the past, 67% (12/18) of whom had performed pain management activities or exercises >10 times in the past month. Regarding goal-setting behavior, 50% (10/18) of patients reported previously setting a goal to improve their KOA symptoms. Of those patients, 60% (6/10) reported setting at least one goal over the 6-week evaluation period, of whom 50% (3/6) indicated a success rate >50%.

Quality of Life and Patient Activation
Between the baseline and follow-up evaluations, patient activation, measured using the PAM, increased significantly from a mean of 80.4 (SD 9.1) to 87.9 (SD 9.7; Wilcoxon signed-rank test, P=.01). Patient quality of life, as measured by the EQ-5D-5L, changed from a mean of 0.77 (SD 0.13) to 0.67 (SD 0.26) between week 0 and week 6 evaluations, corresponding to a significant improvement in quality of life (Wilcoxon signed-rank test, P=.04; Table 5). The change in EQ-5D-5L score from baseline to follow-up exceeded the minimal important difference (MID) of 0.056 based on the Canadian population [39] but did not exceed the MID of 0.20 based on the Canadian degenerative knee disease population [40].

App Quality and Usability
Following the 6-week pilot evaluation, patients were provided with a follow-up questionnaire to assess app usability and quality, quality of life, and patient activation. According to the responses to the ACDC assessing app quality, 53% (9/17) of patients with complete data reported that the app facilitated appropriate navigation, 65% (11/17) indicated that it was reasonably efficient, 71% (12/17) reported that it was user-friendly, and 88% (15/17) indicated that the app was free from confusing terms or jargon. Regarding the appropriate display of data and information, 88% (15/17) of patients reported that the app produced appropriate graphs or statistics for clinical data, and 77% (13/17) indicated that the app displayed correct and relevant information regarding their chronic condition and that visual explanations of concepts were clear, logical, and correct. In addition, the ACDC identified some areas of improvement for the app, with all patients indicating that the app had none or limited tactile, visual, or sound feedback, and 53% (9/17) reporting that the app did not facilitate ease of entering information. Patient responses to the MARS indicated an overall average rating of 3.1 out of 5 stars (rated on a scale of worst to best app ever used). As measured by the SUS, the mean score for perceived app usability was 57.8 (scale 0-100), indicating marginal usability according to SUS cutoffs [41].

Patient Metadata
During the 6-week observation period, user metadata were collected from the 18 participating patients, including symptoms of pain, stiffness and functional impairment, number of goals set, and number of activities completed toward knee OA self-management. The mean number of goals set during the usability period was 2.47 (SD 3.08), and the median was 2 (IQR 1.0-3.0). The mean and median number of activities completed for knee OA self-management during the study period were 22.2 (SD 17.8) and 18.0 (IQR 5.0-41.0), respectively. Spearman rank correlation (r s ) calculations demonstrated that follow-up PAM scores were weakly correlated (r s =−0.32) with the number of goals achieved and the number (r s =0.19) of activities performed during the 6-week usability period.

Qualitative Component
A detailed summary of perceived likeability and usability, suggestions for improvement, and implications for HCP visits are summarized in Table 6 (see the detailed ACTT report in Multimedia Appendix 4). Overall, HCPs expressed support for the app features and functionalities, favoring the Goals, Activities, and Resources tabs. HCPs identified elements of each main app feature that would be useful for patient self-management and that could be useful in the context of a clinical visit. However, HCPs largely held the perspective that the app was too detailed and cumbersome for the patient population, typically comprising older individuals perceived as having limited technological literacy. Suggestions made by the physicians to make the app more user-friendly to this patient population included increasing the use of lay language to reduce high-level language and the reading level (grade 7 or lower; 3/7, 43%), increasing color contrast and using a colorblind-friendly palette (3/7, 43%), and providing areas to add notes or free text (3/7, 43%). There could be more clarity on how to use the feature-HCPs thought it might be too complex for patients to follow Use of the SMART a goal-setting framework • Incorporation of an assessment of confidence in achieving goals (ie, "how confident are you that you will be able to complete this goal?")-marked on a 5-point Likert scale from not confident (1) to very confident (5) • Improve visuals for easier reading-increase font size and color contrast • Too many categories of goals • Achieved goals should be removed • Keep the page to one screen so that scrolling is not required • Summary of goals and prompts for next scheduled activity is useful

Activities
User-friendly and straightforward data entry Need to include an option to go back to completed activities to discuss with HCP

App Quality and Usability
During a 1-time app evaluation, HCPs (n=7) completed a questionnaire assessing app usability and quality from the patient perspective. According to the responses to the MARS, 71% (5/7) of respondents agreed that the app was likely to increase knowledge of knee OA, 86% (6/7) of respondents indicated that the app was likely to increase motivation to address knee OA, and 71% (5/7) of respondents considered that the app was likely to improve self-management practices. When asked if they would recommend the app to patients with OA who might benefit from it, 57% (4/7) of HCPs responded affirmatively. Overall, HCPs rated the app as 3.0 on a scale of 1.0-5.0, indicating the potential for improvement. As measured by the SUS, the mean score for perceived app usability was 39.1, pertaining to unacceptable usability according to SUS cutoffs.

Principal Findings
We have demonstrated that an app prototype co-designed with end users has the potential to successfully deliver self-management guidance to patients with knee OA. During the 6-week observation period, patients experienced significant improvements in patient-reported quality of life and patient activation and exhibited high levels of engagement with the app, as demonstrated by a high number of activities completed and goals achieved during the usability period. However, usability scores were reported to be in the marginal range for patients and an unacceptable range for HCPs. The overall average app ratings from patients and HCPs demonstrated the potential opportunity for further quality improvement.
The qualitative assessment revealed that, from the patient perspective, the app features, including the dashboard, goals, activities, flags, and resources, were useful and user-friendly but could be expanded to include functions that are more personalized and specific to each patient's lived experiences, such as the ability to add notes and reminders. Patients indicated that they were most likely to use the resources feature, emphasizing that the knowledge aspect of the tool was important to their self-management. Furthermore, patients largely viewed the app as a catalyst to increased autonomy in self-management. Many elements of the app features were viewed as useful by HCPs, particularly those related to the inclusion of evidence-based information; specific, measurable, attainable, relevant, time-based goal setting, and exercise therapies for OA.
However, there were gaps between patients' and HCPs' perceptions of app usability and quality, with HCPs expressing greater concern than patients about the patient's ability to effectively navigate and use the app features. This is consistent with our previous work [22], where physicians and patients expressed different views on the seriousness of knee OA and their approach to its management and also their perceptions of whether those diagnosed with knee OA can manage their disease using an app or other mHealth tools. Despite patient receptiveness to the prospect of using an mHealth tool such as an app, patients and physicians held diverging views, where physicians were concerned about the technological literacy of the conventionally older OA population. Negotiating consensus during co-design about the app features beneficial to both patients and physicians may help align patient and physician perspectives. Furthermore, improved patient-HCP communication and the discussion of priorities and best-available evidence may help bolster shared decision-making [42].

Limitations
We recognize several limitations of our study design. The evaluation sample size was small but aligned with the suggestion by Nielsen et al [43] that for projects of medium to large size, 15-20 test users are optimal to balance evaluation costs with the benefits of finding usability problems in testing. Furthermore, our patients were predominantly female and had higher educational attainment. Thus, our patient sample may not be representative of the target early knee OA population, as research has shown that active participants research may have different motivations and priorities compared with those who are less engaged [44], and departures from representativeness may be amplified with increasing age [45]. This may be reflected by our study findings, in which 88% (15/17) and 100% (17/17) of participants reported PAM scores in the highest range of patient activation at baseline and follow-up assessments, respectively, despite the use of multiple recruitment mechanisms intended to capture a range of patient perspectives.
Similarly, the HCPs involved in the design and evaluation of the mHealth app were primarily family physicians. Thus, our findings may not reflect the diversity of perspectives among HCPs that are typically involved in guiding self-management for patients with knee OA. Further studies will expand the focus of the evaluation to assess its applicability in a more diverse group of HCPs (eg, physiotherapists). In addition, the inclusion of patient partners with a wider range of experiences related to education, health literacy, and technological proficiency in future app development and evaluation is essential to address potential health inequities [46].
There were shortcomings in app usability, as demonstrated by the low SUS scores reported by both patients and HCPs. However, usability challenges were of greater concern among HCPs in comparison with patients who reported higher usability scores and identified minimal to no usability challenges in the qualitative assessment. This discrepancy between patient and HCP perceptions of usability is aligned with the findings from our preliminary work [22], where HCPs largely underestimated patients' receptiveness and ability to use mHealth technologies for OA self-management. It is possible that HCPs evaluated the app usability with a broader OA patient population in mind than that represented in our sample, who were both highly educated and demonstrated high levels of activation in their care.
This study had a relatively short duration of follow-up of 6 weeks. As such, the emphasis on the evaluation of patient-reported outcomes against clinical outcomes was suitable, as it was unlikely that we would observe clinically meaningful changes in OA symptoms during the short observation period. However, although significant improvements in health-related quality of life and patient activation were reported during the study period, the change in EQ-5D-5L scores did not surpass the MID established in the degenerative knee population. Thus, the length of follow-up may have been insufficient to establish meaningful changes in some patient-reported outcomes. Given the chronic nature of OA, using a longer evaluation period for a future iteration of the app would provide more relevant data on clinical outcomes to augment the patient-reported outcomes featured here and additional context for whether long-term engagement with the app could be sustained. Further app development involving an evaluation that is more inclusive of diverse patient and HCP perspectives, with a longer follow-up duration and a wider range of outcomes measured, will be an essential next step in this study. In addition, we are exploring opportunities to integrate self-monitoring data with advanced machine learning analytics to provide an intelligent platform to assist patients with knee OA in initiating and sustaining self-management activities.

Comparison With Previous Studies
To our knowledge, this is the first mHealth app developed specifically to aid in knee OA self-management, although several digital self-management programs have been developed for OA [47] and other chronic diseases, including diabetes [48,49] and heart failure [50]. A 2020 systematic review on digital health interventions for people with OA identified that most (5/8, 63%) studies were primarily focused on health education (n=5), whereas some incorporated additional self-management elements, such as goal setting (n=6), action planning (n=4), physical activity (n=6), weight management (n=5), and pain management (n=6) [47]. A 2017 systematic review focusing specifically on mHealth technologies identified a lack of emphasis on tracking OA symptoms and self-management behavior that could be useful for shared decision-making [9]. The review by Choi et al [9] provided a framework for developing mHealth apps for OA management, describing the need for patient-facing mobile apps with capabilities such as symptom monitoring, activity monitoring, joint function measurement, physical activity guidelines, educational content, and provision of data visualization and summary reports for shared-decision-making. Thus, our app addressed many of the identified market gaps for apps in the OA self-management domain by incorporating capabilities that have been established as integral to facilitating self-management, decision support, and shared decision-making.
Regarding methods for evaluating mHealth tools, a recent scoping review summarizing quality assessment methods for mobile apps in chronic disease management found minimal agreement on the most appropriate criteria for evaluating mobile apps, with only 18% (12/65) of apps including evidence-based health information, 22% (14/65) using a behavioral change framework, and 37% (24/65) applying usability metrics as quality criteria for app assessment [51]. On the basis of the gaps identified in the analysis and practice methods for evaluating mobile apps, the authors proposed 3 primary goals for building quality criteria for app assessment: (1) prioritize existing evidence and knowledge against ease of assessment (eg, using patient-reported outcomes vs app ratings), (2) emphasize principles of behavior change theory, and (3) explicitly incorporate the patient perspective. Our study begins to address these identified gaps in quality assessment methods using a range of evidence-based patient-reported outcomes, incorporating behavioral change principles such as goal setting and activity monitoring, and integrating the patient perspective from early planning to app assessment phases.

Conclusions
This pilot study provides support for the use of evidence-based, patient-centered mHealth apps to improve patient-reported outcomes by encouraging patient self-efficacy and improving patient-HCP communication, ultimately promoting conservative management of knee OA. These findings, along with findings from previous study phases, provide a framework from which app developers and researchers can co-design and evaluate mHealth apps targeting self-management in a way that is inclusive of all stakeholders and reflects diverse user perspectives. Further development of the app to address usability and feasibility in the context of a larger evaluation trial, including a more diverse and representative population sample and a longer period of evaluation, will be instrumental in understanding the impact of the app on self-management and a broader range of clinical outcomes such as pain and disability. In addition, our findings can provide a basis for developing best-practice reporting standards and practices to evaluate evidence-based mHealth technologies that target self-management of chronic conditions. Future app development may involve the integration of machine learning to provide personalized self-management recommendations to patients with knee OA to address individual needs and priorities.