Feasibility testing of a standardised virtual clinic for follow-up of patients after hip and knee arthroplasty

Introduction Over 200,000 hip and knee total joint arthroplasties (TJAs) are performed annually in England and Wales. UK guidelines recommend regular follow-up because missed early failure can result in complex revision surgery, which places additional burden on overstretched orthopaedic services. This study evaluated the feasibility and acceptability of an expert, consensus-based, standardised virtual clinic (VC) approach for TJA follow-up. Methods Five UK secondary care orthopaedic centres implemented a standardised VC. Feedback was obtained through patient satisfaction questionnaires and telephone interviews with arthroplasty care practitioners. Key stakeholders subsequently attended an expert discussion forum to achieve consensus on the final VC format and to address obstacles identified during testing. Results From 19 June 2018 to 11 December 2018, 561 TJA patients [mean age (SD) 70 (9.4) years, 57.8% female, 69.0% hip TJA, 1–28 years postsurgery (median 5 years)] completed a VC. Of these 561 patients, 82.2% were discharged without attending an outpatient appointment and 46 (8.8%) required early face-to-face consultant review. Patient satisfaction with the VC was high (156/188; 83.0%); over 70% of patients indicated a preference for the VC. Discussion This feasibility study suggested significant resource savings, including time spent by consultant orthopaedic surgeons in outpatient clinics, hospital transport and an estimated saving of up to two-thirds of usual clinic-allotted time. The expert discussion forum provided helpful feedback for supporting more efficient implementation of the VC. Conclusions A standardised VC is a feasible alternative to outpatient clinics for the follow-up of hip and knee TJA patients, and is acceptable to key stakeholders, including patients.


Introduction
Hip and knee total joint arthroplasties (TJAs) are increasingly common surgeries performed in the UK. Although rare, TJA failure can occur, and in some cases is asymptomatic. Whereas guidelines recommend regular TJA follow-up, 1 with over 200,000 TJAs now conducted annually the burden of maintaining traditional outpatient TJA follow-up is significant and growing. 2,3 Some orthopaedic centres restrict TJA follow-up to the first year postsurgery or omit it entirely, accepting the risks to the patient and the costs and demands on orthopaedic centres of missing patients who require revision surgery. 4 Outpatient reform is a priority for the NHS, highlighted by the Royal College of Physicians, the NHS 10-year plan and Getting it Right First Time. [5][6][7] Both primary and secondary care were forced to rapidly implement remote consultations for most nonurgent appointments during the COVID-19 pandemic. This could pave the way for a permanent shift in the delivery of many NHS services, particularly using versions of 'virtual clinics' (VCs) using patient-reported outcome measures (PROMs) and standardised forms. 8,9 We evaluated the feasibility and acceptability for patients and orthopaedic staff of implementing a standardised approach to a VC followup approach for the follow-up of patients with hip and knee TJA, 10 and provide considerations for wider implementation in orthopaedic centres.

Setting
Five large orthopaedic centres participated in this study (see Table 1). Sites were approached using snowball sampling to provide a mix of regional teaching hospitals and nonacute outpatient clinics, 11 with variation in their clinical administration and practice.

Procedures
Each site used VC documentation consisting of a patient-completed questionnaire, standardised radiology report and clinical management algorithm. 10 Sites were requested to use the VC on 100 consecutive patients due for TJA follow-up.
Sites could adapt the guidance provided for implementing the VC to account for local variations in clinical practice, but were not permitted to change the content of the VC documentation. Arthroplasty care practitioners (ACPs) implemented the VC at each site as usual care. Patients were sent the questionnaire to complete at home with a leaflet explaining the VC and an invitation to arrange an x-ray at a convenient x-ray department. Patients returned the questionnaire when attending for the x-ray. Site 3 (see Table 1) adapted the VC questionnaire, without changing its content, for completion on MyPathway, 12 a system for coordination of orthopaedic care pathways using phones, tablets and personal computers. Patients had the option to download a hard copy, or request one from their ACP. Follow-up telephone interviews with ACPs enabled any problems with either the documentation, procedure or implementation to be discussed.
Following completion of the VC, patients were sent a patient satisfaction questionnaire consisting of four yes/ no questions, and five Likert responses (using smiley faces). Patients could write comments in two 'free text' spaces to describe why they were dissatisfied with the VC and for patients to suggest improvements to the VC experience, respectively.

Expert discussion forum
An expert discussion forum was held with the aim of achieving consensus from a range of experts on the final design of the VC, and for gathering their advice on implementation in UK clinical practice. Invitations were sent through relevant orthopaedic and radiology associations, to patients through our PPIE networks and NHS clinics and to health professionals from each of the five sites who had implemented the VC and those who had previously helped to develop it. 10 ACPs from each participating site described benefits and challenges with the VC, and how they overcame any difficulties. The expert discussion closed with a debate and anonymous vote on each of six questions (given in Results) to support the study aims.

Statistics
Statistical analyses were carried out with SPSS Statistics for Windows, version 22.0. Descriptive statistics are presented as arithmetic mean, standard deviation (SD) or median and interquartile range as appropriate, and absolute and relative frequencies as appropriate for demographic and categorical variables on the patientreported questionnaire, standardised radiology report and patient satisfaction questionnaire. Outcomes of the VC are presented as absolute and relative frequencies.
Exploration of potential associations between VC item responses and patient satisfaction questionnaire feedback were decided a priori, but the association between satisfaction (with the joint) and a preference for face-  to-face clinic appointments was requested by the expert discussion forum delegates. Chi-square test p-values were adjusted to account for multiple testing using a two-stage false-discovery-rate-controlling procedure, 13,14 with alpha set at 0.05. These corrections were performed using R version 3.5.2 (package multtest v2.28.0). 15 Feedback from interviews with the ACPs, from patient satisfaction questionnaires and from the expert discussion feedback was summarised according to key issues.

Results
From 19 June 2018 to 11 December 2018, 561 TJA patients (42.2% male) aged between 28 and 92 years (mean (SD) age 70 (9.4) years) completed a VC (see Table 1). Some sites had lower numbers of patients than anticipated because of delays in obtaining approval from their Research & Innovation (R&I) departments or clinical managers. The shortfall was made up by other sites. Patients demographics were similar to national averages for hip and knee TJA patients. 2 Length of time between surgery and follow-up ranged from 11 months to 28 years (mean (SD) 6 years 5 months (5 years 6 months); median 5 years 3 months). Overall, more patients were followed up for a total hip replacement (THA, 387/561, 69.0%) than a knee replacement (TKA, 174/561, 31.0%), although there was some variation across sites. Of 387 hip TJAs, there were 333 cemented acetabular components (86%), 54 uncemented acetabular components (14%), 315 cemented femoral components (81.4%) and 72 uncemented femoral components (18.6%).

Patient-reported questionnaire
A full breakdown of VC responses for all 561 patients is shown in Table 2. For example, pain (indicated on an Tibiofemoral joint space or displacement 1 Periprosthetic fracture 0 1 One referred for urgent follow-up, probably in combination with pain score. 2 One referred for urgent follow-up, probably in combination with pain score. 3 Two referred for urgent follow-up. 4

Radiology report
Problems on x-ray were identified for 54 (9.6%) patients, with 4 patients reported as having two problems. X-rays were examined at each site by ACPs using their usual means of x-ray review, but reported using the VC standardised x-ray report developed as part of the VC. 16 No problems occurred in the 54 uncemented acetabular components but 25 problems were identified in the 333 cemented acetabular components. There were 5 problems identified in the 72 uncemented femoral components of the hip TJA compared with 11 in the 315 cemented femoral components.

VC outcome decisions
The VC decision algorithm ( Figure 1) defines three possible outcomes, including long-term routine follow-up or discharge in line with British Orthopaedic Association (BOA) guidelines. 1 In the 523 patients for whom the outcome was recorded, VC outcome decisions were similar for patients with hip and knee arthroplasty ( Table 3). The majority of patients (hip: 308/369 (83.5%); knee: 122/154 (79.2%)) were either discharged or scheduled for long-term follow-up (eg, in 2 years), and 30/369 (8.1%) of patients with hip arthroplasty and 17/154 (11.0%) of patients with knee arthroplasty were listed as 'review within 3-12 months'. This suggests that, in these patients, no problems were identified on x-ray and that either the patients indicated no problems on the questionnaire, or that any points raised on the questionnaire were addressed satisfactorily by a phone call from the ACP.
Of the 523 patients, 46 (8.8%) were recalled for a consultation at the next available clinic (31/369 (8.4%)/ 15/154 (9.7%) of hip/knee patients, respectively). These appointments are initiated if (1) a problem is identified on the x-ray (note that this would be checked by the orthopaedic surgeon after being flagged by the ACP); (2) there is pain of 4/10 or greater (but only after symptoms are checked through a phone call by the ACP); or (3) any questionnaire items are endorsed (again, only after a phone call to check by the ACP).

Expert discussion outcomes
Voting results of the expert discussion forum, feedback and considerations for implementation are presented in Tables 4 and 5. Important recommendations for improving communication with patients were noted and included in the ongoing use of the VCs at sites.

Feedback from patients using the VC
Of 561 patients, 189 (33.7%) returned VC satisfaction questionnaires, and their feedback and suggestions about the VC are shown in Table 4; 156/188 patients (83.0%) were satisfied with the VC, and this was similar for hip and knee patients. Seven (4.9%) hip patients and four (8.6%) knee patients were dissatisfied with the VC. Over 70% of patients (n=129) said that they would prefer a VC to a face-to-face appointment.

Planned examination of associations between VC items and outcomes
Results of planned investigations into associations between VC items and/or patient satisfaction are listed in Table 6. There were associations between radiological problems and patients who reported falling, an unstable joint or joint pain.
Patients who reported dissatisfaction with the VC were more likely to have had pain, to have had concerns with other joints or had not received a letter with feedback from the VC review.

Discussion
Information gathered from five diverse orthopaedic centres suggests that, logistically, the standardised approach for VC follow-up of TJA and management of patients 'at risk' could be integrated feasibly into routine clinical practice. One site showed that adaptation of the VC for online use to permit fully automated administration, with computerised preparation and printing of the clinic outcome letter, and thus reduced administrative workload, was also feasible. These benefits are similar to other online PROM capturing systems. 8 Paper versions are likely to be necessary for the foreseeable future, as the site reported that 30% of its patients did not have online access and, in one of its local economically deprived satellite towns, 40% of patients had 'no broadband or desire to get online'. However, research suggests that the older population are increasingly moving online and will adapt to new technology. 8 Overall, 8.8% of patients for whom outcomes were recorded were recalled for a consultation at the next available clinic. This figure is lower than a previously published single-site study where 15% were recalled, and an Australian study of virtual hip and knee follow-up using a radiograph combined with an Oxford Hip or Knee Score rather than a consensus-designed tool, where 25% were recalled. 9,16 However, it is still much higher than the cumulative percentage probability of revision at any of the time points for which this is recorded (0.78% and 0.47% for hips and knees, respectively, at 1 year, and 6.83% and 3.82% for hips and knees, respectively, at 13 years). 2 There was a large difference between sites in the number of patients recalled for urgent follow-up. This could be explained by previous experience of sites using this VC, eg, site 1 had used a similar version and recalled no patients urgently, but did instead list at least four times as many patients for 3-12 month review.
Based solely on the algorithm, we identified 75% patients who met the algorithm criteria for discharge or future long-term follow-up, either immediately or following telephone review. In reality, 77% patients were discharged or placed on long-term follow-up, indicating that sites were following the algorithm to determine the ongoing management of patients reviewed through the VC. A 1-year follow-up audit of our previous single-centre VC study did not identify any patients that had re-presented with problems following discharge from the VC. Future work will audit long-term follow-up of patients reviewed through the standardised VC to ensure that patients with potential problems are not missed.
Most (13/19 68%) of the expert discussion forum attendees supported a national roll out of the VC, but 5 (26.3%) did not. One surgeon suggested that, for some orthopaedic centres, successful implementation of the VC will require a cultural change. However, he felt that his

VC timings
ACPs reported that the new VC ranged from two minutes for patients reporting no problems to ten minutes for complex patients, and that it proved a useful training aid for ACPs. ACPs found the flexibility of the VC helpful; however, they emphasised that the VC requires allocated time in the working timetable, and that the time spent assessing VC patients (up to 25 per hour) is demanding and needs intense focus and concentration.
VC assessment should be scheduled into the clinic timetable.

Administration difficulties
ACPs reported occasionally forgetting to include the information leaflet with the clinic 'appointment' letter, causing problems such as patients arriving at outpatients departments for their 'virtual' clinic appointment, some having arranged hospital transport.
Communication with patients at time of surgery regarding likely VC follow-up mechanism would help to ensure that patients are not confused by the information received.

Urgent recall appointments
One unexpected problem reported by surgeons was the increased length of face-to-face orthopaedic clinic sessions (for patients who were recalled to be seen at the earliest opportunity). Recalled patients often presented with numerous potential areas of concern, causing each appointment to take longer than the standard clinic time. This had a knock on effect for the clinic as there were no 'straightforward' patients on the clinic list who help overrunning clinics to make up time.
Clinic schedules should be adapted to allow longer appointments for recalled patients.

Use of hospital resources
ACPs reported that, with fewer clinic appointments, use of hospital clinic facilities was greatly reduced, as were hospital transport costs, and the VC saved up to two thirds of usual clinic-allotted time. However, there was a significant increase in administrative burden for ACPs.
Employment of dedicated administrative assistants to support the delivery of VCs underpins their success. The increased cost of this administrative support is counteracted by savings in AHP and surgeon time, as well as hospital infrastructure costs.
Feedback on x-ray difficulties Two sites reported difficulty with capacity in x-ray departments, and a third experienced an additional administrative burden by having to agree appointments for x-rays in advance on three days per week; in contrast, in clinics where patients were able to attend radiology on any day, within a window, and have an x-ray at their convenience, the VC worked more efficiently. Patients also reported increased satisfaction with this arrangement.
Radiology should consider whether flexibility can be provided for patients to attend and how they can encourage attendance at times when the service is less busy.

Potential problems of radiology report
Radiologists pointed out that surgeons and ACPs will need to take responsibility for identifying metastases, hernias and other problems, because the radiologists see only a small fraction of TJA x-rays. However it should be noted that the VC formalises and standardises a practice currently carried out across the UK and is no worse than the current outpatients system for follow-up of TJA in which x-rays are not reported on by radiologists. Nevertheless, the forum discussed a number of options designed to address this issue: • x-rays of the approximately 12% of patients not discharged by the VC could be sent for x-ray reporting by radiologists; • patients that report pain but have x-rays on which no problems are identified could have a second x-ray report, but completed by radiologists; Radiology reporting procedures and a system for checking and flagging other radiological problems should be considered and agreed at a local level. Flexibility is key.

258
Ann R Coll Surg Engl 2023; 105: 252-262 • that the radiology report includes a check box to indicate that other radiological problems ('incidental findings') have not been identified.
Radiologists further suggested inclusion of an item to check for loss of concentricity of acetabulum, which can be the only x-ray finding for cases of ceramic head failure. Ceramic head failure is known to be more common than ceramic liner fracture, another potential mode of failure and a radiologic red flag which was an item in the radiology report but which did not occur in the x-rays of any of the 561 patients.
This item and the checkbox will be included in future versions of the VC.
Patient feedback on satisfaction questionnaire Two-thirds of patients who had expressed satisfaction with the clinic still expressed a preference for a face-to-face appointment. However, the results showed that patients who expressed preference for a face-to-face clinic had not received a clinic outcome letter, and half of the patients who received no feedback (22/44) about clinic outcomes left comments suggesting that this was a major problem with the VC.
A letter reporting joint status and clinic outcome should be generated for every patient and a copy sent to the patient as well as the GP Patients suggested that the questionnaire should ask about their other joints. The covering letter should include reassurance that, if the patient has other TJAs, they will be evaluated in a separate future VC. In addition, the covering letter should state that if the patient was experiencing symptoms in a joint not previously referred for an orthopaedic opinion, the patient should visit their GP.
Fourteen patients were confused about the VC, with some not understanding what one was or expressing confusion that they were not going to be seen for up to five years, or why they were not able to discuss their other joints. There were 28 patients who wanted to see a specialist under any circumstances, whether or not they were having difficulties or pain. Some patients complained of the impersonal nature of the VC, and the lack of opportunity to discuss other medical problems and medication.
A VC leaflet should be sent out with all clinic administration letters and the VC questionnaire. It should emphasise that this is about a single joint (and indicate which one); what the patient should do if they have any other joint problems or concerns that they wished to discuss with a medical professional; that testing of the VC shows that if the patient needs a face-to-face clinic appointment with a surgeon, the VC will identify this and will trigger that appointment.
Patients appreciated the convenience of the VC, for example when attending for x-rays; some expressed satisfaction that it took less than one hour, and it had benefits for work and their family. However, for others, x-ray waiting times were a huge source of dissatisfaction, and they would have like to have been able to have their x-ray locally rather than at the central hospital.
See 'Feedback on x-ray difficulties above.' Flexible radiology appointments worked very well in those sites that implemented such a system, and were associated with higher satisfaction with the VC by patients Patients suggested that an online version of the questionnaire would have been welcome. An online version of the questionnaire was successfully implemented at one site. However, this large city-based orthopaedic service found that many patients had no internet access and no plans to get online. ACP = arthroplasty care practitioner; TJA = total joint arthroplasty; VC = virtual clinic assessments and surgery would improve as he would be challenged more often by patients with complications requiring greater experience and skill while his juniors would deal with and benefit from more time with routine assessments and surgeries. Other surgeons recognised the potential pitfall of the VC for placing departments at a possible economic disadvantage, in that the VC could see orthopaedic teams seeing over 80% fewer TJA patients, with a corresponding loss of income per patient not physically seen in clinic. Six respondents suggested that important items were omitted, and that these need to be added. No suggestions were left as to what these items were, however, and we feel it is important to emphasise the extensive iterative process by which the questionnaire was developed by leading experts, 10 both in the forum and from the important societies that are needed to give final approval. This paper is intended to be the first step towards acquiring this approval. As with VCs in other medical specialities, 17-19 the increased administrative burden was the largest difficulty, but this was resolved by dedicated administration support staff. The VC required adaptation of administration pathways, creation of new clinic codes and changes to Patient Administration System pathways.
Patients dissatisfied with the VC suggested that this dissatisfaction could usually be resolved with an information leaflet enclosed with the first 'appointment' letter, and by a clinic outcome letter.
As reported by clinicians in other new VCs, 18 patients identified as having potential problems who are recalled for face-to-face appointments are patients with complex needs who take much longer to assess and consult. We recommend extending the length of appointments allocated to these patients.
We recognise that only five sites took part in the service evaluation. However, each orthopaedic centre across the UK differs in their clinical administration and practice, and the five sites ranged from community clinics to large teaching hospitals. We emphasise that, while the VC documents and algorithm have been developed using expert opinion from senior orthopaedic specialists and patients from across the UK, sites can adapt its implementation to fit with their site's clinical practice.

Conclusion
This feasibility study illustrated that a standardised VC is a feasible alternative to outpatient clinics for the follow-up of patients with hip and knee TJA, and is acceptable to key stakeholders. Orthopaedic centres presented solutions to minor difficulties identified when initially implementing the VC. VC users suggested significant benefits, including savings in resources (time spent by consultant orthopaedic surgeons in outpatient clinics, hospital transport and clinic space), convenience to patients and high patient satisfaction.
NP carried out the day-to-day conduct of the study and prepared the manuscript. EH advised on the appropriate statistical analyses and supervised the statistical procedures and results. PO and AG provided expertise for musculoskeletal radiographical queries and support throughout development and testing of the virtual clinic. MS is the orthopaedic consultant surgeon who has initiated and driven development and testing of the VC, and provided support, guidance and expertise in orthopaedic practice and conduct of the study. SK, PC and GM provided close mentorship, management and supervision of the conduct of the study.

Funding
This study has been part-funded by Versus Arthritis (formerly Arthritis Research UK; grant 21409), and the National Institute for Health Research (NIHR) through the Leeds Biomedical Research Centre. The views expressed are those of the authors and not necessarily those of the NHS, the NIHR or the Department of Health.

Role of the funding source
The funding body had no role in the study design; in the collection, analysis and interpretation of data; in the writing of the report or in the decision to submit the article for publication. All authors are independent from funders and all authors had full access to all of the data (including statistical reports and tables) in the study and can take responsibility for the integrity of the data and the accuracy of the data analysis is also required.

Ethics approval
The VC replaced standard follow-up care at all sites for the duration of the study and was classified as a service evaluation, therefore research ethics approval was not required. Local R&I and/or service manager approval was obtained at each site.
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this license, visit http://creativecommons.org /licenses/by/4.0/.