FormalPara Key Points for Decision Makers

The clinical evidence supports the use of EXOGEN bone healing system in non-union long bone fractures; i.e., fractures which have not healed after 9 months. The use of EXOGEN in these cases is associated with a cost saving of £1,164 per patient, due to the avoidance of surgery.

There is substantial uncertainty surrounding the use of EXOGEN bone healing system for the treatment of delayed union long bone fractures; i.e., those showing no radiological evidence of healing after 3 months. The uncertainty surrounding the rate of bone healing and the necessity of surgery results in a range of potential cost consequences, some of which are cost saving and some which are not.

1 Introduction

This article presents a summary of the External Assessment Centre (EAC) report commissioned by the National Institute for Health and Care Excellence (NICE) for the EXOGEN ultrasound bone healing system for long bone fractures with non-union or delayed healing. It is part of a series of NICE Medical Technology Guidance summaries being published in Applied Health Economics and Health Policy under the remit of NICE’s Medical Technology Evaluation Programme (MTEP) [1–4].

2 The Decision Problem

2.1 Disease Overview

This guidance relates to long bone fractures with non-union or delayed healing. For the purpose of this evaluation, long bones were defined as the humerus, ulna, radius, femur, tibia and fibula. The time that it takes for a fracture to heal varies from patient to patient. ‘Delayed union’ is said to occur when there is no radiological evidence of healing within 3 months of fracture. ‘Non-union’ is established when 9 months have elapsed since the original fracture, with no visible signs of healing in the past 3 months. Delay in fracture healing reduces patients’ quality of life and general well-being. Treatment may be long and complex, resulting in high costs for the NHS. Donaldson et al. [5] used data from the Health Survey for England 2002–2004 to estimate the incidence of long bone fractures at 1.2 and 0.8 per 100 person-years for men and women respectively, about 5–10 % of which will not heal as expected [6].

2.2 Current Treatment Options

Patients are usually treated immediately after fracture, with open or closed reduction (realignment of the bone ends). The limb is immobilised using a plaster or splint, and possibly with insertion of internal or external fixings. X-rays are used to verify alignment of the bone and to assess progress towards healing through bridging of the gap between the fractured bone ends with new bone cortex. Patients not showing progression to healing by 3 months (delayed union) do not usually receive further surgery at this stage unless they have particular indications, such as an unstable or misaligned fracture or a large inter-fragment gap. Surgery may take place between 3 and 9 months after fracture, but clinical practice varies and decisions about the timing of surgery are made on an individual patient basis. If the bone fails to heal by 9 months after the original injury (non-union), surgery will usually be required. Surgery for delayed or non-union usually involves internal or external fixation and bone grafting (with harvesting from the patient’s iliac crest).

2.3 EXOGEN Ultrasound Bone Healing System: Device

The EXOGEN ultrasound bone healing system (referred to hereafter as EXOGEN) is manufactured by Smith and Nephew. It delivers low-intensity pulsed ultrasound waves, and is licensed for healing non-union fractures and accelerating the healing of fresh fractures [7]. It is claimed to promote bone healing by stimulating the removal of old bone, increasing the production of new bone and increasing the rate at which fibrous matrix at a fracture site is converted to mineralised bone. Successful use of EXOGEN may eliminate the need for surgery and its associated complications. Quicker healing may also have a positive impact on a patient’s quality of life and functional capacity. Long bone fractures are suitable for treatment if the fracture is stable, well aligned and well reduced. It is not indicated for use in fractures of the skull or vertebrae, or in children or adolescents due to skeletal immaturity [7].

EXOGEN is available as two disposable devices. The EXOGEN 4000+ is intended for use in patients with non-union fractures (fractures that have failed to heal after 9 months). The device delivers a minimum of 191 treatments (more than 6 months’ treatment). The EXOGEN Express is intended for use in patients with delayed healing fractures (fractures that have no radiological evidence of healing after 3 months). It delivers a maximum of 150 treatments (less than 5 months’ treatment).

The devices consist of a main operating unit with a permanently connected transducer and a separate fixture strap. The strap is placed around the fractured bone, coupling gel is applied to the transducer head and the transducer is secured directly over the fracture site by a fixture on the strap. If the patient’s limb is immobilised in a cast, a hole is cut to allow access of the transducer to the skin. The device is programmed to deliver ultrasound in 20-minute sessions, self-administered daily by the patient in their home [7].

2.4 National Institute for Health and Care Excellence Scope

The scope specified by NICE defined the decision problem as follows [7]:

  • Population: Patients with long bone fractures with non-union (failure of healing after 9 months) or delayed healing (no radiological evidence of healing after approximately 3 months).

  • Intervention: EXOGEN ultrasound bone healing system.

  • Comparator: Surgical treatment with internal or external fixation, and with or without bone grafting.

  • Outcomes: Bridging on radiograph (three out of four cortices bridged); fracture healing time; return to painless weight bearing; avoidance of further surgery, and device-related adverse events.

3 External Assessment Centre Review

The Birmingham and Brunel Consortium was commissioned by NICE to act as the EAC in the assessment of EXOGEN. The EAC’s role is to review and critique the sponsor’s submission, and to produce a report for the Medical Technologies Advisory Committee (MTAC).

As per NICE requirements, the submission on the EXOGEN device, from Smith and Nephew (the sponsor), was based on the decision problem defined in the scope, and followed a set template [8]. The submission comprised a description of the technology under assessment and the clinical context of its use, followed by a review of the available clinical literature relating to the effectiveness of the intervention and comparator technologies; and an economic submission, with a review of relevant economic evidence, a de novo cost analysis, and a spreadsheet model.

3.1 Clinical Effectiveness Evidence

The clinical evidence submitted by the sponsor was based on 18 studies. There were four randomised control trials (RCTs): two comparing EXOGEN with placebo (sham device) in delayed union of long bone fractures; one comparing surgery with shockwave (which is a different but related intervention) in long bone non-union fractures; and one comparing two different types of graft in surgery for non-union patients. Another non-randomised study compared different types of surgery [9]. The comparisons in the surgical studies were not relevant to the decision problem defined in the scope, and so were treated as case series for the purposes of the evaluation. The remaining 13 studies were case series. The sponsor classified four of the case series [10–13] as ‘self-paired’ studies, as the participants were diagnosed with a non-union fracture with no expectation of healing and EXOGEN was the only change in the treatment regimen.

No evidence was found comparing EXOGEN directly with surgery in the treatment of delayed or non-union fracture. For non-union fractures, there were independent estimates of healing rates associated with both EXOGEN and surgery available from non-comparative case series. The majority of studies reported fracture healing rate and time, but evidence on the other outcomes requested in the scope (evidence of bridging on radiograph, return to painless weight bearing, avoidance of further surgery, and device-related adverse events) was limited. The age of study participants varied (13–92 years), and follow-up times ranged from 2 months to 6 years. None of the studies submitted had been carried out in the UK.

For non-union long bone fracture, Mayr et al. [14] reported a mean healing rate of 84 % (216 out of 256) and mean healing time of 5.3 months from an international register of patients treated with EXOGEN. Similarly, Gebauer et al. [10] presented a healing rate of 90 % for a case series of 51 patients with a minimum fracture age of 8 months treated with EXOGEN. Mean healing time in this latter cohort was 178 days (ranging from 86 to 375 days). A third case series reported a mean healing rate of 66 % and mean healing time of 5.9 months with EXOGEN (range 2.9–12.5 months) [11]. For non-union long bone fractures treated by surgery, healing rates ranged from 62 to 100 %, and healing time from 2.25 [15] to 6 months [16].

The principal trial used to provide evidence of clinical effectiveness in delayed union was Schofer et al. [17], an RCT of 101 patients with delayed healing fractures of the tibia (defined as lack of clinical and radiologic evidence of union, bony continuity or bone reaction at the fracture site no less than 16 weeks from the index injury or the most recent intervention) treated by EXOGEN (n = 51) or placebo (n = 50). There was no significant difference between the groups in healing rate (judged by clinician criteria for healing, not otherwise described) over a 4-month follow-up period: 65 % (33/51) versus 46 % (23/50), hazard ratio 1.69 (p = 0.07). However, significant improvements in bone mineral density and bone gap area at the fracture site (both indicators of progression towards healing) were reported. Mayr et al. [14] reported on 696 patients from the EXOGEN registry with delayed union (3–9 months post-fracture). In this group, 90 % of all long bone fractures healed, with an average reported healing time of 4.4 months. Another case series [11] reported a healing rate of 83 % (follow up not stated) among 40 patients treated with EXOGEN. No studies reporting post-surgery healing rates in patients with delayed healing long bone fractures were presented by the sponsor.

The sponsor cited evidence of adverse events from the Manufacturer and User Facility Device Experience (MAUDE) database. Over a 1-year period (April 2011–April 2012), when approximately 55,000 EXOGEN devices were used by patients in the USA, it reported three incidences of skin irritation due to sensitivity to the coupling gel and one report of increased chest pain caused by potential interference with a cardiac pacemaker. No clinical study reported device-related adverse events and no significant safety concerns were identified in relation to EXOGEN. In contrast, several surgical papers reported adverse events, including postoperative wound infection, osteomyelitis and pain.

3.1.1 Critique of Clinical Effectiveness Evidence

The EAC noted that the sponsor had not always followed recommended practice in searching for clinical evidence. For example, of the databases recommended within the MTEP sponsor’s submission (MEDLINE, EMBASE, MEDLINE In-Process and the Cochrane Library), only MEDLINE via Pubmed was searched, increasing the risk of publication bias. However, the EAC found a similar yield on repeating and extending the sponsor’s search strategy to include EMBASE and CENTRAL, and did not identify any relevant clinical studies—published or unpublished—which had not been included in the sponsor’s submission.

Overall, the EAC considered the inclusion criteria used by the sponsor for the selection of studies to be consistent with the decision problem and appropriate. One of the studies included by the sponsor [18] was excluded by the EAC, because it reported outcome measures outside the scope. The EAC noted substantial heterogeneity between studies. The patient population varied in terms of age, fracture age, fracture type, and smoking status. The position of long bone fractures also varied—the most common were the tibia and femur. The definitions given of delayed union and non-union also varied between studies, some falling outside the scope. In addition, the variation in study location (conducted in twelve countries, and not including the UK) may have implications in terms of transferability to an NHS context, given the different characteristics exhibited by different healthcare systems. This heterogeneity was cited by the sponsor as the reason for not undertaking a meta-analysis. Having reviewed the submitted papers, the EAC agreed that a meta-analysis would not be appropriate.

The EAC agreed, broadly, with the absolute healing rates the sponsor quoted for EXOGEN, (90 % [87–92 %] and 84 % [80–89 %] for delayed and non-union, respectively [14]). These estimates came from a large registry study, which used definitions of delayed union and non-union appropriate to the scope, and reported results separately for different long bones. However, the EAC could not fully support the sponsor’s claim that EXOGEN achieved faster progression to healing than placebo in the case of delayed union. This was because the trial on which this claim was based (Schofer et al. [17]) included patients who, according to the scope, would be defined as non-union (more than 9 months post-fracture), as well as delayed union fractures (3–9 months post-fracture). In addition, although intermediate measures of bone healing (bone marrow density and bone gap) were significantly better among the EXOGEN group, differences in actual healing rates were not statistically significant.

The EAC also concluded that it was difficult to compare healing rates between surgery and EXOGEN in non-union fractures, due to differences in the duration of follow-up. These different lengths of follow-up also made it difficult to summarise and draw any conclusion across the range of surgery studies, despite the fact that most reported high rates of healing.

Regarding adverse events, the EAC agreed that none were reported in the included clinical trials, and that there were few reports of possible device-related adverse events from the MAUDE database. Some details surrounding the sponsor’s search strategy for adverse events were not felt to be transparent; for example, searches of internal EXOGEN complaint databases. No explanation was given of why the search was restricted to the period April 2011–12. Furthermore, the EAC felt that EMBASE, the Cochrane Library and MEDLINE In Process should have been searched, in addition to PubMed. Despite this, the EAC did not identify any additional reports of adverse events from these other sources.

3.2 Economic Evidence

The sponsor identified three economic studies related to the decision problem, including a cost-effectiveness analysis [19] and two costing studies [20, 21]. The two models used by the sponsor for the cost analysis in delayed and non-union were adapted for submission from a model developed by Taylor et al. [19]. This was a cost-effectiveness analysis comparing different treatment options for fresh tibial fractures (which was not in the NICE scope) and also comparing surgery with ultrasound for delayed union. The analysis was based on a Markov model, using monthly cycles over a horizon of 1 year. It adopted an NHS perspective and costs were estimated at 2005/6 prices. The study concluded that for delayed union fractures, the most cost-effective strategy was to postpone surgery in favour of a course of ultrasound therapy, resulting in an equivalent rate of healing at a lower cost (£3,926 for EXOGEN and £6,718 for surgery.)

The sponsor’s submitted cost models adopted an NHS perspective in 2012 prices. For non-union fractures, the submitted cost model evaluated the associated costs and consequences of using EXOGEN (4,000+) at diagnosis of non-union, followed by further surgery if the fracture did not heal within 6 months. The comparator was surgery at diagnosis of non-union fractures, followed by repeat surgery if the fracture had not healed within 6 months. The model contained four health states: non-union fracture; healed fracture; infection and post-infection. All patients began in the non-union fracture health state, receiving either treatment with EXOGEN or surgery. If healing had not occurred within 6 months, it was assumed that further surgery was needed. Only patients in the surgery arm were considered at risk of infection. A similar cost model was submitted for delayed union fractures, which contained five health states: delayed union fracture; healed fracture; non-union fracture; infection and post-infection.

In the delayed healing model, healing rates at 4 months were estimated at 92 % in the EXOGEN (Express) arm [14] and 65 % in the surgery arm [17]. These rates were extrapolated over a 6-month period. In the non-union model, healing rates for both the EXOGEN and surgery arms were estimated at 86 % at 6 months [10]. A monthly infection rate of 1.4 % was used to inform transitions to the infection state following surgery for non-union [22]. Costs were estimated using a micro-costing approach. Evidence on resource use was taken from a range of sources, including expert opinion and existing NICE guidance—The Management of Hip Fracture in Adults (CG124) [23]. The sources for unit costs included existing NICE guidance [23], personal correspondence with the sponsor, expert opinion, and existing published cost estimates.

The results of the sponsor’s base-case analyses found EXOGEN to be cost saving for both delayed union (cost saving of £684 relative to control) and for non-union (cost saving of £2,310 relative to surgery). Deterministic sensitivity analysis was carried out to explore the impact of parameter uncertainty on the incremental cost of EXOGEN, varying the healing and infection rates. For the non-union model, EXOGEN remained cost saving for all scenarios tested. For the delayed union model, EXOGEN ceased to be cost saving when its healing rate was reduced from 92 % to less than 82.8 % (assuming a rate of 69 % for the control group).

3.2.1 Critique of Cost Evidence

The search methods used by the sponsor to identify economic evidence were run on PubMed only. The EAC noted that a broader strategy, using MEDLINE and EMBASE, as well as searching NHS EED or EconLit is usually recommended. In addition, the searches were limited to English language (as with the searches for clinical evidence), which may possibly have led to relevant studies being overlooked. The EAC, however, did not find any additional relevant studies when running these additional searches. The studies included by the sponsor were all consistent with the scope of the study and the clinical evidence.

The EAC noted a lack of clarity in the submission surrounding certain aspects of model development; for example, how expert opinion was elicited when checking the face validity of modelling and the clinical pathway. The EAC questioned the use of certain sources used in the models to identify and value resources, in particular the use of existing NICE clinical guidelines on hip fractures (CG124) [23], which may not be relevant to this patient population. The EAC suggested that routinely available reference cost data might have been a more appropriate source of cost information to the NHS.

The EAC considered a number of assumptions in the sponsor’s models to be unjustified, when reviewed in relation to available evidence. The most significant issue related to the way in which clinical data was used to estimate healing rates for the models, and the way in which they were extrapolated over the modelled time horizon. The EAC accepted the assumption of equal healing rates for surgery and EXOGEN in the non-union model [10], but argued that this assumption should be tested in sensitivity analysis, given the lack of comparative evidence on this point. However, they challenged the approach taken in the submitted model for delayed union. This used a healing rate for the control intervention (wait until non-union for surgery) from the control arm of the Schofer et al. RCT [17], while the healing rate for the EXOGEN arm was taken from the Mayr et al. registry study [14]. This coupling of absolute event rates from independent studies is susceptible to bias due to differences in the underlying study populations and methods of outcome assessment: it ‘breaks randomisation’. The EAC argued that a more robust approach would be to define a ‘baseline risk’ likely to be relevant for the clinical scenario of interest (e.g., from the EXOGEN registry data [14]) and then to “model backward”, applying a relative risk adjustment from comparative trial evidence to estimate risk in the control arm [17]. The EAC also questioned the way in which 4-month healing rates from the literature had been used in the delayed union model to estimate the monthly healing rates, and to extrapolate up to the point of non-union (6 months after onset of delayed union).

In addition, a number of coding errors were identified in the submitted model. For example, the submission stated that non-procedure costs should be equal in both modelled arms; however, resource use differed between arms in the non-union model, increasing the costs of each health state in the surgery arm (by £100 in all health states). The EAC also noted that the costs of the EXOGEN devices in the model differed from those stated in the submission.

The EAC made a number of changes to the sponsor’s models, correcting inconsistencies and errors, and conducting additional sensitivity analysis. The results of the EAC sensitivity analyses in the non-union model found that EXOGEN remained cost saving for all scenarios tested, even when the healing rate with surgery was increased to over twice that of EXOGEN. However, the magnitude of the estimated saving was somewhat less than that estimated in the submission; a base-case saving of £1,164 instead of £2,310.

In the delayed-union model, the EAC estimated results for eight scenarios, based on healing curves resulting from a range of plausible interpretations of the evidence. These included different sources of healing rates with EXOGEN (92 % [14] vs. 65 % [17] healed at 4 months after onset of delayed union), different assumptions about the minimum time to heal (0 vs. 2 months after onset of delayed union), and the persistence of the relative benefits of EXOGEN (hazard ratio of 0 vs. 1.69 [17] between 4 and 6 months after onset of delayed union). The EAC specified their ‘preferred scenario’, which they considered to be most plausible. It assumed a healing rate with EXOGEN of 92 % at 4 months [14], and 1.69 hazard ratio [17] for EXOGEN versus placebo, with a 2-month delay before healing is observed, and no persistence of benefit of EXOGEN treatments after 4 months. Based on this preferred scenario, the EAC estimated that using EXOGEN at delayed union would be £504 more expensive per patient than waiting and then treating surgically if necessary at non-union. The scenario most favourable to EXOGEN assumes a healing rate of 46 % for controls and 65 % for EXOGEN at 4 months, with a 2-month delay before healing is observed and a persistent enhanced healing rate of EXOGEN beyond the end of treatment. Under this scenario, early use resulted in a cost saving of £390.

3.3 Conclusions of the EAC

The EAC found the clinical evidence supporting the use of EXOGEN to be limited. There was no direct or indirect evidence comparing healing rates associated with the treatment of delayed union fractures of long bones between EXOGEN and surgery. Consequently, the comparison requested in the scope could not be directly evaluated. However, a randomised trial [17] comparing early use of EXOGEN (at 3 months) and observation followed by surgery (if needed at 9 months) was available, and this was thought to be a clinically appropriate comparison in the NHS. This trial found a significant improvement in measures of bone healing, but was not powered to detect a difference in healing rates. The trial was of a good quality, but it included a mix of patients with delayed union and non-union fractures, with no sub-group analysis. It therefore could not be said with certainty how applicable the results were to the specific context of delayed union. The sponsor reported a cost saving of £684 per patient associated with early use of the EXOGEN system (at 3 months post-fracture). The EAC found this result not to be robust to a range of plausible interpretations of the clinical evidence. Under their preferred analysis, the EAC estimated that early use of EXOGEN would cost around £500 more per patient than waiting for surgery at non-union. However, under an optimistic interpretation of the clinical data, early treatment would save about £390 per patient.

There was also no direct evidence comparing healing rates for EXOGEN and surgery in non-union fractures of long bones. There was a realistic estimate of absolute healing rate with EXOGEN from a large registry study [14], supported by evidence from smaller case series. Similarly, there were case series estimates of the healing rate associated with surgery, although these were subject to more uncertainty due to possible reporting bias. For the non-union costing model, equal healing rates were assumed for surgery and EXOGEN. The sponsor reported a cost saving from the use of EXOGEN in non-union fractures of around £2,310. This is a much larger difference than in the delayed union model, as the assumption is that without EXOGEN all patients with non-union fractures will require surgery. The EAC estimated a more modest cost saving of £1,164 per patient with EXOGEN.

Adverse events related to EXOGEN are rare, and it seems likely to carry a much lower risk of adverse events than surgery.

4 NICE Guidance

4.1 Draft Recommendations

MTAC met in October 2012. Following review of both the sponsor’s submission and the EAC report [24], in conjunction with evidence from expert advisers and patient testimony, the following provisional recommendations were issued [25].

  1. 1.

    “The case for adopting the EXOGEN ultrasound bone healing system to treat long bone fractures with non-union is supported by the clinical evidence which shows high rates of fracture healing and by the cost consequences of an estimated saving of £1164 per patient compared with current management, through avoidance of surgery.

  2. 2.

    There is some radiological evidence of improved healing when the EXOGEN ultrasound bone healing system is used for long bone fractures with delayed healing, but there is high uncertainty about the rate at which healing progresses between 3 and 9 months after fracture and about whether or not surgery would otherwise be necessary. These uncertainties result in a range of cost consequences, some of which are cost-saving and others more costly than current management.”

4.2 Committee Considerations

The Committee considered it acceptable for the cost models to be limited to tibial fractures, as the tibia is the most common long bone for which treatment of non-union is needed. In the case of delayed healing, the Committee was advised by the EAC that the methods used by the sponsor to derive healing rates from the clinical studies [14, 17] were likely to represent an overestimate of the effectiveness of EXOGEN relative to the control arms [25].

For long bone fractures with non-union, the Committee accepted that treatment with EXOGEN resulted in cost savings. It was also advised that the costs associated with surgery in the cost models may be underestimated—cost savings could be even greater in practice. This was also the case for long bone fractures with delayed healing [25].

Overall, the Committee considered the EAC’s approach to scenario analyses to be reasonable. For long bone fractures with delayed healing, the Committee accepted the EAC’s preferred scenario as the most likely. The provisional recommendations were subsequently updated to reflect some of the uncertainty expressed in the EAC’s analysis [24] and incorporated into the final guidance, as outlined below.

The Committee questioned whether the 12-month time horizon used in the cost models might be too short, that is, insufficient to reflect differences between the technologies. The EAC advised that extending the time horizon would be likely to have minimal impact on the results—most fractures would heal in 12 months irrespective of intervention [25].

In terms of equality considerations, the Committee considered if the fact that the device is self-administered would render it unsuitable for certain patients. A patient and clinical expert reassured the Committee that it could be easily administered by a carer instead, and did not pose an equality issue [26].

4.3 Final Guidance

The final Medical Technology Guidance document on EXOGEN ultrasound bone healing system for the treatment of long bone fractures with non-union or delayed healing was published by NICE on 9 January 2013 as MTG12. As a result of changes suggested during the consultation process, one of the recommendations was updated to provide greater clarity surrounding uncertainty about the rate at which bone healing progresses. The final guidance now reads:

“There is some radiological evidence of improved healing when the EXOGEN ultrasound bone healing system is used for long bone fractures with delayed healing (no radiological evidence of healing after approximately 3 months). There are substantial uncertainties about the rate at which bone healing progresses without adjunctive treatment between 3 and 9 months after fracture, and about whether or not surgery would be necessary. These uncertainties result in a range of cost consequences, some cost-saving and others that are more costly than current management.”

5 Challenges

One of the primary challenges identified in the course of this evaluation was how to interpret limited comparative evidence of effectiveness. This is not uncommon in the evaluation of medical devices, where comparative trial evidence can be scarce. In this case, there was an RCT of acceptable quality [17], which did provide some comparative evidence of progress towards healing, along with case series showing healing rates in intervention and control populations. However, it was not clear how well this evidence applied to a relatively early use of the device in patients whose fractures might heal without further intervention, due to the heterogeneous nature of the trial population. A further difficulty related to the need to estimate the shape of the healing curve (not just the proportion of fractures healed at one given time point) in order to estimate the impact of the device on NHS costs. The EAC dealt with this challenge by testing a range of scenarios with differing assumptions. Various plausible interpretations of the data led to very different cost estimates. The disparate nature of the study data introduced further uncertainties. None of the study data were based on patients in the UK, coming instead from countries as varied as Egypt, Japan, and the USA. Differences in clinical practice in different healthcare systems can lead to non-representative data, which may not be most appropriate for application to a UK healthcare environment.