The most common use of real world research was to test the outcomes of a clinical intervention or program outside the limitations of a randomised controlled trial by offering it to a geographic set of all eligible patients. This method was used to test the clinical outcomes of a diabetes programme for children and adolescents in Mexico City (Ramírez-Mendoza et al., 2020); renal denervation in two United Kingdom centres (Burchell et al., 2016); telemedicine for chronic wound healing in Normandy, France (Le Goff-Pronost et al., 2018); a multiple sclerosis (MS) performance test technology in a large MS centre in the US (Rhodes et al., 2019); right anterior mini-thoracotomy for isolated aortic valve replacement (isoAVR) (Rodriguez et al., 2014) and implantable defibrillators in a region of Italy (Ghislandi, Torbica, & Boriani, 2013) and nationwide in Brazil (Migowski et al., 2015). Whilst both of the defibrillator studies (Ghislandi et al., 2013; Migowski et al., 2015) analysed hospital data to assess outcomes, and Rhodes et al. (2019) analysed user testing data, the remaining three studies were conducted as clinical studies collecting primary data. Whilst conducting the same model of study, it is interesting to note that each of the authors described the real world element of their study in different way: using RWD (Ramírez-Mendoza et al., 2020), real world experience (Burchell et al., 2016), real world evaluation (Le Goff-Pronost et al., 2018), real world outcomes (Rodriguez et al., 2014) and RWE (Rhodes et al., 2019), with both defibrillator studies described as using a real world setting (Ghislandi et al., 2013; Migowski et al., 2015).
Similarly, real hospital medical record data was used to validate tools, tests and technology, including the use of Israeli health maintenance organization patient data to validate a commonly used Fracture Risk Assessment tool (Goldshtein, Gerber, Ish-Shalom, & Leshno, 2018); analysis of data from the Shanghai Public Health Clinical Centre to assess the effectiveness of an assay test for the detection of childhood PTB among HIV negative children in China (Xia et al., 2020); data from seven UK clinics to assess complications associated with different intraocular lenses used in cataract surgery (Ursell, Dhariwal, O’Boyle, Khan, & Venerus, 2020); and a US hospitals database to evaluate powered versus manual staplers for patients undergoing laparoscopic bariatric surgery (Roy et al., 2017). Rather than using organisational medical record data, Spataru et al. (2021) analysed data uploaded to the digital platform to analyse the effectiveness of an ecosystem (autoinjector and web interface) for human growth hormone therapy. Whilst Goldshtein et al. (2018) and Spataru et al. (2021) both described their studies as using RWD, the other authors described them as using real world setting (Roy et al., 2017), RWE (Ursell et al., 2020) and real world performance (Xia et al., 2020).
Four studies testing the outcomes of a clinical intervention did so through the creation and analysis of real world registry data (Fokkert et al., 2019; Goudman et al., 2021; Lambiase et al., 2014; Rovaris et al., 2017). Fokkert et al. (2019) analysed data from all Netherlands hospitals to assess well-being and decreased disease burden after 1-year use of flash glucose monitoring. Goudman et al. (2021) analysed data from 15 Belgian and 1 French neuromodulation centres to assess the effectiveness of high-dose spinal cord stimulation for patients with failed back surgery syndrome and to develop a model to predict those who would respond well to treatment. Lambiase et al. (2014) analysed data from 29 sites in Europe and New Zealand to assess experience with a totally subcutaneous implantable defibrillator. Rovaris et al. (2017) analysed data from five cardiology centres in Italy to assess clinical outcomes of AF patients treated with the first and second-generation of circular mapping and ablation catheter. Lambiase et al. (2014) and Rovaris et al. (2017) both described their studies as using real world experience. Fokkart et al (2019) and Goudman et al. (2021) however described them as using RWD and a real world setting respectively.
Five studies were conducted to test the effectiveness of a product on the general population (Giraldo-O'Meara & Doron, 2020; Inkster, Sarda, & Subramanian, 2018; McCall, Helgadottir, Menzies, Hadjistavropoulos, & Chen, 2019; Wiecek, Torres-Robles, Cutler, Benrimoj, & Garcia-Cardenas, 2020; Xie et al., 2013). Four of these studies analysed data collected through their online App, publicly available through the Apple store or Google Play Store, designed to provide interventions to address medication adherence (Wiecek et al., 2020); social anxiety (McCall et al., 2019); mental well-being (Inkster et al., 2018) and self-esteem (Giraldo-O'Meara & Doron, 2020). Xie et al. (2013) analysed data from a US national managed-care insurance claims database to examine the effectiveness of switching from needle and vial to disposable insulin pen technology for the control of Type II Diabetes Mellitus. With the exception of Inkster et al (2018) who described their study as being a real world evaluation, the studies (Giraldo-O'Meara & Doron, 2020; McCall et al., 2019; Wiecek et al., 2020; Xie et al., 2013) were described as examining or analysing real world evidence, data or practice.
Rather than conducting studies to test the effectiveness of a product, three studies used their publicly available online App as a tool to collect individual data on sleep habits and daytime functioning and investigate participant characteristics; and to examine data validity, user retention patterns and data-sharing preferences (Deering et al., 2020) and Allergic Rhinitis (AR) and asthma symptoms, and associated medication use and the impact of AR symptoms on work productivity (Bousquet, Arnavielhe, et al., 2018; Bousquet, Devillier, et al., 2018). Each of these studies was described as using RWD.
Two studies assessed both clinical outcomes and costs associated with clinical programs (Queirós et al., 2021; Wang et al., 2019). To do so, both studies analysed hospital data and primary patient data to assess a diabetes lifestyle program implemented in one region of China (Wang et al., 2019) and to describe cataracts-related clinical outcomes, quality of life and costs of the first patients with cataracts who were treated in Portugal after the implementation of the International Consortium for Health Outcomes Measurement standards (Queirós et al., 2021). The authors of both studies described them as using RWD. In addition to these two studies, five others (Bousquet, Arnavielhe, et al., 2018; Ghislandi et al., 2013; Le Goff-Pronost et al., 2018; Rodriguez et al., 2014; Roy et al., 2017; Xie et al., 2013) assessed costs associated with use of the intervention.
Seven of the studies (Chaudhury et al., 2016; Ehwerhemuepha et al., 2020; Li et al., 2021; Lough et al., 2018; Mikheev et al., 2020; Prince & Smith, 2017; Stevens, Rees, & Polman, 2019) analysed could be considered ‘proof of concept’ studies with five using real patient/participant data to develop and/or test the concept/intervention, a fifth using a simulation based on RWD, and one testing the utility of the product with clinicians. Chaudhury et al. (2016) undertook experimental laboratory testing of a new piston-based pulsatile flow pump system against known criteria gathered from real world patient assessment of aortic flows. Ehwerhemuepha et al. (2020) demonstrated that cloud computing solutions that integrate directly with the electronic medical records could be developed for application of data science algorithms in building and deploying predictive models back to the electronic medical record (EMR) through analysis of real EMR data. Li et al (2021) used real patient corneal images to develop and test a computing solution that could be applied as an effective pre-screening tool to eliminate poor quality corneal images prior to diagnostic assessment. Mikheev et al. (2020) tested smoking topography recording devices for capturing e-cigarette vaping behaviour with real users. Prince and Smith (2017) identified how software could simulate mass spectrometry molecular samples comparable to a RWD set. Through analysis of 33 sets of real patient case notes, Lough et al. (2018) tested the utility of a non-invasive urine test with physicians to determine whether making the results of this test available would modify their diagnostic behaviour reflecting the risk of disease for the patient. Stevens et al. (2019) tested the concept of relationships between group identification, participation, and a range of exercise-specific outcomes and broad health indicators in a real-world setting by analysing primary data collected by questionnaire from participants. With the exception of Stevens et al. (2019), Li et al. (2021) and Mikheev et al. (2020) who described their studies as using a real world setting, real world images and a real world setting respectively, the remaining authors (Chaudhury et al., 2016; Ehwerhemuepha et al., 2020; Lough et al., 2018; Prince & Smith, 2017) described their studies as using RWD.