The Quintet Recruitment Intervention supported five randomized trials to recruit to target: a mixed-methods evaluation

Objective: To evaluate the impact of the QuinteT Recruitment Intervention (QRI) on recruitment in challenging randomized controlled trials (RCTs) that have applied the intervention. The QRI aims to understand recruitment difﬁculties and then implements ‘‘QRI actions’’ to address these as recruitment proceeds. Study Design and Setting: A mixed-methods study, comprising (1) before-and-after comparisons of recruitment rates and the numbers of patients approached and (2) qualitative case studies, including documentary analysis and interviews with RCT investigators. Results: FiveUK-basedpubliclyfundedRCTswereincludedintheevaluation.Allrecruitedtotarget.Randomizedcontrolledtrial2andRCT5 bothreceivedup-frontprerecruitmenttrainingbeforetheinterventionwasapplied.Randomizedcontrolledtrial2didnotencounterrecruitmentissuesandrecruitedabovetargetfromitsoutset.Recruitmentdifﬁculties,particularlycommunicationissues,wereidentiﬁedandaddressedthroughQRIactionsinRCTs1,3,4,and5.RandomizationratessigniﬁcantlyimprovedafterQRIactioninRCTs1,3,and4.QuintetRecruitmentIntervention actionsaddressedissueswithapproachingeligiblepatientsinRCTs3and5,whichbothsawsigniﬁcantincreasesinthenumberofpatientsapproached.TrialinvestigatorsreportedthattheQRIhadunearthedissuestheyhadbeenunawareofandreportedlychangedtheirpracticesafterQRIaction. Conclusion: There is promising evidence to suggest that the QRI can support recruitment to difﬁcult RCTs. This needs to be substantiated with future controlled evaluations. (cid:1) 2018 The Authors. Published by Elsevier Inc. This is an open access article under the CC BY license (http://creativecommons.org/licenses/by/4.0/).

What is new?

Key findings
The QuinteT Recruitment Intervention (QRI) supported five challenging randomized controlled trials (RCTs) in surgery/oncology to recruit to target. The QRI operated by first investigating the sources of recruitment difficulty in each RCT and then implementing tailored ''QRI actions'' to address the issues as recruitment proceeded.
Four of the five RCTs encountered recruitment challenges, underpinned by difficulties in communicating the trial to potential participants and issues around approaching eligible patients. These issues were addressed through QRI actions delivered as the RCTs were recruiting.
Randomization rates significantly improved after QRI action in three trials that encountered communication difficulties. There was no significant difference in two RCTs, which both recruited above target from their outsets and had received previous QRI support. Two trials that received QRI actions designed to address issues around approaching patients saw significant improvements in the numbers of patients approached per site/month.

What this adds to what was known?
There is a growing body of empirical research devoted to explaining why recruitment can be challenging, but a dearth of effective solutions. Numerous systematic reviews have repeatedly highlighted the paucity of broadly applicable, effective strategies to improve trial recruitment. This initial evaluation suggests that the QRI can effectively identify and address recruitment issues, leading to improvements in recruitment in a range of RCTs.

What is the implication and what should change now?
This evidence provides a platform for trialists to consider adopting the QRI (or similar approaches) in their RCTs, particularly if recruitment issues are anticipated. Future controlled evaluations of the QRI are needed to build upon this promising evidence.

Introduction
High-quality evidence about treatment effectiveness is crucial for improving patient outcomes and promoting judicial use of resources. Well-conducted randomized controlled trials (RCTs) can provide this evidence, but one of the biggest threats to an RCT's success is suboptimal recruitment. This can compromise a trial's statistical power, delay implementation of findings, and lead to costly extensions or early closure [1,2]. Various guidance and tools (e.g.,  are now available to support trial design and planning [3,4], but only half of RCTs meet their recruitment targets [5], and recruitment issues remain the top reason for premature closure [1,6]. The threat of recruitment difficulties can also discourage important RCTs from being attempted.
There is a growing body of empirical research devoted to understanding RCT recruitment issues, but a paucity of effective solutions [7e10]. The little work that has been conducted has predominantly focused on lessons learned from previous RCTs [10]. Promising initiatives, such as the MRC ''Start'' program and the clinical trials transformation initiative, aim to empirically test interventions designed to support recruitment [11,12], but interventions evaluated to date have had a narrow field of application or emerged as ineffective [8]. There remains an urgent need for widely applicable interventions to facilitate RCT recruitment [7,8], particularly for challenging RCTs, such as those comparing very different approaches to managing life-changing conditions.
The QuinteT (qualitative research integrated into trials) Recruitment Intervention (QRI) was designed to optimize RCT recruitment [13]. It evolved over the course of 2 decades since its inception in the ProtecT studyda challenging RCT that randomized men to surgery, radiotherapy, or monitoring for localized prostate cancer [14,15]. Rather than relying on fixed strategies, the QRI delivers tailored solutions to improve recruitment, based on a detailed understanding of trial-specific challenges. The intervention focuses on how trial information is communicated to potential participants and seeks to identify and address other factors that may undermine recruitment (e.g., difficulties approaching potential participants). The QRI has been implemented in a growing number of challenging RCTs [16e20], some of which have now completed and reported [21e23].
The aim of this study was to evaluate the impact of the QRI on RCT recruitment. We undertook a mixedmethods evaluation of all main and pilot RCTs that have integrated the intervention and completed (or stopped) recruitment to date. Our primary objective was to investigate the impact of the QRI on recruitment rates. Secondary objectives were to explore the QRI's likely mechanisms of action and identify opportunities for its refinement.

Overview of the QRI
The QRI is designed to be integrated with an RCT's recruitment plans. It aims to rapidly understand recruitment issues and then implement solutions to address these while recruitment is underway. This is achieved through two iterative phases [13]. In brief: Phase 1 is a rapid investigation of the RCT's recruitment processes using four core data collection processes: (1) audio-recording appointments where recruiters discuss the trial with potential participants; (2) interviews with recruiters and trial management group (TMG) members; (3) mapping eligibility and recruitment pathways; and (4) scrutinizing trial documentation. The findings from each source are triangulated to determine the key areas of recruitment difficulty and fed back to the chief investigator and TMG. Phase 2 involves designing and implementing a ''plan of action'' to address recruitment difficulties. The plan comprises one or more ''QRI actions,'' typically including confidential feedback for recruiters who audio-record their appointments, group feedback/ training, written ''Tips and Guidance,'' and changes to trial documentation and processes (e.g., patient information leaflets, recruitment pathways). QuinteT Recruitment Intervention actions are designed and delivered collaboratively by the QRI researcher(s) and the TMG.

Study design and sampling
A mixed-methods evaluation of the QRI was undertaken, comprising a quantitative before-and-after comparison of recruitment rates and a qualitative case study analysis of each RCT's recruitment process.
Randomized controlled trials eligible for inclusion needed to have integrated the full QRI and be closed to recruitment (or have completed pilot/feasibility stages). All RCTs that had collaborated with the QuinteT team between 2005 and 2017 were assessed, and fulfillment (or not) of each QRI component was recorded by L.R. and cross-checked by researchers who worked on each RCT (see Supplementary Information 1).

Documentary analysis
The documentary analysis was conducted first, to produce an overview of the recruitment issues identified through the QRI and the QRI actions implemented to address these. Each eligible RCT's protocol, funding application, and articles/reports were consulted for information about the trial and QRI (see Supplementary Information 2).

Quantitative analysis
Recruitment logs maintained by clinical trials units (CTUs) recorded the number of eligible patients approached per month, and the number of those who went on to be randomized, for each site within each RCT. Site-specific data were not available for RCT 4, so total monthly recruitment figures were used instead. Data from only one of the two sites in RCT 2 were included because recruitment log information was not available for the other site. All RCTs recorded the number of patients randomized per month, apart from RCT 3, which referred to the ''number of patients consented''; we therefore used ''consent'' as a surrogate for randomization for this trial.
''QuinteT Recruitment Intervention actions'' were defined as any initiatives to optimize recruitment that had been informed by phase 1 QRI findings, including studywide actions (e.g., dissemination of written guidance on how to explain the trial) or site-specific activities (e.g., feedback to a recruiter). As the timing and nature of the first QRI action could differ across sites in a given RCT, patients approached in the time up to and including the month of the first ''QRI action'' (in their trial and site) were considered part of the ''preintervention'' period; patients approached in any month after were part of the ''postintervention'' period. In the absence of site-specific data for RCT 4, we conservatively assumed that all QRI actions impacted all sites.
The analysis was conducted by an independent statistician (L.J.S.) blind to the RCTs' identities. The analysis plan was determined before accessing the data. All statistical analyses were performed in Stata 15.1. Analysis 1 investigated randomization success (whether a patient who was eligible and approached went on to be randomized). The association between preintervention/postintervention and randomization success was analyzed using a mixed-effects logistic regression model, with preintervention/postintervention and trial fitted as fixed effects, and site as a random effect. Results are presented as odds ratios (ORs) with associated confidence intervals (CIs). Whether the effect of preintervention/postintervention differed by trial was investigated by fitting a ''preintervention/postintervention by trial'' interaction term in the model; if the interaction was significant at the P 5 0.05 level, then results were presented by trial rather than overall.
Analysis 2 examined the number of patients approached per site in the preintervention and postintervention periods. The association between preintervention/postintervention and the number of patients approached per site/month was analyzed using a mixed Poisson regression model, with preintervention/postintervention and trial fitted as fixed effects, adjusting for the number of months of recruitment at a site as the exposure, and site fitted as a random effect. Results were presented as incident rate ratios (IRRs) with associated CIs. Again, results were presented by trial if the preintervention/postintervention by trial interaction was significant at the P 5 0.05 level.

Qualitative analysis
Semistructured interviews were conducted to investigate how the QRI had been implemented in each RCT as a means of triangulating the documentary analysis findings. Interviews also explored trialists' perceptions of the strengths and limitations of the QRI. A key informant sampling approach was used, where RCT TMG members who had been involved in recruitment were identified and invited (via email) to participate. Interviews were conducted between November 2016 and August 2017 via telephone or in person and audio-recorded with permission. A topic guide was used to consistently cover topics aligned with the objectives (Supplementary Information 3). Written consent had already been obtained through the QRIs, and interviewers (L.R. and S.H.) verbally confirmed individuals were happy to participate.
Interviews were transcribed verbatim and analyzed through constant comparative approaches adopted from Grounded Theory [24,25]. This was an iterative process, involving line-by-line coding of data and arrangement of codes into themes and subthemes as new data were obtained and previously coded transcripts were revisited. NVivo (V10, QSR International) was used to organize the data. Interviews were analyzed by L.R., with a subset double-coded by a researcher who had not worked on any of the trials (C.C.). ''Negative'' cases that appeared to contradict emerging themes were sought and presented in the findings where apparent.

Results
Thirty RCTs were considered for inclusion, with start dates spanning 2005-2017. Fifteen RCTs were excluded because they had not finished recruitment at the time of the evaluation, and 10 were ineligible because they did not implement the full QRI (see Supplementary Information 4 for reasons why).
Five RCTs were included in the evaluation. All were pragmatic, publicly funded trials in UK secondary care settings, spanning different clinical specialties and comparing very different treatments, including less/no treatment (Table 1). Four had integrated the QRI from the outset due to anticipated recruitment challenges, and one RCT integrated the QRI partway through, in response to recruitment difficulties. Figure 1 shows the timeline for each RCT, including when the QRI started (initiation of phase 1) and when QRI actions started to be implemented (phase 2). The timing of the first QRI action ranged from month 3 to month 14 of the RCT's recruitment period.
Two trials (RCT 2 and RCT 5) had received support from QuinteT before recruitment started. Randomized controlled trial 5 received up-front training/guidance, including a generic presentation on recruitment challenges/solutions at the trial's launch, customized ''tips and guidance'' documents for explaining the RCT, refinements to patient-facing literature, and some recruiters had attended a generic QuinteT recruitment workshop [26]. Randomized controlled trial 2 recruiters had been exposed to QRI feedback in a previous trial [27]. This prior support did not fulfill the definition of ''QRI actions'' because activities were not informed by trial-specific investigation of recruitment issues. Therefore, the periods up to the first ''QRI action'' were still considered ''preintervention'' in these trials.

Recruitment outcomes
All RCTs successfully recruited their target samples. Four achieved this on time or earlier than planned, without opening additional sites (RCTs 1, 2, 3, and 5). Randomized controlled trial 4, which integrated the QRI partway through its funded period, had an extension and opened eight additional sites.

Analysis 2: changes in the number of patients approached per site/month
The pooled data across all RCTs showed that 1.7 patients were approached per site/month in the preintervention periods, compared with 1.3 patients per site/month in the postintervention periods (Table 3). Considering the trials separately (preintervention/postintervention by trial interaction P ! 0.001; Figure 2, panel 2), the number of patients approached significantly decreased after intervention in RCT 4 (IRR 0.43, 95% CI 0.37-0.51), did not change in RCT 1 (IRR 1.10, 95% CI 0.79-1.53) and RCT 2 (IRR 1.04, 95% CI 0.62-1.75), and showed a significant improvement in RCT 3 (IRR 1.17, 95% CI 1.01-1.36) and RCT 5 (IRR 1.66, 95% CI 1.21-2.28). Recruitment to most trials (RCTs 1, 3, 4, and 5) was undermined through difficulties in explaining the RCT to potential participants (''communication issues''). These included equipoise issues, apparent through recruiters' difficulties in articulating uncertainty, and imbalanced descriptions of the trial arms (where particular treatments were conveyed as being superior or more appropriate). Difficulties addressing patient preferences for particular treatments/approaches were also common, but most problematic in RCT 1, where the majority (O87%) of patients approached declined the trial in favor of a preferred treatment. Audio-recorded appointments revealed a tendency for recruiters to elicit patients' preferences at the start of the consultation and then accept these without discussing the RCT.

Recruitment obstacles
More specific issues arose in individual trials, such as difficulties in articulating trial designs. For example, RCT 3 randomized patients to routine treatment versus ''test-directed'' treatment, whereby patients received a test that was hypothesized to inform their optimal treatment (i.e., ''personalized treatment''). Patients sometimes found it difficult to comprehend the distinction between randomization and ''testing,'' both of which played a role in determining final treatment prescribed. Design-related issues also arose in RCT 4, where surgical recruiters encountered difficulties in articulating placebo and nonsurgical arms.
As an exception, RCT 2 revealed few communication issues to address in the preintervention stage; QRI actions therefore encouraged recruiters to continue their practices, with some suggestions for additional information provision geared toward better informing patients.

Issues pertaining to the number of patients approached
Recruitment pathways varied from one site to another and were inefficient in some sites, as in RCT 5. Interviews  with recruiters from RCT 5 and RCT 3 also indicated that some recruiters felt uncomfortable offering the trial to eligible patients who they felt would be more appropriate for one treatment over another; this appeared to restrict the numbers of eligible patients approached.

QRI actions to optimize recruitment
The QRI actions implemented in each RCT are shown in Table 4. Feedback on recruiters' communication practices was at the heart of each trial's ''plan of action'' to optimize recruitment, delivered to individuals or groups of recruiters (e.g., a single center or multiple centers). Feedback used anonymized extracts from recruitment appointments/interviews to illuminate difficulties and potential solutions, focusing on issues around equipoise communication, articulation of RCT designs, and management of patient preferences. Group feedback sessions offered opportunities for peer-to-peer discussion; individual feedback offered more specific advice, bespoke to a recruiter's practices. Study-wide QRI actions to address communication issues were delivered in all RCTs through trial-specific ''tips and guidance'' documents. These included recommendations for structuring consultations and explaining common concepts (e.g., ''randomization''). Changes to patientfacing documentation were also implemented in some RCTs with a view to conveying equipoise and trial design more clearly (e.g., RCT 3).
Issues around approaching patients were addressed by reviewing summaries of screening logs and exploring cases where eligible patients had not been approached. In RCT 3, hypothetical vignettes were used to prompt discussion around contentious aspects of the eligibility criteria to encourage peer-to-peer discussion. The intention here was to enable recruiters who were less comfortable approaching patients with certain characteristics to gain reassurance from peers who were more comfortable [17]. Scrutiny of patient pathways and suggestions for refining how patients were identified/approached were also discussed in sitespecific feedback sessions in RCT 5.

Overall impact of QRI actions
Randomization success improved in RCTs 1, 3, and 4, all of which appeared to be compromised by communication issues. Communication issues were also apparent in RCT 5, but there was no difference in randomization success (although the trial recruited above target throughout its duration). There was also no significant improvement in randomization success in RCT 2, although the documentary analysis indicated that QRI actions encouraged recruiters to maintain their current successful practices. Randomized controlled trial 2 and RCT 5 recruited above target throughout and both had been exposed to prior QuinteT training.
The numbers of patients approached per site/month improved after intervention in RCTs 3 and 5, in line with documented strategies designed to address issues around approaching patients. No significant improvements were observed in the other RCTs, but there were no QRI actions that specifically targeted approach issues. The number of eligible patients approached per site/month significantly decreased in RCT 4. Informal correspondence with the TMG indicated that fewer eligible patients were being identified in the later stages of this trial, reportedly due to depletion of prevalent cases. The qualitative interviews also indicated that the new centers opened during the postintervention period did not engage as well as original centers, which diluted the number of patients approached per site/month.

Trialists' perspectives on the QRI
Seven trial investigators representing all five RCTs participated in interviews. Their accounts of the QRI activities generally complemented the documentary data. The one exception to this was RCT 2. Although the documentation indicated that the mandatory components of the QRI had been conducted, interviews suggested that the QRI was not fully implemented as it was not deemed necessary (following observation of successful recruitment trends). Randomized controlled trial 2 did not have a dedicated QRI researcher, and the QRI activities were not conducted intensively. This was corroborated by researchers who contributed to RCT 2's QRI, both of whom questioned whether RCT 2 was eligible for the evaluation. Informants' accounts from other trials indicated that the QRI had been delivered as reported in the documentary sources. Trial investigators were forthright about the difficulties of quantifying the impact of the QRI but most felt that QRI actions translated into improved recruitment (Web Appendix 1). Most discussed how the QRI had introduced them to sources of difficulty that they had previously been unaware of, and those with direct experience of recruiting gave specific examples of how the QRI had changed their discussions with patients. Individual/group feedback and written guidance were thought to have catalyzed these changes: Investigator 4: In the very early days, I was a very poor recruiter. [.]. .. And actually I learnt from [the QRI], and the work [QRI researcher] did. Because toward the end I was much more successful, and I had really started to follow the guidance in the tips document. And I think it was extremely useful.
Investigator 5: The problem-solving nature of it was just a different level of consideration on what the issues were.
[.] No questioneno question [in response to being asked if they felt the QRI had an impact]. So this language that I'm speaking now is a direct result of that thinking. I don't think we'd delineated it to the same extent then at all. It seems very straight forward and obvious now [.]dbut this was a completely different world to us. We were just clinicians in a clinic, saying ''there's a trial here [.]ddo you want to go in it?'' [.] I expected, in the beginning, I'll be honestdI expected ''We'll do all this talk and it won't make a difference''dbut it did make a difference.
One informant stood apart, having indicated that the feedback of QRI findings had not been ''surprising,'' although still mentioned specific issues that had compromised recruitment in sites other than their own: Investigator 9: There's nothing that came as a surprise, but I think it was good. I mean, I think it had an impact. I think recruitment suddenly improved somewhere else.
Interviewer: In what way do you think? Several informants highlighted the necessity for chief investigators, CTUs, and recruiters to engage with the QRI and for its processes to be integrated with the trial to facilitate success: Investigator 5: The key thing for me was getting the chief investigator and the team around them engaged with it. It will never work if the chief investigator doesn't get it.
Some questioned whether QRI actions had reached all sites, particularly those less engaged with the RCT, but there were also suggestions that feedback/training was unlikely to be restricted to sites the QRI team directly worked with (Web Appendix 2). When asked to summarize their impressions of what the QRI entailed, most emphasized the feedback/training elements, informed by audiorecordings of recruiters' actual practices. Informants tended to discuss the QRI as a ''complex intervention,'' hesitating to pin-point any one aspect that was more effective than another: Investigator 5: The whole is greater than the sum of its parts.
Investigator 2: I wouldn't want to put it down to one particular thing. I think it was a combination of things that had an effect.
[.] I think it's much better if it's integrated into the whole structure.

Discussion
This article reports the first empirical evaluation of the QRIdan intervention designed to optimize recruitment in challenging RCTs. The QRI's impact on recruitment was examined in five RCTs from oncology and surgical specialties. The QRI identified issues that appeared to undermine recruitment to most of the RCTs, particularly in relation to how the RCT was communicated to patients. QuinteT Recruitment Intervention actions sought to address these difficulties through feedback/training, dissemination of written guidance, and changes to patient-facing documentation. There was evidence of improvement in randomization success after intervention in three RCTs; the two RCTs that did not see an improvement both recruited above target throughout their durations (and had received up- Abbreviations: RCT, Randomized controlled trial; E&A, eligible and approached. a Site data available from one of the 2 recruiting sites. Table 4. Summary of key sources of recruitment difficulty based on phase 1 findings and timing/type of QRI actions to address difficulties

RCT identifier
Key sources of recruitment difficulties to emerge from phase 1 investigation Timing (recruitment mo) and type of phase 2 QRI actions RCT 1 1) Difficulties integrating the RCT into existing clinical service: a Complex recruitment pathways (from point of identifying eligible patients to the recruitment appointment) b Structure of consultations was often not conducive to explaining the RCT rationale; RCT participation was not presented as a viable option.2) Difficulties conveying equipoise when describing trial arms to patients 3) Tendency to accept patient preferences, without exploration 4) Difficulties articulating and explaining trial concepts (e.g., ''randomization'')

RCT 2
Recruitment was perceived to be progressing well throughout, and recruiters were encouraged to continue their practices. Additional tips were provided in relation to: 1) Conveying equipoise during recruitment appointments (when describing trial arms) 2) Explaining trial concepts (e.g., ''randomization'' and ''blinding'') 3) Addressing patient preferences 1) Clinicians' discomfort approaching the full spectrum of eligible patients (clinicians' individual sense of equipoise). 2) Clinicians' and research nurses' discomfort around introducing the study, based on assumptions that patients would not be able to process the offer of trial participation. 3) Structure of consultations was not conducive to explaining the RCT rationale; RCT participation was not presented as a viable option. 4) Difficulties articulating uncertainty and conveying equipoise during recruitment appointments 5) Difficulties explaining the trial design clearly 6) Tendency to accept patient preferences, without exploration 7) Difficulties articulating and explaining trial concepts (e.g., ''randomization'' and ''blinding'') front/prior QRI recruitment training). The QRI also identified issues around approaching eligible patients in some RCTs, resulting in QRI actions designed to improve opportunities for approaching patients. The numbers of eligible patients approached per site/month significantly increased in two trials that specifically implemented these QRI actions. The number of patients approached did not change, or significantly decreased, in the postintervention periods of other RCTs without these QRI actions. Trialists perceived a change in recruitment practices after QRI actions, particularly in relation to how they communicated with patients, with most reporting that the QRI had unearthed issues they had not previously considered. Full integration of the QRI into the RCTs, with support from chief investigators and CTUs, was deemed integral to the QRI's success.
The study had several limitations. We cannot establish cause-effect relationships between QRI actions and recruitment and cannot account for other factors that might have explained changes observed. To minimize these limitations, we examined preintervention/postintervention periods at the level of each site within each trial, rather than defining blanket preintervention/postintervention cutoff periods at the overall trial level. The improvements in randomization success in multiple RCTs support the notion that the QRI actions were genuinely influencing and changing recruitment, although it is possible that recruiters naturally improved their practices over time. The changes observed are unlikely to be due to spontaneous improvements, given that the QRI identified issues that recruiters were not aware of. Furthermore, QRI actions (and improvements) occurred more than 1 year into the recruitment periods of some RCTs, by which point recruiters' practices had likely stabilized.
The evaluation was only undertaken with five UK RCTs and thus would be strengthened by inclusion of more trials spanning a diversity of settings. There were also limitations in the quality and completeness of data available, leading to several a priori assumptions. We assumed that QRI actions affected all recruiters in any given site, and that all sites and recruiters had been exposed to every ''study-wide'' QRI action (e.g., ''tips and guidance'' documents). The large RCTs had many centers that never directly engaged with the QRI, but we counted their recruitment activity as ''postintervention'' following any study-wide QRI actions. These limitations are likely to have resulted in a more conservative estimate of QRI impact, as recruiters not truly exposed to QRI actions may have diluted the ''postintervention'' effect. The qualitative evaluation was limited by the possibility that informants' responses were influenced by knowledge that the interviewer was affiliated with QRI researchers who worked on their trials. As such, the interviews primarily focused on examining how the QRI operated in each RCT, rather than trialists' perceived value of the QRI.
Strengths of the study included the use of mixed methods, which allowed for a detailed understanding of whether (and how) the QRI was likely to have impacted recruitment. The independent documentary analysis and qualitative interviews triangulated our understanding of the potential mechanisms of change in each RCT, and thus plausibility of effect.
The QRI has previously been recognized as a promising intervention, in a field that is largely devoid of effective solutions to addressing recruitment issues [28e30]. When faced with the pressures of improving recruitment, trialists often try multiple strategies concurrently, making evaluation difficult. Randomized evaluations have tended to test relatively simple interventions, such as altering patient information materials or arranging site visits [31e36]. These have largely been shown to have little impact on recruitment, although increasing patients' knowledge about the clinical condition, telephone reminders, monetary incentives, and ''opt out'' processes can be effective [7,8]. However, most of these strategies are unlikely to improve recruitment in complex trials involving life-changing conditions or very different interventions, as considered in this evaluation.
The extensive literature on reasons for poor recruitment supports plausible mechanisms of action for the QRI. Recruiters can experience complex intellectual challenges, relating to struggles with equipoise, role conflict, and concerns about their relationships with patients [37e39]. Retrospective analyses of reasons for trial failure have also indicated that most issues are ''preventable,'' with recruiters' and patients' preconceptions about treatments frequently underpinning reasons for discontinuation [40]. The QRI recognizes that recruiters' attitudes, beliefs, and skills can undermine recruitment at several stages from identifying eligible patients through to approaching them and presenting the option of trial participation. Although not exclusively a ''recruiter-targeted'' intervention, most QRI actions are geared toward altering recruiters' practices through guidance and feedback. Other ''recruiter-targeted'' interventions have taken the form of one-off training [41], although those that have shown positive effects have been context specific [42,43]. Similar to the QRI, one potentially transferable initiative trained recruiters to elicit and respond to patients' views about trial participation; this appeared to be associated with recruitment success when evaluated in a single trial [44]. The theoretical and empirical literature on clinicians' learning underlines the importance of specific, preplanned, data-driven feedback [45,46]. In particular, cognitive learning theory proposes that individuals learn by organizing and relating new information to existing knowledge, with feedback presented as one way of achieving this [47].
Application of the QRI to a growing number of RCTs has provided opportunities to identify common recruitment issues [37,38,48,49] and design pre-emptive training [26]. Randomized controlled trials that have most recently integrated the QRI have included up-front training/guidance, as in RCT 5, which did not see a significant improvement in randomization success, although the numbers of patients approached per site/month increased following QRI actions designed to address issues around approaching patients. It is possible that the up-front training may have enhanced recruitment in the ''preintervention'' period, or RCT 5 might have recruited well without the QRI. Randomized controlled trial 2, which had prior exposure to QRI actions, also recruited above target with no notable issues. Further work is required to investigate the role of up-front generic training versus responsive feedback and any sustainable effects of QRI feedback. Prerecruitment training might address some generic issues (e.g., suggestions for explaining randomization) but is unlikely to address more complex challenges such as equipoise communication, which can be disrupted by trial-specific contextual factors.
Future evaluations of the QRI need to include individual recruiter-level data. Examination of the cost-effectiveness of the QRI would also be valuable, given that the QRI requires time and resources (although likely to be small relative to the costs associated with suboptimal recruitment) [13] and should consider the transferability of QRI feedback/training to other RCTs. Ideally, future evaluations of the QRI should incorporate randomization. Randomizing sites in a given RCT to receive the QRI (or no QRI) would be challenging due to the ''ripple effect'' of QRI actions and cross-site contamination. Randomization could occur at the RCT level, although this would be practically difficult and require large numbers of trials. Despite these complications, the promising evidence needs to be substantiated by future more robust evaluations of the QRI and its components.

Conclusion
The QRI supported recruitment in five challenging RCTs, all of which successfully achieved their recruitment targets. There was strong evidence that QRI actions improved randomization success in the RCTs, with the exception of two RCTs that had received prerecruitment QRI training/ support, which recruited well from the outset. Improvement in the number of eligible patients approached was also apparent in two RCTs where these issues were specifically addressed. Tailored feedback to recruiters was at the heart of the QRI actions and was perceived by recruiters to have had an impact on their practices. While more robust evaluation is required, there is evidence that the QRI has a positive effect on recruitment in a range of RCTs.