A multiple criteria decision analysis to establish the use cases and candidate point of care tests to enter into a platform trial of multiple in vitro diagnostic point of care tests in the prehospital environment

Background There are increasing demands on Emergency Medical Services. More efficient treatment pathways are required to support conveyance decision making and patient referral in prehospital care. Point of Care testing is increasingly available and utilised across the NHS to support optimal ways of working. We aimed to design and conduct a Multiple Criteria Decision Analysis to prioritise in vitro point of care tests and use cases for inclusion in a platform trial of in vitro point of care testing in UK Emergency Medical Services. Methods We designed a Multiple Criteria Decision Analysis that included systematic scoping reviews stakeholder recruitment, two stakeholder surveys and two stakeholder workshops to scope the use cases, explore criteria and map use cases, evaluate the criteria and measure the use cases against the criteria. Results We recruited 32 stakeholders. We developed a scoring matrix with 4 criteria for scoring the use cases and 8 criteria for scoring the point of care tests and applied weighting determined from survey results. Use cases were scored by the stakeholders against 4 criteria. The 3 highest scoring use cases were point of care troponin testing in: possible Acute Myocardial Infarction, lactate testing in suspected sepsis and in trauma. We developed the process for scoring the point of care tests to be completed close to a proposed trial to allow for a changes in technology. Conclusions We successfully designed a Multiple Criteria Decision Analysis to identify use cases and candidate tests for inclusion in a future platform trial of in vitro point of care testing in UK Emergency Medical Services. We identified 3 use cases for evaluation in a platform trial of in vitro point of care testing: troponin testing in possible acute myocardial infarction, lactate testing in suspected sepsis and lactate testing to identify occult haemorrhage in trauma.


Introduction
Demand on Emergency Medical (Ambulance) Services (EMS) continues to increase 1 .Lengthy delays in patient handover at crowded Emergency Departments (EDs) compromises the ability of EMS to respond to new emergency calls, leading to prolonged ambulance response times 2 .Strategies to improve efficiency by maximising the safe avoidance of patient conveyance to hospital wherever possible, while also ensuring patients who are transported receive the best care and reach the most appropriate facility, are paramount.
An interview study with medical cardiologists and oncologists identified that laboratory testing guides 60-70% of clinical decisions in these contexts 3 .Emergency Departments utilise in vitro diagnostics for swift identification of patient conditions and to support decision making.In vitro Point of Care (POC) testing could enhance EMS by achieving shorter response times, less hospital transfers and reduced ED waits, with potential for substantial cost savings.POC testing is increasingly available and utilised across the NHS.Rapid growth in the POC testing market is predicted and may rival the pharmaceutical industry within 20 years 4 .
Platform trials are designed to test multiple interventions at the same time 5 and in 2023 the UK National Institute of Health and Care Research (NIHR) encouraged funding applications for its Health Technology Assessment (HTA) programme for platform trials of strategic priority 6 .While planning a platform randomised evaluation of clinical and cost effectiveness of in vitro POC testing in EMS in the UK, we required a robust and transparent process to triage POC tests and associated "use cases" for inclusion in a platform trial.Multiple criteria decision analysis (MCDA) is a commonly used method to guide complex decision making in HTA 7 when there are multiple options and multiple relevant criteria to be taken into account during the decision-making process 8 .In particular, MCDA has been most frequently used within HTA to guide resource priority decision-making 9 .This paper describes the process and outcomes of MCDA methodology in the prioritisation of candidate in vitro POC tests and use cases for potential inclusion in a proposed future platform randomised controlled trial of in vitro POC testing in UK EMS.In this paper the term POC tests refers to in vitro POC testing.

Patient and Public Involvement
This work benefitted from a PPI co-applicant who had prior experience of relevant high-quality point of care testing research and who was involved in all aspects of the research.A wider group of Public Advisory Group members were involved in the following ways: Public Contributors (PCs) helped us to understand the range of variation in service provision and patient circumstances.This included looking at the potential alternative care pathways that can be accessed by testing and describing the facilitators and barriers to pre-hospital testing from a patient/ carer perspective.They also advised on the pre-requisites that make pre-hospital testing and alternative care pathways acceptable from a patient and carer perspective.
PCs were involved in the development of the Stakeholder Survey 1 to create the longlist of 'criteria' for assessment and the Stakeholder Survey 2 to define the weighting of criteria.PCs also participated fully in both stakeholder workshops.
We conducted an MCDA to prioritise candidate in vitro POC tests and use cases for inclusion in a platform trial of POC testing in UK ambulance services.The MCDA was conducted in accordance with the principles described in the good practice guidelines for the conduct of MCDA published by the International Society for Health Economics and Outcomes Research (ISPOR) 10 .
The MCDA was conducted between March and September 2023 and the process is illustrated in Figure 1.

Identification of potential use cases
The options (use cases) were identified through systematic literature search and selection process following principles of

Amendments from Version 1
Thank you for taking the time to review this paper and for your helpful comments.
We have added more context to the reference to Rohr et al.'s paper in the Background section.
Thank you for highlighting the point regarding the gold standard for two reviewers to review articles for inclusion in a scoping review.We have added an additional paragraph to the discussion section to address this point.
The study team included clinical leaders in prehospital and emergency care, and also clinicians and researchers with expertise in prehospital POCT.Via personal networks and snowball sampling we identified and recruited stakeholders that we believe were satisfactorily representative, though we accept that it is challenging to determine whether a sample is truly representative and includes the widest possible range of expertise.We have therefore added the following to the "limitations" section.
"Whilst we aimed to recruit a genuinely representative group of stakeholders, using personal networks and snowball techniques, the participants are unlikely to wholly reflect the full range of expertise relevant to pre-hospital POC testing, and as a result some views and areas of knowledge will have been under-represented in this research".
We agree that limitations regarding evidence for POCTs could be more clearly emphasised, and have added the following text to the Discussion section.
"POC testing is a relatively new development in EMS, and as a result the quantity and quality of evidence is limited currently.Even the POC tests that scored most highly in this research, such as troponin in chest pain, have relatively little evidence of their clinical effectiveness and more trials are required 24 .This field is anticipated to develop rapidly over the next few years 4 , and we have established a methodology by which POC tests may continue to be evaluated and selected for further assessment in research".
Any further responses from the reviewers can be found at the end of the article scoping review 11 .We did not assess the bias in the studies identified, nor did we attempt any formal synthesis of the evidence as these are outside of the methods used for scoping reviews 12 .

Searching for relevant studies
To identify POC tests used within, or with potential for use in the EMS we searched three electronic bibliographic databases (Medline and EMBASE on Ovid and CINAHL on Ebscohost).The search was developed by an information specialist (SD) and systematic reviewers (TM,JS) in liaison with the rest of the team.The strategy combined terms for point of care tests with terms for EMS.We did not apply a filter to restrict types of study design.The search strategies are available in an online data repository 13 .We searched reference lists of key systematic reviews.We restricted to English language and research published after 2000 and focused the search on adults.Following an initial broad search for POC tests in the EMS, additional supplementary searches for specific potential use cases were completed between April and August 2023.A table summarising the searches is presented in Extended Data File 1. Titles and abstracts of records were screened by one author (TM) with all excluded records checked for exclusion by a second member of the wider team (see acknowledgements).Full-text versions of the records marked "retrieve" by either reviewer were screened by one author (TM) with all excluded papers checked for exclusion by a second member of the wider team.Strict inclusion and exclusion criteria were used to guide selection, criteria included: reporting on a POC test used in the EMS, setting was the EMS, POC test was using bodily fluids (saliva, blood, serum, breath, sweat).We excluded studies where the focus was use of equipment to obtain a diagnosis from a body scan, such as ultrasound.We excluded tests used in specialist services such as mobile stroke units because of the reliance on specialist personnel and scanning equipment in conjunction with any POC tests.Search data were managed using Rayyan and Endnote software 14,15 .Screening was managed using the Microsoft software Access and Excel 16,17 .

Preparation of stakeholder summaries
Data extraction from studies identified in the scoping exercise was done by one researcher (TM) and checked by a second (KM,RP).Data extracted included: Type of test(s), test device name and manufacturer, study design, condition the test was being used for, biomarkers being tested.Information for potential use cases was organised into summaries for stakeholders into 6 types of information, 1) systematic review of primary research, 2) Economic evaluation of use of POC tests in the EMS, 3) ability to make causal inference for a change in patient pathway, from a) experimental studies with interventions assigned at random, or b) from non-randomised studies with a comparator group or controlled before and after studies 4) demonstration of the POC test being used in the EMS from observational studies describing the use of a POC test (single arm observation studies) 5) Views of paramedics or clinicians or patients or hospital staff on the use of POC tests in the EMS to change the patient pathway (qualitative studies or surveys).Studies investigating the diagnostic test accuracy of POC tests were documented and listed but not summarised as this information is not sufficient to ascertain if a change in patient pathway has potential.The number of records for each use case, and the type of information identified for each were organised into tables and stakeholder summaries that were provided at two workshops.The summaries presented at Workshop 2 are listed in (Extended Data File 1).

Stakeholder recruitment
The MCDA focused on UK EMS care, but recruited national and international stakeholders with the expertise to meet the inclusion criteria described in Figure 1.Stakeholders were identified and recruited during June and July 2023.Eligible stakeholders, who had been identified as lead authors of research found during the scoping review, or whose work and expertise was familiar to the study team, were contacted.Snowball sampling was then employed to recruit additional stakeholders.Initial contact was via an email providing information about the study.Where stakeholders expressed an interest in participating, they were provided with a participant information sheet and a consent form and asked to consent to participate in the study.In addition to seeking topic experts, we also recruited a sample of patient and public stakeholders from an established patient and public research advisory group.

Public contributor workshops
To support the full engagement of public contributors in the MCDA process we convened two virtual public contributor workshops.The initial workshop was designed to provide contributors with training on background information about the topic, the proposed platform trial, and the MCDA methodology.A member of the study team (KK) and an expert in Patient and Public Engagement (AG) presented information to the group and answered questions.Participants were also provided with written information.A subsequent workshop was convened and supported by JC, HN and AG to inform public contributors about the process that would be used in Workshop 2, and to ensure there was adequate support to enable patient and public contributors to engage fully in all activities.

Exploring the criteria and mapping the use cases Stakeholder Survey 1
Stakeholder Survey 1 (Extended Data File 2) was disseminated to non-public contributor stakeholders to explore candidate criteria that should be considered when deciding whether a POC test is suitable for use in EMS.The survey contained questions focused on 'a priori' themes of promise, plausibility, risk, costs, the nature of the condition being investigated, the technology, the adopter system, the wider institutional and societal context, the evidence quality and other criteria which we selected from reviews of pertinent research by Campbel and Knox 18 and Greenhalgh and colleagues 19 .Stakeholders were asked to rate the overall importance of each of the criteria using a five-point Likert scale but were also asked to provide free text responses to elaborate upon their answers.
Responses to the survey were collated and reviewed by the research team.Likert scale responses were summarised using frequencies.Survey 1 results informed an update of a priori themes, producing a new shortlist of criteria to take forward into Stakeholder Workshop 1.

Stakeholder Workshop 1
Prior to the workshop summaries of the evidence available for different use cases were provided to the stakeholders.Responses to Survey 1 were visually summarised using an online interface (Miro board 20 ) and formed the basis of a subsequent discussion.During the online workshop, stakeholders were invited to map clinical pathways for candidate use cases (mapping both the current care pathway and the proposed amendments to the care pathway that introduction of a POC test would allow), discuss the importance of each shortlisted criterion and make suggestions or alterations to the shortlist.All ideas were captured on the Miro board and displayed to the stakeholders, who were asked to agree on the final content of the Miro board by the end of the workshop.The workshop was recorded using Microsoft Teams.
We then compared the refined shortlist of criteria against the Evidence and Value: Impact on Decision Making (EVIDEM) framework 21 , which has previously been used for MCDA involving health technologies 22 .Following an iterative process of coding and refinement within the study team, we reached a final consensus that the list of criteria identified was truly representative of stakeholder opinions, while being sufficiently specific and measurable to use in the MCDA process.The study team also prepared a draft rating system with which to score each criterion.We searched for published examples of established scoring systems and used these wherever possible.Where that was not possible, the team agreed threeor four-point scoring systems to put to the stakeholders.

Evaluating the criteria Stakeholder Survey 2
To validate and finalise the list of criteria to be used for scoring and to determine the weightings that would be used to score those criteria, we disseminated a second survey (Extended Data File 2) to non-public contributor stakeholders.Each of the shortlisted criteria was presented back to the stakeholders.They were asked three questions about each criterion.First, they were asked to score the importance of the criterion on an 11-point scale.This would later be used to assign weightings to the final criteria.Second, they were asked to provide feedback on the proposed rating system.They were asked if they felt that the rating system was appropriate.If not, we asked stakeholders to explain why not and to propose alternatives.Finally, we asked the stakeholders if any of the categories should be used to exclude a POC test from being considered in a future platform trial.This allowed us to identify the 'deal breakers' or satisfice criteria, which would exclude a POC test from further consideration.
The responses to the survey were collated and analysed within the study team.The list of criteria was finalised and stratified into two sections: criteria for prioritising use cases; and criteria for prioritising individual POC tests or assays within those use cases.Scoring systems and satisfice criteria were finalised based on consensus of the stakeholders, with review within the study team for every instance of disagreement.Outputs were taken forward into Stakeholder Workshop 2.

Measuring the use cases against the criteria and achieving consensus on scoring POC tests Stakeholder Workshop 2
Both expert and public contributor stakeholders were invited to participate in the online Stakeholder Workshop 2. Prior to the workshop, all stakeholders were provided with the final list of options (use cases) identified alongside their associated care pathways (current and POC test-driven), which had been mapped earlier in the project (Extended Data File 4).A summary of the evidence identified in the updated systematic scoping exercise was also provided for each use case (Extended Data File 1).
During the first half of the second workshop, the final list of criteria for scoring use cases was presented back to all stakeholders, who were given an opportunity to make any final adjustments.Once consensus had been achieved, we proceeded to the second stage of the workshop, during which the options (candidate use cases) were presented and scored.The relevant care pathways were explained to stakeholders, and further comment was invited.The summary of evidence for each use case was then presented.The summary was provided in language that was comprehensible to a lay person, and there was an opportunity for all stakeholders to ask questions.
Once we had confirmed that the stakeholders had understood the information and after answering any questions, we proceeded to invite the stakeholders to score the options against each of the agreed criteria using the scoring systems developed earlier in the project.This was done via Mentimeter 23 .Average scores from Survey 2 provided the weighting figure assigned to each criteria.The pre-determined weightings and satisfice criteria were used to create a template scoring matrix prior to the workshop.Once stakeholders had provided a score, this was averaged (mean) and added to the matrix by a study team member during the workshop.The weighting and average figures were multiplied for each criteria, and this provided a final score for each use case.
By collating the scores contemporaneously, we were then able to present the final results back to the stakeholders and ensure that consensus had been achieved about the final scores and the ranking of use cases.Having ranked the use cases, we selected the top three scoring use cases for inclusion in a future platform trial.

Achieving consensus on scoring POC tests
Prior to completing Stakeholder Workshop 2, we also presented the agreed criteria for scoring in vitro POC tests and scoring systems for rating individual tests/assays within the relevant use cases.Consensus was achieved for each prior to the end of the workshop.We then identified all eligible commercially supplied, CE-or UKCA-marked assays for the relevant test.This list will be updated and POC tests scored prior to a platform trial of in vitro POC tests in EMS to allow for industry advancements and for the study team to systematically review each device before scoring.

Scoping the use cases Options (use case) -Identification
Systematic scoping reviews of POC tests in EMS supplemented by surveys of industry organisations, identified the candidate use cases shown in Table 1.

Stakeholder recruitment
We recruited 32 stakeholders to participate in the study and their expertise and country of origin is shown in Table 2.

Achieving consensus on scoring POC tests
The satisfice criteria for evaluating the POC tests detailed was agreed with the stakeholders during Stakeholder Workshop 2 and this is detailed within Table 4.

Discussion
We developed an MCDA process for identifying the most appropriate options (use cases and in vitro POC tests) to enter into a platform trial of POC testing in UK EMS.The process consisted of identifying relevant use cases and POC tests through systematic searching of the literature and industry engagement.We identified a relevant stakeholder group and worked with these stakeholders to: scope the use cases; explore the criteria and map the use cases; evaluate the criteria; measure the use cases against the criteria and achieve consensus on scoring in vitro POC tests.Following this process, we identified that the most appropriate use cases to progress into a subsequent platform trial are POC troponin testing in possible AMI, POC lactate testing in suspected sepsis and POC lactate testing in trauma.
POC testing is a relatively new development in EMS, and as a result the quantity and quality of evidence is limited currently.
Even the POC tests that scored most highly in this research, such as troponin in chest pain, have relatively little evidence of their clinical effectiveness and more trials are required 24 .This field is anticipated to develop rapidly over the next few years 4 , and we have established a methodology by which POC tests may continue to be evaluated and selected for further assessment in research.
There is a general recognition that there are few resources to support organisations in evaluating POC tests for implementation, and that often POC tests are developed that appear to meet clinical requirements, but then fail to be adopted successfully into practice 25 .There are no studies that investigate an MCDA in relation to POC testing specifically.This study has developed a formal and reproducible methodology to evaluate alternatives and priorities for use cases and POC tests that may be included in a platform trial of POC testing in UK EMS, facilitating transparency in decision-making.Our study presents a quantitative MCDA as opposed to qualitative, or MCDA with decision rules 10 .Quantitative MCDA is considered more robust because the use of scores and weightings improves the consistency of recommendations.
There are well-documented benefits of using MCDA for HTA.These include a sound structuring of the decision problem, avoidance of bias in complex decision-making, and a transparent, consistent and reproducible process 26 .The method is particularly attractive when there are many important criteria to consider when triaging the available options.These qualities are important in the prioritisation of POC tests to enter into a platform trial of POC testing in UK EMS.In addition, our  process of MCDA has included a wide range of stakeholders, including public contributors allowing greater participation in decision-making.
An argument against using MCDA in HTA, and a factor that we recognise as a potential limitation in our methods, is that the management of the scoring and weighting of factors by researchers can inhibit debate, rather than stimulate it 26 .However, our stakeholders were informed of all decision-making and indicated throughout the process that they were content with the methods used and the outcomes.Whilst we aimed to recruit a genuinely representative group of stakeholders, using personal networks and snowball techniques, the participants are unlikely to wholly reflect the full range of expertise relevant to pre-hospital POC testing, and as a result some views and areas of knowledge will have been underrepresented in this research.Due to resource constraints, during the scoping review of the literature only one reviewer screened articles for inclusion.We recognise that the gold standard for a scoping review is to have two, or more reviewers screening articles for inclusion 27 .
Moving forward we will further refine and develop this MCDA process and continue to utilise it to determine which POC tests are appropriate for inclusion in an EMS platform trial.This research will also be informative for healthcare organisations and guideline decision makers on the appropriateness of POC tests for EMS and related settings, recognising that this MCDA was completed with UK EMS in mind and may not be generalisable to international settings.Continuation of research in this area in a future trial of POC testing in EMS is warranted to understand the factors that influence successful adoption and implementation of POC testing in UK EMS.

Conclusion
We report on the development and use of MCDA methodology to identify candidate tests for inclusion in a future platform trial of POC testing in UK EMS.After identifying stakeholders and candidate tests (options) through systematic scoping reviews and surveys of industry organisations, we used a series of stakeholder surveys and workshops to define the criteria for selection of use cases and in vitro POC tests, and determine the appropriate weighting and scoring criteria.Through this process, we identified three use cases for evaluation in a platform trial: troponin testing in possible AMI, lactate testing in suspected sepsis and lactate testing to identify occult haemorrhage in trauma.

Ethics and consent
Ethics committee approval was granted on 2

Daniel Ebbs
Yale University, New Haven, Connecticut, USA Thank you for this opportunity to review your article, A Multiple criteria decision analysis to establish the use cases and candidate point of care tests to enter into a platform trial of multiple in vitro diagnostic point of care tests in the prehospital environment.This is an important and timely study.The use of POCTs has the potential to serve as early triage for several medical conditions as discussed in this article.This manuscript aims to design an MCDA to set priority to potential relevant POCTs in the prehospital setting which can guide future stakeholders for implementation.
Intro: Reliance on in vitro lab for medical decision making is highly over-exaggerated, as referenced by Rohr in "The Value of In Vitro Diagnostics.." in this introduction.This one citation is used as a supporting argument in the introduction, which interviews highly sub-specialized physicians (cardiologist/oncologists) who rely on POCTs for very specific chronic medical conditions.Although medical providers are highly reliant on these diagnostics, 60-70% is extreme, and this is stated without any context.
Methods: The use of multiple criteria decision analysis (MCDA) is both innovative and appropriate for this study.Appropriate guidelines followed via ISPOR.
For article review, the gold standard for scoping reviews as outlined by more contemporary sources, such as Joanna Brigg Institute, typically requires two reviewers to review articles for inclusion; having one reviewer select all inclusion can introduce bias which should be noted in discussion.It is noted that a second reviewer was utilized for exclusion review.
The stakeholder survey methods and results are not representative of experts; the breadth of POCTs in the prehospital setting, particularly among EMS, is novel and new to the last decade, and there is limited data in the form of studies, as you noted (cited in extended file 1) , on most prehospital POCTs.Please expand on how reaching out to random authors who publish on this or applying snowball methods can be synonymous to expertise in the field.Please expand on the limitation section that describes stakeholders.
Discussion should also discuss serious limitations in clinical utility/effectiveness of each POCT and discuss the lack in quality evidence overall in the impact assessment.For example, Troponin, which scored upward of 3 on the evidence of clinical effectiveness, has only three RCTs, and limited overall evidence towards its benefit; yet it is cited among experts that this is above moderate.Reviewer Expertise: Prehospital emergency services, Prehospital POCTs, Prehospital Predictive Modeling I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above.

Is
The researchers did a very thorough job and have also given an excellent coverage of this in this article.In developing an MDCA, the researchers, with input from Public Contributors and stakeholders, have been able to identify 3 key use cases that could be further explored in a platform trial in the (near) future.The topic focuses on important developments in prehospital care, namely the use of point-of-care diagnostics.The use of these diagnostics helps in the decision-making of healthcare professionals, strives for the right care in the right place for patients and offers opportunities to reduce overcrowding in emergency departments and improve ambulance availability.

Is the work clearly and accurately presented and does it cite the current literature? Yes
Is the study design appropriate and is the work technically sound?Yes

Are sufficient details of methods and analysis provided to allow replication by others? Yes
If applicable, is the statistical analysis and its interpretation appropriate?

Not applicable
Are all the source data underlying the results available to ensure full reproducibility?Yes

Are the conclusions drawn adequately supported by the results? Yes
Competing Interests: No competing interests were disclosed.
Reviewer Expertise: Prehospital care, riskstratification, point of care testing in healthcare I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.

Figure 1 .
Figure 1.Multiple Criteria Decision Analysis flow diagram.
the work clearly and accurately presented and does it cite the current literature?YesIs the study design appropriate and is the work technically sound?YesAre sufficient details of methods and analysis provided to allow replication by others?PartlyIf applicable, is the statistical analysis and its interpretation appropriate?PartlyAre all the source data underlying the results available to ensure full reproducibility?YesAre the conclusions drawn adequately supported by the results?PartlyCompeting Interests: No competing interests were disclosed.
Ethics committee approval was granted by the University of the West of England, Bristol Faculty Research Ethics Committee reference: UWE REC REF No: HAS.23.04.101.

Table 3 .
23e cases were mapped and discussed in Stakeholder Workshop 1 and through consensus the most appropriate use cases for further analysis were identified, based principally on available technology and pathway readiness.Identified use cases were ketone testing in diabetic ketoacidosis, NT-proBNP testing in heart failure, lactate testing in trauma, lactate testing in sepsis, bacterial or viral testing in respiratory tract infection, biomarker testing in traumatic head injury and troponin testing in chest pain.Use case mapping is included in Extended Data File 4. The full Mentimeter23results are included in Extended Data File 7 and Table6illustrates the results of the use case scoring.

Table 5 . Scoring of use case criteria.
nd June 2023 by the University of the West of England, Faculty of Health and Applied Sciences Faculty Research Ethics Committee, refer- ence: UWE REC REF No: HAS.23.04.101.All participants in this research provided written informed consent prior to their participation.

have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard. Version 1
This is an open access peer review report distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.