An overview of critical decision-points in the medical product lifecycle: Where to include patient preference information in the decision-making process?

Background: Patient preference (PP) information is not effectively integrated in decision-making throughout the medical product lifecycle (MPLC), despite having the potential to improve patients’ healthcare options. A ﬁrst step requires an understanding of existing processes and decision-points to know how to incorporate PP information in order to improve patient-centric


Introduction
The pharmaceutical industry, regulatory authorities, and reimbursement/Health Technology Assessment (HTA) bodies (including payers) generally agree that the use of patient preference information (PP information) could be beneficial to decision-making throughout the medical product lifecycle (MPLC) in Europe and the US [1][2][3].PP information is defined as information resulting from "assessments of the relative desirability or acceptability to patients of specified alternatives or choices among outcomes or other attributes that differ among alternative health interventions" [4].PP information can be determined through qualitative and quantitative methods, and includes the relative importance of what matters most to patients, enabling the examination of trade-offs that patients are willing to make between benefits and harms.Therefore, PP information is different to patient reported outcomes (PROs), but it can provide through which outcomes can be prioritised.It not only captures patient needs and concerns, but it also provides a significant opportunity for patients to express important preferences and have this information incorporated into decisionmaking [5].Patient-centric decision-making not only results in better transparency and accountability of medical product development, but may also result in better outcomes for patients, improved quality of research and study outcomes more relevant to patients, more products developed in line with patients' needs, and increasing overall well-being [5][6][7].
Before PP information can be formally integrated within decision-making, a clear overview of the current decision-making processes, including critical decision-points, along the MPLC must be formulated.We define a critical decision-point as an identified fixed moment where a decision influences the course of the medicine development, authorisation or reimbursement process.These are essentially "go-or-no-go" decisions.Therefore, this study will be focusing on stakeholders that directly affect the progression of medical products along the MPLC (i.e.industry, regulatory authorities, and reimbursement/HTA).In addition, it needs to be determined which decisions can potentially benefit from the inclusion of PP information.To date, there is no consolidated, published overview concerning the critical decision-points along the MPLC for the different stakeholders involved, as well as how, and based on what information and criteria, these stakeholders make their decisions.
The current study aims to: 1) identify the decision-making processes and critical decision-points throughout the MPLC for industry, regulatory authorities, and reimbursement/HTA, and 2) determine which of these critical decision-points have the potential to include PP information.

Materials and methods
A four-step approach was used in this study, including a scoping literature review (step 1), semi-structured interviews (step 3), and validation meetings (steps 2 & 4) (see Appendix VII).

Scoping literature review
In step 1, a literature search was performed to identify relevant white and grey literature [8].White literature included relevant, peer-reviewed articles published in scientific/academic journals.The literature was retrieved via five scientific databases: Guidelines International Network, Embase, PubMed (including Cochrane Central and Medline), PsycINFO and EconLit.Search queries consisted of MeSH terms and free text words in order to optimise the breadth of results (e.g.decision making, patient preference, decision-point, and drug life cycle).The databases were searched for relevant titles and abstracts published between January 2011 and April 2017.Grey literature was collected in order to incorporate the most current information and knowledge written by industry, regulatory authorities, and reimbursement/HTA.Three researchers (CW, KSB, JV) independently reviewed the abstracts of the literature and applied inclusion and exclusion criteria.Inclusion criteria were: conceptual or applied descriptions of (i) decisions made by industry or regulatory authorities or reimbursement/HTA related to medical products; (ii) the use of patient preferences in decision-making by industry or regulatory authorities or reimbursement/HTA regarding medical products.The following exclusion criteria were applied: (i) not written in English, (ii) no full text article available, (iii) published before 2011, (iv) country outside of US/EU because of the focus on EU decision-processes and differences compared to the US, (v) conference abstracts, conference notes, book reviews, and presentations.Data extraction was conducted by four researchers (CW, KSB, JV, SW), by assessing the full texts of white and grey literature based on the same inclusion and exclusion criteria as the abstract screening.Through this process, key decision-points and decision processes were identified for each stakeholder group, making a preliminary list to be confirmed in the next steps of the methodology.

Semi-structured interviews
In step 3, interviewees representing one of the stakeholder groups (industry, regulatory authorities and reimbursement/HTA) were recruited via purposive sampling and snowballing (i.e.asking confirmed interviewees to suggest others).The interviews were conducted between April 2017 and August 2017 in Sweden, the UK, Italy, the Netherlands, France, Germany and Romania which provided representation from all different cardinal regions in Europe.Interviews were also conducted in the US to examine perspectives outside the EU.Potential interviewees received information on the study and provided informed consent.This study was approved by the Ethics Committees in each of the countries where interviews were conducted.
An interview guide was developed based on topics that had emerged from the literature review detailed in step 1. Interviews were conducted by five researchers and took approximately one hour and were conducted via telephone or face-to-face.Interviews were conducted in English, audio recorded, and transcribed verbatim.Transcripts were analysed through framework analysis [9] using NVivo software [10], where the data were interpreted for consensuses and observations across the stakeholder groups, which created thematic 'codes'.After a 'familiarisation process' [9] in which the coders (CW, KSB, RH) examined transcripts from each stakeholder group, open-coding was applied to three transcripts, along with deductive codes reflecting key stages of the MPLC identified from the scoping review.The success of these codes encapsulating the themes within each transcript were compared, and then served as the basis of the codes that were then applied to the entire 70 transcript sample.The sections of transcript that corresponded to a code were indexed into a chart that identified the coded material for each participant's stakeholder group and country.These charts were analysed by four researchers (CW, KSB, JM, SW) for common attitudes and opinions of the respondents, comparing similarities between stakeholder groups and countries.

Validation meetings
In steps 2 and 4, meetings with representatives from each of the stakeholder groups (industry, regulatory authorities, reimbursement/HTA) were scheduled to validate the results from both the literature review and the semi-structured interviews.These stakeholder representatives, from both the EU and US, were not participants in the semi-structured interviews.During this step, information was retrieved on differences in decision-making processes between EU and US.The first round of meetings with each stakeholder group (one meeting with two industry representatives, four meetings with four regulatory representatives, one meeting with one reimbursement/HTA representative) was conducted after the scoping literature review to confirm the identified decisionmaking processes.The second round of separate meetings with each stakeholder group (two meetings with two industry representatives, two meetings with two regulatory representatives, one meeting with two reimbursement/HTA representatives) took place after the semi-structured interviews to discuss the results of the interviews.

Results
During step 1, 723 records were screened on title, abstract, or table of contents (see Appendix VII).From these, 223 records were selected for full-text screening (of which 32 related to industry, 89 to regulatory authorities, and 102 to reimbursement/HTA) and 57 records were selected from the full text screening (of which 10 related to industry, 14 to regulatory authorities and 33 to reimbursement/HTA).General decision processes, chronology of decision-making, and type of information required during these processes, was extracted.Not all works are directly cited in the results, because most articles named the decision-points but did not describe the content of the decision-points, or the required information, in detail.
During step 2, seven validation meetings were conducted (n = 2 industry, n = 4 regulatory, n = 1 reimbursement/HTA).The decisionmaking processes from step 1 were confirmed, the differences between EU and US decision-making processes was examined.
During step 3, 70 interviews were conducted with representatives from industry (n = 24), regulatory authorities (n = 23) (including US regulators, European-level regulators, and national EU regulators) and reimbursement/HTA (n = 23).In this step, six industry decision-points, four regulatory decision-points, and six reimbursement/HTA decision points were identified.
During step 4, six validation meetings were conducted (n = 2 industry, n = 2 regulatory, n = 2 reimbursement/HTA).The decisionpoints identified in step 3 were confirmed, and one regulatory decision-point was removed.Experts confirmed that 'orphan designation' is a designation process which creates separate processes and timelines, and not a decision-point (go-or-no-go decision) that exists for every medical product.
Conclusively, six industry decision-points, three regulatory decision-points, and six reimbursement/HTA decision points were identified.

Decision-points within industry decision-making
We identified six critical decision-points in the industry processes (Fig. 1).These decision-points start immediately after pre-discovery, and run through the MPLC, with the final decision-point concluding in post-approval.In general, product development and decisions whether to proceed are in the context of regulatory requirements, which may differ between products or regulatory designations (e.g.orphan status).The most commonly followed decision-points are described below.
Select & prioritise targets and leads: this decision-point is generally based on biology data such as the extent to which human and animal disease pathology overlap.The candidate selection decision is based on animal efficacy and toxicity, pharmacology, pharmacokinetics, and drug metabolism characterised using in vitro assays and efficacy using animal models of disease or in vitro tests on human cells or tissue [11,12].The semi-structured interviews with industry representatives in the EU and US revealed a positive perspective towards patient preference information (PP information) in early stages of the MPLC.An EU interviewee suggested early integration of PP information will help "[identify] fields in which new therapies, whether it be medical devices or medicines, should be developed" (Netherlands).
Prioritise studies (Early clinical development): data and conclusions on whether to enter clinical development are peerreviewed by technical and operational management committees that verify safety, quality, regulatory documentation and resource availability.The decision to enter clinical development is based on pre-clinical evidence supporting confidence in the biologic target, literature data, manufacturing data, operational feasibility and verification that the medicine candidate could meet the needs of the target product profile.Representatives from the US and EU suggested that companies which routinely engage patients better inform and communicate their decisions.One representative from EU said "patient preference on, for example, target product profiles [. ..] inform go/no-go [development] decisions" (UK).
Prioritise assets (Early clinical development): data and conclusions on whether to enter Phase 2 of clinical development are peer reviewed by technical and operating management committees to ensure safety, quality and favourable medical benefit-risk to support continued development in a larger clinical trial [14].The decision to enter Phase 2 development is based on evidence that the medicine is having a pharmacologic effect on the target organ of normal health volunteers (or in patients for oncology products) operational feasibility, and confidence the medicine candidate's performance could hit the target product profile [13,14].During the interviews, a number of industry representatives said that PP information informs trials by "look[ing] at the endpoints of your study, defining them" (Netherlands), by assessing "clinical trial feasibility" (US), and by translating PP information "into an outcome measure" (UK).
Optimise & Prioritise assets (Late clinical development): data and conclusions on whether and how to enter Phase 3 of clinical development are peer reviewed by technical and operating committees to ensure safety, quality and favourable medical benefit-risk to support continued development in a substantially larger Phase 3 patient study.The benefit-risk profile is further developed and may include risk management planning to mitigate safety risks and increase the probability that the benefit-risk profile remains positive [15].Technical performance includes data from Phase 2 with sufficient dose-response evidence to support Phase 3 dose-selection, safety, quality and favourable medical benefitrisk in the appropriate patient population [13,14].Widespread PP information integration by industry will improve healthcare by increasing resource allocation efficiency and by developing products with stronger value propositions, or as one EU industry representative explained "better decisions will be taken and [. ..] patient value will increase" (Sweden).
Regulatory Submission & Launch: committees review efficacy and safety data to ensure evidence supports a favourable medical benefit-risk profile and planned label claims.Technical performance includes Phase 3 data from pivotal registration studies demonstrating efficacy and safety.The decision to apply for regulatory marketing authorisation typically requires: Phase 3 technical performance that supports the target product profile; desired label claims; and commercial opportunity or considerations [13].An industry representative stated that including PP information in "the dossiers that you put together for regulatory authorities or HTAs[. ..] provides context to either the company narrative or company conclusions around the datasets" (UK).
Manage MPLC & Prioritise opportunities: includes decisions that are made after commercialisation.Products may be enhanced to further satisfy medical needs.Ideas may arise from many potential sources including observational studies (e.g., to comply with regulatory commitments), investigator-initiated studies, patient advisory boards, focus groups, surveys, and structured patient interviews [16].A number of representatives identified the value of PP information to inform decisions in the post-approval setting, including reimbursement, comparative effectiveness, addressing new safety signals and informing shared-decisions between a patient and their health care provider.An EU industry representative suggested that "if patient preferences are taken into account, the cost effectiveness of a therapy might be better and reimbursement might be easier to decide on" (Netherlands).

Decision-points within regulatory decision-making
Three critical decision-points were identified within regulatory decision-making, defined as a fixed moment where a decision is taken that influences the course of the authorisation process (Fig. 2).The identified critical decision-points are submission and validation, scientific opinion, and commission decision.Other regulatory processes, defined as activities that do not need a go-or-no-go decision, but are conducted in order to inform future decision-points, were also identified (in white in Fig. 2).Depending on the product, as well as whether the medical product will go through the centralised procedure of the EMA or the FDA's regulatory decisionmaking, some decision-points may vary.For example, separate processes and timelines exist for products that are designated to be orphan products for rare diseases or paediatric products.A comparison of the decision-points of the FDA and EMA can be found in Fig. 3.
Submission & validation happens when companies submit a Marketing Authorisation Application dossier for the approval of a medicine.Specifics depend on the application: for new applications, a full dossier (electronic Common Technical Document (eCTD)) needs to be completed, for a variation, new changes need to be submitted only.If applicable, a renewal confirms that all information is up to date and a Periodic Safety Update Report adds new information to the original dossier.For medicines to go through to scientific evaluation, submitted materials are assessed for completeness (eCTD) and they have to meet all the legal requirements [18].Regulatory representatives in the EU and US said that they occasionally get PP information when pharmaceutical companies apply for Marketing Authorisation and that their agency supports this information: "We get them occasionally, less frequently than I would like [. ..].So if we get them, we have to take them into account" (Germany).
Scientific opinion is a positive or negative recommendation given by the Committee for Medical Products for Human Use (CHMP), on whether to authorise a medicine based on the scientific evaluation [17,18].All information gathered during the pre-submission is needed to make a scientific opinion, including a complete submission dossier (eCTD).A positive opinion is issued when a positive benefit-risk balance, including efficacy, is sufficiently demonstrated and when the dossier meets all legal requirements [5].A 'summary of opinion' is immediately published after the opinion is submitted to the European Commission [18].During the interviews, regulatory representatives recognised the value of including patients as experts in discussions that lead to the scientific opinion.However, representatives in both the EU and US were sure that there are no formal or systematically integrated protocols for including PP information in the regulatory process at this moment: "There are not formal protocols for that, I think much yet.But I think that that will be important to provide" (US).
A commission decision, is when the European Commission, grants, refuses, changes, suspends or revokes marketing authorisation.The commission decision is based on the 'summary of opinion' and legal requirements [18].The summary is replaced by a full European Public Assessment Report (EPAR) once the European Commission has decided to approve marketing authorisation or not [18].The safety monitoring and the ongoing benefit-risk assessments might feedback to submission and validation if there is; a variation, a renewal, need for a periodic safety update report (PSUR), a referral or switch to Over the Counter (OTC) for the medicine [18].Some of the representatives in both the EU and US expressed a limited acceptance for PP information and that there is a need for a structured way to include PP information in commission decisions.One respondent stated, "All these things are very important but you have to create a way of measuring the impact of taking into account patient preference" (Italy).

Decision-points within reimbursement/HTA decision-making
There are six critical decision-points during the reimbursement/HTA decision-making processes (Fig. 4).These include filtration, prioritisation, and appraisal, and also when these three decision-points are repeated for reassessment.Other HTA processes, defined as activities that do not need a go-or-no-go decision, but are conducted in order to inform future decision-points, were also identified (in white in Fig. 4).The process by which different countries conduct these decision-points is generally similar in practice, although also depends on the country's unique healthcare system [26].EU countries operate under procedural rules and timelines set by the European Commission, although methodological and procedural differences exist between nation states [27,28].In the US, health payers and organisations, including both commercial health payers and government payers make their own decisions regarding reimbursement [29].
The filtration of potential assessment topics is often conducted in order to narrow down prospective assessment topics to a manageable number, although not always relevant to reimburse-   ment/HTA bodies that address all medicines [1,[30][31][32].Medicines that are expected to have a limited impact on the healthcare system or patients are considered to be a lower priority.Filtration selects products by applying pre-established criteria, which do not vary widely across the EU and North America [33].The criteria often address whether the technology is new and innovative, is a modification of an existing product, or is an existing product being used for a new indication.Further criteria are often related to the associated disease burden; whether there are existing treatments for the condition; the anticipated clinical, economic, or societal impacts; the appropriateness for relevant stakeholders or healthcare system; or the timeframe that it would take the product to be commercialised and incorporated into practice [34].
The prioritisation of potential assessment topics, if applicable to the particular reimbursement/HTA body, aims at determining the significance of the filtered technologies for the healthcare system, and deciding which technologies will be invested in with limited assessment resources [35].Filtration and prioritisation are often conducted through horizon scanning or early awareness and alert activities.In the US, this is often the AHRQ (Agency for Healthcare Research and Quality), or private sector companies [29].The majority of explicit criteria utilised by reimbursement/HTA con-cern patient group sizes and the burden of disease; the potential clinical benefit on morbidity, mortality or quality of life; the cost or economic impact, both to the patient and to stakeholders; social impact including ethical or legal concerns; the anticipated speed of adoption; and the availability of evidence or additional input from patient groups [36].However, one EU representative stated that frequently "one patient is sitting in one big HTA big [sic] decisionmaking body and they don't believe them because -'oh yeah one single patient'" (Germany), indicating that one patient might not carry much weight for decision-makers.
After evidence is obtained during the assessment step, the critical decision-point of the appraisal occurs, where the evidence is reviewed and a decision is made regarding reimbursement.Generally, assessments collect scientific clinical and economic evidence: safety and efficacy information (often relative to available alternatives), clinical effectiveness (often relative to available alternatives), time required for diffusion, costs, or financial impact [30,36,37].This can be included in form of literature reviews, clinical evidence from clinicians or manufacturers, cost-effectiveness analysis, estimated QALYs, observational studies, or combined sources.Additional evidence from patients and patient organisations can also be submitted.The appraisal committee can consider social or ethical impacts, equity issues, the product's degree of innovation, the burden of disease and projected epidemiological trends, and other patient issues [38].All stakeholder representatives mentioned that PP information is not required or implemented systematically, with cost-effectiveness and efficacy given priority instead.An EU representative said, "By the time it comes to HTA bodies it's a bit too late to start thinking about patient preferences" (UK).There is limited guidance for PP information inclusion, as an EU representative indicated, "We don't have anything, any explicit criteria [. ..] that specifies "this is the weight of patient preference we should take in the decision" (France).Although all representatives accurately understood the concept of PP information, many had a misconception that patient preferences are sufficiently accommodated through QALYs, despite their calculation frequently incorporating public preferences, and not patient preferences.An EU participant described QALYs as "implicit" patient preferences and stated, "we suppose that patient preferences are included in this tool" (France).

Discussion
This paper represents a significant first attempt to identify 15 critical decision-points from key stakeholders with the objective to incorporate patient preference information (PP information) in the MPLC.An overview of all identified critical decision-points is given in Appendix VIII.Each of the critical decision-points requires different information, based on pre-determined decision criteria, to be submitted to the decision-makers.A description of the information needs and decision criteria for each decision-point can be found in Appendix I-III.Some decision criteria already allow for PP information to be incorporated more readily than others, but PP information is currently not routinely considered one of the requirements for decision-making.However, within all stakeholder groups this has been recognised as a valuable component to inform decision making across the MPLC in the near future.
In general, industry representatives spoke positively about increasing the integration of PP information to inform decisionmaking throughout the MPLC, especially in the development of a new medicines since it provides context which informs and helps communicate their decisions.Regulatory representatives expressed that there is limited acceptance for PP information within their decision-making processes since there is currently no recognised nor structured way to include and/or value such information.However, some regulators stated that PP information could be more important in specific situations, like for rare diseases.The EMA and FDA value the perspectives of patients and are committed to encouraging patient input throughout medicine development and product reviews [4,39,40].Both EMA and FDA play an important role in providing guidance to industry and reimbursement/HTA on how to best incorporate PP information in future assessments [4,39].All reimbursement/HTA representatives agreed that PP information is sometimes included in assessment dossiers, but not required.According to representatives from EUPATI (European Patients Academy on Therapeutic Innovation) and HTAi (Health Technology Assessment international), mechanisms exist to collect preferences from patients that help identify and select potential reimbursement/HTA topics that are most important or pressing.
Greater discussion and collaboration is required between key stakeholders, especially regulators and reimbursement/HTA, in order to consolidate efforts to integrate PP information.Despite PP information having the potential to be integrated at numerous stages of the MPLC, there still are various barriers preventing its inclusion.The integration of PP information into industry decisionmaking appears readily feasible, whereas reimbursement/HTA and regulatory authorities first need to decide how much weight should be given to PP information compared to other required information.Although not the objective of this paper, timelines, budgets, and other issues of feasibility (e.g., method selection) need to be appraised and resolved by these stakeholders before PP information can be integrated.Recommendations are needed to inform all decision-makers about how best to capture patient preferences, which methods to use, who should best conduct patient preference studies in order to avoid potential bias, how to interpret the results, and satisfy particular decision criteria for each decision-point.It was also not the objective of this study to determine where PP information should be integrated, or assess at which decision-points it would be more valuable.However, identifying decision-points where PP information can be integrated serves as an important first step.
Our results focused on medicines instead of medical devices, although the latter adds other important dimensions and nuances to this discussion.Within industry, medical device development is highly variable with different company procedures, depending on regulatory authorities' assigned risk level and the intended use of the device.However, decisions are governed by the same principles as medicine development.High risk-level medical device development can be analogous to medicine decision-making from Phase III onwards [17].For reimbursement/HTA in the EU, most international guidelines for economic evaluation are written to be applicable to both medicines and medical devices.Some EU countries appraise medical devices through specialist reimbursement agencies, separate from medicines.The US appraises medical devices similar to medicines, but has separate processes: private and government payers decide upon reimbursement decisions of medical devices by conducting a technology assessment which is largely dependent on clinical impact or utility and costeffectiveness [29].The starkest difference between medicines and medical devices occurs during regulatory decision-making.All devices are classified based on the risk level.A clinical investigation can be assigned and approved by the applicable authority before submission.For all devices in EU, a European CE Marking Certificate is issued for the device after successful completion of a Notified Body audit [18].In the US, the probable benefits should outweigh the risks with oversight from an Institutional Review Board or an appropriate local committee [22].Current literature suggests PP information should be seen as additional data in the development and submission of medical devices [23][24][25].

Strengths and weaknesses
This study included a comprehensive four-step approach where international representatives (n = 70) from all the three stakeholder groups in the MPLC were included, creating a novel and highly representative overview that has not been outlined in previous literature.The scoping literature review included grey literature which enriched the collected information with current knowledge and practices.The literature review only included English papers and documents, which could be a potential limitation.However, the validation meetings confirmed the findings of both the literature review and the semi-structured interviews, while clarifying differences between EU and US.A potential limitation was the snowballing recruitment technique of the interviewees, the majority of whom were found through connections with the PREFER consortium, which may inadvertently introduce a sampling bias.Some participants may have wanted to participate because they already found the topic interesting or valuable.In addition, five interviewers conducted the interviews in eight different countries, meaning there could be variation in the conduct of the interviews.We expect this variation to be minimal, however, because all interviewers used the same interview guide and instruction manual.This study examined three stakeholders directly involved with PP information integration because it was focusing on policy and MPLC decision-making.However, the perspectives of patients, patient organisations, academics, clinicians, and other stakeholders are also vital in the successful use and integration of PP information.

Conclusion
Patient preference (PP) information is currently not routinely considered one of the requirements for decision-making.With support already being generated by all these stakeholders, this study provides an overview of 15 decision-points with the potential to include PP information.This roadmap, combined with continued discussion between key stakeholders, is needed to successfully implement PP information into decision-making, and strengthen a crucial path forward into patient-centric healthcare.

Declaration of Competing Interest
The Patient Preferences in Benefit-Risk Assessments during the Drug Life Cycle (PREFER) project has received funding from the Innovative Medicines Initiative 2 Joint Undertaking under grant agreement No 115966.This Joint Undertaking receives support from the European Union's Horizon 2020 research and innovation programme and EFPIA.This text and its contents reflect the PRE-FER project's view and not the view of IMI, the European Union or EFPIA.JM and SW are employees of Pfizer, Inc. CAP is an employee of Merck & Co, Inc. (Kenilworth, NJ, USA), and is a stockholder in this company.RH is an employee of AstraZeneca.All other authors have no competing interests to declare.

Fig. 1 .
Fig. 1.The core industry decision-making process, including the critical decision-points (in black arrows), along the medical product lifecycle.

Fig. 2 .
Fig. 2. The core regulatory decision-making process, including the critical decision-points (in black arrows), along the medical product lifecycle.

Fig. 3 .
Fig. 3. Comparison of EMA and FDA decision-points, including the critical decision-points (in black arrows), along the medical product lifecycle.

Fig. 4 .
Fig. 4. The core HTA decision-making process, including the critical decision-points (in black arrows), along the medical product lifecycle.