Inconsistent approaches of the G-BA regarding acceptance of primary study endpoints as being relevant to patients - an analysis of three disease areas: oncological, metabolic, and infectious diseases

Background Previous evaluations of oncological medicines in the German early benefit assessment (EBA) procedure have demonstrated inconsistent acceptance of endpoints by regulatory authorities and the Federal Joint Committee (G-BA). Accepted standard endpoints for regulatory purposes are frequently not considered as patient-relevant in the German EBA system. In this study the acceptance of clinically acknowledged primary endpoints (PEPs) from regulatory trials in EBAs conducted by the G-BA was evaluated across three therapeutic areas. Methods Medicines for oncological, metabolic and infectious diseases with EBAs finalised before 25 January 2016 were evaluated. Respective manufacturer’s dossiers, regulatory assessments, G-BA appraisals and oral hearing minutes were reviewed, and PEPs were examined to determine whether they were considered relevant to patients by the G-BA. Furthermore, the acceptance of symptomatic vs asymptomatic PEPs was also analysed. Results A total of 65 EBAs were evaluated. Mortality PEPs were widely accepted as patient-relevant but were only used in a minority of EBAs and exclusively in oncological diseases. Morbidity PEPs constituted around 72 % of assessed PEPs, but were excluded from the EBA in over half of the corresponding assessments as they were not considered patient-relevant. Symptomatic endpoints were largely deemed patient-relevant, whereas acceptance of asymptomatic endpoints varied between therapeutic areas. Conclusions This evaluation identified inconsistencies in patient relevance of morbidity-related PEPs as well as in acceptance of asymptomatic endpoints by the G-BA in all three disease areas examined. Better harmonisation between the regulatory authorities and the G-BA is still required after 5 years of AMNOG health technology assessment in Germany. Electronic supplementary material The online version of this article (doi:10.1186/s12913-016-1902-8) contains supplementary material, which is available to authorized users.


Background
Since 2011, all new medicines in Germany are required to undergo an early benefit assessment (EBA) in comparison to a pre-specified appropriate comparator [1]. At market entry, the pharmaceutical manufacturer (PM) needs to submit an EBA dossier evaluating the additional benefit of the drug, based on available clinical trial data, to the Federal Joint Committee (Gemeinsamer Bundesausschuss, G-BA).
The extent of additional benefit is classified by the G-BA as major (1), considerable (2), minor (3), non-quantifiable (4), no additional benefit (5), and less benefit (6) [2]. The benefit rating serves as the main basis for subsequent price negotiations for reimbursement between the PM and statutory health insurance providers.
For the determination of additional benefit the G-BA requests efficacy data in three different categories: mortality, morbidity and health-related quality of life (HRQoL) [2,3]. However, the G-BA only takes into account endpoints deemed to be patient-relevant. Importantly, the opinion of the G-BA on patient relevance of endpoints frequently diverges from those of regulatory authorities [4][5][6][7][8].
A previous analysis of endpoints in oncology indicated acceptance by the G-BA of endpoints related to mortality, while morbidity endpoints were largely disregarded [4]. The current analysis extends the scope beyond oncology to evaluate if the identified trends also apply to other disease areas. Benefit assessments in the three most frequently assessed therapeutic areas (with ≥10 finalised and evaluable assessments) were analysed: oncological, metabolic, and infectious diseases. For all primary endpoints (PEPs), patient relevance was evaluated and used to determine whether these PEPs would be taken into consideration in an EBA by the G-BA.

Methods
Analysis set: EBA dossiers submitted by the PMs, the corresponding G-BA appraisals, and oral hearing minutes, all obtained from the G-BA website, were used as sources of data [9]. Dossiers were not included in the analysis if the G-BA's view on patient relevance of endpoints was not discernible. This could be due to i) no dossier being submitted by the PM, ii) inability to evaluate acceptance of PEP by the G-BA (due to use of inappropriate comparator, inadequate indirect comparison, or dossiers/studies being incomplete), or iii) if a more recent assessment for the same medicine and indication was available.
Analysis of patient relevance of primary endpoints in the benefit assessment by the G-BA: The resulting dataset was used to analyse whether PEPs used in the clinical studies and reported in the PM dossiers went on to be accepted as patient-relevant in the G-BA appraisals. PEPs were recorded as: patient-relevant when this was explicitly mentioned in the G-BA appraisal, or the corresponding data were used to justify an additional benefit, partially patient-relevant when the PEP was a composite or co-primary endpoint with both patient-relevant and non-patient-relevant components, or not relevant to patients when the G-BA clearly questioned the validity of the endpoint or the appraisal did not mention the PEP data or contained no clear statement of its inclusion.
Following determination of the G-BA's view on patient relevance of each PEP, they were categorised as either symptomatic or asymptomatic. To ensure this categorisation of the endpoints was unbiased and meaningful, we utilised multiple sources of information integral to the EBA process to inform this procedure. Specifically, the categorisations were based on definitions and descriptions of the endpoints and the respective discussions within the dossiers, the assessments by the Institute for Quality and Efficiency in Healthcare (Institut für Qualität und Wirtschaftlichkeit im Gesundheitswesen, IQWiG), and the G-BA appraisals. The oral hearing at the G-BA aims to provide clarification on critical and controversial topics. Key representatives of the relevant medical and scientific societies are always attending those hearings. Thus, the oral hearing minutes served as a source of expert opinion which we referred to for categorisation of certain endpoints along with supporting scientific statements.
The medicines were divided into therapeutic areas, as categorised by the G-BA, and only therapeutic areas with ≥10 finalised and evaluable EBAs as of 25 January 2016 were considered, which led to the inclusion of oncological, metabolic and infectious diseases in the final analysis set.

Analysis set
Inclusion of therapeutic areas with ≥10 finalised and evaluable EBAs (oncological, metabolic, and infectious disease areas) resulted in 97 EBAs which were considered for evaluation. Of these, 65 (67 %) were evaluable according to the inclusion criteria. Table 1 summarises the number of EBAs included in the analysis and the main reasons for exclusion of EBAs from the analysis set by therapeutic area.
A high proportion of EBAs in metabolic diseases (19 of 34, 56 %) were not evaluable, mainly because the G-BA objected to the methodological execution of the studies for reasons including use of inappropriate comparator or indirect comparison, and incomplete dossiers or studies. A list of EBAs and the respective PEPs evaluated in the analysis is presented in Table 2.
Analysis of patient relevance of primary endpoints in the benefit assessment by G-BA All therapeutic areas This analysis set consisted of 65 PEPs obtained from 65 EBA dossiers. The G-BA deemed the PEPs as relevant to patients in 31 (48 %) of 65 EBAs, not relevant to patients in 25 (38 %), and partially patient-relevant in nine (14 %) assessments (refer to sections on individual therapeutic areas for more details on specific PEPs).
Analysis by endpoint category showed that the majority of PEPs were morbidity endpoints (47 of 65, 72 %). Mortality PEPs were used in 11 of 65 PM submissions (17 %), all of which concerned oncological diseases and were deemed patient-relevant by the G-BA (Fig. 1a). Morbidity PEPs were considered patient-relevant in 20 of 47 (43 %) EBAs and partially patient-relevant in two cases (4 %). In seven EBAs the PEP was a composite endpoint of mortality and morbidity and in each case deemed partially patient-relevant. Six of these cases were in oncological diseases and one was in the metabolic disease area (see also sections on individual therapeutic areas). No assessment included HRQoL as a PEP.

Oncological diseases
In oncological diseases 36 of 47 PM submissions were analysed (Table 1). Mortality PEPs were reported in 11 of the 36 EBAs (all overall survival [OS]) and regarded as patient-relevant in the benefit assessment by the G-BA in all cases (Fig. 1b). Fifteen of 19 (79 %) morbidity PEPs (mostly progression-free survival [PFS]) were deemed not patient-relevant. The three patient-relevant morbidity PEPs were major cytogenic response (MCR) (bosutinib, chronic myeloid leukaemia), reduction of spleen volume by >35 % (ruxolitinib, chronic myeloproliferative diseases), and haematocrit control without phlebotomy and reduction of spleen volume by >35 % (ruxolitinib, polycythaemia vera). For one composite morbidity endpoint, namely partial and complete durable tumour and symptomatic response, only complete response was considered patient-relevant and included in the G-BA appraisal (siltuximab, multicentric Castleman's disease). Six assessments had co-primary endpoints, comprising mortality and morbidity (all OS and PFS). In all these cases only OS was considered in the appraisal.

Metabolic diseases
In metabolic diseases, 34 PM submissions were available, 15 of which were analysed ( Table 1). All PEPs were morbidity endpoints, apart from one composite mortality/ morbidity PEP (Fig. 1c) and three morbidity PEPs were deemed patient-relevant in the benefit assessments. One of these PEPs was glycated haemoglobin (HbA1c) in type 1 diabetes (insulin degludec), which was explicitly specified by the G-BA as a validated and patient-relevant endpoint. The others were 6-minute walk test (6MWT) in type IVA mucopolysaccharidose (elosulfase alfa), and median urinary free cortisol level (mUFC) in pituitary gland dysfunction (pasireotide).
In the ten EBAs where PEPs were not considered patient-relevant by the G-BA, PEPs were HbA1c and FEV1 (forced expiratory volume in 1 second) in type 2 diabetes and cystic fibrosis, respectively. In eight benefit assessments (albiglutide, dulaglutide, linagliptin [re-assessment], saxagliptin, saxagliptin/metformin, sitagliptin, sitagliptin/metformin, vildagliptin [re-assessment]) the PEP HbA1c was not explicitly mentioned as patient-relevant and not considered in the G-BA benefit evaluation, and therefore assumed to be not relevant to patients according to the evaluation criteria of this analysis stated above. In the two assessments of ivacaftor, FEV1 data was not taken into consideration by the G-BA as there were different opinions within the G-BA regarding its patient relevance.
In two EBAs the PEP was only partially accepted as patient-relevant by the G-BA. For eliglustat (Gaucher disease type 1) the PEP was a co-primary endpoint, consisting of two endpoints. The first was the composite morbidity endpoint stable health status (spleen volume, liver volume, haemoglobin and thrombocyte count) and the second percentage change in spleen volume, where only the change in spleen volume was deemed patientrelevant. For pasireotid (acromegaly) the PEP was biochemical control (defined as a combination of reduction of the mean growth hormone [GH] level below 2.5 μg/l and normalisation of the age-and sex-adjusted insulin-

Infectious diseases
In infectious diseases 14 of 16 PM submissions were included in the analysis set (Table 1). There were no mortality PEPs in infectious diseases. All 14 PEPs concerned morbidity (Fig. 1d), and all were accepted as patientrelevant by the G-BA. In most cases (13 of 14) the PEP was viral response (sustained viral response [SVR] in hepatitis C or viral response [VR] in HIV-infected patients), and in one case, the PEP was overall cure (fidaxomicin, clostridium infection).

Acceptance of symptomatic vs asymptomatic PEPs
PEPs were categorised as symptomatic or asymptomatic according to the definition of the PEP and based on the appraisal documents as well as scientific discussions during the hearing process. A list of therapeutic area expert panels attending the oral hearings is shown in Table 2. Table 3 summarises each evaluated PEP and the results of the categorisation by therapeutic area (see Additional file 1: Table S1 for a table listing the rationale of categorisation for each endpoint). Asymptomatic PEPs mostly comprised laboratory parameters (e.g. HbA1c in diabetes) or endpoints involving imaging outcomes (e.g. PFS or objective response rate [ORR]) [9]. Symptomatic endpoints included outcomes directly experienced by the patient, for example OS in oncological conditions and 6MWT in mucopolysaccharidose [9]. For other PEPs (complete/partial durable tumour and symptomatic response, overall cure), symptomaticity was self-explanatory by their definitions [9]. Regarding the symptomaticity of the PEPs related to reduction in spleen or liver volume, an independent Gaucher disease type 1 expert (a member of the German Working Group for Congenital Metabolic Disorders in Internal Medicine [Arbeitsgemeinschaft für angeborene Stoffwechselstörungen in der Inneren Medizin, ASIM]) stated in the minutes of the oral hearing procedure for eliglustat that the spleen volume in patients prior to treatment is typically increased 15-fold to ca. 2 L [9]. Since the measured reduction with eliglustat treatment was around 30 % and therefore easily palpable we assigned the reduction in spleen volume as a symptomatic endpoint. A similar reasoning applies to the reduction of ≥35 % of spleen volume in the EBA of ruxolitinib in the indications chronic myeloproliferative diseases and polycythaemia vera. In contrast, a closer analysis of the data for Gaucher disease showed that the relative enlargement and reduction in liver volume before and after eliglustat treatment were too small to be palpable and thus deemed not symptomatic.
Subsequently, we compared the consideration of symptomatic vs asymptomatic PEPs by the G-BA in evaluating additional benefit. For three PEPs the oral hearing minutes were consulted in order to categorise them as symptomatic or asymptomatic, as described above. Symptomatic endpoints were mostly regarded as patient-relevant (Table 3 A), apart from partial durable tumour and symptomatic response (siltuximab, multicentric Castleman's disease) ( Table 3

Discussion
This analysis demonstrates that specific morbidity PEPs from the EBA are categorically excluded by the G-BA across major disease areas, irrespective of indication and disease stage. Typically, these are PEPs used in pivotal studies for marketing authorisation. The respective clinical studies were specifically designed to show clinical benefit based on these endpoints. The aim of the EBA is to assess the additional benefit of a new medicine compared to the prespecified comparative treatment. This generally differs from the aim of regulatory authorities, which is to evaluate the benefit vs risk profile of a new medicine. Nevertheless, the methods and standards of evidence-based medicine Fig. 1 Patient relevance of PEPs according to the G-BA by endpoint dimension. a All three therapeutic areas, b oncological diseases, c metabolic diseases, and d infectious diseases. HRQoL was not used as PEP for any assessment, therefore it is not shown in the graphs apply to both evaluation procedures. Furthermore, primary endpoints of pivotal studies which led to EMA approval of the investigated drug should be accepted as patient-relevant, irrespective of whether 'benefit' or 'additional benefit' is under consideration. To suggest that these primary study endpoints are not relevant to patients, as the G-BA does, creates a major dilemma in clinical development and implicitly questions the ethical conduct of studies. Our analysis by outcome dimension showed a strong dominance of morbidity over mortality PEPs (47 vs 11). While the mortality PEPs were accepted as patientrelevant without exception, less than half of the morbidity PEPs were deemed patient-relevant. Interestingly, the acceptance varied by therapeutic area, revealing a disadvantage for oncological and metabolic disease indications, where morbidity PEPs were predominantly regarded as not patient-relevant. Considering that the majority of these PEPs were accepted by regulatory authorities for marketing authorisation, the classification of such a large proportion of PEPs as not relevant to patients by the G-BA highlights marked differences in data interpretation by regulatory and health technology assessment (HTA) bodies. A similar finding was also demonstrated in a comparative analysis of parallel scientific advice of different European HTA bodies and the EMA [10].
The PEPs that were most frequently dismissed as not patient-relevant were PFS in oncological diseases and HbA1c in metabolic diseases (except for type 1 diabetes). Despite opposing views on patient relevance of PFS within the G-BA (e.g. afatinib, axitinib, crizotinib, eribuline) [9] data on this endpoint were systematically excluded from benefit assessments, based on the justification that PFS evaluation utilises asymptomatic endpoint assessment techniques such as imaging. Conversely, in infectious diseases, the PEP SVR was accepted as patient-relevant in hepatitis C despite being an asymptomatic surrogate endpoint lacking formal validation. PEPs in neurological diseases were also evaluated but ultimately excluded from the analysis set as only six assessments were evaluable. Here, morbidity PEPs were largely accepted as patientrelevant by the G-BA (5 of 6 EBAs).
The results of our analysis clearly demonstrate inconsistency in the G-BA's approach to judging the patient relevance of PEPs between disease areas. For example, in infectious diseases, the G-BA has shown some flexibility in accepting the asymptomatic endpoint SVR as patientrelevant. On the other hand, in oncological and metabolic diseases some asymptomatic endpoints (PFS in cancers and HbA1c in type 2 diabetes) are categorically dismissed, without taking into consideration the different disease profiles of the indications. This is in contrast to the EMA, which adopts a broader approach when evaluating patient relevance and takes indication, difficulty in obtaining mortality data, and priority of accelerating patient access into account.
We welcome the flexibility the G-BA has shown regarding the acceptance of SVR as patient-relevant in hepatitis C, but identify a need to expand this flexibility to other endpoints. The general dismissal of PFS is questionable since in some cancers, such as ovarian cancer, it has been suggested that improved PFS is associated with clinical benefit and is a valid surrogate for extended OS [11]. Nevertheless the G- BA did not accept PFS as patient-relevant in the EBA of olaparib for the treatment of ovarian cancer [9]. The success of novel anticancer drugs in recent years has led to a classification shift of various oncological conditions from acutely fatal to chronic disease. Similar to chronic diseases such as diabetes, it is challenging to obtain mortality data showing significant differences in those oncological diseases with slow progression and low death rates. For these reasons, PFS has gained importance as a PEP in clinical trials because it is an early detectable and meaningful endpoint for disease progression [12,13].
Similarly to PFS, HbA1c is routinely used as a surrogate endpoint in diabetes mellitus and widely accepted by regulatory authorities [14]. For example the EMA states in its "Note for guidance on clinical investigation of medicinal products in the treatment of diabetes mellitus" that HbA1c is the most widely accepted measure of overall, long-term blood glucose control in type 1 and type 2 diabetes, and therefore an appropriate primary study endpoint [15]. Moreover, this guidance states that reduction of HbA1c is directly related to a reduced risk of developing vascular complications. The G-BA, however, only accepts HbA1c as a patient-relevant endpoint in type 1 diabetes, but not in type 2.
For the patient relevance of morbidity endpoints, clarification of its definition and determination criteria by the G-BA are urgently needed, particularly in oncological diseases and type 2 diabetes, as currently no clear criteria for patient relevance are listed in the G-BA rules of procedure [2]. Representatives from patient advocacy groups and other external experts are consulted at several stages of the assessment procedure, potentially also concerning the patient relevance of endpoints, but their influence on the benefit appraisal is not transparent and has been criticised as insufficient [16,17]. In a recent press release from the German Society for Hematology and Medical Oncology (Deutsche Gesellschaft für Hämatologie und Medizinische Onkologie, DGHO) a DGHO board member criticised the fact that market authorisation, EBA, and treatment guidelines often come to different conclusions, despite being based on the same clinical data, making it difficult for doctors to make treatment decisions [18].
Alignment of study requirements between the G-BA and regulatory authorities is necessary in order to optimise trial design and reduce patient sample sizes, particularly in rare indications, in order to allow timely access to new treatments. A recent publication has identified considerable heterogeneity in regulatory and HTA approaches, even among different European HTA bodies [10]. Regulatory authorities have adapted their approval pathways for innovative and promising new medicines to facilitate early patient access to new treatments, for example via conditional approval or adaptive pathways [10]. In order to effectively establish early access to medicines, HTA procedures need to follow the EMA's footsteps and provide harmonised, transparent, flexible, conditional, and adaptive methods that adopt the level of evidence accepted by medicines agencies [10].
The UK National Institute for Health and Clinical Excellence (NICE) uses a less restrictive approach in their HTA procedure, where appropriate methods may be used to extrapolate data from less than ideal study designs regarding study type, study duration, patient population, choice of comparator, and type of outcomes [19]. In contrast to the approach adopted by the G-BA, where data that do not fully comply with requirements are largely rejected, NICE is more receptive to data derived from clinical trials which are for instance nonrandomised, non-comparative or employ modelling.
Our analysis has revealed that asymptomatic endpoints were frequently disregarded by the G-BA, while symptomatic endpoints were largely deemed patient-relevant. However, acceptance of asymptomatic endpoints varied by therapeutic area. A major achievement in modern medicine is the ability to capture aspects of morbidity prior to their transformation in symptomatic disease in diagnostic and prognostic examinations. For example, in the assessment of axitinib for renal cell carcinoma, PFS was used to assess the delay in onset of metastases by imaging [9]. It is evident that early detection of metastases before the onset of symptomatic pain or even vertebral fractures is essential and of utmost relevance to the patient. In this case, the strict insistence on symptomatic endpoints may lead to a delay in complementary treatment options, such as bisphosphonates, which could prevent occurrence of vertebral fractures [20]. This would contradict the underlying ethical principles of diagnosis and therapy in this and many other oncological diseases.
Currently, the G-BA demands patient-relevant outcome data for the three endpoint categories mortality, morbidity and HRQoL, and weighs each with equal importance for all assessments. However, a number of reasons may lead to attenuation of mortality data, for instance, i) clinical advancement of available therapies, ii) change in disease status from 'untreatable' to 'chronic' , and iii) differences in the course of diseases themselves. Given the differences in mortality and progression between different diseases, for example some forms of cancer vs non-fatal chronic diseases, equal weighting of endpoint categories is not informative. Instead, differential weighting of each endpoint category in the EBA, taking into account disease type and severity/stage, as outlined in Fig. 2 would be advisable.

Limitation
For each EBA evaluated, only the respective PEP was included, as this is the parameter which provides the most clinically relevant and convincing evidence, based on the statistical principles of clinical trials [21], and clinical trials are explicitly designed to show efficacy in terms of the PEP. A more comprehensive approach would also include secondary endpoints. Nevertheless, our approach sufficiently highlights the clear inconsistencies that exist between endpoint acceptance by regulatory authorities and the G-BA.

Conclusions
There is a need for the G-BA to define the morbidity endpoints that are considered patient-relevant, particularly in indications such as diabetes and oncological diseases. This will allow PMs to target clinical trial designs towards requirements of the G-BA at an early clinical development stage to ensure an efficient pipeline for bringing innovative and efficacious treatments to patients. In addition, further harmonisation between regulatory bodies and G-BA with regards to acceptance of PEPs as relevant to patients is urgently required. In particular, this applies to morbidityrelated PEPs that are not (yet) symptomatic and tangible to patients.