Abstract
Background
The decision matrix applied by the Institute for Quality and Efficiency in Health Care (IQWiG) for the quantification of added benefit within the early benefit assessment of new pharmaceuticals in Germany with its nine fields is quite complex and could be simplified. Furthermore, the method used by IQWiG is subject to manifold criticism: (1) it is implicitly weighting endpoints differently in its assessments favoring overall survival and, thereby, drug interventions in fatal diseases, (2) it is assuming that two pivotal trials are available when assessing the dossiers submitted by the pharmaceutical manufacturers, leading to far-reaching implications with respect to the quantification of added benefit, and, (3) it is basing the evaluation primarily on dichotomous endpoints and consequently leading to an information loss of usable evidence.
Objective
To investigate if criticism is justified and to propose methodological adaptations.
Methods
Analysis of the available dossiers up to the end of 2016 using statistical tests and multinomial logistic regression and simulations.
Results
It was shown that due to power losses, the method does not ensure that results are statistically valid and outcomes of the early benefit assessment may be compromised, though evidence on favoring overall survival remains unclear. Modifications, however, of the IQWiG method are possible to address the identified problems.
Conclusion
By converging with the approach of approval authorities for confirmatory endpoints, the decision matrix could be simplified and the analysis method could be improved, to put the results on a more valid statistical basis.
Similar content being viewed by others
References
Skipka, G., Wieseler, B., Kaiser, T., Thomas, S., Bender, R., Windeler, J., Lange, S.: Methodological approach to determine minor, considerable, and major treatment effects in the early benefit assessment of new drugs. Biom J. Biometrische Zeitschrift. 58(1), 43–58 (2016). https://doi.org/10.1002/bimj.201300274
IQWiG.: Documentation and evaluation of comments on the Draft of the General Methods 4.1. Institute for Quality and Efficiency in Health Care (2013). https://www.iqwig.de/download/Dokumentation_und_Wuerdigung_der_Stellungnahmen_IQWiG_Methoden_4-1.pdf. Accessed 6 Nov 2017
AMNOG.: Act to Reorganize the Pharmaceuticals’ Market in the SHI System (Arzneimittelneuordnungsgesetz-AMNOG) (2010). http://www.bgbl.de/xaver/bgbl/start.xav?startbk=Bundesanzeiger_BGBl#__bgbl__%2F%2F*%5B%40attr_id%3D%27bgbl110s2262.pdf%27%5D__1431036981006. Accessed 6 Nov 2017
IQWiG.: General Methods. Version 4.2 of 22 April 2015 (2015). https://www.iqwig.de/download/IQWiG_General_Methods_Version_4-2_no_longer_valid.pdf. Accessed 6 Nov 2017
Vach, W.: Quantifying the additional clinical benefit of new medicines: little-considerable-significant-6 remarks from a biometrician’s point of view]. Gesundheitswesen (Bundesverband der Arzte des Offentlichen Gesundheitsdienstes (Germany)). 76(11), 757–762 (2014). https://doi.org/10.1055/s-0033-1363682
Röhmel, J.: Gutachten zum Vorschlag des IQWiG zur Bewertung des Ausmaßes des Zusatznutzens im Rahmen der Nutzenbewertung von Arzneimitteln nach § 35a SGB V, Bremen (2012)
Witte, J., Greiner, W.: Problembefunde der Quantifizierung des Zusatznutzens im Rahmen der frühen Arzneimittelnutzenbewertung. Gesundh ökon Qual manag. 18(05), 226–234 (2013). https://doi.org/10.1055/s-0033-1350539
Ruof, J., Dintsios, C.-M., Schwartz, F.W.: Questioning patient subgroups for benefit assessment: challenging the German Gemeinsamer Bundesausschuss approach. Value Health. 17(4), 307–309 (2014). https://doi.org/10.1016/j.jval.2014.05.001
Rasch, A., Dintsios, C.-M.: Subgruppen in der frühen Nutzenbewertung von Arzneimitteln: eine methodische Bestandsaufnahme. Zeitschrift für Evidenz, Fortbildung und Qualität. im Gesundheitswesen. 109(1), 69–78 (2015). https://doi.org/10.1016/j.zefq.2015.01.001
AM-NutzenV.: Verordnung über die Nutzenbewertung von Arzneimitteln nach § 35a Absatz 1 SGB V für Erstattungsvereinbarungen nach § 130b SGB V. https://www.bgbl.de/xaver/bgbl/start.xav#__bgbl__%2F%2F*%5B%40attr_id%3D%27bgbl110s2324.pdf%27%5D__1523126802497 (2010). Accessed 09 Apr 2018
Djulbegovic, B., Kumar, A., Soares, H.P., Hozo, I., Bepler, G., Clarke, M., Bennett, C.L.: Treatment success in cancer: new cancer treatment successes identified in phase 3 randomized controlled trials conducted by the National Cancer Institute-sponsored cooperative oncology groups, 1955 to 2006. Arch. Internal Med. 168(6), 632–642 (2008). https://doi.org/10.1001/archinte.168.6.632
IQWiG.: IQWiG reports. https://www.iqwig.de/en/projects-results/publications/iqwig-reports.1071.html. Accessed 09 Apr 2018
Bender, H., Dintsios, C.M.: [Health-related quality of life in the context of early benefit assessment of drugs according to section sign 35a of the German social code book V: a challenging endpoint for all the involved stakeholders]. Gesundheitswesen (Bundesverband der Arzte des Offentlichen Gesundheitsdienstes (Germany)). 80(2), 132–143 (2018). https://doi.org/10.1055/s-0042-116433
Niehaus, I., Dintsios, C.M.: Confirmatory versus explorative endpoint analysis: decision-making on the basis of evidence available from market authorization and early benefit assessment for oncology drugs. Health Policy (Amst. Neth.) (2018). https://doi.org/10.1016/j.healthpol.2018.03.017
FDA.: Guidance for industry: providing clinical evidence of effectiveness for human drugs and biological products. U.S. Department of Health and Human Services Food and Drug Administration, Center for Drug Evaluation and Research (CDER), Center for Biologics Evaluation and Research (CBER) (1998)
EMA.: 11.14. CPMP/EWP/2330/99: points to consider on validity and interpretation of meta-analyses, and one pivotal study (2001). http://www.ema.europa.eu/docs/en_GB/document_library/Scientific_guideline/2009/09/WC500003657.pdf. Accessed 6 Nov 2017
Lange, S.: Challenges for clinical trials in oncology within the scope of early benefit assessment of drugs. Z Evid Fortbild Qual Gesundhwes. 109(6), 417–430 (2015). https://doi.org/10.1016/j.zefq.2015.08.003
Marcus, R., Peritz, E., Gabriel, K.R.: On Closed testing procedures with special reference to ordered analysis of variance. Biometrika. 63(3), 655–660 (1976). https://doi.org/10.2307/2335748
Sonnemann, E.: Kombination unabhängiger tests. In: Vollmer, J. (ed.) Biometrie in der chemisch-pharmazeutischen Industrie, 4, Stand und Perspektiven. Gustav-Fischer, Stuttgart (1991)
Holm, S.: A simple sequentially rejective multiple test procedure. Scand. J. Stat. 6(2), 65–70 (1979)
Budde, M., Bauer, P.: Multiple test procedures in clinical dose finding studies. J. Am. Stat. Assoc. 84(407), 792–796 (1989). https://doi.org/10.1080/01621459.1989.10478839
Horn, M., Volland, R.: Multiple Tests und Auswahlverfahren. Spektrum Akademischer Verlag, Heidelberg (1999)
Wahl, M.: Proposal of a Two Stage Design for Dose Finding Studies. University of Dortmund, Dortmund (2001)
Erklärung der UPDRS auf parkinson/web. http://www.parkinson-web.de/content/was_ist_parkinson/beurteilungs__skalen/updrs_skala/index_ger.html. Accessed 09 April 2018
EMA.: EPAR Rotigotine (2006). http://www.ema.europa.eu/docs/en_GB/document_library/EPAR_-_Scientific_Discussion/human/000626/WC500026394.pdf. Accessed 6 Nov 2017
CADTH.: Common drug review–clinical review report—Rotigotine (2016). https://www.cadth.ca/sites/default/files/cdr/clinical/SR0344_Neupro_CL_Report.pdf. Accessed 6 Nov 2017
ICH E9 Expert Working Group.: Statistical principles for clinical trials: ICH harmonized tripartite guideline. Stat Med 18, 1905–1942 (1999)
Ruof, J., Staab, T., Dintsios, C.M., Schroter, J., Schwartz, F.W.: Comparison of post-authorisation measures from regulatory authorities with additional evidence requirements from the HTA body in Germany—are additional data requirements by the Federal Joint Committee justified? Health Econ Rev. 6(1), 46 (2016). https://doi.org/10.1186/s13561-016-0124-4
Dintsios, C.M.: Nutzen-Schaden-Abwägung zur Herleitung eines Gesamtnutzens im Rahmen der frühen Nutzenbewertung: Kritische Würdigung und alternative Lösungsansätze. 4–14 (2013). https://doi.org/10.3205/13dkvf040. http://www.egms.de/static/en/meetings/dkvf2013/13dkvf040.shtml. Accessed 6 Nov 2017
Dintsios, C.M.: Patientenpräferenzbasierte Nutzen-Schaden-Abwägung für die frühe Nutzenbewertung von Arzneimitteln: ein Erweiterungsvorschlag. 17 (2017). https://doi.org/10.3205/17ebm066. http://www.egms.de/en/meetings/ebm2017/17ebm066.shtml. Accessed 6 Nov 2017
Egbrink, M.O., IJzerman M.: The value of quantitative patient preferences in regulatory benefit-risk assessment. J Market Access Health Policy 2 (2014). https://doi.org/10.3402/jmahp.v2.22761
Muhlbacher, A.C., Johnson, F.R.: Giving patients a meaningful voice in european health technology assessments: the role of health preference research. Patient. 10(4), 527–530 (2017). https://doi.org/10.1007/s40271-017-0249-5
Hummel, M.J., Volz, F., van Manen, J.G., Danner, M., Dintsios, C.M., Ijzerman, M.J., Gerber, A.: Using the analytic hierarchy process to elicit patient preferences: prioritizing multiple outcome measures of antidepressant drug treatment. Patient. 5(4), 225–237 (2012). https://doi.org/10.2165/11635240-000000000-00000
Muhlbacher, A.C., Bridges, J.F., Bethge, S., Dintsios, C.M., Schwalm, A., Gerber-Grote, A., Nubling, M.: Preferences for antiviral therapy of chronic hepatitis C: a discrete choice experiment. Eur. J. Health. Econ. HEPAC Health Econ. Prev. Care. 18(2), 155–165 (2017). https://doi.org/10.1007/s10198-016-0763-8
Ruof, J., Schwartz, F.W., Schulenburg, J.M., Dintsios, C.M.: Early benefit assessment (EBA) in Germany: analysing decisions 18 months after introducing the new AMNOG legislation. Eur. J. Health Econ. HEPAC Health Econ. Prev. Care. 15(6), 577–589 (2014). https://doi.org/10.1007/s10198-013-0495-y
Funding
The authors did not receive any financial support for the preparation of the manuscript.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
MH is working for ClinStat GmbH, Cologne, Germany, a service provider offering statistical support to different pharmaceutical companies. Next to his academic affiliation CMD is employed by Bayer Vital GmbH, Leverkusen, Germany.
Rights and permissions
About this article
Cite this article
Herpers, M., Dintsios, CM. Methodological problems in the method used by IQWiG within early benefit assessment of new pharmaceuticals in Germany. Eur J Health Econ 20, 45–57 (2019). https://doi.org/10.1007/s10198-018-0981-3
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10198-018-0981-3
Keywords
- IQWiG
- Early benefit assessment
- Benefit quantification
- Dichotomization
- Pivotal trials
- Statistical validity