Skip to main content
Log in

Methodological problems in the method used by IQWiG within early benefit assessment of new pharmaceuticals in Germany

The European Journal of Health Economics Aims and scope Submit manuscript

Abstract

Background

The decision matrix applied by the Institute for Quality and Efficiency in Health Care (IQWiG) for the quantification of added benefit within the early benefit assessment of new pharmaceuticals in Germany with its nine fields is quite complex and could be simplified. Furthermore, the method used by IQWiG is subject to manifold criticism: (1) it is implicitly weighting endpoints differently in its assessments favoring overall survival and, thereby, drug interventions in fatal diseases, (2) it is assuming that two pivotal trials are available when assessing the dossiers submitted by the pharmaceutical manufacturers, leading to far-reaching implications with respect to the quantification of added benefit, and, (3) it is basing the evaluation primarily on dichotomous endpoints and consequently leading to an information loss of usable evidence.

Objective

To investigate if criticism is justified and to propose methodological adaptations.

Methods

Analysis of the available dossiers up to the end of 2016 using statistical tests and multinomial logistic regression and simulations.

Results

It was shown that due to power losses, the method does not ensure that results are statistically valid and outcomes of the early benefit assessment may be compromised, though evidence on favoring overall survival remains unclear. Modifications, however, of the IQWiG method are possible to address the identified problems.

Conclusion

By converging with the approach of approval authorities for confirmatory endpoints, the decision matrix could be simplified and the analysis method could be improved, to put the results on a more valid statistical basis.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

References

  1. Skipka, G., Wieseler, B., Kaiser, T., Thomas, S., Bender, R., Windeler, J., Lange, S.: Methodological approach to determine minor, considerable, and major treatment effects in the early benefit assessment of new drugs. Biom J. Biometrische Zeitschrift. 58(1), 43–58 (2016). https://doi.org/10.1002/bimj.201300274

    Article  PubMed  Google Scholar 

  2. IQWiG.: Documentation and evaluation of comments on the Draft of the General Methods 4.1. Institute for Quality and Efficiency in Health Care (2013). https://www.iqwig.de/download/Dokumentation_und_Wuerdigung_der_Stellungnahmen_IQWiG_Methoden_4-1.pdf. Accessed 6 Nov 2017

  3. AMNOG.: Act to Reorganize the Pharmaceuticals’ Market in the SHI System (Arzneimittelneuordnungsgesetz-AMNOG) (2010). http://www.bgbl.de/xaver/bgbl/start.xav?startbk=Bundesanzeiger_BGBl#__bgbl__%2F%2F*%5B%40attr_id%3D%27bgbl110s2262.pdf%27%5D__1431036981006. Accessed 6 Nov 2017

  4. IQWiG.: General Methods. Version 4.2 of 22 April 2015 (2015). https://www.iqwig.de/download/IQWiG_General_Methods_Version_4-2_no_longer_valid.pdf. Accessed 6 Nov 2017

  5. Vach, W.: Quantifying the additional clinical benefit of new medicines: little-considerable-significant-6 remarks from a biometrician’s point of view]. Gesundheitswesen (Bundesverband der Arzte des Offentlichen Gesundheitsdienstes (Germany)). 76(11), 757–762 (2014). https://doi.org/10.1055/s-0033-1363682

    Article  Google Scholar 

  6. Röhmel, J.: Gutachten zum Vorschlag des IQWiG zur Bewertung des Ausmaßes des Zusatznutzens im Rahmen der Nutzenbewertung von Arzneimitteln nach § 35a SGB V, Bremen (2012)

  7. Witte, J., Greiner, W.: Problembefunde der Quantifizierung des Zusatznutzens im Rahmen der frühen Arzneimittelnutzenbewertung. Gesundh ökon Qual manag. 18(05), 226–234 (2013). https://doi.org/10.1055/s-0033-1350539

    Article  Google Scholar 

  8. Ruof, J., Dintsios, C.-M., Schwartz, F.W.: Questioning patient subgroups for benefit assessment: challenging the German Gemeinsamer Bundesausschuss approach. Value Health. 17(4), 307–309 (2014). https://doi.org/10.1016/j.jval.2014.05.001

    Article  PubMed  Google Scholar 

  9. Rasch, A., Dintsios, C.-M.: Subgruppen in der frühen Nutzenbewertung von Arzneimitteln: eine methodische Bestandsaufnahme. Zeitschrift für Evidenz, Fortbildung und Qualität. im Gesundheitswesen. 109(1), 69–78 (2015). https://doi.org/10.1016/j.zefq.2015.01.001

    Article  Google Scholar 

  10. AM-NutzenV.: Verordnung über die Nutzenbewertung von Arzneimitteln nach § 35a Absatz 1 SGB V für Erstattungsvereinbarungen nach § 130b SGB V. https://www.bgbl.de/xaver/bgbl/start.xav#__bgbl__%2F%2F*%5B%40attr_id%3D%27bgbl110s2324.pdf%27%5D__1523126802497 (2010). Accessed 09 Apr 2018

  11. Djulbegovic, B., Kumar, A., Soares, H.P., Hozo, I., Bepler, G., Clarke, M., Bennett, C.L.: Treatment success in cancer: new cancer treatment successes identified in phase 3 randomized controlled trials conducted by the National Cancer Institute-sponsored cooperative oncology groups, 1955 to 2006. Arch. Internal Med. 168(6), 632–642 (2008). https://doi.org/10.1001/archinte.168.6.632

    Article  Google Scholar 

  12. IQWiG.: IQWiG reports. https://www.iqwig.de/en/projects-results/publications/iqwig-reports.1071.html. Accessed 09 Apr 2018

  13. Bender, H., Dintsios, C.M.: [Health-related quality of life in the context of early benefit assessment of drugs according to section sign 35a of the German social code book V: a challenging endpoint for all the involved stakeholders]. Gesundheitswesen (Bundesverband der Arzte des Offentlichen Gesundheitsdienstes (Germany)). 80(2), 132–143 (2018). https://doi.org/10.1055/s-0042-116433

    Article  CAS  Google Scholar 

  14. Niehaus, I., Dintsios, C.M.: Confirmatory versus explorative endpoint analysis: decision-making on the basis of evidence available from market authorization and early benefit assessment for oncology drugs. Health Policy (Amst. Neth.) (2018). https://doi.org/10.1016/j.healthpol.2018.03.017

    Article  Google Scholar 

  15. FDA.: Guidance for industry: providing clinical evidence of effectiveness for human drugs and biological products. U.S. Department of Health and Human Services Food and Drug Administration, Center for Drug Evaluation and Research (CDER), Center for Biologics Evaluation and Research (CBER) (1998)

  16. EMA.: 11.14. CPMP/EWP/2330/99: points to consider on validity and interpretation of meta-analyses, and one pivotal study (2001). http://www.ema.europa.eu/docs/en_GB/document_library/Scientific_guideline/2009/09/WC500003657.pdf. Accessed 6 Nov 2017

  17. Lange, S.: Challenges for clinical trials in oncology within the scope of early benefit assessment of drugs. Z Evid Fortbild Qual Gesundhwes. 109(6), 417–430 (2015). https://doi.org/10.1016/j.zefq.2015.08.003

    Article  PubMed  Google Scholar 

  18. Marcus, R., Peritz, E., Gabriel, K.R.: On Closed testing procedures with special reference to ordered analysis of variance. Biometrika. 63(3), 655–660 (1976). https://doi.org/10.2307/2335748

    Article  Google Scholar 

  19. Sonnemann, E.: Kombination unabhängiger tests. In: Vollmer, J. (ed.) Biometrie in der chemisch-pharmazeutischen Industrie, 4, Stand und Perspektiven. Gustav-Fischer, Stuttgart (1991)

    Google Scholar 

  20. Holm, S.: A simple sequentially rejective multiple test procedure. Scand. J. Stat. 6(2), 65–70 (1979)

    Google Scholar 

  21. Budde, M., Bauer, P.: Multiple test procedures in clinical dose finding studies. J. Am. Stat. Assoc. 84(407), 792–796 (1989). https://doi.org/10.1080/01621459.1989.10478839

    Article  Google Scholar 

  22. Horn, M., Volland, R.: Multiple Tests und Auswahlverfahren. Spektrum Akademischer Verlag, Heidelberg (1999)

    Google Scholar 

  23. Wahl, M.: Proposal of a Two Stage Design for Dose Finding Studies. University of Dortmund, Dortmund (2001)

    Google Scholar 

  24. Erklärung der UPDRS auf parkinson/web. http://www.parkinson-web.de/content/was_ist_parkinson/beurteilungs__skalen/updrs_skala/index_ger.html. Accessed 09 April 2018

  25. EMA.: EPAR Rotigotine (2006). http://www.ema.europa.eu/docs/en_GB/document_library/EPAR_-_Scientific_Discussion/human/000626/WC500026394.pdf. Accessed 6 Nov 2017

  26. CADTH.: Common drug review–clinical review report—Rotigotine (2016). https://www.cadth.ca/sites/default/files/cdr/clinical/SR0344_Neupro_CL_Report.pdf. Accessed 6 Nov 2017

  27. ICH E9 Expert Working Group.: Statistical principles for clinical trials: ICH harmonized tripartite guideline. Stat Med 18, 1905–1942 (1999)

    Google Scholar 

  28. Ruof, J., Staab, T., Dintsios, C.M., Schroter, J., Schwartz, F.W.: Comparison of post-authorisation measures from regulatory authorities with additional evidence requirements from the HTA body in Germany—are additional data requirements by the Federal Joint Committee justified? Health Econ Rev. 6(1), 46 (2016). https://doi.org/10.1186/s13561-016-0124-4

    Article  PubMed  PubMed Central  Google Scholar 

  29. Dintsios, C.M.: Nutzen-Schaden-Abwägung zur Herleitung eines Gesamtnutzens im Rahmen der frühen Nutzenbewertung: Kritische Würdigung und alternative Lösungsansätze. 4–14 (2013). https://doi.org/10.3205/13dkvf040. http://www.egms.de/static/en/meetings/dkvf2013/13dkvf040.shtml. Accessed 6 Nov 2017

  30. Dintsios, C.M.: Patientenpräferenzbasierte Nutzen-Schaden-Abwägung für die frühe Nutzenbewertung von Arzneimitteln: ein Erweiterungsvorschlag. 17 (2017). https://doi.org/10.3205/17ebm066. http://www.egms.de/en/meetings/ebm2017/17ebm066.shtml. Accessed 6 Nov 2017

  31. Egbrink, M.O., IJzerman M.: The value of quantitative patient preferences in regulatory benefit-risk assessment. J Market Access Health Policy 2 (2014). https://doi.org/10.3402/jmahp.v2.22761

  32. Muhlbacher, A.C., Johnson, F.R.: Giving patients a meaningful voice in european health technology assessments: the role of health preference research. Patient. 10(4), 527–530 (2017). https://doi.org/10.1007/s40271-017-0249-5

    Article  PubMed  Google Scholar 

  33. Hummel, M.J., Volz, F., van Manen, J.G., Danner, M., Dintsios, C.M., Ijzerman, M.J., Gerber, A.: Using the analytic hierarchy process to elicit patient preferences: prioritizing multiple outcome measures of antidepressant drug treatment. Patient. 5(4), 225–237 (2012). https://doi.org/10.2165/11635240-000000000-00000

    Article  PubMed  Google Scholar 

  34. Muhlbacher, A.C., Bridges, J.F., Bethge, S., Dintsios, C.M., Schwalm, A., Gerber-Grote, A., Nubling, M.: Preferences for antiviral therapy of chronic hepatitis C: a discrete choice experiment. Eur. J. Health. Econ. HEPAC Health Econ. Prev. Care. 18(2), 155–165 (2017). https://doi.org/10.1007/s10198-016-0763-8

    Article  Google Scholar 

  35. Ruof, J., Schwartz, F.W., Schulenburg, J.M., Dintsios, C.M.: Early benefit assessment (EBA) in Germany: analysing decisions 18 months after introducing the new AMNOG legislation. Eur. J. Health Econ. HEPAC Health Econ. Prev. Care. 15(6), 577–589 (2014). https://doi.org/10.1007/s10198-013-0495-y

    Article  Google Scholar 

Download references

Funding

The authors did not receive any financial support for the preparation of the manuscript.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Charalabos-Markos Dintsios.

Ethics declarations

Conflict of interest

MH is working for ClinStat GmbH, Cologne, Germany, a service provider offering statistical support to different pharmaceutical companies. Next to his academic affiliation CMD is employed by Bayer Vital GmbH, Leverkusen, Germany.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Herpers, M., Dintsios, CM. Methodological problems in the method used by IQWiG within early benefit assessment of new pharmaceuticals in Germany. Eur J Health Econ 20, 45–57 (2019). https://doi.org/10.1007/s10198-018-0981-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10198-018-0981-3

Keywords

JEL Classification

Navigation