Skip to main content
SpringerLink
Log in

Early Detection of Pharmacovigilance Signals with Automated Methods Based on False Discovery Rates

A Comparative Study

  • Original Research Article
  • Published:
Drug Safety Aims and scope Submit manuscript

Abstract

Background: Improving the detection of drug safety signals has led several pharmacovigilance regulatory agencies to incorporate automated quantitative methods into their spontaneous reporting management systems. The three largest worldwide pharmacovigilance databases are routinely screened by the lower bound of the 95% confidence interval of proportional reporting ratio (PRR02.5), the 2.5% quantile of the Information Component (IC02.5) or the 5% quantile of the Gamma Poisson Shrinker (GPS05). More recently, Bayesian and non-Bayesian False Discovery Rate (FDR)-based methods were proposed that address the arbitrariness of thresholds and allow for a built-in estimate of the FDR. These methods were also shown through simulation studies to be interesting alternatives to the currently used methods.

Objective: The objective of this work was twofold. Based on an extensive retrospective study, we compared PRR02.5, GPS05 and IC02.5 with two FDR-based methods derived from the Fisher’s exact test and the GPS model (GPSpH0 [posterior probability of the null hypothesis H0 calculated from the Gamma Poisson Shrinker model]). Secondly, restricting the analysis to GPSpH0, we aimed to evaluate the added value of using automated signal detection tools compared with ‘traditional’ methods, i.e. non-automated surveillance operated by pharmacovigilance experts.

Methods: The analysis was performed sequentially, i.e. every month, and retrospectively on the whole French pharmacovigilance database over the period 1 January 1996–1 July 2002. Evaluation was based on a list of 243 reference signals (RSs) corresponding to investigations launched by the French Pharmacovigilance Technical Committee (PhVTC) during the same period. The comparison of detection methods was made on the basis of the number of RSs detected as well as the time to detection.

Results: Results comparing the five automated quantitative methods were in favour of GPSpH0 in terms of both number of detections of true signals and time to detection. Additionally, based on an FDR threshold of 5%, GPSpH0 detected 87% of the RSs associated with more than three reports, anticipating the date of investigation by the PhVTC by 15.8 months on average.

Conclusions: Our results show that as soon as there is reasonable support for the data, automated signal detection tools are powerful tools to explore large spontaneous reporting system databases and detect relevant signals quickly compared with traditional pharmacovigilance methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Table I
Table II
Table III
Table IV
Fig. 1
Table V

Similar content being viewed by others

References

  1. van Puijenbroek EP, Bate A, Leufkens HGM, et al. A comparison of measures of disproportionality for signal detection in spontaneous reporting systems for adverse drug reactions. Pharmacoepidemiol Drug Saf 2002 Feb; 11(1): 3–10

    Article  PubMed  Google Scholar 

  2. Evans SJ, Waller PC, Davis S. Use of proportional reporting ratios (PRRs) for signal generation from spontaneous adverse drug reaction reports. Pharmacoepidemiol Drug Saf 2001 Nov; 10(6): 483–6

    Article  PubMed  CAS  Google Scholar 

  3. Bate A, Lindquist M, Edwards IR, et al. A Bayesian neural network method for adverse drug reaction signal generation. Eur J Clin Pharmacol 1998 Jun; 54(4): 315–21

    Article  PubMed  CAS  Google Scholar 

  4. Norén GN, Bate A, Orre R, et al. Extending the methods used to screen the WHO drug safety database towards analysis of complex associations and improved accuracy for rare events. Stat Med 2006 Nov 15; 25(21): 3740–57

    Article  PubMed  Google Scholar 

  5. Szarfman A, Machado SG, O’Neill RT. Use of screening algorithms and computer systems to efficiently signal higher-than-expected combinations of drugs and events in the US FDA’s spontaneous reports database. Drug Saf 2002; 25(6): 381–92

    Article  PubMed  CAS  Google Scholar 

  6. DuMouchel W. Bayesian data mining in large frequency tables, with an application to the FDA spontaneous reporting system. Am Stat 1999 Aug; 53(3): 177–90

    Google Scholar 

  7. DuMouchel W, Pregibon D. Empirical bayes screening for multi-item associations. In: Proceedings of the Seventh ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. San Francisco (CA): ACM, 2001: 67–76 [online]. Available from URL: http://portal.acm.org/citation.cfm?id=502526 [Accessed 2010 Mar 26]

  8. Benjamini Y, Hochberg Y. Controlling the false discovery rate: a practical and powerful approach to multiple testing. J R Stat Soc Series B Stat Methodol 1995; 57(1): 289–300

    Google Scholar 

  9. Ahmed I, Dalmasso C, Haramburu F, et al. False discovery rate estimation for frequentist pharmacovigilance signal detection methods. Biometrics 2010 Mar; 66(1): 301–9

    Article  PubMed  CAS  Google Scholar 

  10. Ahmed I, Haramburu F, Fourrier-Reglat A, et al. Bayesian pharmacovigilance signal detection methods revisited in a multiple comparison setting. Stat Med 2009 Jun 15; 28(13): 1774–92

    Article  PubMed  Google Scholar 

  11. Ahmed I, Thiessard F, Miremont-Salame G, et al. Pharma-covigilance data mining with methods based on false discovery rates: a comparative simulation study. Clin Pharmacol Ther 2010 Oct; 88(4): 492–8

    Article  PubMed  CAS  Google Scholar 

  12. Roux E, Thiessard F, Fourrier A, et al. Evaluation of statistical association measures for the automatic signal generation in pharmacovigilance. IEEE Trans Inf Technol Biomed 2005 Dec; 9(4): 518–27

    Article  PubMed  Google Scholar 

  13. Matsushita Y, Kuroda Y, Niwa S, et al. Criteria revision and performance comparison of three methods of signal detection applied to the spontaneous reporting database of a pharmaceutical manufacturer. Drug Saf 2007; 30(8): 715–26

    Article  PubMed  Google Scholar 

  14. Almenoff JS, LaCroix KK, Yuen NA, et al. Comparative performance of two quantitative safety signalling methods: implications for use in a pharmacovigilance department. Drug Saf 2006; 29(10): 875–87

    Article  PubMed  CAS  Google Scholar 

  15. Lehman HP, Chen J, Gould AL, et al. An evaluation of computer-aided disproportionality analysis for post-marketing signal detection. Clin Pharmacol Ther 2007 Aug; 82(2): 173–80

    Article  PubMed  CAS  Google Scholar 

  16. Lindquist M, Stahl M, Bate A, et al. A retrospective evaluation of a data mining approach to aid finding new adverse drug reaction signals in the WHO international database. Drug Saf 2000 Dec; 23(6): 533–42

    Article  PubMed  CAS  Google Scholar 

  17. Hauben M, Madigan D, Gerrits CM, et al. The role of data mining in pharmacovigilance. Expert Opin Drug Saf 2005 Sep; 4(5): 929–48

    Article  PubMed  CAS  Google Scholar 

  18. Hochberg AM, Hauben M, Pearson RK, et al. An evaluation of three signal-detection algorithms using a highly inclusive reference event database. Drug Saf 2009; 32(6): 509–25

    Article  PubMed  Google Scholar 

  19. Hochberg AM, Hauben M. Time-to-signal comparison for drug safety data-mining algorithms vs. traditional signaling criteria. Clin Pharmacol Ther 2009 Jun; 85(6): 600–6

    Article  PubMed  CAS  Google Scholar 

  20. Alvarez Y, Hidalgo A, Maignen F, et al. Validation of statistical signal detection procedures in eudravigilance post-authorization data: a retrospective evaluation of the potential for earlier signalling. Drug Saf 2010 Jun 1; 33(6): 475–87

    Article  PubMed  Google Scholar 

  21. Thiessard F. Détection des effets indésirables des médicaments par un système de génération automatisée du signal adapté à la base nationale de pharmacovigilance [in French; thesis]. Bordeaux: Université Victor Segalen Bordeaux 2, 2004

    Google Scholar 

  22. Thiessard F, Roux E, Miremont-Salamé G, et al. Trends in spontaneous adverse drug reaction reports to the French pharmacovigilance system (1986–2001). Drug Saf 2005; 28(8): 731–40

    Article  PubMed  Google Scholar 

  23. Miller GC, Britt H. A new drug classification for computer systems: the ATC extension code. Int J Biomed Comput 1995 Oct; 40(2): 121–4

    Article  PubMed  CAS  Google Scholar 

  24. Brown EG, Wood L, Wood S. The medical dictionary for regulatory activities (MedDRA). Drug Saf 1999 Feb; 20(2): 109–17

    Article  PubMed  CAS  Google Scholar 

  25. Ahmed I, Poncet A. PhViD: an R package for pharma-covigilance signal detection [online]. Available from URL: http://cran.r-project.org/web/packages/PhViD/index.html [Accessed 2012 Mar 29]

  26. R Development Core Team. R: a language and environment for statistical computing [online]. Vienna, 2011. Available from URL: http://www.R-project.org [Accessed 2012 Mar 29]

  27. Bégaud B, Evreux JC, Jouglard J, et al. Imputation of the unexpected or toxic effects of drugs: actualization of the method used in France. Therapie 1985 Apr; 40(2): 111–8

    PubMed  Google Scholar 

Download references

Acknowledgements

This work was supported in part by IRESP (Institut de Recherche en Santé Publique) grant A06076LS. The authors have no conflicts of interest that are directly relevant to the content of this study.

Author information

Authors and Affiliations

  1. CESP Centre for Research in Epidemiology and Population Health, U1018, INSERM, Biostatistics, 16, Av P.V. Couturier, 94807, Villejuif, France

    Ismaïl Ahmed & Pascale Tubert-Bitter

  2. Universite’ Paris-Sud, UMRS 1018, Villejuif, France

    Ismaïl Ahmed & Pascale Tubert-Bitter

  3. Department of Epidemiology and Biostatistics, Imperial College, London, UK

    Ismaïl Ahmed

  4. Laboratoire d’Epide’miologie, Statistique et Informatique Me’dicales (LESIM), Institut de Sante’ Publique, d’E’pide’miologie et de De’veloppement (ISPED), Universite’ Victor Segalen Bordeaux 2, Bordeaux, France

    Frantz Thiessard

  5. Service d’Information Me’dicale, Centre Hospitalier Universitaire de Bordeaux, Bordeaux, France

    Frantz Thiessard

  6. Centre Re’gional de Pharmacovigilance, Centre Hospitalier Universitaire de Bordeaux, Bordeaux, France

    Ghada Miremont-Salame & Françoise Haramburu

  7. Bordeaux, INSERM, U657, France

    Ghada Miremont-Salame, Françoise Haramburu & Bernard Be’gaud

  8. De’partement de Pharmacovigilance, AFSSAPS, Saint Denis, France

    Carmen Kreft-Jais

  9. Universite’ de Bordeaux, U657, Bordeaux, France

    Bernard Be’gaud

Authors
  1. Ismaïl Ahmed
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Frantz Thiessard
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ghada Miremont-Salame
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Françoise Haramburu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Carmen Kreft-Jais
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Bernard Be’gaud
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Pascale Tubert-Bitter
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ismaïl Ahmed.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Ahmed, I., Thiessard, F., Miremont-Salame, G. et al. Early Detection of Pharmacovigilance Signals with Automated Methods Based on False Discovery Rates. Drug Saf 35, 495–506 (2012). https://doi.org/10.2165/11597180-000000000-00000

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.2165/11597180-000000000-00000

Keywords

Search

Navigation