Abstract
There has been growing awareness of the importance of the statistical evaluation of drug safety data both in the premarketing and postmarketing settings. Careful and comprehensive approaches are warranted in safety evaluation. This paper offers a high-level review of some key issues and emerging statistical methodological developments. Specifically, the following topics are discussed: prospective program-level safety planning, evaluation, and reporting; the impact of adverse event grouping on statistical analysis; the applications of Bayesian methods in safety signal detection; meta-analysis for analyzing safety data; and safety graphics. Aspects related to benefit-risk assessments are also covered.
Article PDF
Similar content being viewed by others
Avoid common mistakes on your manuscript.
References
ICH. ICH safety guidelines. Available at: https://www.ich.org/products/guidelines/safety/article/safetyguidelines.html. Accessed October 29, 2013.
Council for International Organizations of Medical Sciences (CIOMS) Working Group VI. Management of Safety Information From Clinical Trials. Geneva: CIOMS; 2005.
European Commission. Communication from the commission: detailed guidance on the collection, verification and presentation of adverse event/reaction reports arising from clinical trials on medicinal products for human use (‘CT-3’). 2011. Available at: https://www.kme-nmec.si/Docu/ct-3.pdf. Accessed October 29, 2013.
Crowe BJ, Xia HA, Berlin JA, et al. Recommendations for safety planning, data collection, evaluation and reporting during drug, biologic and vaccine development: a report of the safety planning, evaluation, and reporting team. Clin Trials. 2010;6(5):430–440.
Chuang-Stein C, Xia HA. The practice of pre-marketing safety assessment in drug development. J Biopharm Stat. 2013;23(1):3–25.
Xia HA, Crowe BJ, Schriver RC, Oster M, Hall DB. Planning and core analyses for periodic aggregate safety data reviews. Clin Trials. 2011;8(2):175–182.
Downs M, Wittes J. Data monitoring in practice: making your data monitoring committee effective. ASA Biopharmaceutical Section Webinar. March 10, 2010. Available at: https://www.biopharmnet.com/doc/2010_03_10_webinar.pdf. Accessed October 29, 2013.
O’Neill RT. Assessment of safety. In: Peace KE, ed. Biopharmaceutical Statistics for Drug Development. New York: Marcel Dekker; 1988:543–604.
Solomon SD, McMurray JJ, Pfeffer MA, et al. Cardiovascular risk associated with celecoxib in a clinical trial for colorectal adenoma prevention. N Engl J Med. 2005;352:1071–1080.
Proschan MA, Lan KKG, Wittes JT. Statistical Methods for Monitoring Clinical Trials. New York: Springer; 2006.
MedDRA. MSSO introductory guide: version 16.0. 2013. Available at: https://www.meddra.org/sites/default/files/guidance/file/intguide_16_0_english.pdf. Accessed October 29, 2013.
Gould AL.Drug safety evaluation in and after clinical trials. Presented at: the Deming Conference; December 3, 2002; Atlanta City, New Jersey.
Berry SM, Carlin BP, Lee JJ, Muller P. Bayesian Adaptive Methods for Clinical Trials. Boca Raton, Florida: Chapman and Hall/CRC Press; 2010.
Prieto-Merino D, Quartey G, Wang J, Kim J. Why a Bayesian approach to safety analysis in pharmacovigilance is important. Pharm Stat. 2011;10(6):554–559.
Mehrotra DV, Heyse JF. Use of the false discovery rate for evaluating clinical safety data. Stat Methods Med Res. 2004;13:227–238.
Berry SM, Berry DA. Accounting for multiplicities in assessing drug safety: a three-level hierarchical mixture model. Biometrics. 2004;60:418–426.
Xia HA, Ma H, Carlin BP. Bayesian hierarchical modeling for detecting safety signals in clinical trials. J Biopharm Stat. 2011;21(5):1006–1029.
Benjamini Y, Hochberg Y. Controlling the false discovery rate: a practical and powerful approach to multiple testing. J R Stat Soc Series B Stat Methodol. 1995;57(1):289–300.
Gould AL. Detecting potential safety issues in clinical trials by Bayesian screening. Biom J. 2008;50:837–851.
DuMouchel W. Multivariate Bayesian logistic regression for analysis of clinical study safety issues. Stat Sci. 2011;27(3):319–339.
Bate A, Lindquist M, Edwards IR, et al. A Bayesian neural network method for adverse drug reaction signal generation. Eur J Clin Pharmacol. 1998;54:315–321.
DuMouchel W. Bayesian data mining in large frequency tables, with an application to the FDA spontaneous reporting system. Am Stat. 1999;53:177–190.
Hopstadius J, Norén GK, Bate A, Edwards IR. Impact of stratification on adverse drug reaction surveillance. Drug Saf. 2008;31(11):1035–1048.
Almenoff J, Tonning JM, Gould AL, et al. Perspectives on the use of data mining in pharmacovigilance. Drug Saf. 2005;28:981–1007.
Schuemie MJ. Methods for drug safety signal detection in longitudinal observational databases: LGPS and LEOPARD. Drug Saf. 2011;20:292–299.
Norén G, Hopstadius J, Bate A, Star K, Edwards I. Temporal pattern discovery in longitudinal electronic patient records. Data Min Knowl Discov. 2010;20(3):361–387.
Norén GN, Bate A, Hopstadius J, Edwards IR. Safety surveillance of longitudinal databases: methodological considerations. Pharmacoepidemiol Drug Saf. 2011;20:714–717.
Nissen SE, Wolski K. Effect of rosiglitazone on the risk of myocardial infarction and death from cardiovascular causes. N Engl J Med. 2007;356(24):2457–2471.
Bridge JA, Iyengar S, Salary CB, et al. Clinical response and risk for reported suicidal ideation and suicide attempts in pediatric antidepressant treatment: a metaanalysis of randomized controlled trials. JAMA. 2007;297(15):1683–1696.
ICH. ICH Harmonised Tripartite Guideline: statistical principles for clinical trials E9. 1998. Available at: https://www.ich.org/products/guidelines/efficacy/article/efficacy-guidelines.html. Accessed May 23, 2013.
Egger M, Smith GD, Phillips AN. Meta-analysis: principles and procedures. BMJ. 1997;315:1533–1537.
Hammad TA, Pinheiro SP, Neyarapally GA. Secondary use of randomized controlled trials to evaluate drug safety: a review of methodological considerations. Clin Trials. 2011;8(5):559–570.
Sutton AJ, Cooper NJ, Lambert PC, Jones DR, Abrams KR, Sweeting MJ. Meta-analysis of rare and adverse event data. Expert Rev Pharmacoecon Outcomes Res. 2002;2:367–379.
Deeks JJ, Altman DG. Effect measures for meta-analysis of trials with binary outcomes. In: Egger M, Smith GD, Altman DG, eds. Systematic Reviews in Health Care: Meta-analysis in Context. 2nd ed. London: BMJ Publishing Group; 2008:313–335.
Engels EA, Schmid CH, Terrin N, Olkin I, Lau J. Heterogeneity and statistical significance in meta-analysis: an empirical study of 125 meta-analyses. Stat Med. 2000;19(13):1707–1728.
Localio AR, Margolis DJ, Berlin JA. Relative risks and confidence intervals were easily computed indirectly from multivariable logistic regression. J Clin Epidemiol. 2007;60(9):874–882.
Berlin JA, Crowe BJ, Whalen E, Xia HA, Koro CE, Kuebler J. Meta-analysis of clinical trial safety data in a drug development program: answers to frequently asked questions. Clin Trials. 2013;10(1):20–31.
Parmar MK, Torri V, Stewart L. Extracting summary statistics to perform meta-analyses of the published literature for survival endpoints. Stat Med. 1998;17:2815–2834.
Tierney JF, Stewart LA, Ghersi D, Burdett S, Sydes MR. Practical methods for incorporating summary time-to-event data into meta-analysis. Trials. 2007;8:16.
Borenstein M, Hedges LV, Higgins JPT, Rothstein HR. A basic introduction to fixed-effect and random effects models for meta-analysis. Res Synth Methods. 2010;1:97–111.
Sweeting MJ, Sutton AJ, Lambert PC. What to add to nothing? Use and avoidance of continuity corrections in meta-analysis of sparse data. Stat Med. 2004;23:1351–1375.
Thompson SG. Why sources of heterogeneity in meta-analysis should be investigated. BMJ. 1994;309:1351–1355.
Lewis S, Clarke M. Forest plots: trying to see the wood and the trees. BMJ. 2001;322:1479–1480.
Higgins JP, Thompson SG. Quantifying heterogeneity in a meta-analysis. Stat Med. 2002;21:1539–1558.
Higgins JPT, Green S. Cochrane Handbook for Systematic Reviews of Interventions: Version 5.1.0. Cochrane Collaborations. March 2011. Available at: https://handbook.cochrane.org/. Accessed May 28, 2013.
Bradburn MJ, Deeks JJ, Berlin JA, Localio AR. Much ado about nothing: a comparison of the performance of meta-analytical methods with rare events. Stat Med. 2007;26:53–77.
Tian L, Cai T, Pfeffer M, Piankov N, Cremieux PY, Wei LJ. Exact and efficient inference procedure for meta analysis and its application to the analysis of independent 2 by 2 tables with all available data but without artificial continuity correction. Biostatistics. 2009;10(2):275–281.
Cai T, Parast L, Ryan L. Meta-analysis for rare events. Stat Med. 2010;29(20):2078–2089.
Askling J, Fahrbach K, Nordstrom B, Ross S, Schmid C, Symmons D. Cancer risk with tumor necrosis factor alpha (TNF) inhibitors: meta-analysis of randomized controlled trials of adalimumab, etanercept, and infliximab using patient level data. Pharmacoepidemiol Drug Saf. 2011;20(2):119–130.
Kaizar EE, Greenhouse JB, Seltman H, Kelleher K. Do antidepressants cause suicidality in children? A Bayesian meta-analysis. Clin Trials. 2006;3(2):73–90, discussion 91–98.
Lyman GH, Kuderer NM. The strengths and limitations of meta-analyses based on aggregate data. BMC Med Res Methodol. 2005;5:14.
Hu M, Cappelleri JC, Lan KK. Applying the law of iterated logarithm to control type I error in cumulative meta-analysis of binary outcomes. Clin Trials. 2007;4(4):329–340.
Ibrahim J, Chen M, Xia HA, Liu T. Bayesian meta-experimental design: evaluating cardiovascular risk in new antidiabetic therapies to treat type 2 diabetes. Biometrics. 2012;68(2):578–586.
Chen M, Ibrahim J, Xia HA, Liu T, Hennessey V. Bayesian sequential meta-analysis design in evaluating cardiovascular risk in a new antidiabetic drug development program. Stat Med. Provisionally accepted.
Anziano R, Gordon R. Examples of understanding safety data using graphical methods. Wiley Interdiscip Rev Comput Stat. 2013;5:78–95.
Duke S, Jiang Q, Huang L, Banach M. Safety graphics. In: Jiang Q, Xia HA, eds. Quantitative Evaluation of Safety in Drug Development: Design, Analysis and Reporting. New York: Taylor & Francis. In preparation.
Mt-Isa S, Peters R, Phillips LD, et al. On behalf of PROTECT Work Package 5 participants: review of visualisation methods for the representation of benefit risk assessment of medication. February 2013. Available at: https://www.imi-protect.eu/documents/ShahruletalReviewofvisualisationmethodsfortherepresentationofBRassessmentofmedicationStage1F.pdf. Accessed October 27, 2013.
Zeng D, Chen M, Ibrahim JG, et al. A counterfactual p-value approach for benefit-risk assessment in clinical trials. J Biopharm Stat. In press.
Jiang Q, Shepherd S, Ke C, Ma H, Snapinn S. Considerations for benefit: risk assessment in pharmaceutical drug development. Presented at: ICSA/ISBS; June 9–12, 2013; Bethesda, Maryland.
Chuang-Stein C. A new proposal for benefit-less-risk analysis in clinical trials. Control Clin Trials. 1994;15:30–43.
EMA. Human Medicines Development and Evaluation. Benefit-Risk Methodology Project. Work package 2 report: applicability of current tools and processes for regulatory benefit-risk assessment. August 31, 2010 (EMA/549682/2010). Available at: https://www.ema.europa.eu/docs/en_GB/document_library/Report/2010/10/WC500097750.pdf. Accessed October 27, 2013.
EMA. Human Medicines Development and Evaluation. Benefit-Risk Methodology Project. Work package 4 report: benefit-risk tools and processes. May 9, 2012 (EMA/297405/2012). Available at: https://www.ema.europa.eu/docs/en_GB/document_library/Report/2012/03/WC500123819.pdf. Accessed October 27, 2013.
FDA. Structured approach to benefit-risk assessment in drug regulatory decision-making. Draft PDUFA V Implementation Plan: February 2013, Fiscal Years 2013–2017. Available at: https://www.fda.gov/downloads/forindustry/userfees/prescriptiondruguserfee/ucm329758.pdf. Accessed October 27, 2013.
Walker S. Chairman’s introduction. Presented at: CIRS Technical Workshop on Benefit-Risk Framework for the Assessment of Medicines: Valuing the Options and Determining the Relative Importance (Weighting) of Benefit and Risk Parameters; December 13, 2012; Philadelphia, Pennsylvania.
Levitan BS, Andrews EB, Gilsenan A, et al. Application of the BRAT framework to case studies: observations and insights. Clin Pharmacol Ther. 2011;89(2):217–224.
EMA. Report of the CHMP Working Group on benefit-risk assessment models and methods. January 19, 2007 (EMEA/CHMP/15404/2007). Available at: https://www.ema.europa.eu/docs/en_GB/document_library/Regulatory_and_procedural_guideline/2010/01/WC500069668.pdf. Accessed October 27, 2013.
EMA. ICH guideline E2C (R2) on periodic benefit-risk evaluation report (PBRER). January 2013 (EMA/CHMP/ICH/544553/1998). Available at: https://www.rsihata.com/updateguidance/2013/WC500136402.pdf. Accessed October 27, 2013.
Jiang Q, Ke C, Bloss L, Kouchakji E, Snapinn S, Vega J. Some thoughts on benefit-risk assessment. Presented at: FDA/Industry Statistical Workshop; September 16–18, 2013; Washington, DC.
Ke C, Jiang Q, Snapinn S. Methods for benefit-risk assessments in drug development. In: Jiang Q, Xia HA, eds. Quantitative Evaluation of Safety in Drug Development: Design, Analysis and Reporting. New York: Taylor & Francis. In preparation.
Kouchakji E, Jiang Q, Consuelo B, Bloss L, Ke C. Benefit:risk assessments: a ranking/qualitative approach. Presented at: CIRS Technical Workshop on Benefit-Risk Framework for the Assessment of Medicines: Valuing the Options and Determining the Relative Importance (Weighting) of Benefit and Risk Parameters; December 13, 2012; Philadelphia, Pennsylvania.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Xia, H.A., Jiang, Q. Statistical Evaluation of Drug Safety Data. Ther Innov Regul Sci 48, 109–120 (2014). https://doi.org/10.1177/2168479013510917
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1177/2168479013510917