Abstract
Comparisons between observed data and model simulations represent a critical component for establishing confidence in population physiologically-based pharmacokinetic (Pop-PBPK) models. Numerical predictive checks (NPC) that assess the proportion of observed data that correspond to Pop-PBPK model prediction intervals (PIs) are frequently used to qualify such models. We evaluated the effects of three components on the performance of NPC for qualifying Pop-PBPK model concentration–time predictions: (1) correlations (multiple samples per subject), (2) residual error, and (3) discrepancies in the distribution of demographics between observed and virtual subjects. Using a simulation-based study design, we artificially created observed pharmacokinetic (PK) datasets and compared them to model simulations generated under the same Pop-PBPK model. Observed datasets containing uncorrelated and correlated observations (± residual error) were formulated using different random-sampling techniques. In addition, we created observed datasets where the distribution of subject body weights differed from that of the virtual population used to generate model simulations. NPC for each observed dataset were computed based on the Pop-PBPK model’s 90% PI. NPC were associated with inflated type-I-error rates (> 0.10) for observed datasets that contained correlated observations, residual error, or both. Additionally, the performance of NPC were sensitive to the demographic distribution of observed subjects. Acceptable use of NPC was only demonstrated for the idealistic case where observed data were uncorrelated, free of residual error, and the demographic distribution of virtual subjects matched that of observed subjects. Considering the restricted applicability of NPC for Pop-PBPK model evaluation, their use in this context should be interpreted with caution.
Similar content being viewed by others
References
Maharaj AR, Edginton AN (2014) Physiologically based pharmacokinetic modeling and simulation in pediatric drug development. CPT Pharmacomet Syst Pharmacol 3:e150. https://doi.org/10.1038/psp.2014.45
Willmann S, Hohn K, Edginton A, Sevestre M, Solodenko J, Weiss W et al (2007) Development of a physiology-based whole-body population model for assessing the influence of individual variability on the pharmacokinetics of drugs. J Pharmacokinet Pharmacodyn 34(3):401–431. https://doi.org/10.1007/s10928-007-9053-5
Sager JE, Yu J, Ragueneau-Majlessi I, Isoherranen N (2015) Physiologically based pharmacokinetic (PBPK) modeling and simulation approaches: a systematic review of published models, applications, and model verification. Drug Metab Dispos 43(11):1823–1837. https://doi.org/10.1124/dmd.115.065920
Djebli N, Fabre D, Boulenc X, Fabre G, Sultan E, Hurbin F (2015) Physiologically based pharmacokinetic modeling for sequential metabolism: effect of CYP2C19 genetic polymorphism on clopidogrel and clopidogrel active metabolite pharmacokinetics. Drug Metab Dispos 43(4):510–522. https://doi.org/10.1124/dmd.114.062596
Hornik CP, Wu H, Edginton AN, Watt K, Cohen-Wolkowiez M, Gonzalez D (2017) Development of a pediatric physiologically-based pharmacokinetic model of clindamycin using opportunistic pharmacokinetic data. Clin Pharmacokinet 56(11):1343–1353. https://doi.org/10.1007/s40262-017-0525-5
Salerno SN, Edginton A, Cohen-Wolkowiez M, Hornik CP, Watt KM, Jamieson BD et al (2017) Development of an adult physiologically based pharmacokinetic model of solithromycin in plasma and epithelial lining fluid. CPT Pharmacomet Syst Pharmacol 6(12):814–822. https://doi.org/10.1002/psp4.12252
Jiang XL, Zhao P, Barrett JS, Lesko LJ, Schmidt S (2013) Application of physiologically based pharmacokinetic modeling to predict acetaminophen metabolism and pharmacokinetics in children. CPT Pharmacomet Syst Pharmacol 2:e80. https://doi.org/10.1038/psp.2013.55
Rodgers T, Leahy D, Rowland M (2005) Physiologically based pharmacokinetic modeling 1: predicting the tissue distribution of moderate-to-strong bases. J Pharm Sci 94(6):1259–1276. https://doi.org/10.1002/jps.20322
Rodgers T, Leahy D, Rowland M (2005) Tissue distribution of basic drugs: accounting for enantiomeric, compound and regional differences amongst beta-blocking drugs in rat. J Pharm Sci 94(6):1237–1248. https://doi.org/10.1002/jps.20323
Rodgers T, Rowland M (2006) Physiologically based pharmacokinetic modelling 2: predicting the tissue distribution of acids, very weak bases, neutrals and zwitterions. J Pharm Sci 95(6):1238–1257. https://doi.org/10.1002/jps.20502
PK-Sim® Ontogeny Database (version 7.1) 2017. https://github.com/Open-Systems-Pharmacology/OSPSuite.Documentation/blob/master/PK-Sim%20Ontogeny%20Database%20Version%207.1.pdf
Brendel K, Comets E, Laffont C, Mentre F (2010) Evaluation of different tests based on observations for external model evaluation of population analyses. J Pharmacokinet Pharmacodyn 37(1):49–65. https://doi.org/10.1007/s10928-009-9143-7
Wilkins J, Karlsson M, Jonsson E (2006) Patterns and power for the visual predictive check. PAGE 15 Abstr 1029 [wwwpage-meetingorg/?abstract=1029].
Duffull SB, Wright DF, Winter HR (2011) Interpreting population pharmacokinetic-pharmacodynamic analyses—a clinical viewpoint. Br J Clin Pharmacol 71(6):807–814. https://doi.org/10.1111/j.1365-2125.2010.03891.x
Mould DR, Upton RN (2013) Basic concepts in population modeling, simulation, and model-based drug development-part 2: introduction to pharmacokinetic modeling methods. CPT Pharmacomet Syst Pharmacol 2:e38. https://doi.org/10.1038/psp.2013.14
Huang W, Nakano M, Sager J, Ragueneau-Majlessi I, Isoherranen N (2017) Physiologically based pharmacokinetic model of the CYP2D6 probe atomoxetine: extrapolation to special populations and drug-drug interactions. Drug Metab Dispos 45(11):1156–1165. https://doi.org/10.1124/dmd.117.076455
Lin W, Heimbach T, Jain JP, Awasthi R, Hamed K, Sunkara G et al (2016) A physiologically based pharmacokinetic model to describe artemether pharmacokinetics in adult and pediatric patients. J Pharm Sci 105(10):3205–3213. https://doi.org/10.1016/j.xphs.2016.06.026
Nguyen TH, Mouksassi MS, Holford N, Al-Huniti N, Freedman I, Hooker AC et al (2017) Model evaluation of continuous data pharmacometric models: metrics and graphics. CPT Pharmacomet Syst Pharmacol 6(2):87–109. https://doi.org/10.1002/psp4.12161
Zhao P (2016) Application of physiologically-based pharmacokinetic modeling to support dosing recommendations—the US Food and Drug Administration Experience. 2016 EMA Workshop on PBPK Guideline2016 [cited 2018 07/23]; Available from: http://www.ema.europa.eu/docs/en_GB/document_library/Presentation/2016/12/WC500217569.pdf
Funding
This study was funded by the National Institutes of Health (1R01-HD076676-01A1; M.C.W.).
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflicts of interest
Anil R. Maharaj and Huali Wu have no conflicts of interest to declare. Michael Cohen-Wolkowiez receives support for research from the NIH (5R01-HD076676), NIH (HHSN275201000003I), NIAID/NIH (HHSN272201500006I), FDA (1U18-FD006298), the Biomedical Advanced Research and Development Authority (HHSO1201300009C), and from the industry for the drug development in adults and children (www.dcri.duke.edu/research/coi.jsp). Christoph P. Hornik receives salary support for research from National Institute for Child Health and Human Development (NICHD) (K23HD090239), the U.S. government for his work in pediatric and neonatal clinical pharmacology (Government Contract HHSN267200700051C, PI: Benjamin, under the Best Pharmaceuticals for Children Act), and industry for drug development in children.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Maharaj, A.R., Wu, H., Hornik, C.P. et al. Pitfalls of using numerical predictive checks for population physiologically-based pharmacokinetic model evaluation. J Pharmacokinet Pharmacodyn 46, 263–272 (2019). https://doi.org/10.1007/s10928-019-09636-5
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10928-019-09636-5