Skip to main content
Log in

Pitfalls of using numerical predictive checks for population physiologically-based pharmacokinetic model evaluation

  • Original Paper
  • Published:
Journal of Pharmacokinetics and Pharmacodynamics Aims and scope Submit manuscript

Abstract

Comparisons between observed data and model simulations represent a critical component for establishing confidence in population physiologically-based pharmacokinetic (Pop-PBPK) models. Numerical predictive checks (NPC) that assess the proportion of observed data that correspond to Pop-PBPK model prediction intervals (PIs) are frequently used to qualify such models. We evaluated the effects of three components on the performance of NPC for qualifying Pop-PBPK model concentration–time predictions: (1) correlations (multiple samples per subject), (2) residual error, and (3) discrepancies in the distribution of demographics between observed and virtual subjects. Using a simulation-based study design, we artificially created observed pharmacokinetic (PK) datasets and compared them to model simulations generated under the same Pop-PBPK model. Observed datasets containing uncorrelated and correlated observations (± residual error) were formulated using different random-sampling techniques. In addition, we created observed datasets where the distribution of subject body weights differed from that of the virtual population used to generate model simulations. NPC for each observed dataset were computed based on the Pop-PBPK model’s 90% PI. NPC were associated with inflated type-I-error rates (> 0.10) for observed datasets that contained correlated observations, residual error, or both. Additionally, the performance of NPC were sensitive to the demographic distribution of observed subjects. Acceptable use of NPC was only demonstrated for the idealistic case where observed data were uncorrelated, free of residual error, and the demographic distribution of virtual subjects matched that of observed subjects. Considering the restricted applicability of NPC for Pop-PBPK model evaluation, their use in this context should be interpreted with caution.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

References

  1. Maharaj AR, Edginton AN (2014) Physiologically based pharmacokinetic modeling and simulation in pediatric drug development. CPT Pharmacomet Syst Pharmacol 3:e150. https://doi.org/10.1038/psp.2014.45

    Article  CAS  Google Scholar 

  2. Willmann S, Hohn K, Edginton A, Sevestre M, Solodenko J, Weiss W et al (2007) Development of a physiology-based whole-body population model for assessing the influence of individual variability on the pharmacokinetics of drugs. J Pharmacokinet Pharmacodyn 34(3):401–431. https://doi.org/10.1007/s10928-007-9053-5

    Article  PubMed  Google Scholar 

  3. Sager JE, Yu J, Ragueneau-Majlessi I, Isoherranen N (2015) Physiologically based pharmacokinetic (PBPK) modeling and simulation approaches: a systematic review of published models, applications, and model verification. Drug Metab Dispos 43(11):1823–1837. https://doi.org/10.1124/dmd.115.065920

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  4. Djebli N, Fabre D, Boulenc X, Fabre G, Sultan E, Hurbin F (2015) Physiologically based pharmacokinetic modeling for sequential metabolism: effect of CYP2C19 genetic polymorphism on clopidogrel and clopidogrel active metabolite pharmacokinetics. Drug Metab Dispos 43(4):510–522. https://doi.org/10.1124/dmd.114.062596

    Article  CAS  PubMed  Google Scholar 

  5. Hornik CP, Wu H, Edginton AN, Watt K, Cohen-Wolkowiez M, Gonzalez D (2017) Development of a pediatric physiologically-based pharmacokinetic model of clindamycin using opportunistic pharmacokinetic data. Clin Pharmacokinet 56(11):1343–1353. https://doi.org/10.1007/s40262-017-0525-5

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  6. Salerno SN, Edginton A, Cohen-Wolkowiez M, Hornik CP, Watt KM, Jamieson BD et al (2017) Development of an adult physiologically based pharmacokinetic model of solithromycin in plasma and epithelial lining fluid. CPT Pharmacomet Syst Pharmacol 6(12):814–822. https://doi.org/10.1002/psp4.12252

    Article  CAS  Google Scholar 

  7. Jiang XL, Zhao P, Barrett JS, Lesko LJ, Schmidt S (2013) Application of physiologically based pharmacokinetic modeling to predict acetaminophen metabolism and pharmacokinetics in children. CPT Pharmacomet Syst Pharmacol 2:e80. https://doi.org/10.1038/psp.2013.55

    Article  CAS  Google Scholar 

  8. Rodgers T, Leahy D, Rowland M (2005) Physiologically based pharmacokinetic modeling 1: predicting the tissue distribution of moderate-to-strong bases. J Pharm Sci 94(6):1259–1276. https://doi.org/10.1002/jps.20322

    Article  CAS  PubMed  Google Scholar 

  9. Rodgers T, Leahy D, Rowland M (2005) Tissue distribution of basic drugs: accounting for enantiomeric, compound and regional differences amongst beta-blocking drugs in rat. J Pharm Sci 94(6):1237–1248. https://doi.org/10.1002/jps.20323

    Article  CAS  PubMed  Google Scholar 

  10. Rodgers T, Rowland M (2006) Physiologically based pharmacokinetic modelling 2: predicting the tissue distribution of acids, very weak bases, neutrals and zwitterions. J Pharm Sci 95(6):1238–1257. https://doi.org/10.1002/jps.20502

    Article  CAS  PubMed  Google Scholar 

  11. PK-Sim® Ontogeny Database (version 7.1) 2017. https://github.com/Open-Systems-Pharmacology/OSPSuite.Documentation/blob/master/PK-Sim%20Ontogeny%20Database%20Version%207.1.pdf

  12. Brendel K, Comets E, Laffont C, Mentre F (2010) Evaluation of different tests based on observations for external model evaluation of population analyses. J Pharmacokinet Pharmacodyn 37(1):49–65. https://doi.org/10.1007/s10928-009-9143-7

    Article  PubMed  Google Scholar 

  13. Wilkins J, Karlsson M, Jonsson E (2006) Patterns and power for the visual predictive check. PAGE 15 Abstr 1029 [wwwpage-meetingorg/?abstract=1029].

  14. Duffull SB, Wright DF, Winter HR (2011) Interpreting population pharmacokinetic-pharmacodynamic analyses—a clinical viewpoint. Br J Clin Pharmacol 71(6):807–814. https://doi.org/10.1111/j.1365-2125.2010.03891.x

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  15. Mould DR, Upton RN (2013) Basic concepts in population modeling, simulation, and model-based drug development-part 2: introduction to pharmacokinetic modeling methods. CPT Pharmacomet Syst Pharmacol 2:e38. https://doi.org/10.1038/psp.2013.14

    Article  CAS  Google Scholar 

  16. Huang W, Nakano M, Sager J, Ragueneau-Majlessi I, Isoherranen N (2017) Physiologically based pharmacokinetic model of the CYP2D6 probe atomoxetine: extrapolation to special populations and drug-drug interactions. Drug Metab Dispos 45(11):1156–1165. https://doi.org/10.1124/dmd.117.076455

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  17. Lin W, Heimbach T, Jain JP, Awasthi R, Hamed K, Sunkara G et al (2016) A physiologically based pharmacokinetic model to describe artemether pharmacokinetics in adult and pediatric patients. J Pharm Sci 105(10):3205–3213. https://doi.org/10.1016/j.xphs.2016.06.026

    Article  CAS  PubMed  Google Scholar 

  18. Nguyen TH, Mouksassi MS, Holford N, Al-Huniti N, Freedman I, Hooker AC et al (2017) Model evaluation of continuous data pharmacometric models: metrics and graphics. CPT Pharmacomet Syst Pharmacol 6(2):87–109. https://doi.org/10.1002/psp4.12161

    Article  CAS  Google Scholar 

  19. Zhao P (2016) Application of physiologically-based pharmacokinetic modeling to support dosing recommendations—the US Food and Drug Administration Experience. 2016 EMA Workshop on PBPK Guideline2016 [cited 2018 07/23]; Available from: http://www.ema.europa.eu/docs/en_GB/document_library/Presentation/2016/12/WC500217569.pdf

Download references

Funding

This study was funded by the National Institutes of Health (1R01-HD076676-01A1; M.C.W.).

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Michael Cohen-Wolkowiez.

Ethics declarations

Conflicts of interest

Anil R. Maharaj and Huali Wu have no conflicts of interest to declare. Michael Cohen-Wolkowiez receives support for research from the NIH (5R01-HD076676), NIH (HHSN275201000003I), NIAID/NIH (HHSN272201500006I), FDA (1U18-FD006298), the Biomedical Advanced Research and Development Authority (HHSO1201300009C), and from the industry for the drug development in adults and children (www.dcri.duke.edu/research/coi.jsp). Christoph P. Hornik receives salary support for research from National Institute for Child Health and Human Development (NICHD) (K23HD090239), the U.S. government for his work in pediatric and neonatal clinical pharmacology (Government Contract HHSN267200700051C, PI: Benjamin, under the Best Pharmaceuticals for Children Act), and industry for drug development in children.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Maharaj, A.R., Wu, H., Hornik, C.P. et al. Pitfalls of using numerical predictive checks for population physiologically-based pharmacokinetic model evaluation. J Pharmacokinet Pharmacodyn 46, 263–272 (2019). https://doi.org/10.1007/s10928-019-09636-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10928-019-09636-5

Keywords

Navigation