Abstract
Given an (optimal) dynamic treatment rule, it may be of interest to evaluate that rule – that is, to ask the causal question: what is the expected outcome had every subject received treatment according to that rule? In this paper, we study the performance of estimators that approximate the true value of: (1) an a priori known dynamic treatment rule (2) the true, unknown optimal dynamic treatment rule (ODTR); (3) an estimated ODTR, a so-called “data-adaptive parameter,” whose true value depends on the sample. Using simulations of point-treatment data, we specifically investigate: (1) the impact of increasingly data-adaptive estimation of nuisance parameters and/or of the ODTR on performance; (2) the potential for improved efficiency and bias reduction through the use of semiparametric efficient estimators; and, (3) the importance of sample splitting based on the cross-validated targeted maximum likelihood estimator (CV-TMLE) for accurate inference. In the simulations considered, there was very little cost and many benefits to using CV-TMLE to estimate the value of the true and estimated ODTR; importantly, and in contrast to non cross-validated estimators, the performance of CV-TMLE was maintained even when highly data-adaptive algorithms were used to estimate both nuisance parameters and the ODTR. In addition, we apply these estimators for the value of the rule to the “Interventions” study, an ongoing randomized controlled trial, to identify whether assigning cognitive behavioral therapy (CBT) to criminal justice-involved adults with mental illness using an ODTR significantly reduces the probability of recidivism, compared to assigning CBT in a non-individualized way.
Funding source: National Institute of Allergy and Infectious Diseases
Award Identifier / Grant number: F31AI140962
Funding source: National Institutes of Health
Award Identifier / Grant number: R01AI074345
-
Author contribution: All the authors have accepted responsibility for the entire content of this submitted manuscript and approved submission.
-
Research funding: None declared.
-
Conflict of interest statement: The authors declare no conflicts of interest regarding this article.
References
1. Khoury, MJ, Iademarco, MF, Riley, WT. Precision public health for the era of precision medicine. Am J Prev Med 2016;50:398–401. https://doi.org/10.1016/j.amepre.2015.08.031.Search in Google Scholar PubMed PubMed Central
2. Laber, EB, Davidian, M. Dynamic treatment regimes, past, present, and future: a conversation with experts. Stat Methods Med Res 2017;26:1605–10. https://doi.org/10.1177/0962280217708661.Search in Google Scholar PubMed PubMed Central
3. Bembom, O, van der Laan, MJ. A practical illustration of the importance of realistic individualized treatment rules in causal inference. Electronic Journal of Statistics 2007;1:574–96. https://doi.org/10.1214/07-EJS105.Search in Google Scholar PubMed PubMed Central
4. Chakraborty, B, Laber, EB, Zhao, Y. Inference for optimal dynamic treatment regimes using an adaptive m-out-of-n bootstrap scheme. Biometrics 2013;69:714–23. https://doi.org/10.1111/biom.12052.Search in Google Scholar PubMed PubMed Central
5. Chakraborty, B, Murphy, SA. Dynamic treatment regimes. Annual Review of Statistics and its Application 2014;1:447–64. https://doi.org/10.1146/annurev-statistics-022513-115553.Search in Google Scholar PubMed PubMed Central
6. Robins, JM. A new approach to causal inference in mortality studies with a sustained exposure period - application to control of the healthy worker survivor effect. Math Model 1986;7:9–12. https://doi.org/10.1016/0270-0255(86)90088-6.Search in Google Scholar
7. van der Laan, MJ, Petersen, ML. Causal effect models for realistic individualized treatment and intention to treat rules. Int J Biostat 2007;3:1557–4679. https://doi.org/10.2202/1557-4679.1022.Search in Google Scholar PubMed PubMed Central
8. Moodie, EE, Richardson, TS, Stephens, DA. Demystifying optimal dynamic treatment regimes. Biometrics 2007;63:447–55. https://doi.org/10.1111/j.1541-0420.2006.00686.x.Search in Google Scholar PubMed
9. Murphy, SA. Optimal dynamic treatment regimes. J Roy Stat Soc B 2003;65:331–55. https://doi.org/10.1111/1467-9868.00389.Search in Google Scholar
10. Robins, JM. Optimal structural nested models for optimal sequential decisions. In: Proceedings of the Second Seattle Symposium in Biostatistics. New York: Springer; 2004:189–326 pp.10.1007/978-1-4419-9076-1_11Search in Google Scholar
11. Kosorok, MR, Laber, EB. Precision medicine. Annual Review of Statistics and its Application 2019;6:263–86. https://doi.org/10.1146/annurev-statistics-030718-105251.Search in Google Scholar PubMed PubMed Central
12. Kosorok, MR, Moodie, EE. Adaptive Treatment Strategies in Practice: Planning Trials and Analyzing Data for Personalized Medicine. Philadelphia: Society for Industrial and Applied Mathematics; 2015.10.1137/1.9781611974188Search in Google Scholar
13. Tsiatis, AA, Davidian, M, Holloway, ST, Laber, EB. Dynamic Treatment Regimes: Statistical Methods for Precision Medicine. Boca Raton: Chapman and Hall/CRC Press; 2019.10.1201/9780429192692Search in Google Scholar
14. Coyle, JR. Computational considerations for targeted learning [Ph.D. thesis]. Berkeley: University of California; 2017.Search in Google Scholar
15. Luedtke, AR, van der Laan, MJ. Super-learning of an optimal dynamic treatment rule. Int J Biostat 2016;12:305–32. https://doi.org/10.1515/ijb-2015-0052.Search in Google Scholar PubMed PubMed Central
16. van der Laan, MJ, Polley, EC, Hubbard, AE. Super learner. Stat Appl Genet Mol Biol 2007;6. https://doi.org/10.2202/1544-6115.1309.Search in Google Scholar PubMed
17. Montoya, L, van der Laan, MJ, Luedtke, AR, Skeem, JL, Coyle, JR, Petersen, ML. The optimal dynamic treatment rule superlearner: considerations, performance, and application. Int J Biostat [Forthcoming].Search in Google Scholar
18. Hernan, MA, Robins, JM. Estimating causal effects from epidemiological data. J Epidemiol Community Health 2006;60:578–86. https://doi.org/10.1136/jech.2004.029496.Search in Google Scholar PubMed PubMed Central
19. Rosenbaum, PR, Rubin, DB. The central role of the propensity score in observational studies for causal effects. Biometrika 1983;70:41–55. https://doi.org/10.1093/biomet/70.1.41.Search in Google Scholar
20. Robins, JM. Robust estimation in sequentially ignorable missing data and causal inference models. In: Proceedings of the American Statistical Association. American Statistical Association; 1999;6–10 pp.Search in Google Scholar
21. Robins, JM, Rotnitzky, A, Zhao, LP. Estimation of regression coefficients when some regressors are not always observed. J Am Stat Assoc 1994;89:846–66. https://doi.org/10.1080/01621459.1994.10476818.Search in Google Scholar
22. Scharfstein, DO, Rotnitzky, A, Robins, JM. Adjusting for nonignorable drop-out using semiparametric nonresponse models: Rejoinder. J Am Stat Assoc 1999;94:1135–46. https://doi.org/10.2307/2669930.Search in Google Scholar
23. Rosenblum, M, van der Laan, MJ. Targeted maximum likelihood estimation of the parameter of a marginal structural model. Int J Biostat 2010;6:1557–4679. https://doi.org/10.2202/1557-4679.1238.Search in Google Scholar PubMed PubMed Central
24. van der Laan, MJ, Rose, S. Targeted learning: causal inference for observational and experimental data. New York: Springer; 2011.10.1007/978-1-4419-9782-1Search in Google Scholar
25. van der Laan, MJ, Luedtke, AR. Targeted learning of the mean outcome under an optimal dynamic treatment rule. J Causal Inference 2015;3:61–95. https://doi.org/10.1515/jci-2013-0022.Search in Google Scholar PubMed PubMed Central
26. van der Laan, MJ, Rose, S. Targeted learning in data science. New York: Springer; 2018.10.1007/978-3-319-65304-4Search in Google Scholar
27. Zheng, W, van der Laan, MJ. Asymptotic theory for cross-validated targeted maximum likelihood estimation. In: U.C. Berkeley division of biostatistics working paper series; 2010.10.2202/1557-4679.1181Search in Google Scholar PubMed PubMed Central
28. Laber, EB, Qian, M. Evaluating personalized treatment regimes. New York: Chapman and Hall/CRC; 2017: 483–97 pp. chapter 15.Search in Google Scholar
29. Chakraborty, B, Murphy, S, Strecher, V. Inference for non-regular parameters in optimal dynamic treatment regimes. Stat Methods Med Res 2010;19:317–43. https://doi.org/10.1177/0962280209105013.Search in Google Scholar PubMed PubMed Central
30. Sies, A, Van Mechelen, I. Estimating the quality of optimal treatment regimes. Stat Med 2019;38:4925–38. https://doi.org/10.1002/sim.8342.Search in Google Scholar PubMed
31. Hubbard, AE, Kherad-Pajouh, S, van der Laan, MJ. Statistical inference for data adaptive target parameters. Int J Biostat 2016;12:3–19. https://doi.org/10.1515/ijb-2015-0013.Search in Google Scholar PubMed
32. Skeem, JL, Manchak, S, Peterson, JK. Correctional policy for offenders with mental illness: creating a new paradigm for recidivism reduction. Law Hum Behav 2011;35:110–26. https://doi.org/10.1007/s10979-010-9223-7.Search in Google Scholar PubMed
33. Skeem, JL, Winter, E, Kennealy, PJ, Louden, JE, Tatar, JRII. Offenders with mental illness have criminogenic needs, too: toward recidivism reduction. Law Hum Behav 2014;38:212–24. https://doi.org/10.1037/lhb0000054.Search in Google Scholar PubMed
34. Petersen, ML, van der Laan, MJ. Causal models and learning from data: integrating causal modeling and statistical estimation. Epidemiology 2014;25:418–26. https://doi.org/10.1097/ede.0000000000000078.Search in Google Scholar
35. Petersen, ML, Porter, KE, Gruber, S, Wang, Y, van der Laan, MJ. Diagnosing and responding to violations in the positivity assumption. Stat Methods Med Res 2012;21:31–54. https://doi.org/10.1177/0962280210386207.Search in Google Scholar PubMed PubMed Central
36. Luedtke, AR, van der Laan, MJ. Statistical inference for the mean outcome under a possibly non-unique optimal treatment strategy. Ann Stat 2016;44:713–42. https://doi.org/10.1214/15-AOS1384.Search in Google Scholar PubMed PubMed Central
37. Gruber, S, van der Laan, MJ. A targeted maximum likelihood estimator of a causal effect on a bounded continuous outcome. Int J Biostat 2010;6:1557–4679. https://doi.org/10.2202/1557-4679.1260.Search in Google Scholar PubMed PubMed Central
38. van der Laan, MJ, Robins, JM. Unified methods for censored longitudinal data and causality. New York: Springer; 2003.10.1007/978-0-387-21700-0Search in Google Scholar
39. van der Vaart, AW. Asymptotic statistics. Cambridge: Cambridge University Press; 1998.10.1017/CBO9780511802256Search in Google Scholar
40. Bickel, PJ, Klaassen, CAJ, Ritov, Y, Klaassen, JA. Efficient and adaptive estimation for semiparametric models. Baltimore: Johns Hopkins University Press; 1993, volume 4.Search in Google Scholar
41. R Core Team. R: A Language and Environment for Statistical Computing. Vienna, Austria: R Foundation for Statistical Computing; 2020. Available from: https://www.R-project.org/.Search in Google Scholar
42. Coyle, J. tlverse: Umbrella package for targeted learning in R; 2021. Available from: https://github.com/tlverse/tlverse, r package version 0.0.1.Search in Google Scholar
43. Friedman, JH. Multivariate adaptive regression splines. Ann Stat 1991;19:1–67. https://doi.org/10.1214/aos/1176347963.Search in Google Scholar
44. Ripley, BD. Pattern Recognition and Neural Networks. Cambridge: Cambridge University Press; 1996.10.1017/CBO9780511812651Search in Google Scholar
45. Chang, C-C, Lin, C-J. Libsvm: a library for support vector machines. ACM Transactions on Intelligent Systems and Technology 2011;2:1–27. https://doi.org/10.1145/1961189.1961199.Search in Google Scholar
46. Breiman, L, Friedman, JH, Olshen, RA, Stone, CJ. Classification and Regression Trees. New York: Routledge; 1984.Search in Google Scholar
47. Polley, E, LeDell, E, Kennedy, C, van der Laan, M. SuperLearner: Super Learner Prediction; 2020. Available from: https://CRAN.R-project.org/package=SuperLearner, r package version 2.0-28.Search in Google Scholar
48. Breiman, L. Random forests. Mach Learn 2001;45:5–32. https://doi.org/10.1023/a:1010933404324.10.1023/A:1010933404324Search in Google Scholar
49. Rosenblum, M, van der Laan, MJ. Using regression models to analyze randomized trials: asymptotically valid hypothesis tests despite incorrectly specified models. Biometrics 2009;65:937–45. https://doi.org/10.1111/j.1541-0420.2008.01177.x.Search in Google Scholar PubMed PubMed Central
50. Almirall, D, Nahum-Shani, I, Sherwood, NE, Murphy, SA. Introduction to SMART designs for the development of adaptive interventions: with application to weight loss research. Translational Behavioral Medicine 2014;4:260–74. https://doi.org/10.1007/s13142-014-0265-0.Search in Google Scholar PubMed PubMed Central
51. Lei, H, Nahum-Shani, I, Lynch, K, Oslin, D, Murphy, SA. A ”SMART” design for building individualized treatment sequences. Annu Rev Clin Psychol 2012;8:21–48. https://doi.org/10.1146/annurev-clinpsy-032511-143152.Search in Google Scholar PubMed PubMed Central
Supplementary Material
The online version of this article offers supplementary material (https://doi.org/10.1515/ijb-2020-0128).
© 2022 Walter de Gruyter GmbH, Berlin/Boston
