Abstract
Measuring treatment effects are a complicated task as the outcomes of receiving and not receiving the treatment cannot be observed simultaneously. Thus, the issue of obtaining accurate measurement is an issue of predicting the counterfactuals accurately. In this study, we explore the suitability of using time-varying parameter models in a panel to generate robust measure of counterfactuals, hence robust measure of treatment effects. We suggest some within-sample tests for constant parameter versus time-varying parameter models and diagnostic tools based on time-varying parameter framework as a flexible alternative to predict missing data. Monte Carlos and two empirical studies are examined in this framework. The results appear to show that if the focus is on minimizing the mean square error of the predicted treatment effects, a “straitjacket” approach relying on the best selected model from the pre-treatment data remains the best option in view of the missing post-treatment information on counterfactuals. On the other hand, the confidence band based on the time-varying parameter model provides a more robust inference to hedge against possible changes of the relations between the treated units and the controls in the post-treatment period.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
As pointed out by A. Pagan in a private correspondence, our alternative way to differentiate CPM from TVM is to test whether \(var\left({x}_{t}\right)var\left({\beta }_{t-1}\right)\) is constant or increasing over time. For instance, consider the first-differenced model \(\Delta {y}_{t}=\Delta {a}_{t}+\Delta {x}_{t}{\beta }_{t}+\Delta {u}_{t}\), its \(var\left({x}_{t}\right)var\left({\beta }_{t-1}\right)\) must increase with \(t\) if \({\beta }_{t}\) follows a random walk hypothesis.
Pagan and Wickens (2019) have explored using Bayesian estimates for the dynamic stochastic general equilibrium model as a way to “modify the jacket, not the model” has also failed in most of the criteria they suggested.
References
Abadie A, Diamond A, Hainmueller J (2010) Synthetic control methods for comparative case studies: estimating the effect of California’s tobacco control program. J Am Stat Assoc 105:493–505
Bai J (1997) Estimating multiple breaks one at a time. Econ Theory 13:315–352
Cooley TF, Prescott EC (1976) Estimation in the presence of stochastic parameter variation. Econometrica 44:167–184
Dell M, Jones BF, Olken BA (2014) What do we learn from the weather? The new climate-economy literature. J Econ Lit 52:740–798
Diebold FX, Mariano R (1985) Comparing predictive accuracy. J Bus Econ Stat 13:253–265
Gardeazabal J, Vega-Bayo A (2017) An empirical comparison between the synthetic control method and Hsiao et al.’s panel data approach to program evaluation. J Appl Econ 32(5):983–1002
Haar RJ, Lacopino V, Ranadive N, Weiser SD, Dandu M (2017) Health impacts of chemical irritants used for crowd control: a systematic review of the injuries and deaths caused by tear gas and pepper spray. BMC Public Health 17:831. https://doi.org/10.1186/s12889-017-4814-6
Harvey AC (1978) The estimation of time-varying parameters from panel data. Annales de TINSEE, Special Issue Econ Panel Data 30–31, 203–226
Harvey AC, Phillips GDA (1982) The estimation of regression models with time-varying parameters. In: Deistler M, Fürst E, Schwödiauer G (eds) Games, economic dynamics, and time series analysis. Physica, Heidelberg. https://doi.org/10.1007/978-3-662-41533-7_18
Heckman JJ (1979) Sample selection bias as a specification error. Econometrica 47(1):153–161
Heckman JJ, Vytlacil EJ (2005) Structural equations, treatment effects, and econometric policy evaluation. Econometrica 73:669–738
Heckman JJ, Vytlacil EJ (2007) Econometric evaluation of social programs, part II: using the marginal treatment effect to organize alternative economic estimators to evaluate social programs and to forecast their effects in new environments. In Heckman JJ, Leamer EE (eds) Handbook of econometrics, edn 1, vol 6, chapter 71, Elsevier
Hsiao C (2014) Analysis of panel data, 3rd edition, econometric society monographs, 54. Cambridge University Press, Cambridge
Hsiao C, Ching H, Wan SK (2012) A panel data approach for program evaluation: measuring the benefits of political and economic integration of Hong Kong and mainland China. J Appl Econ 27(5):705–740
Hsiao C, Wan SK (2014) Is there an optimal forecast combination? J Econ 178:294–309
Hsiao C, Zhou Q (2019) Panel parametric, semiparametric, and nonparametric construction of counterfactuals. J Appl Econo 27:705–740
Hsiao, C, Shen Y, Zhou Q (2020) Panel approach to measure average treatment effects when there are multiple treatment units-to aggregate or not? mimeo
Kalman RE (1960) A new approach to linear filtering and prediction problems. Transactions of the ASME. J Basic Eng 82:35–45
Li KT, Bell DR (2017) Estimation of average treatment effects with panel data: asymptotic theory and implementation. J Econ 197:55–75
Little RJA, Rubin DB (1987) Statistical analysis with missing data. Wiley, New York
Pagan A, Wickens MR (2019) “Checking if the Straitjacket Fits”, CEPR Discussion Paper 14140 (forthcoming in Advances in Econometrics)
Pesaran MH, Smith RP (2011) Beyond the DSGE Straitjacket. Manch Sch 79:5–16
Rosenbaum PR, Rubin DB (1983) The central role of the propensity score in observational studies for causal effects. Biometrika 70:41–55
Wan SK, Xie Y, Hsiao C (2018) Panel data approach versus synthetic control method. Econ Lett 164:121–123
Xu Y (2017) Generalized synthetic control method: causal inference with interactive fixed effects models. Political Anal 25:57–76
Acknowledgements
We are grateful for the helpful comments by the guest editors, referees and A. Pagan on the early version. The second author also wishes to thank China NSF grant #71131008 for partial research support. Zhou’s research is supported by the National Natural Science Foundation of China (No. 71431006).
Funding
This study was partly funded by Cheng Hsiao’s China NSF (No. 71631004), and partly by Qiankun Zhou’s National Natural Science Foundation of China (No. 72033008).
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no conflict of interest.
Ethical approval
This article does not contain any studies with human participants performed by any of the authors.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Wan, S.K., Hsiao, C. & Zhou, Q. Can a time-varying structure provide a more robust panel construction of counterfactuals-straitjacket or straitjackets?. Empir Econ 60, 113–129 (2021). https://doi.org/10.1007/s00181-020-01978-1
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00181-020-01978-1