A Primer on Propensity Score Analysis

doi:10.1053/j.nainr.2009.12.010

Newborn and Infant Nursing Reviews

Volume 10, Issue 1, March 2010, Pages 19-26

https://doi.org/10.1053/j.nainr.2009.12.010 Get rights and content

This article discusses the role that propensity score analysis can play in assessing the effects of interventions. It mostly focuses on identifying the range of solutions to practical problems that occur in propensity score analysis, especially with regard to propensity score construction (logistic regression, classification trees, ensemble methods), balancing (significance tests, other metrics), and analysis (matching, stratifying, weighting, covariance). Throughout, the article will identify particularly important or common pitfalls that need to be avoided in these analyses. The article ends with a discussion of the comparative advantages and disadvantages of propensity scores compared to alternative analytic and design options.

Section snippets

Estimating Propensity Scores

Three issues emerge in estimating propensity scores: the quality of the pretest measures, the statistical method for estimating propensity scores from those measures, and demonstrating balance using the propensity scores.

Using Propensity Scores to Estimate Treatment Effects

Once a best set of propensity scores is identified, those scores are used to estimate adjusted effects from nonrandomized experiments in one of several different ways—matching or stratifying on the propensity score, using the propensity score as a covariate in an ordinary regression, or weighting by some function of the propensity score. All these different propensity score techniques for estimating the treatment effect can be implemented using the regression framework. This has the advantage

Discussion

Propensity score analysis is increasingly popular for adjusting nonrandomized experiments. But does it work? The answer may be a qualified yes.⁵ The crucial qualification was mentioned previously—high-quality measurement of the selection process is crucial. Propensity score analysis is likely to work best when careful measurement of the selection process is planned before the study begins. This should include interviewing potential participants to find out what factors they consider in deciding

References (45)

CookT.D.
“Waiting for life to Arrive”: a history of the regression-discontinuity design in psychology, statistics and economics
J Econometrics
(2008)
ShahB.R. et al.
Propensity score methods gave similar results to traditional regression modeling in observational studies: a systematic review
J Clin Epidemiol
(2005)
ShadishW.R. et al.
Experimental and ouasi-experimental designs for generalized causal inference
(2002)
RosenbaumP.R. et al.
The central role of the propensity score in observational studies for causal effects
Biometrika
(1983)
RosenbaumP.R. et al.
Reducing bias in observational studies using subclassification on the propensity scores
J Am Stat Assoc
(1984)
ShadishW.R. et al.
Can nonrandomized experiments yield accurate answers? A randomized experiment comparing random to nonrandom assignment
J Am Stat Assoc
(2008)
Steiner PM, Cook TD, Shadish WR, et al. The importance of covariate selection in controlling for selection bias in...
Cook TD, Steiner PM. Case Matching and the reduction of selection bias in quasi-experiments: the relative importance of...
Steiner PM, Cook TD, Shadish WR. On the importance of reliable covariate measurement in selection bias adjustments...
HastieT. et al.
The elements of statistical learning
(2001)

BerkR.A.

Statistical learning from a regression perspective

(2008)

StoneR.A. et al.

Propensity score adjustment for pretreatment differences between hospitalized and ambulatory patients with community-acquired pneumonia

Medical Care

(1995)

BerkR.A.

An introduction to ensemble methods for data analysis

Sociol Methods Res

(2006)

McCaffreyD.F. et al.

Propensity score estimation with boosted regression for evaluating causal effects in observational studies

Psychol Methods

(2004)

LuellenJ.K. et al.

Propensity scores: an introduction and experimental test

Evaluation Review

(2005)

BerkR. et al.

Statistical difficulties in determining the role of race in capital cases: a re-analysis of data from the state of Maryland

J Quant Criminol

(2005)

R Development Core Team

R: a language and environment for statistical computing

(2009)

ShadishW.R. et al.

Propensity scores and quasi-experimentation

HoD. et al.

Matching as nonparametric preprocessing for reducing model dependence in parametric causal inference

Political Anal

(2007)

RubinD.B.

Using propensity scores to help design observational studies: application to the tobacco litigation

Health Serv Outcome Res Methodol

(2001)

RubinD.B.

For objective causal inference, design trumps analysis

Ann Appl Stat

(2008)

RobinsJ.M. et al.

Semiparametric efficiency in multivariate regression models with missing data

J Am Stat Assoc

(1995)

Cited by (85)

Alternatives to traditional outcome modelling approaches in applied linguistics: A primer on propensity score matching
2023, Research Methods in Applied Linguistics
Observational studies and quasi-experiments are commonplace in applied linguistics. Causal inference based on such a design hinges on different assumptions, one of which is commonly referred to as the assumption of no unmeasured confounding. To address this assumption, researchers typically consider different confounders using regression adjustment methods (i.e., entering the confounders into the model as covariates). Although these methods are theoretically sound procedures, they are not without criticism, for instance, regarding potentially extrapolated effect estimates or implausible linear functional form assumptions. In this article, we introduce an alternative statistical method, known as propensity score matching (PSM), which can help address the limitations of traditional regression adjustment methods. We use an example case study, where we first discuss the challenges applied linguists face when they intend to investigate causal effects. We illustrate how PSM can be used with applied linguistics data and examine theoretical and practical issues around each analytical step, including electing relevant covariates, estimating the propensity score, performing matching, assessing the matches, and estimating the treatment effects. Our overall goal is to provide applied linguists with an alternative method (PSM), which might be beneficial and reasonably applied in several practical situations.
Causal models
2022, International Encyclopedia of Education: Fourth Edition
The study of cause-effect relationships is central to education research. Well-designed randomized experiments, while ideal for evaluating the causal impact of an intervention, are often practically unfeasible due to ethical concerns or logistical challenges. In observational studies where each participant's exposed treatment is non-randomly assigned or selected, as is common in most education research settings, the association between the treatment and the outcome need not accurately describe the causal effect. Causal models provide a framework for accurately evaluating the causal impact of the intervention by seeking to obviate spurious associations between the treatment and the outcome arising from simultaneous influences.
Women's participation in microfinance: Effects on Women's agency, exposure to partner violence, and mental health
2021, Social Science and Medicine
Citation Excerpt :
The extent of overlap in the propensity score distributions indicates the likelihood that each participant ever enrolled or not in microfinance also has a certain likelihood of enrollment in the other group. The greater the overlap of the propensity-score distributions across groups, the greater is the common support and the likelihood of balance (Shadish and Steiner, 2010). When balance is achieved, pre-program distributions on the propensity scores and the covariates are likely to be similar, and the dataset provides support for causal inference (Shadish and Steiner, 2010).
The health and social effects of women's microfinance participation remain debated.
Using propensity-score methods, we assessed effects of microfinance participation on novel measures of agency; intimate partner violence (IPV) exposure; and depressive symptoms in 930 wives in Matlab, Bangladesh interviewed 11/2018–01/2019.
Participants, versus non-participants, were married younger (16.7 vs. 17.4 years), more often Muslim (90.7% vs. 86.2%), less schooled (5.4 vs. 6.8 grades), and more often had husbands (27.0% vs. 19.6%) and mothers (63.2% vs. 50.5%) without schooling. Participants and non-participants had similar unadjusted mean scores for prior-week depressive symptoms, prior-year IPV, and intrinsic attitudinal agency (gender-equitable attitudes; non-justification of wife beating). Participants had higher unadjusted mean scores for intrinsic voice/mobility; instrumental agency (using financial services, voice with husband, voice/mobility outside home); and collective agency. Average adjusted treatment effects were non-significant for depressive symptoms, IPV, and attitudinal intrinsic agency, and significantly favorable for other agency outcomes.
Microfinance participation had no adverse health effects and favorable empowerment effects in Bangladeshi wives. Policy Implications. Microfinance can empower women without adverse health effects. Social-norms programming with men and women may be needed to change gendered expectations about the distribution of unpaid labor and the rights of women.
Comparison of Valve Durability and Outcomes of Transcatheter Aortic Valve Implantation Versus Surgical Aortic Valve Replacement in Patients With Severe Symptomatic Aortic Stenosis and Less-Than-High-Risk for Surgery
2020, American Journal of Cardiology
Citation Excerpt :
Categorical variables are presented as absolute number with frequency (%) and were compared using the χ2 test (or Fisher's exact test where appropriate). Propensity score matching was used to adjust for differences in baseline characteristics and potential confounders.16 First, a propensity score for each patient was built using multivariable logistic regression, with the type of intervention as the end point (TA vs TF).
This study aimed to investigate the rate of severe structural valve deterioration (SVD) and long-term outcomes of patients with severe symptomatic aortic stenosis undergoing transcatheter aortic valve implantation (TAVI) or surgical aortic valve replacement (SAVR). Propensity score matched analysis of patients who underwent TAVI (n = 216) and SAVR (n = 216) between 2008 and 2012. Long-term echocardiographic parameters and clinical outcomes were assessed after more than 6 years after TAVI/SAVR. Rate of severe SVD was 10.5% versus 4.5% in the TAVI and SAVR groups, respectively, but the difference was not statistically significant (hazard ratio [HR] 2.5; 95% confidence interval [CI] 0.7 to 8.3; p = 0.159). This was largely driven by higher rates of mean transprosthetic gradient ≥40 mm Hg (7.0 vs 3.4%; p = 0.327) and aortic regurgitation (4.7% vs 0%; p = 0.058). TAVI patients had lower survival rates at 6 years than SAVR patients (40.7% vs 59.6%, respectively, p <0.001, HR 2.15; 95% CI 1.45 to 3.20). Rate of cardiovascular events (14.4% TAVI vs 18.2% SAVR, HR 0.8; 95% CI 0.4 to 1.3; p = 0.347) and permanent pacemaker implantation (PPI; 16.0% TAVI vs 9.2% SAVR, p = 0.234) was similar between the 2 groups. In conclusion, incidence of moderate and severe SVD was not statistically different between TAVI and SAVR. Rate of moderate or severe aortic regurgitation was significantly higher in the TAVI group with predominant use of first-generation valves. Reintervention rate was low in both groups. Survival rate was lower after TAVI, probably because of higher frailty index, but incidence of cardiovascular events, PPI, and SVD was similar in both groups.
Evaluation of a pragmatic trial of a collaborative school-based obesity prevention intervention in a low-income urban district
2020, Preventive Medicine
Citation Excerpt :
Some schools offered incentives to increase participation. We used propensity score matching methods with mixed models accounting for clustering to estimate the relative change in BMI between participants in intervention and control schools (Austin, 2011; Shadish and Steiner, 2010; Thoemmes and Kim, 2011). We first predicted the likelihood of receiving the intervention based on the following variables measured at baseline: sex, age at baseline, race, special needs status, ELL status, baseline BMI z-score, and baseline height.
Broader adoption of effective school-based obesity prevention interventions is critical to the success of ongoing efforts to address the childhood obesity epidemic. School-level barriers to adopting evidence-based interventions may be overcome by empowering school-level leaders to select appropriate intervention components. We used a quasi-experimental pragmatic trial design to evaluate a tailored obesity prevention intervention in 9 schools in a mid-sized urban school district in upstate New York from fall 2013 to spring 2016. We analyzed repeated height and weight measurements from an existing district screening system on 5882 students from intervention and control schools matched using propensity score methods. We assessed diet and physical activity changes in intervention schools using surveys and direct observation. The intervention led to a change of −0.27 (p = 0.026, 95% Confidence Interval (CI): −0.51, −0.03) and −0.28 (p = 0.031, 95% CI: −0.54, −0.03) BMI units in spring 2014 and fall 2014, respectively. There were no significant differences between intervention and control from spring 2015 to spring 2016. Despite the lack of sustained effects on BMI, we demonstrated the potential of supporting school leaders in a low-income district to implement supportive policy and practice changes and of using an existing BMI screening system to reduce the burden of health promotion evaluation.
Propensity Score Weighting with Missing Data on Covariates and Clustered Data Structure
2024, Multivariate Behavioral Research

View all citing articles on Scopus

^☆: The authors were supported in part by grant R305U070003 from the Institute for Educational Sciences, U.S. Department of Education. The second author was also supported by grants from the Spencer Foundation and W.T. Grant Foundation.

View full text

A Primer on Propensity Score Analysis☆

Section snippets

Estimating Propensity Scores

Using Propensity Scores to Estimate Treatment Effects

Discussion

J Econometrics

J Clin Epidemiol

Experimental and ouasi-experimental designs for generalized causal inference

The central role of the propensity score in observational studies for causal effects

Biometrika

Reducing bias in observational studies using subclassification on the propensity scores

J Am Stat Assoc

Can nonrandomized experiments yield accurate answers? A randomized experiment comparing random to nonrandom assignment

J Am Stat Assoc

The elements of statistical learning

Statistical learning from a regression perspective

Propensity score adjustment for pretreatment differences between hospitalized and ambulatory patients with community-acquired pneumonia

Medical Care

An introduction to ensemble methods for data analysis

Sociol Methods Res

Propensity score estimation with boosted regression for evaluating causal effects in observational studies

Psychol Methods

Propensity scores: an introduction and experimental test

Evaluation Review

Statistical difficulties in determining the role of race in capital cases: a re-analysis of data from the state of Maryland

J Quant Criminol

R: a language and environment for statistical computing

Propensity scores and quasi-experimentation

Matching as nonparametric preprocessing for reducing model dependence in parametric causal inference

Political Anal

Using propensity scores to help design observational studies: application to the tobacco litigation

Health Serv Outcome Res Methodol

For objective causal inference, design trumps analysis

Ann Appl Stat

Semiparametric efficiency in multivariate regression models with missing data

J Am Stat Assoc