From a single-level analysis to a multilevel analysis of single-case experimental designs☆
Introduction
The use of single-case designs in a variety of different research fields in education as well as the suggested methods to analyze these types of designs have been expanding for decades. In this article, we describe and illustrate one method, namely, the use of multilevel modeling, which provides an appropriate method to analyze and summarize single-case designs (Moeyaert et al., 2013a, Owens and Ferron, 2012, Van den Noortgate and Onghena, 2008). In a single-case study, usually multiple cases, subjects, or participants are involved and repeatedly measured over time (Shadish & Sullivan, 2011). Therefore, in addition to the case-specific estimates, it is useful to develop methods to summarize the results over cases within a particular study. In the first part of this article, we present the basic two-level regression modeling framework that can be used to estimate the treatment effect across cases within studies and the between-case variance of this treatment effect (Van den Noortgate & Onghena, 2003a). We suggest and illustrate a sensitivity analysis approach in which multiple alternative specifications of this basic two-level model are examined. For illustration, we use the dataset of Lambert, Cartledge, Heward, and Lo (2006). In order to allow further examination of external validity and contribute to evidence-based research (Shadish & Rindskopf, 2007), multiple single-case studies measuring the same outcome variable can be combined using the three-level model, which is a straightforward extension of the two-level model. Thus, the second part of this article focuses on the three-level model. We present the basic three-level model assuming no linear trends in which the treatment effect across cases and across studies can be estimated as well as the between-case and between-study variances of this estimate. We will discuss the flexibility of this three-level modeling framework by suggesting multiple alternatives to the basic three-level model. The basic three-level model and alternative specifications of this basic three-level model will be illustrated by summarizing five studies in which a multiple-baseline across participants design was used to investigate the effects of pivotal response training with children with autism.
Section snippets
Two-level model
In single-case experiments, usually more than one case is the focus of interest (Shadish & Sullivan, 2011), such as in the replicated ABAB reversal designs and the multiple-baseline across participants designs. In this first design, there are multiple baseline phases (A phases) and multiple treatment phases (B phases), and the same ABAB design is implemented simultaneously to different participants (see Fig. 1a). In the multiple-baseline across participants design, an AB phase design (with one
Three-level model
The number of published single-case studies is growing rapidly during the last decade, and therefore there is an increasing interest in meta-analyzing these types of studies in order to estimate average treatment effects. The three-level model can be used to synthesize data across cases and across studies. If we pool several studies together, we can examine the generalizability of the results. The synthesis of the studies can inform policy and important decisions can be made based on these
Summary of three-level analysis of single-case experimental data
Table 14 provides a summary of the immediate treatment effect estimates for each proposed model (Models 1 through 4). Notwithstanding each suggested model has its own assumptions, we found similar results for the treatment effect estimates across Model 1 and Model 2 on the one hand and Model 3 and Model 4 on the other hand. The reason is that, in Model 3 and Model 4, a time trend during the treatment phase is modeled, which is not the case in Models 1 and 2. The positive estimated time trend
Discussion
Using the multilevel model (either the two-level or three-level model) to summarize single-case results over cases, over studies, or both has multiple advantages. Multilevel models can provide detailed information regarding the treatment effects (e.g., estimates of case-specific immediate treatment effects, case-specific trend shifts, level shifts across cases and across studies, average trend shifts across cases and across studies, and variance in effects across participants and studies). The
References (44)
- et al.
Using the natural language paradigm (NLP) to increase vocalizations of older adults with cognitive impairments
Research in Developmental Disabilities
(2007) - et al.
A generalized least squares regression approach for computing effect sizes in single-case research: Application examples
Journal of School Psychology
(2011) - et al.
Brief report: toward refinement of a predictive behavioral profile for treatment outcome in children with autism
Research in Autism Spectrum Disorders
(2009) Information theory and an extension of the maximum likelihood principle
- et al.
Making treatment effect inferences from multiple baseline data: The utility of multilevel modeling approaches
Behavior Research Methods
(2009) - et al.
Estimating causal effects from multiple-baseline studies: Implications for design and analysis
(2013) Multilevel statistical models
(1995)- et al.
Multilevel time series models with applications to repeated measures data
Statistics in Medicine
(1994) - et al.
A standardized mean difference effect size for single case designs
Research Synthesis Methods
(2012) - et al.
Two biased-reduced autocorrelation estimators: rF1 and rF2
Perceptual and Motor Skills
(1994)
Irrelevant autocorrelation in least-squares intervention models
Psychological Methods
Unbalanced repeated-measures models with structured covariance matrices
Biometrics
Setting generalization of question-asking by children with autism
American Journal on Mental Retardation
Parent education for families of children with autism living in geographically distant areas
Journal of Positive Behavior Interventions
The effect of tier 2 intervention for phonemic awareness in a response-to-intervention model in low-income preschool classrooms
Language, Speech, and Hearing Services in Schools
Determining the efficacy of intervention: The use of effect sizes for data analysis in single-subject research
Journal of Experimental Education
Effects of response cards on disruptive behavior and academic responding during math lessons by fourth-grade urban students
Journal of Positive Behavior Interventions
Training parents to use the natural language paradigm to increase their autistic children's speech
Journal of Applied Behavior Analysis
A double bootstrap method to analyze linear models with autoregressive error terms
Psychological Methods
Three-level analysis of single-case experimental data: Empirical validation
The Journal of Experimental Education
The influence of the design matrix on treatment effect estimates in the quantitative analyses of single-case experimental design research
Three-level analysis of standardized single-case experimental data: Empirical validation
Multivariate Behavior Research
Cited by (0)
- ☆
This research is funded by the Institute of Education Sciences, U.S. Department of Education, Grant number R305D110024 and Research Foundation - Flanders. The opinions expressed are those of the authors and do not represent views of the Institute or the U.S. Department of Education or the Research Foundation - Flanders.