Using assessment to individualize early mathematics instruction

Abstract

Accumulating evidence suggests that assessment-informed personalized instruction, tailored to students' individual skills and abilities, is more effective than more one-size-fits-all approaches. In this study, we evaluate the efficacy of Individualizing Student Instruction in Mathematics (ISI-Math) compared to Reading (ISI-Reading) where classrooms were randomly assigned to ISI-Math or ISI-Reading. The literature on child characteristics X instruction or skill X treatment interaction effects point to the complexities of tailoring instruction for individual students who present with constellations of skills. Second graders received mathematics instruction in small flexible learning groups based on their assessed learning needs. Results of the study (n = 32 teachers, 370 students) revealed significant treatment effects on standardized mathematics assessments. With effect sizes (d) of 0.41–0.60, we show that we can significantly improve 2nd graders' mathematics achievement, including for children living in poverty, by using assessment data to individualize the mathematics instruction they receive. The instructional regime, ISI-Math, was implemented by regular classroom teachers and it led to about a 4-month achievement advantage on standardized mathematics tests when compared to students in control classrooms. These results were realized within one school year. Moreover, treatment effects were the same regardless of school-level poverty and students' gender, initial mathematics or vocabulary scores.

Introduction

Children who develop strong mathematics skills in their early school years generally experience greater long term academic achievement than do their peers with poor numeracy skills (Duncan et al., 2007, Siegler et al., 2012), which in turn predicts later school completion and occupational success, (Geary et al., 2013, National Mathematics Advisory Panel, 2008, Ritchie and Bhatia, 1999, Rivera-Batiz, 1992, Siegler et al., 2012). However, according to the National Assessment of Educational Progress (NAEP, 2015), only about 40% of United States 4th graders attain proficiency in mathematics, and this percentage drops to 24% for children living in poverty. These rates represent an enduring problem, because achievement levels have held fairly steady over the last decade (NAEP 2005 to 2015). This means that a significant number of current and future U.S. students will enter school on a path towards underachievement in mathematics. At issue, then, is the need to identify ways to reverse this path and to do so as early as possible because early mathematics underachievement does not simply correct itself with time (Aunola et al., 2002, Mazzocco and Myers, 2003). One way, we hypothesize, is to improve the early mathematics instruction students receive, so that it is better aligned with their individual learning strengths and weaknesses in general, and their mathematical knowledge and skills specifically. However, the features of such individualized learning needs and instruction (as they pertain to early mathematics) are not yet well understood, and identifying them has been proposed as one of the current “grand challenges” of mathematical cognition research (Alcock et al., 2016, p. 23). In this paper, we describe an approach to mathematics instruction that uses assessment results to individualize (i.e., personalize, differentiate) the instruction students receive. We then test the efficacy of this instructional approach in a randomized controlled trial with teachers randomly assigned to individualized student instruction in either mathematics or reading.

Accumulating evidence suggests that differences in mathematics achievement levels are related to multiple sources of influence (Berch and Mazzocco, 2007, Bronfenbrenner and Morris, 2006) including family, community, and student characteristics. These sources of influence are also likely to influence mathematics development and children's kindergarten entry quantitative skills, which are highly predictive of later mathematics achievement (e.g., Nunes et al., 2015, Purpura et al., 2013). Although early mathematical skills include spatial and geometric skills, much of the research on early mathematical thinking focuses on number (e.g., Mazzocco & Räsänen, 2013), and number is a primary focus for early mathematics in the Common Core State Standards (CCSS, Common Core State Standards Initiative, 2010). There is evidence of individual differences emerging from studies of very basic, primitive measures of magnitude judgment (Halberda, Mazzocco, & Feigenson, 2008) to highly formalized representations of number relations and the ability to fluently execute simple operations (Nunes et al., 2015). For instance, kindergartner's understanding of enumeration, symbolic numbers, and the relations between numbers are strong predictors of their math achievement level at the end of third grade (Mazzocco & Thompson, 2005) or fourth grade (Jordan et al., 2010, Morgan et al., 2009). Their ability to compose and decompose numbers at school entry predicts numeracy levels in high school (Geary et al., 2013). Moreover, individual differences persist throughout the school age years, when they may manifest as qualitative errors (e.g., place value errors, atypical computational errors; Mazzocco et al., 2013a, Mazzocco et al., 2013b) or differences in fluency versus accuracy (Petrill et al., 2012, Price et al., 2013). Importantly, these differences are apparent in early childhood (e.g., Desoete et al., 2012, Murphy et al., 2007), which means that we cannot assume children arrive at school with equivalent foundational mathematics skills.

Even if children do enter school with solid foundational mathematics skills, the path to their mathematics competence may not be maintained throughout primary school. Moreover, efforts to promote mastery are likely to support achievement only for those children who have not yet achieved it, and efforts to maintain mastery will do little to advance mathematics achievement of students who have already attained age-appropriate early mathematics skills (Engel, Claessens, & Finch, 2013). This dilemma is the logical outcome of the heterogeneity in mathematics skills seen in early childhood, which continues through elementary school and beyond. Hence for this study, we chose second grade as an important grade for improving mathematics achievement and for testing the causal influence of children's individual differences on mathematics achievement (i.e., child X instruction interactions).

In addition to early mathematics skills, accumulating research suggests that other child characteristics including cognitive, linguistic, and social-emotional processes, may interact reciprocally and synergistically with instruction to impact how students respond to instruction (Connor et al., 2016). Based on bio-ecological theories (Bronfenbrenner & Morris, 2006) and dynamic systems (Yoshikawa & Hsueh, 2001), the lattice model (Connor, 2016) has been applied to literacy achievement. We have adapted this model for mathematics achievement as depicted in Fig. 1. Although there are important distinctions between mathematics and reading skills, the rationale for applying this framework to mathematics emerges from evidence for shared variance among early literacy and numeracy (Davidse, De Jong, & Bus, 2014), from findings that select literacy measures predict later mathematics achievement (Purpura, Hume, Sims, & Lonigan, 2011), and for well documented comorbidity of mathematics and reading disorders (Willcutt & Pennington, 2000). Applying the lattice framework to studies of early mathematics interventions may shed additional light on shared and non-shared aspects of early learning in mathematics and reading. This also justifies including other sources of influence on children's mathematics development including gender, language skills, and socio-economic status in our models.

In the lattice model, proficient mathematics achievement relies on developing mathematics-specific processes, such as numerical processing (Mazzocco et al., 2011a, Mazzocco et al., 2011b), other numeracy skills (Geary et al., 2013), number knowledge (Purpura et al., 2013), linguistic skills (LeFevre et al., 2010, Purpura et al., 2011), and social-emotional skills (Jones et al., 2011, Rimm-Kaufman et al., 2009). These skills may interact with any one of a wide range of factors but particularly with instruction. For example, whereas we would not expect gender differences between boys and girls based on biological, linguistic, cognitive, or math-specific processes; it may be the case, that social-emotional attitudes and anxiety might indirectly contribute to gender differences (Beilock, Gunderson, Ramirez, & Levine, 2010). This may be one reason that gender differences do not emerge consistently on measures of the early and later numeracy and other basic mathematics skills (e.g., Geary et al., 2013, Halberda et al., 2008, Purpura et al., 2013) – the influence of gender on mathematics is indirect.

The key to our conceptual framework is that it is dynamic. That is, children bring a constellation of skills, aptitudes, attitudes, and beliefs to the process of learning mathematics and to the instructional environment. This means that what is effective mathematics instruction for one child might be less effective for a child with a different constellation of skills, aptitudes, and attitudes – and that personalizing instruction, taking into account potential child X instruction interactions, should contribute to stronger mathematics achievement for children overall. Hence in this study, in addition to a range of mathematics processes, we consider students' gender, beginning of grade vocabulary knowledge (a linguistic process), and school-level poverty (as measured by the percentage of children qualifying for the US National School Lunch Program (NSLP), a widely-used indicator of family poverty). It was beyond the scope of this study to specifically test the cognitive and social-emotional aspects of the model.

In the lattice model, the instructional environment is an important source of influence on students' mathematics achievement; we would expect that student achievement would be greater when these environments are aligned with students' educational needs. In light of individual differences, children with weaker or stronger mathematics skills may not receive instruction optimally aligned with their potential achievement levels – that is, there may be child-characteristic-by-instruction (CXI) interactions. Also called skill-by-treatment interactions (Burns, Codding, Boice, & Lukito, 2010), CXI interaction effects are recognized in reading (Connor et al., 2013) but have not been as widely or as explicitly tested in elementary mathematics although there are notable exceptions (Burns et al., 2010). For example, teacher-led mathematics instruction in first grade was generally more effective for students with difficulties in mathematics whereas child-centered instruction was more effective for students with more typical mathematics skills (Morgan, Farkas, & Maczuga, 2015). Plus, there is research on personalized mathematics instruction using technology; for example, cognitive tutor technology, which is individualized based on students' performance, has been effective (Corbett et al., 2001, VanLehn, 2011).

Other studies of early mathematics instructional content have found less evidence of instruction aligned to students' learning needs. For example, one study found that teachers spent disproportionate time on concepts that fell either above or below a students' current level of mathematical understanding. Specifically, Engel et al. (2013) found that the kindergarten teachers in their study reported devoting most of their mathematics instructional time to counting and shape identification; but most kindergartners already had this knowledge when they entered school. Not surprisingly, Engel and colleagues also found that the children with the lowest levels of math skills were the only ones to show gains in mathematics achievement, whereas most of the other children made greater mathematics achievement gains when they were exposed to mathematical content that went beyond this basic level. Focus on basic skills continues to consume instructional time in the early grades, at least in the United States (Li, Chi, DeBey, & Baroody, 2015).

Drawing broadly from the literature on personalized or individualized instruction, (Connor et al., 2007, Connor et al., 2011, Connor et al., 2011), an essential feature of individualized instruction is targeting instruction based on children's current performance on component skills (e.g., in reading: decoding and comprehension). A components-based approach should also work for early mathematics (e.g., Burns et al., 2010). As demonstrated in an early randomized controlled study on this approach, first graders with stronger incoming fall mathematics skills demonstrated greater mathematics gains when they were in randomly assigned classrooms where they were provided with supplementary peer-assisted learning opportunities (Fuchs et al., 1997). However, children with weaker initial skills made greater gains when they were in the control condition, which did not include peer-assisted learning opportunities (Mazzocco, Crowe, Calhoon, & Connor, in preparation). These CXI interactions are among the first to be explicitly documented for mathematics achievement (Clements et al., 2013, Morgan et al., 2015).

As noted previously, despite documented individual child differences in mathematics achievement, research shows that elementary mathematics instruction is rarely designed to accommodate these differences (Arnup, Cheree, John, & Louise, 2013). For example, many widely-used mathematics curricula (e.g., Saxon Math) are designed for whole-class implementation. At the same time, numerous advocates for the use of differentiated mathematics instruction recognize its effectiveness for children at various levels, from students with disabilities to those gifted in mathematics (Dee, 2010, Edwards, 2006). Despite such recommendations, many pre-service and practicing teachers are ill-prepared to differentiate instruction in their classrooms (Dee, 2010, Edwards, 2006). Given the reported absence of relevant coursework and training on children's mathematical thinking for pre-service elementary teachers (Ma, 1999), many in-service teachers appear to lack an understanding of the strategies and techniques that emphasize student development and learning that can be used to individualize instruction. Importantly, teachers' knowledge of children's mathematical learning is predictive of mathematics achievement gains among first and third graders (Hill, Rowan, & Ball, 2005). Hence, in order for an early mathematics instructional program to be effective, it should include professional development designed to support general education classroom teachers' gains in understanding children's mathematics trajectories (Turner et al., 2012).

The following two research questions guided the design and implementation of this study.

• To what extent are CXI interactions causally implicated in early mathematics achievement? That is, if we design an individualized mathematics program, ISI Math, using assessment to guide instruction, will individualized mathematics instruction be more effective than an alternative treatment that is not individualized?

In the present study, to begin to test whether CXI interactions were causally implicated in early mathematics achievement, we conducted the cluster-randomized controlled study described here. The purpose of this study was to examine whether 2nd grade mathematics instruction that takes into account individual student differences in mathematics component skills was more effective than more typical whole group mathematics instruction provided to students in the control ISI-Reading classrooms. To test whether individualizing math instruction was generally effective, we compared students' mathematics outcomes for students in treatment (ISI-Math) vs. control (ISI-Reading) classrooms hypothesizing that mathematics instruction individualized based on students' assessed skills would be more effective in improving student outcomes than instruction provided classroom-wide to all students, regardless of skills.

• To what extent does the ISI-Math intervention change the effect that child level gender, initial math skill, and vocabulary and school level poverty have on mathematical growth?

  The lattice model would support an association between mathematics and language (e.g., vocabulary). Thus, since we designed ISI-Math as an individualized assessment-based intervention, we hypothesized that finding CXI interactions would suggest that we had not succeeded in identifying important aspects of mathematics instruction that aligned with students' skills and abilities. Rather, other dimensions of instruction might be important to consider. Finding no CXI interactions would suggest we were on the right track in identifying child characteristics that interacted with mathematics instruction, and that our developmental model was supported. We included vocabulary skills as a proxy for the broader construct of linguistic skills, following the lattice model, and because, as noted previously, there is evidence of shared variance in language and mathematics skills. As noted previously, we also included gender X treatment interaction effects because there is some (albeit inconsistent) evidence that girls may have weaker math skills than do boys. Finally, we included the contextual characteristic of school-wide poverty levels. Again, if ISI-Math does consider child characteristics that impact learning, it should be effective regardless of student and school characteristics.