Examining the effects of adult and peer mediated goal setting and feedback interventions for writing: Two studies
Introduction
With the distribution of information from the National Commission on Writing (2003) and writing assessment results from the National Assessment of Educational Progress, 2002, National Assessment of Educational Progress, 2012, educators are beginning to uncover the significant gaps in students' writing skills that currently exist in today's schools (Graham, Harris, & Herbert, 2011). For example, as many as 70% of students are not considered proficient in their writing (NAEP, 2012). Writing is a complex and intentional process used as a tool to communicate ideas, develop and support an argument, and is an indicator of content knowledge (Graham et al., 2007, Rowland, 2014). Writing difficulties have been associated with greater risk for high school and college dropout, lower academic performance in other curricular areas, and has been noted by many employers as a critical skill in post-secondary success (Graham, 2006, Graham and Perin, 2007a, Graham et al., 2007). Considering the negative outcomes associated with writing deficits and the fact that early writing difficulties have been linked with writing difficulties in subsequent grade levels, it is important to determine strategies to remediate writing deficits early (Juel, 1988). The purpose of this two-part study was to investigate the utility of two practical and efficient interventions in writing that could be used to help improve students' writing in the later elementary and early middle school years.
Section snippets
Writing intervention and assessment
Intervention efforts in schools tend to focus on subjects such as reading and mathematics, which are subject areas that have gained more attention on state and national accountability tests (Cutler and Graham, 2008, Malecki, 2014, National Commission on Writing in America’s Schools and Col). However, with the adoption of the Common Core State Standards, more states are beginning to assess their students' writing skills; therefore, interventions designed to improve writing may begin to compete
Curriculum Based Measurement in Written Expression
One effective tool that can address the issues of complexity and subjectivity in the assessment and monitoring of writing is Curriculum Based Measurement in Written Expression (CBM-WE). CBM-WE produces scores shown to be reliable and valid indicators of general writing achievement that are sensitive to change over time and have been found to increase student performance when used as a progress monitoring tool (Deno et al., 2001, Fuchs and Fuchs, 1986). CBM-WE is a General Outcome Measure (GOM)
Goal setting and feedback intervention strategies
One practical and effective set of intervention strategies often used to improve students' academic performance across subject areas includes goal setting, feedback, and self-monitoring (Graham, Harris, & Troia, 1998). Setting ambitious content-area goals and providing specific feedback regarding individual progress towards these established targets could be a powerful intervention approach for educators to utilize with their students. However, to utilize this approach, feedback on progress and
Self-regulated learning and goal setting/feedback interventions
Goal setting and performance feedback interventions are considered to be effective tools to promote learning across subject areas in that they promote self-regulation and self-monitoring of performance. Self-regulated learning is a defined as “…the degree to which students are meta-cognitively, motivationally, and behaviorally active participants in their own learning process” (Zimmerman, 2008, p.167). It is a self-directed process that focuses on using specific learning strategies, actively
Research on self-regulated learning
Several studies have examined the effectiveness of strategies that focus on Self-Regulated Strategy Development (SRSD) as a writing intervention tool (Graham & Harris, 1997). In a recent meta-analysis on writing instruction strategies, writing instruction that focused on SRSD techniques, including individual self-assessment (self-evaluation) and goal setting components, had a significantly larger effect size than studies that used a non-SRSD approach (ES = 1.14 and ES = .69, respectively) (Graham &
Goal setting and performance feedback interventions
The use of goal setting and performance feedback interventions has been effective in improving student performance in academic areas such as reading, mathematics, and spelling (Codding et al., 2005, Conte and Hintze, 2000, Figarola et al., 2008, Fuchs et al., 1989, Fuchs et al., 1991, Graham and Perin, 2007a, Harris et al., 1994). Conte and Hintze (2000) examined the effects of performance feedback and goal setting on CBM oral reading fluency and found that students who were given feedback
Peer-mediated interventions
The overwhelming majority of performance feedback studies have been teacher led and rarely have researchers examined the utility of self-monitoring interventions when feedback is given by peers in the classroom. However, researchers suggest that feedback or progress towards attaining a goal can be provided by teachers, peers, or through self-monitoring (Graham and Harris, 2005, Harris et al., 2008). Students are an abundant resource that can be effectively utilized to construct practical
Peer-mediated interventions in writing
A meta-analysis of 123 studies examining effective writing instruction techniques found that collaborative writing, or writing as a team, had an average effect size of .75, suggesting that working with peers on writing tasks may serve as an effective tool to improve students' overall writing abilities (Graham & Perin, 2007b). In another small study of first graders (n = 20), students were able to respond to teacher and peer feedback (evidenced by their ability to revise their writing in response
Goals of the current studies
Intervention strategies that focus on performance feedback techniques have been found to enhance students' writing by allowing for frequent feedback on writing performance, teaching students how to evaluate their own writing, and monitoring their own progress in the skill over time. However, it is important to extend the literature on performance feedback and goal setting interventions and consider whether involving students in the entire intervention process (both receiving and delivering the
Participants
Upon approval by the University Institutional Review Board, participants were recruited from one Midwestern elementary school. Participants included 114 fourth grade students in four classrooms from one participating suburban elementary school. Of the 114 students, 56 (49%) were male, 58 (51%) were female, 103 (90%) were in general education classrooms, and 11 (10%) received special education services. Demographic information regarding student gender and special education status was collected
Analysis strategy
Means and standard deviations for all dependent variables at baseline and the last time point are presented in Table 1. Preliminary analyses confirmed the multilevel nature of the student data. Student scores across time showed strong and significant intraclass correlations (see Table 2). Accordingly, a two-level multilevel model was specified for each outcome variable to examine the effectiveness of the intervention (Level 2) on student scores across time (Level 1). Following steps outlined by
Discussion: Study 1
At the end of the ten-week intervention time period, students in the teacher-provided performance feedback condition produced significantly more TWW, WSC, and CWS than students in the control group. In addition to these significant differences between groups post-intervention, students in the experimental group also made significantly greater gains in their rate of improvement at each time point for all three production dependent indices, suggesting that providing students with goals and
Study 2
Results from the first study support the use of feedback and goal setting as an intervention to help promote students writing ability on curriculum based measures of written expression. The purpose of Study 2 was to extend these findings by examining the effectiveness of a similar intervention strategy when the intervention was delivered by classroom peers rather than the classroom teacher. One of the main criticisms of class-wide interventions is the amount of time and teacher investment
Analysis strategy
Means and standard deviations for production-dependent variables at baseline and the final time point are provided in Table 3. Similar to Study 1, multilevel models were tested with random intercepts, random slopes, and controlling for student gender and baseline scores. Different from Study 1, time points were nested within teacher and week such that students from three different classrooms provided scores at two time points per week across eight weeks. Intraclass correlations were computed
Discussion: Study 2
Students receiving the peer-mediated intervention in Study 2 demonstrated greater post-intervention performance on all three production dependent indices (TWW, WSC and CWS), suggesting that knowledge of goals and receiving feedback on performance from peers led to greater performance on measures of writing fluency. In addition to these post-intervention differences, students in the intervention group also improved their performance on the TWW index and CSW index at a significantly faster rate
General discussion
At the end of the intervention time frame, the students who received the performance feedback and goal setting intervention from teachers in Study 1 and peers in Study 2 produced significantly more TWW, WSC, and CWS than students in the control groups. Results indicated that both interventions that provided students with performance feedback and goals positively affected student performance on the fluency measures compared to control groups. These findings are consistent with prior research on
Conclusions
The current study investigated the effects of goal setting and performance feedback on production-dependent CBM indices of written expression. Students receiving the performance feedback and goal setting intervention from classroom teachers and from peers outperformed students in the control group on TWW, CSW, and CWS post-intervention, suggesting that goal setting and performance feedback are effective intervention tools.
These findings are extremely important in the area of CBM written
References (99)
- et al.
Self-regulation and writing: Where do we go from here?
Contemporary Educational Psychology
(1997) A practical guide to multi-level modeling
The Journal of School Psychology
(2010)- et al.
Intrinsic and extrinsic motivations: Classical definitions and new directions
Contemporary Educational Psychology
(2000) - et al.
Goals and progress feedback: Effects of self-efficacy and writing achievement
Contemporary Educational Psychology
(1993) Research for the future: Research on cooperative learning and achievement: what we know, what we need to know
Contemporary Educational Psychology
(1996)- et al.
Evaluating the impact of feedback on elementary aged students' fluency growth in written expression: A randomized controlled trial
The Journal of School Psychology
(2014) - et al.
Becoming a self-regulated writer: A social cognitive perspective
Contemporary Educational Psychology
(1997) - et al.
The predictive validity of CBM writing indices for eighth-grade students
The Journal of Special Education
(2011) - et al.
The contributions of educational psychology to school psychology
- et al.
Probing interactions in fixed and multilevel regression: Inferential and graphical techniques
Multivariate Behavioral Research
(2005)
Language development and language disorders
Best practices in evaluating the effectiveness of interventions using case study data
Effects of a peer-mediated phonological skill and reading comprehension program for middle school students with reading disabilities
Journal of Learning Disabilities
The effects of peer-assisted learning strategies and curriculum-based measurement on the mathematics performance of secondary students with disabilities
Remedial and Special Education
Best practices in fostering student engagement
Examining the efficacy of performance feedback and goal setting interventions in children with ADHD: A comparison of two methods of goal setting
Journal of Evidence-Based Practices for Schools
CBM reading, mathematics, and written expression at the secondary level: Examining latent composite relations among indices and unique predictions with a state achievement test
Journal of Educational Psychology
Statistical power analysis for the behavioral sciences
Curriculum-based measurement of writing in kindergarten and first grade: An investigation of production and qualitative scores
Exceptional Children
The effects of performance feedback and goal setting on oral reading fluency within curriculum-based measurement
Diagnostique
Primary grade writing instruction: A national survey
Journal of Educational Psychology
Developments in curriculum based measurement
The Journal of Special Education
Using curriculum based measurement to establish growth standards for students with learning disabilities
School Psychology Review
Relationships among simple measures of written expression and performance on standardized achievement tests (rep. no. 22)
Behavioral conceptions and applications of acceptability: Issues related to service delivery and research methodology
School Psychology Quarterly
The relationship between curriculum based measurement in written expression and quality and completeness of expository writing for middle school students
The Journal of Special Education
Identifying indicators of written expression proficiency for middle school students
The Journal of Special Education
The effects of peer-assisted learning strategies on the beginning reading skills of young children with emotional or behavioral disorders
Behavioral Disorders
Effects of reciprocal peer tutoring on academic achievement and psychological adjustment: A component analysis
Journal of Educational Psychology
Effects of self-graphing and goal setting on the math fact fluency of students with disabilities
Behavioral Analysis in Practice
Assessing the social validity of behavioral interventions: A review of treatment acceptability measures
School Psychology Quarterly
The past, present and future of curriculum based measurement research
School Psychology Review
Effects of systematic formative evaluation: A meta-analysis
Exceptional Children
Effects of ongoing curriculum-based measurement on student awareness of goals and progress
Education and Treatment of Children
Effects of goal line feedback on level, slope, and stability of performance within curriculum-based measurement
Learning Disabilities Research and Practice
Enhancing kindergartners' mathematical development: Effects of peer-assisted learning strategies
The Elementary School Journal
Effects of peer-assisted learning strategies on high school students with serious reading problems
Remedial and Special Education
Peer-assisted learning strategies: Making classrooms more responsive to diversity
American Educational Research Journal
Acquisition and transfer effects of classwide peer-assisted learning strategies in mathematics for students with varying learning histories
School Psychology Review
Moving beyond total words written: The reliability, criterion validity, and time cost of alternative measures for curriculum-based measurement in writing
School Psychology Review
Effects of two types of self-regulatory instruction programs on students with learning disabilities in writing products, processes and self-efficacy
Learning Disability Quarterly
Strategy instruction and the teaching of writing: A meta-analysis
Missing data analysis: Making it work in the real world
Annual Review of Psychology
Improving the writing performance of young struggling writers
The Journal of Special Education
Writing next: Effective strategies to improve writing of adolescents in middle and high schools — A report to Carnegie Corporation of New York
A meta-analysis of writing instruction for adolescent students
Journal of Educational Psychology
Informing writing: The benefits of formative assessment. A Carnegie Corporation time to act report
Writing and self-regulation: Cases from the self-regulated strategy development model
Best practices in writing instruction
Cited by (10)
Understanding and supporting the use of feedback from mobile applications in the learning of vocabulary among young adolescent learners
2023, Studies in Educational EvaluationExploring self-regulated learning through differentiated feedback
2023, Cogent Education