Recent Research in Science Teaching and Learning

SENSE OF BELONGING INFLUENCES PARTICIPATION

Penuel, W. R., Krumm, A. E., Pazera, C., Singleton, C., Allen, A. R., and Deverel-Rico, C. (2023). Belonging in science classrooms: Investigating their relation to students’ contributions and influence in knowledge building. Journal of Research in Science Teaching. Online Early View. https://doi.org/10.1002/tea.21884

One of the common challenges in active learning classrooms is encouraging students to share their ideas in front of the whole class or in small groups. In either case, speaking up is risky behavior − a student could answer wrong and feel dumb and/or their contribution could be ignored by their classmates (Nasir and Hand 2008; Clarke 2015). For students with marginalized identities, there can be an additional concern that what they say may be viewed as representing their group and risk confirming stereotypes (Steele et al., 2002). Penuel and colleagues hypothesize that sense of belonging (defined as feeling accepting, valued, and respected in a setting) can buffer students against these participation risks and increase class participation.

Penuel and colleagues utilized data from an ongoing study of 146 middle school classrooms across 10 U.S. states. In these classes, students engaged in an intervention called storylines where they collaborate with each other to build knowledge across a series of activities in each lesson. At the end of each course day students completed an “exit ticket” where they reflected on the day’s experience. Exit tickets had one yes/no question about whether they shared ideas with peers in class (participation), one question on whether they believed their participation influenced their peers’ thinking, and a three-item scale measuring belonging (Likert scale). Researchers also collected student demographic information to explore differences in outcomes across various student identities. Using this event sampling method, they collected 9725 complete exit tickets from 146 teachers’ classes. They sampled an average of 2.2 lessons per teacher and an average of ∼30 student exit tickets for each lesson.

Researchers used generalized linear mixed-effect models to assess the role of belonging on participation and to identify potential disparities in participation for particular student identities. Logistic regressions allowed them to model their binary outcome variable (participation). Mixed-effect models allowed them to account for the nested structure of their data (multiple exit tickets per lesson and multiple lessons per teacher) that could bias their results.

They explored four nested models to address their hypothesis. First, they ran a baseline regression model that examined participation predicted only by student race/ethnicity and gender. Next, they added an interaction term between race/ethnicity and gender to make room for the impact of intersectional identities on participation. In the third model, they tested their hypothesis that belonging influences participation by adding two belonging variables: 1) individual student sense of belonging, and 2) the average sense of belonging across the class. Each student’s sense of belonging was standardized such that the researchers were modeling how different a focal student’s belonging was from the average belonging in their class. Researchers were able to examine how a more positive or negative sense of belonging than the rest of the class influenced participation after accounting for variation in average sense of belonging by the teacher. In the final model, researchers added interaction terms between belonging and race/ethnicity as well as belonging and gender so they could explore how the relationship between belonging and participation varied based on these aspects of identity.

Results from the initial two models without belonging demonstrated disparities in participation by race/ethnicity and gender: Asian American, multiethnic, and White female students were more likely to talk in class than Black, Hispanic, American Indian, and White male students. These effects were slightly reduced but did not go away when belonging was added, which means differences in belonging do not fully account for these differences in participation. However, researchers did find that belonging influenced the probability of participation for everyone. Students whose sense of belonging was one SD above their peers had an 84% chance of participation in a given class day and students with a sense of belonging one SD below their peers had a 65% chance. No interactions between race/ethnicity, gender, and belonging were supported by the models, meaning belonging impacted the participation of all students in a similar way.

Researchers also used a similar set of four models to explore whether belonging was related to a student’s belief that other students were influenced by their participation. The sample used in this analysis was a subset of their total dataset, only including students who reported participating in class. The variation in this outcome explained by their models (i.e., R² values) was much smaller than for the participation models overall. However, they did find that belonging was positively related to whether students felt their contributions in class influenced their peers. In addition, Black students perceived their contributions to be more influential than Latinx students (who were the reference level for race/ethnicity in these models).

Overall researchers found a relationship generally between belonging and participation, though it was not a stronger predictor of participation for students with marginalized identities than their majority peers. They also observed that belonging varied from lesson to lesson within the same class, suggesting that building belonging is not just a first-day task for an instructor, but a continuous one. Thus, this study suggests that one route for increasing participation in class is structuring instruction to support students’ sense of belonging. One limitation of this work is that the measures were collected simultaneously (i.e., students answered about the predictor and outcome variables on the same exit ticket). This means causation cannot be assessed and must be argued for based on theory. The next study addresses this challenge with a novel statistical technique.

CLASSROOM RELATIONSHIPS ARE IMPORTANT FOR INSTRUCTORS TOO

Li, X., Bergin, C., and Olsen, A. A. (2022). Positive teacher−student relationships may lead to better teaching. Learning and Instruction, 80, 101581.

The education literature is teeming with studies on the importance of instructor−student relationships for student outcomes ranging from performance and persistence to affective outcomes like pleasure and confidence (Wubbels and Brekelmans 2005; Quin, 2017). Less has been explored about the influence of instructor−student relationships on instructors and their outcomes. In this study, Li and colleagues used archival data from a state-wide teacher evaluation program to test whether positive instructor−student relationships predict increased use of evidence-based teaching practices.

Researchers drew on a student questionnaire measuring their experiences of high-impact teaching practices used in 285 school districts in the U.S. state of Missouri across multiple years. The questionnaire also included a five-item scale measuring instructor−teacher relationships. Each school district chose up to six teaching practices from the Teacher Effectiveness Student Survey to measure out of a pool of 26 possible practices. This study focused on elements of the questionnaire related to four of these practices measured in 4th through 10th grades: 1) cognitive engagement, 2) problem-solving and critical thinking, 3) affective engagement, and 4) instructional monitoring. The cognitive engagement element (four-item scale) measures students’ active mental involvement during lessons that can be elicited by particular teacher practices during lessons. Problem solving and critical thinking (four-item scale) asks students about opportunities to engage in applying, analyzing, and evaluating during lessons. Affective engagement (five-item scale) measures how instructors helped students motivate to engage in lesson content. Finally, instructional monitoring (four items) measures how commonly teachers apply formative assessment to support student learning. Because different school districts may choose different practices to measure, the size of the sample varied from 217 classrooms to 733 classrooms across the four teaching practices.

Researchers ran four regressions with each teaching practice as an outcome and instructor−student relationship as the predictor as well as grade level and an interaction term between grade level and instructor−student relationship. Additional control variables such as a teacher’s years of experience, whether the course was a core subject area (such as English or math), school-level student demographics, and school-level achievement on state proficiency exams were included in the models.

Collecting teaching practices and instructor–student relationship at the same time created a challenge for researchers: they wanted to be able to describe both correlation and directionality of the relationship between these variables. To determine directionality, they employed direction dependence analysis, a family of multiple statistical tests that can be used in combination with regression to assess the likelihood that different directional relationships best fit the data (e.g., whether instructor–student relationships drive teaching practices or vice versa) or if a third, confounding, variable is involved. Because directional dependence analysis cannot currently be implemented with multilevel models, researchers accommodated their multilevel data structure by employing linear regression with cluster robust standards errors.

Using these techniques, researchers found that instructor–student relationships drove three teaching practices: 1) cognitive engagement, 2) problem-solving and critical thinking, and 3) instructional monitoring. However, affective engagement showed the opposite pattern: here teaching practice drove the student–teacher relationship. In addition, grade level interacted with several teaching practices to increase the influence of the relationship on teaching practices at higher grade levels.

Using directional dependence analysis and regressions, researchers were able to build a model that suggests teaching practices that support affective engagement influence the instructor–student relationship and that, in turn, the instructor–student relationship has beneficial impacts on teaching quality by increasing the use of more complex teaching practices that encourage cognitive engagement, critical thinking, and instructional monitoring. This research has implications for teacher professional development: it suggests supporting instructors to introducing changes that increase student affective engagement may be a high impact practice that could motivate instructors to change more aspects of their teaching as their relationships with students improve.

RUBRICS VERSUS EXEMPLARS: WHICH SUPPORTS STUDENT WRITING BETTER?

Lipnevich, A. A., Panadero, E., and Calistro, T. (2023). Unraveling the effects of rubrics and exemplars on student writing performance. Journal of Experimental Psychology: Applied, 29(1), 136–148.

Feedback is key for student learning, yet providing quality feedback to students is one of the most time-consuming tasks for instructors, especially on writing. One way to address the challenge of feedback is to involve students in generating it through assessment of peers or self-assessment. Tools for helping novices learn to provide quality feedback include rubrics (which provide the criteria on which a task will be assessed and what different performance levels look like on those criteria) and exemplars (i.e., examples of the task that illustrate successful or unsuccessful attempts). In this study, Lipnevich and colleagues explore which of these tools for self-assessment has the largest effect on student performance and how important training is for students to use these tools effectively.

Two-hundred high-school students at one school participated in this study by writing and revising two SAT-type essays over multiple weeks. On the first day, teachers introduced the task to their students and described the criteria on which they would be evaluated. Next, participants were asked to read a passage and then draft an in-class essay describing how the author built their argument using evidence from the passage. Participants were then randomly assigned to one of four treatments: 1) rubrics, 2) exemplars, 3) both rubrics and exemplars, and 4) control. In the rubric condition, they were given three rubrics used to score writing tasks on the SAT. Each rubric scores a different aspect of the assignment: 1) reading (understanding the passage), 2) writing (organization and precision of essay), and 3) analysis (how well students can characterize author’s use of evidence and reasoning). In the exemplar condition, students were given three exemplars demonstrating different levels of proficiency which also came from the SAT website. In the combined condition, students received both exemplars and rubrics. Finally, in the control condition students received instructions to reread the prompt and their essay and revise their draft. Participants were directed to use these materials in the next class session to revise their essay draft.

For the first essay assignment, participants received no training with their assigned rubrics and/or exemplars. One week after submitting their first revised draft, participants received training on how to use their assigned tools. The control group received instruction in study strategies. Then they were given a second essay assignment, writing a first draft in class and using their assigned tools to revise their essay the next class day. Two teachers graded all four student drafts using the SAT rubrics. These teachers had served as scorers for SAT exams before so were well-trained and calibrated on the use of these rubrics.

Researchers found that before receiving any treatment (i.e., on the first draft of the first essay) there were no differences in essay performance between students in the different conditions. Differences in performance did appear after the first revision: students using any tool for self-assessment outperformed the control condition on the reading and writing outcomes, but students using rubrics outperformed everyone on these measures. Students in the rubrics as well as the exemplars condition performed the same on the analysis outcome. This implied that even without training students were able to engage in meaningful self-assessment of their own writing and this was particularly true for students using rubrics alone.

After the training session, students using exemplars had greater gains in performance such that their scores on reading caught up to that of the students using rubrics alone. However, students in the rubric alone condition still outperformed them on writing. Thus, it seems training was particularly beneficial when asking students to use exemplars to self-evaluate their work.

This study suggests that students are able to give themselves meaningful self-feedback on writing if they are provided tools to scaffold that feedback and they may do an even better job at it if they receive training on use of the tools (especially if exemplars are used). Interestingly, receiving both rubrics and exemplars did not help students nearly as much as receiving just one of these. Researchers suggest that the 50-min time limit to revise their essays may have influenced this. Students who had to look at both rubrics and exemplars in that time may not have been able to deeply engage with either. Alternatively, being asked to engage with both could have led to too much cognitive load, reducing students’ capacity to learn.

One limitation of this study was that there was not a teacher feedback condition to evaluate how well self-assessment supports revision relative to teacher assessment. This would be an interesting follow-up study. Whatever the outcome, the current study demonstrates that students can provide meaningful and actionable feedback on writing which supports the use of writing in large classes where instructor feedback may be nearly impossible.

REFERENCES