Peer-Modeled Mindsets: An Approach to Customizing Life Sciences Studying Interventions

INTRODUCTION

Despite a growing demand for professionals with skills in the science, technology, engineering, and mathematics (STEM) fields (National Science Board, 2014), many undergraduate STEM majors drop out or change majors before completing college (see Seymour and Hunter, 2019). This attrition, which is especially pronounced among underrepresented minority and first-generation college students (Radford et al., 2010; Shaw and Barbuti, 2010; National Science Board, 2014; Riegle-Crumb et al., 2019; Seymour and Hunter, 2019), results in a national workforce that is weaker and less diverse than it otherwise could be. This challenging problem demands innovative solutions, as reflected by persistent calls from policy makers, researchers, and funding agencies to find ways to improve college students’ experiences and promote diversity in STEM (e.g., President’s Council of Advisors on Science and Technology, 2012; Asai et al., 2022). “Mindset interventions,” which shift students’ beliefs about themselves as learners and about their classroom experiences (e.g., growth mindset, belonging mindset, interest or relevance mindsets), are a promising solution, because they are brief, scalable, and can improve undergraduate students’ academic outcomes (see Tibbetts et al., 2016; Harackiewicz and Priniski, 2018; Walton and Wilson, 2018; Richardson et al., 2020). Mindset interventions often take the form of short, self-administered exercises that are carefully crafted by social psychologists to precisely target and address students’ most pronounced fears (Yeager and Walton, 2011; Tough, 2014).

Mindset interventions have been incorporated into undergraduate courses, including biology (e.g., Harackiewicz et al., 2016; Hecht et al., 2019a), statistics and developmental math (e.g., Bryk et al., 2015), and physics (e.g., Miyake et al., 2010), with increasing regularity over the last decade or so. Nevertheless, the full potential of mindset interventions is not yet known. In particular, psychologists have emphasized the importance of customization for mindset interventions to be maximally effective (e.g., Yeager and Walton, 2011; Harackiewicz and Priniski, 2018; Murphy et al., 2020). Indeed, the specific challenges students face and resources available to them vary greatly between different courses and institutions, making it unlikely that an established intervention will fit perfectly into any given context without modification (see Yeager and Walton, 2011). Therefore, there is a pressing need for a method to guide the process of developing customized mindset interventions for distinct college contexts.

In the present research, we developed a protocol to guide this customization process. The end result of the protocol is a “peer-modeled” mindset intervention, so called because it uses a targeted series of questions, rooted in social-psychological theory, to elicit advice from former students. This allows peers to model (or demonstrate) the changes in thinking that helped them to be successful in a specific course. Therefore, this type of intervention merges the psychologically precise and targeted approach of mindset interventions (see Walton, 2014) with the existing, context-sensitive approach of providing advice from former students to current students (see Lane et al., 2021).

To develop and test the protocol, we formed a research–practice partnership team of three biology instructors (A.G.L., R.E.B., D.R.H.) and two social psychologists (C.A.H., D.S.Y.). We developed a peer-modeled mindset intervention that was customized to our (A.G.L., R.E.B., D.R.H.) sections of Introductory Biology I. We then evaluated this intervention using a well-powered randomized controlled trial (RCT). We showed that the peer-modeled mindset intervention significantly impacted students’ approaches to learning, feelings of belonging in the course, and academic outcomes. Consistent with past mindset interventions, the most pronounced benefits appeared for students who have historically been underrepresented in STEM fields (see Yeager and Dweck, 2020; Hecht et al., 2021), highlighting the potential of our approach to broaden inclusion and participation in STEM. We include all tools and protocols in the Supplemental Material to support future researchers and instructors who want to do the same.

This study makes two primary contributions to research on undergraduate biology education. First, it offers an innovative and validated way to promote more effective approaches to learning among undergraduate students in biology classrooms, which has long been identified as an important goal by researchers and educators in the life sciences (e.g., Sebesta and Bray Speth, 2017). Second, this work provides a needed approach for developing customized mindset interventions that has the benefits of a structured protocol, but also draws centrally on the insights of former students and thus is inherently flexible in reflecting the distinct resources, needs, and demands in a particular life sciences course.

Background

The peer-modeled mindset intervention is rooted in a social-psychological analysis of disparities in STEM degree attainment that are accelerated by large “gateway” courses. It is well known that many students leave STEM fields, including the life sciences, soon after taking their first few large, introductory college science courses (Koch, 2017; Seymour and Hunter, 2019). Barriers to succeeding in these challenging courses are often structural (Shukla et al., 2022). For example, students from lower-income families often work one or more jobs to meet tuition and living expenses (Warburton et al., 2001; Phinney and Haas, 2003), leaving them less time to dedicate to course work (Pascarella et al., 2004). There are also barriers related to students’ mindsets: their assumptions, beliefs, or perspectives that shape how they interpret and respond to the academic environment (see Walton and Wilson, 2018; Hecht et al., 2021).

Students’ mindsets can be the accumulated result of prior experiences with discrimination or underrepresentation, and they can also be proximal, psychological causes of ongoing underperformance (see Ross and Nisbett, 1991; Steele, 1997; Cohen and Sherman, 2014; Hecht et al., 2021). For example, when students perform poorly at the outset of a gateway science course, they often form the belief that they simply do not belong in the sciences or are incapable of being highly successful in these fields. These doubts, in turn, can lead them to prematurely leave STEM for a different field of study (Ost, 2010; Seymour and Hunter, 2019; Rosenzweig et al., 2021).

This view of struggle is cyclical and pernicious (see Yeager et al., 2016). It is also possible to address because students are apt to misperceive the causes of their struggles and failures, through a process called “misattribution” (Weiner and Sierad, 1975). For most undergraduate science students, struggle and failure are temporary, and often simply reflect an overreliance on surface-level study strategies learned in high school, such as rote memorization, rather than the higher-order thinking skills and study strategies (e.g., mapping complex processes, comparing and contrasting related concepts) that are needed to master college-level science concepts (Karpicke et al., 2009; Cook et al., 2013). Indeed, previous research has found that students perform better in introductory biology courses when they are more metacognitively aware and tend to self-regulate their learning, intentionally assessing their progress and reflecting on which strategies have been most effective for them (Sebesta and Bray Speth, 2017; Osterhage et al., 2019). When students do not do this—when they rely too heavily on the surface-level strategies that may have helped them succeed in high school—they may perform poorly in these courses and mistakenly doubt their potential in the field. This prevents them from realizing that they could succeed by developing new and more appropriate approaches to studying.

Mindset interventions have been proposed as a way to stop or reverse this pernicious cycle of worry, failure, and disengagement. These interventions focus on the meaning of ongoing experiences of struggle (Wilson and Linville, 1985) rather than the tool kit of study strategies (although these may be complementary; see Hattie et al., 1996; Dignath and Büttner, 2008). The mindset approach rests on the hypothesis that, if students view initial struggle in an introductory course as an indicator that they lack potential to be successful in the field, then they may not adopt and benefit from the effective study strategies that instructors provide. Meanwhile, students may experience less doubt about their potential and become more strategic about their learning if they attribute struggling in a course to the need to develop better study strategies (see also Chen et al., 2017).

Our approach built on the insights of previous mindset interventions, and in particular, attributional retraining interventions. Attributional retraining changes how students explain the causes of struggles in college, thereby moving students away from fixed, uncontrollable explanations such as a lack of ability (see Perry et al., 1993; Haynes et al., 2009). We aimed to develop an attributional retraining approach that was customizable to the needs of each life sciences classroom. We hypothesized that normalizing challenge and shifting students toward the belief that struggle indicates a need to study differently (rather than a lack of potential to succeed) would increase students’ uptake of effective learning strategies, alleviate their doubts about belonging in science, and improve their academic outcomes (e.g., performance, persistence). Although attributional retraining interventions have been tailored for specific science courses in some past research (e.g., Okolo, 1992; Ziegler and Heller, 2000), the process for doing this was not repeatable.

Using Peer Modeling to Tailor Mindset Interventions.

The peer-modeled approach to intervention development is grounded in social cognitive theories of learning (see Bandura, 1986), which posit that people can learn from observing others’ behaviors and from witnessing the consequences of those behaviors. We extend this principle to suggest that individuals can similarly learn about others’ mental states (e.g., their mindsets) and change their own mental states accordingly. Peer modeling may provide an effective approach to designing context-customized mindset interventions for two reasons. First and foremost, the approach lends itself naturally to customization. That is, former students can 1) help to identify and articulate the changes in beliefs or perspectives that are most relevant to success in a particular course and 2) provide details about specific challenges in the course (e.g., particularly difficult exams or concepts) and specific resources in the context that are useful for overcoming those challenges (e.g., structured review sessions, practice problems). Mindset interventions that do not draw on former students’ experiences in this way may not have access to such context-specific information. Second, drawing on peers’ experiences may have additional social-psychological benefits for persuasion and behavior change (see Walton and Wilson, 2018; Wilson, 2011). For example, former students may serve as particularly credible sources of information for current students (see Hovland and Weiss, 1951; Petty and Cacioppo, 1986; Cialdini and Goldstein, 2004) and may also communicate helpful social norms about struggling and improving in the course (see Sherif, 1936; Cialdini and Goldstein, 2004; McDonald and Crandall, 2015; Tankard and Paluck, 2016).

The Present Research

Here we 1) developed a protocol for creating customized peer-modeled mindset interventions in the context of a research–practice partnership and 2) used this protocol to develop a customized intervention for Introductory Biology I at a large, public university in a southern state in the United States. The targeted biology course was the first of a two-course sequence, and it is typically taken by life sciences students in their first semester of college.

Importantly, the course was already rich with instructional supports to help students improve their approach to studying, such as guides, reflection activities, and discussion sections. We (A.G.L., R.E.B., D.R.H.) had furthermore built existing supports into the course for students’ sense of belonging, such as personal stories about coming to find a sense of belonging in biology over time. We also had supports for students’ focus on learning (rather than performance) goals, such as “second-chance” opportunities to receive additional points on exams and assignments. Mindset interventions tend to be more effective in supportive learning environments like these, because these environments allow changes to students’ beliefs and attitudes to be readily translated into impactful changes in behavior (see Walton and Yeager, 2020; Bryan et al., 2021; Hecht et al., 2021). When such supports and opportunities are unavailable, changes in beliefs or attitudes may do little to affect students’ outcomes. Thus, our course was a useful first setting to evaluate peer-modeled mindset interventions.

Our collaborative research comprised two phases: a design phase and an evaluation phase. During the design phase, we created the intervention development protocol and used it to devise a peer-modeled mindset intervention that was customized to our introductory biology course. The intervention consisted of three ∼15-minute online activities that focused on the experiences of former students (presented as interview video clips).

In the evaluation phase, we used an RCT to test the intervention with new students in the same introductory biology course. From a causal inference perspective, RCTs are the preferred method for testing the effects of mindset interventions because they address shortcomings of observational research and pre–post designs (e.g., that correlation does not imply causation; Schneider et al., 2007; Campbell and Stanley, 1963). Importantly, we randomized students to condition within course sections, which means that half of the students in each section received the intervention and half received a control activity. Although this raised the possibility that benefits could spill over from the treatment group to the control group within classes, which would reduce estimated treatment effects, the within-class randomization afforded greater statistical power. In summary, effects were expected to be smaller than what could be achieved with a more comprehensive, class-wide version of the intervention, but statistical inferences were expected to be more precise.

METHODS

Design Phase

Our collaborative team of biology educators and social psychologists created a protocol for developing a customized peer-modeled mindset intervention that consisted of four discrete stages (summarized in Figure 1).

Stage 1: Draw on Prior Experiences Teaching the Class.

The first stage involved reflecting on our (A.G.L., R.E.B., D.R.H.) experiences with former students to try to identify 1) potential mindsets (i.e., beliefs, assumptions, or perspectives) students might hold that may harm their experiences and performance in the course, 2) positive changes to these mindsets that can help students adapt to the course, and 3) a set of former students who may have experienced this change in mindset.

By sharing insights between the social psychologists and biology instructors on the research team, we identified the mindset described earlier as a potential contributor to students’ struggle to adapt to introductory biology. Many students struggled early in the course (e.g., failing the first exam), even if they had been successful in high school biology, and, attributing this struggle to a lack of ability or potential, came to doubt whether they belonged in the biological sciences. We hypothesized that a key change that enabled some students to overcome this barrier was coming to understand struggle as resulting from an overreliance on surface-level strategies that were effective in high school but not college biology (e.g., rote memorization), and to see developing a better set of strategies as the path to success. Note that, although we identified attributions as a high-leverage mindset to target in this particular context, other mindsets (e.g., beliefs about social belonging, perceptions of relevance; see Walton and Cohen, 2007; Hulleman and Harackiewicz, 2009) might be more operative in other contexts and, in such cases, could similarly be targeted by a peer-modeled mindset intervention.

Finally, we (A.G.L., R.E.B., D.R.H.) identified a set of former students who had overcome struggle in the course and may have undergone the hypothesized change in mindset. Our assessment was based on both an inspection of our gradebooks and our recollection of informal conversations with students (e.g., during office hours). We identified most students based on their grades, targeting students who had failed (or done very poorly on) the first exam but ultimately earned an “A” or an “A−” in the course. We also identified some students with whom we had formed relationships (e.g., students who went on to be course assistants). In identifying potential interviewees, we sought a diverse group of students (in terms of gender and race/ethnicity), ensuring that the most vulnerable groups of students (e.g., those historically underrepresented in the field) were well represented. We contacted several of these former students, and eight arranged times to be interviewed and recorded by the research team.

Stage 2: Prepare to Interview Former Students.

In the second stage, we (C.A.H., D.S.Y.) prepared to interview these former students. Our strategy to evoke relevant stories from students that would model the targeted change in mindset was to develop a prototype of an intervention that described this change. We planned to have the former students complete this intervention prototype and then tell us what did (or did not) match their own experiences in the course. We drafted a prototype describing 1) that many previous students struggled in the course, even if they had been successful in high school biology; 2) that they often interpreted this struggle as a sign that they might not be cut out for a career in the biological sciences; and 3) that many of these students found better ways to learn the material and ended up being successful in the course. The intervention prototype also included quotations that were written by the research team based on conversations with previous students. These quotations were intended to vividly convey students’ experiences of overcoming struggle. Finally, the prototype included a short writing activity to facilitate reflection on the intervention messages. The intervention prototype from the present study, which can be adapted to other settings, is included in the Supplemental Material.

When the intervention prototype was finalized, we scheduled recorded videoconferences with small groups of two to three former students each. We chose to schedule multiple small focus groups (targeting no more than five students) rather than a single, large focus group so that 1) individual students would have more time to talk about their experiences and 2) insights from earlier focus group sessions could be used to inform subsequent ones. Former students were offered $40 for their participation in the focus groups, informed that their participation could help improve the course for future students, and asked to sign a release allowing us to use video footage of the interviews.

Stage 3: Conduct Recorded Interviews with Former Students.

In the third stage, one of the social psychologists (C.A.H.) conducted three ∼90-minute recorded Zoom interviews with these small groups of former students. Following our interview protocol (included in the Supplemental Material), we took steps to create an environment in which students would feel comfortable being open and authentic while sharing their experiences. Note that, although instructors may be able to conduct interviews themselves, it may be better if they are conducted by a “neutral third party” so that students do not feel pressured to censor descriptions of their experiences in the course.

The interviewees spent ∼15 minutes going through the intervention prototype, off camera. They were told that this activity was only a draft that was intended to prime their thinking for the rest of the conversation. As noted, when they finished, students were asked what had resonated with them and what experiences were missing from the prototype activity. This prompted students to tell stories about their own initial struggles in the course and their experiences overcoming them. As the students relayed their experiences, the interviewer listened and probed to encourage students’ elaboration on 1) their experiences of struggle and belief that this indicated a lack of capacity to succeed in biology; 2) the shift in their thinking about struggle over the course of the semester, coming to understand the need to improve their approaches to studying; and 3) their eventual adaptation to the demands of the course and experiences of success. These probes typically involved asking students to elaborate, to provide specific stories, and to “riff” on one another’s stories (e.g., acknowledging when a student had been nodding along with another student’s story and asking if that second student had experienced something similar). In each of the three interviews, this one question with subsequent probing proved to be the only prompting necessary to facilitate the entire focus group conversation.

At the end of each interview, we generated content for the control group. We did so by asking several questions related to other topics (i.e., specific study strategies, developing an interest in biology, and adjusting to college), which allowed us to control for any effects of simply hearing from the same group of former students.

Note that, after interviews, researchers and instructors may realize that they had not identified the most relevant harmful mindset(s) and/or change in mindset(s) among their students. Although this did not happen in the present study, this situation might necessitate a return to stage 1, equipped with this new understanding of students’ psychological journey through the course (see Figure 1).

Stage 4: Develop Intervention Activities.

In the fourth stage, we chose clips from these interviews that vividly and impactfully communicated messages 1–3 listed earlier and then used these clips to create three ∼15-minute intervention activities to be completed by students throughout the semester. Each activity presented some background on the interviews, summarized the core themes, and then showed students a video (5–7 minutes for each activity) that assembled relevant interview clips to provide a clear narrative that conveyed the intervention message about reattributing struggle. The activities then included a short writing exercise that asked students to describe 1) why it is common for students to struggle in the course and 2) how and why students overcome these struggles as they find better ways to learn the material (i.e., the metacognitive process of testing and evaluating different study strategies; see Bakracevic Vukman and Licardo, 2010). The writing exercises were intended to promote internalization of the intervention message through the “saying-is-believing” effect (Aronson et al., 2002), in which individuals come to more strongly endorse a message for which they have freely advocated. The intervention activities from the present study are included in the Supplemental Material.

The intervention activities were designed to be given at strategic time points in the term. These were times that came before or after expected points of vulnerability in the semester, as identified by our (A.G.L., R.E.B., D.R.H.) experiences and the interviews with former students. These time points were: 1) before the first exam, to prepare students for the possibility of initial struggle; 2) soon after the first exam, to help students reattribute challenges they encountered on the first exam; and 3) before the final exam, which was cumulative and would require students to integrate the many concepts they learned throughout the term. Each activity emphasized the core intervention message but varied somewhat to remain relevant to the point in the term at which it was shown.

Note that, while developing the intervention activities, researchers and instructors may realize that the interviewees did not elaborate sufficiently on a specific theme. In this case, it might be necessary to return to stage 3 and conduct additional interviews that guide students to speak about the targeted topic.

Evaluation Phase

Participants.

We evaluated this intervention in a double-blind RCT conducted in our (A.G.L., R.E.B., D.R.H.) three sections of introductory biology (N = 917). This study was conducted in the Fall of 2021, the first semester in which students had returned to in-person classes at this university after the onset of the COVID-19 pandemic. Across these three sections, 70% of students were women, 36% were from racial/ethnic groups that are underrepresented in the biomedical fields (i.e., Black, Hispanic/Latinx, Native American/Alaska Native, or Native Hawaiian/Pacific Islander), and 29% were first-generation college students (i.e., neither parent held a 4-year college degree).

Procedure.

The study procedure is summarized in Figure 2. Students were randomly assigned to either an intervention or a control condition within each course section. Before completing any experimental activities, students completed a short baseline survey (for a small amount of course credit) assessing their attributions of struggle to strategy usage, reliance on memorization in high school biology, feelings of belonging in the course, uncertainty about belonging in the biological and health sciences, and confidence about performing well in the course (see the Supplemental Material for items and scale reliabilities). In both conditions, students completed the three experimental activities over the course of the semester, which were included in the course as graded assignments for a small amount of completion credit.

In the intervention condition, students were shown the three intervention activities described earlier. In the control condition, students were shown three activities that were similar to the intervention activities in terms of surface features (each including background on the student interviews, a 5- to 7-minute video featuring interview clips from the same group of former students, and a brief writing exercise). However, the control activities did not include the critical theme of reattributing struggle in the course to developing study strategies, and instead focused on related topics about succeeding in and beyond the course. Notably, the first control activity focused on study strategies that helped former students to learn the material. The control condition was therefore conservative, ruling out exposure to improved study strategies (rather than reattributing struggle) as a mechanism of intervention effects. The second control activity focused on how students’ academic interests develop over time, and the third control activity focused on how students adjust to college life.

After the third experimental activity and before the final exam, students completed a survey (for a small amount of course credit) that measured their attributions for struggle, approach to studying in the course, and feelings of belonging. After the term, we (A.G.L., R.E.B., D.R.H.) provided students’ letter grades for analysis, as well as records of whether students took Introductory Biology II (i.e., the second course in the introductory biology sequence) the following semester.

Measures.

The end-of-semester survey was completed by approximately three-quarters of the sample (missingness = 26–27%, depending on the outcome) and there was no differential attrition for any of the outcomes (see the Supplemental Material). All items were measured on a six-point strongly disagree–strongly agree Likert-type scale, unless otherwise noted. Consistent with recommended best practices (see Lovelace and Brickman, 2013), we ensured that internal consistency (Cronbach’s alpha) was high for multi-item scales. Note that we used short scales for each outcome (one to four face-valid items), because the survey was included as a brief course assignment and therefore had strict length limitations.

Attributions for struggle were measured on a five-point not at all likely to think this–extremely likely to think this Likert-type scale (adapted from Yeager et al., 2016). Students were instructed to: “Pretend that, later today or tomorrow, you got a bad grade on a very important assignment in this class. Honestly, if that happened, how likely would you be to think these thoughts?” Two items measured attributions to strategy usage (“I can get a higher score next time if I find a better way to study,” “I will need to change the way I prepare for exams in this class if I want to get a higher score”; r = 0.70, α = 0.82).

We tested two novel measures of students’ approaches to studying that were designed specifically for this study. First, we assessed students’ perceptions that it was important to experiment with study strategies in order to be successful in the course using one item (“It is important to experiment with new study strategies to be successful in [Introductory Biology I]”). Second, we assessed students’ reliance on the strategies they had used to study for high school biology with one item (“My approach to studying in this class has been similar to how I studied for biology in high school”).

Feelings of belonging were measured both at the level of the course and the field more generally. Course belonging was measured with four items on a six-point strongly disagree–strongly agree Likert-type scale (“I feel comfortable in this class,” “I feel accepted in this class,” “I feel like I can be myself in this class,” “I feel like I belong in this class”; α = 0.89; PERTS, 2022). Uncertainty about belonging in the biological and health sciences was measured with two items (“I don’t know if I really belong in the biological and health sciences,” “Sometimes I’m not sure if I really belong in the biological and health sciences”; r = 0.82, α = 0.90; adapted from Harackiewicz et al., 2014).

Performance in the course was assessed with letter grades obtained from official transcripts (“A” = 4.00, “A−” = 3.67, “B+” = 3.33, “B” = 3.00, “B−” = 2.67, “C+” = 2.33, “C” = 2.00, “C−” = 1.67, “D+” = 1.33, “D” = 1.0, “D−” = 0.67, “F” = 0.00). Thirty students in the sample did not have course grades because they withdrew from the course. Persistence was measured by assessing whether students took the second course in the introductory biology sequence the subsequent semester (continued to next course = 1, did not continue = 0), also obtained from official records.

Correlations and descriptive statistics for all outcomes are presented in Table 1. Correlations between survey measures, including the new measures designed for this study, were consistent with the patterns that would be expected with valid measures of the theoretical constructs (see the Supplemental Material for a deeper discussion of these associations).

Analysis Plan

The preregistered analysis plan for this study can be found here: https://osf.io/ywp9q. Here, we analyze four preregistered outcomes (attributions of struggle to strategies, reliance on high school strategies, field belonging uncertainty, and course grade); one preregistered exploratory outcome (belonging in the course) that was administered as part of pre-existing surveys in the course; one outcome that was added to the survey after the study was preregistered (perceived importance of experimenting with new strategies); and one outcome that was not preregistered, because we did not know that we would have the opportunity to collect it (continuation to the second course in the introductory biology sequence).

To help account for the inclusion of exploratory outcomes, we used Bayesian causal forest (BCF) analysis to test intervention effects. BCF is a conservative machine-learning algorithm that is effective at minimizing the likelihood of finding spurious intervention effects, as well as identifying robust patterns of moderation (Hahn et al., 2020). BCF is effective for these purposes because it uses the built-in prior belief that treatment effects are null (centered at zero) and not moderated, which shrinks the average treatment effect size and its variability across levels of a moderator toward zero. Therefore, only strong evidence will lead BCF to identify treatment effects and moderators, reducing type I error rates. In addition, BCF allows for nonparametric relationships between variables, and thus makes fewer assumptions about the nature of the data, such as the assumption of linear relationships.

BCF estimates the treatment effect for a given outcome (i.e., difference between the treatment and control group) for each individual in the sample thousands of times, forming a posterior distribution of estimates. We summarize posterior distributions by presenting the average of the distribution (i.e., the average treatment effect [ATE]) and the average of the distribution for particular subgroups (i.e., the conditional ATEs [CATEs]). In addition, we report the proportion of estimated effects in the distribution that are greater than zero, which can be interpreted as the probability that a given effect is greater than zero (reported as “pr()”). Finally, we report the interval of the posterior distribution from the 10th to 90th percentile to complement the probability estimate by providing a more complete sense of the full distribution.

Our reporting of posterior probabilities, as opposed to frequentist p values, is consistent with calls to report continuous probabilities that a hypothesis is true, rather than to rely on binary significance thresholds (Gelman, 2016; McShane et al., 2019; for other examples of research that has used this approach, see Bryan et al., 2019; Yeager et al., 2019, 2021). Following our preregistered standards (see https://osf.io/ncxtm), we do not interpret any effect below a 75% posterior probability (i.e., interquartile range includes 0) to be meaningful, and posterior probabilities above 75% are reported continuously (Gelman, 2016; McShane et al., 2019), with higher probabilities indicating greater confidence in the effect. Results from ordinary least-squares regression, which are consistent with the BCF results, are presented in the Supplemental Material.

The BCF model for each outcome tested the effect of the peer-modeled mindset intervention and tested student gender, underrepresented racial/ethnic minority status, and generational status (i.e., first-generation vs. continuing generation) as potential moderators. Covariates included an indicator for course instructor as well as the measures from the baseline survey mentioned earlier. Missingness on baseline variables (4–12%, depending on the measure) was imputed with the mean for continuous variables and the mode for categorical variables, and dummy-coded missingness indicators were included as covariates in the model as well. The model pulled 3000 draws to form the posterior distribution (after 10,000 burn-in draws), with a thinning interval of four.

RESULTS

Preliminary Examination of Written Responses

At the end of each intervention activity, students in the treatment condition were asked to describe why they thought it was common for students to struggle in the course, and how previous students had overcome these struggles. Before conducting BCF analyses of treatment effects, we looked at students’ written responses to these questions as a preliminary examination of whether they understood and endorsed the intervention messages. This examination allowed us to peer into how students thought about and experienced the intervention activities.

Selected quotations from students in the intervention and control conditions are provided in Table 2. In each of the three intervention activities, more than 92% of students in the intervention condition (from a random 10% of the full sample) wrote responses that articulated the central intervention message: Struggle in the course did not point to a lack of capacity to be successful, but rather a need to develop better study strategies. In addition, many students described specific study strategies that no longer seemed to work for them in college biology and pointed to new and more effective strategies that they were beginning to use.

As expected, control participants described how specific study strategies could be helpful for learning challenging concepts. Therefore, both the treatment and control activities led students to reflect on their use of effective learning strategies. In addition, students in the control condition reported that they enjoyed hearing from former students and articulated new insights they gained from the interviews, which suggests that the control activities may have conferred some benefits. Thus, the control condition offered a conservative test of our hypothesis.

BCF Analyses

CATEs for each outcome, by generational status and race/ethnicity are displayed in Figure 3.

Attributions for Struggle.

The intervention increased attribution of struggle to strategy usage by 0.10 SD [0.01, 0.20], pr(ATE > 0) = 0.93. This effect was somewhat larger for first-generation college students, CATE_First-Gen = 0.17 SD [0.02, 0.34], pr(CATE_First-Gen > 0) = 0.94, than for continuing-generation students, CATE_Cont-Gen 0.08 SD [−0.01, 0.18], pr(CATE_Cont-Gen > 0) = 0.86, pr(Difference_CATEs > 0) = 0.76. The estimated treatment effect did not meaningfully differ as a function of race/ethnicity or gender, pr(Difference_CATEs > 0) < 0.62 for each moderator.

Approach to Studying.

The intervention increased the perceived importance of experimenting with new study strategies by 0.11 SD [0.01, 0.20], pr(ATE > 0) = 0.94. This effect was somewhat larger for first-generation college students, CATE_First-Gen = 0.17 SD [0.02, 0.35], pr(CATE_First-Gen > 0) = 0.94, than for continuing-generation students, CATE_Cont-Gen = 0.09 SD [−0.00, 0.18], pr(CATE_Cont-Gen > 0) = 0.89, pr(Difference_CATEs > 0) = 0.78. The effect was also somewhat larger for underrepresented racial/ethnic minority students, CATE_URM = 0.17 SD [0.02, 0.34], pr(CATE_URM > 0) = 0.95, than for racial/ethnic majority students, CATE_Majority = 0.08 SD [−0.01, 0.18], pr(CATE_Majority > 0) = 0.88, pr(Difference_CATEs > 0) = 0.79. The estimated treatment effect did not meaningfully differ as a function of gender, pr(Difference_CATEs > 0) = 0.53.

Similarly, the intervention reduced students’ reliance on the study strategies they had used to study for high school biology by 0.15 SD [−0.24, −0.06], pr(ATE < 0) = 0.98. This effect was larger for first-generation college students, CATE_First-Gen = −0.30 SD [−0.47, −0.14], pr(CATE < 0) = 0.99, than for continuing-generation students, CATE_Cont-Gen = −0.10 SD [−0.20, 0.00], pr(CATE_Cont-Gen < 0) = 0.89, pr(Difference_CATEs > 0) = 0.95. The effect was also larger for underrepresented racial/ethnic minority students, CATE_URM = −0.33 SD [−0.49, −0.17], pr(CATE_URM < 0) = 1.00, than for racial/ethnic majority students, CATE_Majority = −0.07 SD [−0.18, 0.04], pr(CATE_Majority > 0) = 0.80, pr(Difference_CATEs > 0) = 0.98. Finally, the effect was larger for women, CATE_Women = −0.21 SD [−0.31, −0.10], pr(CATE_Women < 0) = 0.99, than for men, CATE_Men = −0.01 SD [−0.15, 0.15], pr(CATE_Men < 0) = 0.54, pr(Difference_CATEs > 0) = 0.95.

Belonging.

The intervention increased feelings of belonging in the course by 0.09 SD [0.01, 0.18], pr(ATE > 0) = 0.93. This effect was somewhat larger for first-generation college students, CATE_First-Gen = 0.16 SD [0.01, 0.34], pr(CATE_First-Gen > 0) = 0.94, than for continuing-generation students, CATE_Cont-Gen = 0.07 SD [−0.01, 0.15], pr(CATE_Cont-Gen > 0) = 0.85, pr(Difference_CATEs > 0) = 0.78. The estimated treatment effect did not meaningfully differ as a function of race/ethnicity or gender, pr(Difference_CATEs > 0) < 0.57 for each moderator.

There was no meaningful main effect of the intervention on uncertainty about belonging in the biological and health sciences, ATE = 0.01 SD [−0.06, 0.08], pr(ATE) > 0 = 0.56. However, effects did differ somewhat as a function of generational status and race/ethnicity. For first-generation college students, the intervention reduced field belonging uncertainty by 0.09 SD [−0.25, 0.03], pr(CATE_First-Gen < 0) = 0.77, whereas it somewhat increased belonging uncertainty for continuing-generation students (though it did not meet the 75% probability threshold for interpretation as meaningful), CATE_Cont-Gen = 0.04 [−0.03, 0.13], pr(CATE_Cont-Gen > 0) = 0.73, pr(Difference_CATEs > 0) = 0.86. Similarly, for underrepresented racial/ethnic minority students, the intervention somewhat reduced field belonging uncertainty by 0.06 SD [−0.20, 0.04], pr(CATE_URM < 0) = 0.73, whereas for racial/ethnic majority students, it somewhat increased belonging uncertainty, CATE_Majority = 0.04 SD [−0.04, 0.13], pr(CATE > 0) = 0.72, pr(Difference_CATEs > 0) = 0.84, though neither CATE met the 75% probability threshold to be interpreted as meaningful.

Performance.

The estimated ATE on course grades was 0.02 SD [−0.03, 0.08], pr(ATE > 0) = 0.66, without meaningful evidence of moderation as a function of generational status, race/ethnicity, or gender; pr(Difference_CATEs > 0) < 0.56 for each moderator. However, descriptively inspecting the data revealed that the treatment effect was most pronounced in terms of helping students to receive an “A” (i.e., a 4.0 on a grade point average scale) in the course. Receiving an “A” in introductory biology is a meaningful outcome, because it sets students on the path toward being highly competitive for careers in the biological and medical fields (e.g., acceptance to medical school), and helping students to earn an “A” may therefore impact their subsequent academic trajectories. In addition, an especially high percentage of students earned at least an “A−” in the course compared with previous semesters, likely as a result of changes to the course and testing structure as a result of COVID-19. Therefore, earning an “A”, as compared with an “A−” or below, distinguished students’ achievement in the course from that of their classmates.

A secondary exploratory BCF model found that the intervention increased the rate at which students received an “A” by 0.07 SD [0.00, 0.15], pr(ATE > 0) = 0.90 (i.e., 3.65 percentage points). Effects on receiving an “A” also did not meaningfully differ as a function of generational status, race/ethnicity, or gender, pr(Difference_CATEs > 0) < 0.66 for each moderator, though descriptively, estimated effects were somewhat larger for first-generation college students, CATE_First-Gen = 0.09 SD [–0.00, 0.20] (i.e., 4.33 percentage points), pr(CATE_First-Gen > 0) = 0.89; racial/ethnic minority students, CATE_URM = 0.10 SD [0.00, 0.21] (i.e., 4.77 percentage points), pr(CATE_URM > 0) = 0.91; and women, CATE_Women = 0.08 SD [0.00, 0.17] (i.e., 3.99 percentage points), pr(CATE_Women > 0) = 0.90; compared with their continuing-generation, majority, and male peers, CATEs < 0.07 SD (i.e., < 3.43 percentage points), pr(CATEs > 0) < 0.88.

Persistence.

The effect of the intervention on continuation to the second biology course in the subsequent semester was ATE = 0.04 SD (i.e., 1.88 percentage points) [−0.02, 0.12], pr(ATE > 0) = 0.78. This is a modest effect that was imprecisely estimated (i.e., with a wide interval of the posterior distribution). Effects did not meaningfully differ as a function of generational status, race/ethnicity, or gender; pr(Difference_CATEs > 0) < 0.63 for each moderator. However, descriptively, estimated effects were somewhat larger for first-generation college students, CATE_First-Gen = 0.06 SD [–0.02, 0.18] (i.e., 2.68 percentage points), pr(CATE_First-Gen > 0) = 0.79; and racial/ethnic minority students, CATE_URM = 0.06 SD [–0.02, 0.16] (i.e., 2.61 percentage points), pr(CATE_URM > 0) = 0.81; compared with their continuing-generation and majority peers, CATEs < 0.04 SD (i.e., 1.61 percentage points), pr(CATEs > 0) < 0.75. Interestingly, effects were also somewhat larger for men, CATE_Men = 0.06 SD [–0.02, 0.16] (i.e., 2.52 percentage points), pr(CATE_Men > 0) = 0.79, than for women, CATE_Women = 0.04 SD [–0.03, 0.11] (i.e., 1.61 percentage points), pr(CATE_Women > 0) = 0.75.

DISCUSSION

Many students struggle at the outset of introductory science courses, because they have not yet adopted the learning strategies required to master the concepts (Tracy et al., 2022). To address this problem, we created a novel intervention development protocol. We then used it to craft a customized peer-modeled mindset intervention, which was designed to shift students’ attributions for struggle in an introductory biology course. We tested the intervention in a preregistered RCT with a large sample of students and assessed treatment effects using a conservative Bayesian machine-learning algorithm (Hahn et al., 2020). The intervention altered students’ reported approaches to studying and alleviated their doubts about belonging (in the course and in the biological sciences more generally); doubts about belonging are a strong predictor of attrition, especially among underrepresented students (Walton and Cohen, 2011; Smith et al., 2013; Thoman et al., 2014; see also Strayhorn, 2012). Finally, the intervention had positive, though modest and imprecisely estimated, downstream implications for students’ performance and persistence, increasing their likelihood of receiving an “A” and enrolling in the subsequent class in the introductory biology sequence.

The effectiveness of the peer-modeled mindset intervention underscores the importance of hearing the right story at the right time from a trusted, credible source (see Wilson, 2011). Along the path to a STEM career, many individuals will encounter a variety of new challenges that may evoke doubts about their potential to succeed in the field. At these times, hearing a story about the steps to overcome that challenge from an individual who has already done so in the same course can vividly convey that a student is not alone in facing this difficulty and that it is possible to overcome the challenge, while providing useful details about how to do so.

Importantly, the effects of the peer-modeled mindset intervention were especially pronounced among first-generation college students and underrepresented racial/ethnic minority students, who have historically been underrepresented in the life sciences and in the STEM fields more broadly (Radford et al., 2010; Shaw and Barbuti, 2010; National Science Board, 2014; Riegle-Crumb et al., 2019; Seymour and Hunter, 2019). The present findings therefore indicate that peer-modeled mindset interventions hold promise for broadening participation in the STEM fields. They also point to the power of drawing on the experiences of former students to help a wide variety of incoming students adjust to introductory courses at the transition to college (for related research incorporating learning assistants in an introductory biology course, see Clements et al., 2022).

There are at least two different plausible mechanisms that could explain our findings. First, by nature of being underrepresented in the STEM fields, first-generation and underrepresented racial/ethnic minority students have less exposure to mentors and role models with similar backgrounds to themselves in these fields, as compared with their continuing-generation and majority peers. The peer-modeled mindset intervention may have been effective for underrepresented students, in part, because it provided them exposure to successful upper-division students from similar backgrounds (for related research on same-race teachers and mentors, see Egalite et al., 2015; Gershenson, 2016). On the other hand, the intervention may have been effective because it included stories not only from former students from underrepresented groups, but also majority students. That is, learning that students from all backgrounds experience struggle in the course may underscore that students are not alone in struggling and that all students can overcome struggle by refining their approach to learning.

The second possibility is that, absent intervention, students from advantaged groups were afforded more opportunities to view instances of struggle as temporary and possible to overcome (see Steele and Sherman, 1999). That is, students from groups that are overrepresented in STEM are exposed to many examples of other individuals from their own backgrounds who were able to overcome challenges and be successful in a STEM field. An attributional retraining intervention may thus be less impactful for more advantaged students, because their cultural experiences may already provide them with strategy-based attributions for struggle.

An important step for future research will be to directly test these two alternative hypotheses for why a peer-modeled mindset intervention may especially benefit students from groups that are underrepresented in the STEM fields. This knowledge would help us to understand the mechanisms by which mindset interventions can improve inclusion and equity in these fields, which is a high priority for the National Science Foundation (NSF, 2016, 2020). By doing so, such research could shed light on other strategies that might help to broaden participation (e.g., hosting diverse panels of successful former students, encouraging teachers to be transparent and explicit about attributions for success and failure).

An exciting implication of the peer-modeled approach to mindset interventions is that it is built for customization to local contexts. Student experiences differ greatly between different college courses, and without customization, mindset interventions are unlikely to fit perfectly into any given context and have their greatest potential impact on student outcomes (see Yeager and Walton, 2011). By drawing on the experiences of former students, future peer-modeled mindset interventions would fit a novel context in at least three important ways. First, the intervention would be customized to the particular subject matter of the course, including stories from former students about learning course-specific topics. Second, the intervention would allow former students to reference whatever resources are available in the context. For example, in the present intervention, former students reflected on optional discussion sections, practice problems, instructors’ office hours, and many other available learning resources. In a novel context, former students could be encouraged to discuss the most relevant and important resources that they drew on to be successful in that particular course. Third, and perhaps most intriguing, interventions could be customized to address completely different psychological barriers in a given setting. In the present research, we identified students’ attributions for struggle as centrally important, but students in other courses may face doubts about social belonging (see Walton and Cohen, 2007), low perceptions of value or purpose for learning (see Yeager et al., 2014; Harackiewicz et al., 2016), or many other psychological barriers. The interviewing procedure can illuminate what the most salient barriers are in a context and provide vivid stories about how previous students overcame those barriers. An exciting step for future research will be to use the peer-modeled mindset intervention protocol in a wide variety of other college science courses to develop and test novel customized interventions.

Another important question to address in future research will be whether the impacts of peer-modeled mindset interventions are sustained over time. The present results indicated that the intervention had modest effects on students’ enrollment in the second course in the introductory biology sequence. It is possible that this intervention might initiate positive recursive processes, or feedback loops, in which improved experiences and an enhanced sense of belonging in biology lead to improved performance and persistence, leading to more positive experiences (see Yeager and Walton, 2011; Hecht et al., 2019b). If this is the case, intervention effects might be expected to amplify over time as the cumulative effects of the intervention compound. On the other hand, many intervention effects have been found to fade over time (see Bailey et al., 2017). A careful investigation of whether peer-modeled mindset intervention effects persist, and the individual and contextual factors that moderate whether these effects persist, will be necessary to understand the long-term implications of the present intervention approach.

Finally, we note an unexpected positive side effect of the present study. By developing the present intervention, we (A.G.L., R.E.B., D.R.H.) gained new insight into our students’ experiences in our courses. In large introductory courses, it can be challenging to effectively gauge the learning experiences of our students. By watching the interviews with former students, we learned about their initial struggles, the doubts these struggles caused, and the ways in which some students were able to overcome these doubts and be successful in the course. These interviews provided a window into students’ lives that is rarely available to us as instructors, giving us a new understanding of how to help our most vulnerable and struggling students. The present intervention strategy is thus an exciting prospect, not only as a treatment for students, but also as a new potential approach to professional learning and development. Encouraging instructors to learn from interviews with their former students could help other science instructors see their courses from their students’ perspectives and thus find new ways to make their courses more emotionally and motivationally supportive and inclusive.

CONCLUSION

The success of the present intervention highlights how science instructors’ and former students’ expertise and knowledge of a particular learning context can be harnessed to create customized and impactful mindset interventions for incoming students. Such messages, which are tailored with intimate knowledge of an introductory science course, can help students adapt to a challenging new academic setting while also generating new insights about attitude and behavior change, making major contributions to both practice and theory.

ACCESSING MATERIALS

All materials referenced in this article can be accessed in the Supplemental Material and at https://osf.io/ywp9q.

ACKNOWLEDGMENTS

REFERENCES