Testing the association of growth mindset and grades across a challenging transition: Is growth mindset associated with grades?

Abstract Mindset theory predicts that whether students believe basic ability is greatly malleable exerts a major influence on their own educational attainment ( Blackwell, Trzesniewski, & Dweck, 2007 ). We tested this prediction in two near-replication studies (total n = 832). In study 1 we tested the association of mindset with university grades in a cross-sectional design involving self-reported grades for 246 undergraduates. Growth mindset showed no association with grades (β = −0.02 CI95 [−0.16, 0.12], t = −0.26, p = .792). In study 2, we implemented a longitudinal design, testing the association of mindset with grade transcript scores across a series of challenging transitions: from high school to university entry, and then across all years of an undergraduate degree (n = 586). Contrary to prediction, mindset was not associated with grades across the challenging transition from high-school to the first year of university (β = −0.05 CI95 [−0.14, 0.05], t = −0.95, p = .345). In addition, mindset was unrelated to entry grades (p = .808). And no support was found for a predicted interaction of mindset with academic disadvantage across the transition (β = −0.03 CI95 [−0.12, 0.07], t = −0.54, p = .592). Follow-up analyses showed no association of mindset with improvement in grades at any subsequent year of the degree (minimum p-value 0.591). Jointly, these two near-replication studies suggest that, even across challenging transitions, growth mindset is either unrelated to educational attainment or has a very small negative influence.


Introduction
Intelligence scores strongly predict educational attainment (Deary, Strand, Smith, & Fernandes, 2007), as well as wealth (Lynn, Vanhanen, & Stuart, 2002;Zagorsky, 2007) and health (Gottfredson & Deary, 2016). These findings have understandably fostered great interest in raising intelligence, including studies testing how education itself raises intelligence test scores (Ritchie, Bates, & Deary, 2015). An alternative theory, however, asserts that a crucial cause of educational attainment is one's beliefs about intelligence (Dweck, 2006). Specifically, mindset theory asserts that a crucial cause of educational outcomes is whether children believe that intelligence is malleable or fixed, with growth beliefs leading to high attainment and fixed beliefs to failure (Blackwell et al., 2007). If true, this theory is clearly of great importance for both intelligence theory (which predicts that it is intelligence itself, not beliefs about the malleability intelligence, that is responsible for learning and problem solving), and for educational practice (where factual information about the consequences of beliefs about intelligence on learning are important for teacher training, for classroom practice, for policy and research funding decisions, as well as for potential effects of learning the science of intelligence, which involves considerable stability). Despite this importance, and wide influence in teaching (Yettick, Lloyd, Harwin, Riemer, & Swanson, 2016), business (Bock, 2015), philanthropy (e.g., Gates, 2015), and the public mind (Dweck, 2006), this radically different model of learning has been subject to little independent replication (for a meta-analysis, see Sisk, Burgoyne, Sun, Butler, & Macnamara, 2018) and in fact appears to rest on shaky foundations (Burgoyne, Hambrick, & Macnamara, 2020;Li & Bates, 2019). Here we report two near-replication studies of Blackwell et al. (2007) study 1, testing if intelligence mindsets are associated with educational attainment, including a unique longitudinal study with a relatively large sample size tracking association of mindset with grades across the challenging transition from high school to first year university and onward.
Growth mindset refers to the belief that intelligence can be greatly changed. It contrasts with a belief that intelligence is fixed or "fixed mindset" (Dweck, 2006). Mindset theory proposes that it is these beliefs that largely determine learning and attainment as a fixed mindset causes people to avoid even attempting difficult tasks as these people view "challenges as a sign that they may lack intelligence-that they may be 'dumb' or might be seen as 'dumb'" (Yeager & Dweck, 2012). Instead, children with a fixed mindset are predicted to "document" https://doi.org/10.1016/j.intell.2020.101471 Received 16 November 2019; Received in revised form 1 July 2020; Accepted 2 July 2020 their success by only engaging in tasks with low likelihood of failure (Yeager & Dweck, 2012). These beliefs about the nature of intelligence are predicted to have both immediate and long-term effects. In the short term, growth mindset is predicted to benefit performance on an IQ test following failure feedback (Mueller & Dweck, 1998). Li and Bates (2019) tested the validity of the classic praise-for-hard-work manipulation of mindset in 624 individually-tested 9-13 years old. Their active-control design found no effect of either growth mindset or mindset manipulation on IQ scores following a challenge. No effect of mindset was found on any of the 7 motivation and attribution measures used by Mueller and Dweck (1998), and no effects of mindset occurred for challenging materials. Compatible with these null results, children's own mindsets were unrelated to resilience to failure for either moderate or difficult IQ test items (ps = 0.673 to 0.888). The sole exception was a significant effect in the reverse direction to prediction found in Study 2 for resilience on more difficult material (p = .007).
If mindset cannot improve performance even in the short term, this clearly raises questions about the other foundational claim of mindset theory: that long-term growth mindset improves real-world educational attainment, at least across challenging transitions (Blackwell et al., 2007). Here we focused on whether beliefs about intelligence show these predicted links to educational attainment.
If growth mindsets help children cope with challenges in learning then, given that it is well known that at any given time, many students are encountering challenges or are struggling with learning, it may be deduced that mindset effects should be present in any cross-sectional data, reflecting the effects of both current and prior un-measured challenges. However, this deduction is contradictory to the findings in Blackwell et al. (2007), who reported no association of mindset with math grades at the baseline of their study. As has been highlighted in research on interactions in psychiatry (Munafo, Zammit, & Flint, 2014), even mechanisms which operate solely via interactions should produce statistical main effects in baseline data in studies like of those of Blackwell et al. (2007). Moreover, any mechanism leading to improved learning, even only for a period of time, should lead to positive, and cumulative gains in learning. If this were not the case, one would have to posit some countervailing negative effect of growth mindset leading to subsequent reduced learning, which mindset theory does not predict. Thus, we wished to both to test the explicit proposed interaction of mindset with an acknowledged challenging transition, but also to document whether mindset shows any evidence of having had a main effect on entry grades and, most importantly in the present data, any evidence of retained benefits beyond the challenging transition. We also wished to test a more recently proposed interaction, namely that mindset is only of value in students who are particularly challenged (Paunesku et al., 2015).
Finally, we wished to consider the question of the falsifiability of mindset theory: Whether it is specified in such a way that not only can testable predictions be derived from it which are improbable under competing theories, but that failure of these predictions falsifies the theory, causing it to be abandoned (Popper, 1963). With an increasing number of questions being raised about, and data being reported leading to doubt over, foundational claims of mindset theory (Burgoyne et al., 2020;Macnamara & Rupani, 2017;Sisk et al., 2018), we wished to assess the scientific status of the theory given well-powered null results for tests of its central predictions and whether the 'hard core' of the theorythat beliefs about intelligence malleability drive up learning-is supported.
We conducted two substantial studies to explore these questions. The data presented constitute all data we have collected on mindset and grades. In addition, we have reported all measures, conditions, data exclusions (if any), and how we determined our sample sizes in the present study. These results, we hope are of direct value, and will be of use in future meta-analyses.
We conducted two near-replication studies of study 1 in Blackwell et al. (2007): Our study 1 tested the cross-sectional association between mindset and self-reported university grades. Our second study used a larger sample size (N = 586) and more rigorous set of analyses than was undertaken by Blackwell et al. (2007), testing the association of mindset with official grades in a longitudinal sample, controlling from baseline (university entrance) exam grades, and testing mindset effects across the challenging transition from high school to university and continuing each year across additional challenging thresholds to graduation. Study 2 thus used repeated measures data with known intervening academic challenges to test the key theorical claim of mindset theory: that growth mindset enhances educational attainment across challenging transitions (Blackwell et al., 2007). In study 2 we were also able to test for an interaction of mindset with entry grades, examining the recent modification of mindset theory, suggesting that its effects might be restricted to students who are most likely to struggle when entering university (Paunesku et al., 2015). Table 1 provides a summary of the similarities and differences between Blackwell et al. (2007) study 1 and the present studies 1 and 2. Both studies were approved by the Psychology Research Ethics Committee at the School of Philosophy, Psychology and Language Sciences, University of Edinburgh.

Study 1: Testing association of mindset and educational attainment in 246 undergraduates
In study 1, we examined the association between undergraduates' mindset and their grades in university. Many students find university to be challenging, especially in the first year. The big transition from high school to university brings difficulties to students both in learning and living, which further causes a high dropout rate in the following year. For example, 6.3% of students who enrolled in UK universities in the academic year 2016/2017 later dropped out altogether (HESA, 2019). The subsequent years in university are also challenging to students seeking grades required to enter their preferred honours course, and a steep learning curve year on year, as new and more complex tasks such as scientific writing, statistics, dissertation projects, and the self-management required to complete course work.
In study 1, we tested the association of mindset with grades, hypothesising that students with more of a growth mindset would have better grades than those with more of a fixed mindset. Regarding the effect size, since university provides what is widely reported as a highly challenging transition, often associated with significant learning-related stress and risk of dropout, we hypothesised that we would find an association of at least r = 0.2 effect ( Blackwell et al. (2007). The study had power of 88% to detect an effect of this size (two sided). Ideally, one would control for university entry grades but in this initial study we simply examined the raw association of mindset with grades among students in the midst of the challenging transitions presented by university (note: we report data using recordbased grades and controlling for official entry grades and tracking progress across multiple transitions below in our much larger, multiyear, study 2). In addition to mindset, we also included three other traits that have been suggested to be associated with educational attainment: Self-esteem (Leary & Baumeister, 2000), locus of control (Findley & Cooper, 1983) and grit (Duckworth, Peterson, Matthews, & Kelly, 2007). We hoped that controlling for some of these non-mindset traits might enhance power to detect a true association between mindset and grades, as well as casting light on their associations with grades in the present data. Our second hypothesis was thus that including self-esteem, locus of control and grit as covariates would allow the predicted association between mindset and educational attainment to emerge more strongly.

Subjects
In total, 308 students were invited to take part in study 1. All students were in their second or subsequent year at a single UK University and were recruited from the university's undergraduate participant pool. Of 246 students who consented, 68 were males and 178 were females (mean age = 21.43, SD = 4.27). There was no compensation for students' participations.

Materials
Mindset was assessed using the 8-item Theories of Intelligence scale (Dweck, 1999). Example items include "You have a certain amount of intelligence, and you can't really do much to change it." Responses were recorded on a Likert scale (from 1: strongly agree to 6: strongly disagree). Responses were reversed where appropriate and summed to form a mindset score for each subject with high scores indicating a growth mindset. Self-esteem was measured using the 10-item self-esteem scale (Rosenberg, 1965). Locus of control was measured using the 29-item locus of control scale (Rotter, 1966) and grit was measured using the 8-item short grit scale (Duckworth & Quinn, 2009). Students entered their most recent year's final letter grades (received 4 months prior to testing). These were recoded into numerical scores corresponding to the university grade bands used in study 2 (ranging from 0 to 100).

Procedure
Students provided consent, and then proceeded to complete the online survey. This included demographic information comprising age and sex. This was followed by the Theories of Intelligence scale, Rosenberg self-esteem scale, Rotter's locus of control scale, and the grit scale followed by the letter grade received for each course, giving their most recent year's grade in each case.

Results
All data and analysis code are open-access and raw data and R analysis scripts used in the two studies are available in supplementary data at https://osf.io/vg87m/. We first tested the hypothesis that growth mindset would be positively associated with higher grades. This was tested using a linear regression, with average grade as the dependent variable and mindset as the independent variable. Contrary to prediction, growth mindset was associated with worse, not better grades (β = −0.02 CI 95 [−0.16, 0.12]) and the effect was not significant (t = −0.26, p = .792; see Fig. 1). Adding age and sex as covariates did not change the null association of mindset and grades (β = −0.02 CI 95 [−0.16, 0.12], t = −0.28, p = .778).

Study 1 Discussion
The major finding of study 1 was that we found a near-zero Table 1 Comparison of Blackwell et al. (2007) study 1 and the present study 1 and 2. Blackwell et al. (2007) The present paper  (Dweck, 1999, p.177) Theories of Intelligence Scale (Dweck, 1999, p.178) Theories of Intelligence Scale (Dweck, 1999, p association of mindset with grade. Indeed, the (non-significant) effect was in the reverse direction to that predicted. This null result was robust to inclusion of covariates. A similar lack of association of mindset with grades was reported by Blackwell et al. (2007), in which they reported a significant association of mindset once students were progressing across the "challenging transition" into 7th grade. This lack of association is also in-line with a growing set of studies reporting no support for any association between growth mindset and better educational attainment (e.g., Bahník & Vranka, 2017;Bazelais et al., 2018). Indeed, the slight negative association has been found in other samples (Li & Bates, 2019). We can be increasingly confident, then, that mindset does not lead to any average increase in grades, with similar results in the present study, and other studies of the cross-sectional association of mindset and attainment, including even Blackwell et al. (2007). As the study covers what many students find to be a highly challenging transition, the null association is, however, even more surprising, and contrary to mindset theory. Selection effects can potentially influence effect sizes. If it were the case, however, that students with a growth mindset are more likely to apply for university, mindset scores in our subjects should tend to be high. But they are not: subjects' scores in our study 1 were normally distributed with a mean of 3.46 (SD = 0.60), i.e., more fixed than was reported for Blackwell et al. (2007), see Fig. 2. It is unlikely, therefore, that the null effect in our study 1 is due to selection. A second possibility is that our results would have been significant if, like Blackwell et al. (2007), we were able to control for entry grades. Also, in this study we relied on self-reported grades rather than official transcripts to measure educational attainment. Self-reported grades have somewhat lower construct validity than transcripts (Kuncel, Credé, & Thomas, 2016). If the challenging transition of university activates effects of growth mindset promoting educational attainment, then this should have been apparent as a main effect of mindset on attainment, independent of initial grades.
To address both these limitations, we conducted a large longitudinal study using nationally recorded high-school exit grades to control for performance prior to entry and using transcript grades to study effects of mindset on grades longitudinally. This allows control for transcriptbased entry grades, recording of mindset in year 1, and tracking transcript-based grades across each of the next four years for three nearcomplete cohorts of students entering a psychology program. These changes allow us to clarify whether the null result of study 1 reflect use of self-reported grades or absence of control for entry grades, or if there is in fact no or even a slight negative association of mindset with grades across a challenging transition.
5. Study 2: Does mindset associate with undergraduate students' performance across a challenging transition (high school to university) and beyond?
In study 1, we tested the association of mindset with university grades, finding a null result. In study 2, as noted above, we set out to test this association using three near-complete cohorts of students enrolled in an introductory psychology course, with available transcript records of their entry grades grade-data recorded each year until they graduated. This offered not only greater power (n = 586; 99% power to detect an effect of r = 0.2) and improved measurement precision, but also an opportunity to test whether mindset relates to specifically to first year grades (putatively the greatest challenge is the transition from high school to university), but also across the challenge of entrance into an honours program. Finally, with entry grades available, we could also test the hypothesis that mindset effects are magnified in students entering with low grades (Paunesku et al., 2015).
The educational system for higher education is varied in the UK. In England, Wales and Northern Ireland, students normally undertake a three-year programme to achieve a bachelor's degree. In Scotland, student undertake a three-year programme for an ordinary bachelor's degree and a four-year programme for an honours' degree. Each of the university years is challenging for students. The first year in university is the perhaps particularly challenging. Compared to high school, university carries a much heavier workload, and students have to work not only in the class but also outside of class and with far less supervision.
The second year is a threshold year for students, as grades in second year determine whether they are able to progress to the honours Fig. 1. The association between students' mindset scores and average grade in university in study 1. Confidence bands indicate 95% confidence intervals.

Table 2
The correlation between mindset, locus of control, grit and self-esteem in study 1.

Y. Li and T.C. Bates
Intelligence 81 (2020) 101471 program, or have to take out a non-honours bachelor degree. The third year, again, presents unique challenges. The material studied increases sharply in complexity and students undertake new and challenging assessments including a literature review and running a project. In fourth year, students are again challenged, now needing to complete a major dissertation project. Thus, each of the four years represents a distinct challenge, with the first year being perhaps the closest match to the challenging transition for our students as identified by Blackwell et al. (2007). Our hypotheses were as follows. First, it is not clear whether there is an association between mindset and entry grades, but we were interested in testing this. Therefore, the first hypothesis tested was that mindset would be associated with students' entry grades. Second, following Blackwell et al. (2007), we predicted that growth mindset would be associated with higher grades at the end of year 1 of university, controlling for student's entry grades (hypothesis two). Extending this hypothesis, because, as noted above, each year of university presents an escalating series of challenges to students, we predicted positive associations of year 2, 3, and 4 grades with growth mindset, controlling for entry grades (hypotheses three, four, and five). Finally, and following the "current era" model of growth mindset (Dweck & Yeager, 2019;Paunesku et al., 2015), we predicted that mindset would most strongly predict grades in those students encountering the greatest challenge (those joining the university with the lowest entry grades), i.e., an interaction between mindset and entry grades predicting students' first year grades (hypothesis six).

Subjects
Our subjects consisted of 586 undergraduate students entering and completing a bachelor's degree at a research-intensive university. Subjects enrolled in a foundational psychology course in their first year of university were invited to participate in the study as part of their course work. In total, 448 female and 138 male students were studied across their 4-year degree (mean age 21.88, SD = 3.17).

Materials
Mindset measure: Mindset was measured using the 8-item Theories of Intelligence scale (Dweck, 1999). Grades: When applying for university, students'' high school exit qualifications were made available. These consist with grades from a range of national tests (e.g. A-level). The letter grades that students achieved were converted to uniform numerical scores based on the tariff table provided by the Universities and Colleges Admissions Service (UCAS, 2019). Grade was calculated for each student for each year using course records (range from 0 to 100).

Procedure
Students provided consent and completed the mindset scale online in the first semester of their degree as part of their undergraduate class work. A further consent was gained from the Psychology Research Ethics Committee at University of Edinburgh to access transcript records. Thus, a data frame consisting of students' mindset scores, entry grades, average grade in each year of university was assembled.

Results
Four students (two female and two male students) were recorded as having first-year numerical grades more than 5 SDs below the mean of the sample (due to personal circumstances). These subjects were removed from all analyses. As in Blackwell et al. (2007), we first tested if students' pre-challenge grades (i.e., entry grades) were associated with their growth mindset. This was done using a linear regression, with entry grades as the dependent variable, mindset as the predictor. Growth mindset was not significantly associated with students' entry grades (β = −0.01 CI 95 [−0.11, 0.09], t = −0.24, p = .808). The association between mindset and students' entry grades remains null after controlling for students' age and sex (β = −0.01 CI 95 [−0.11, 0.09], t = −0.23, p = .822).
Next, we tested hypothesis two, that growth mindset would be associated with higher grades at the end of year 1 of university, controlling for entry grades (i.e., with change in grades across this challenging transition). This was again tested using a linear regression with first year average grade as the dependent variable, mindset scores as the independent variable, and entry grades as covariate. Growth mindset was slightly negatively linked to change in grades (β = −0.05 CI 95 [−0.14, 0.05]), in the reverse direction to expectation, and non-significant (t = −0.95, p = .345). Controlling for student's age and sex did not change the null association of mindset and year 1 average grade (β = −0.04 CI 95 [−0.14, 0.06], t = −0.81, p = .417).
We next tested whether growth mindset was associated with grades in years 2, 3, and 4, controlling for entry grades (hypotheses three, four, and five). These hypotheses were tested using three linear regressions, with average grade in each of years 2, 3, and 4 as the dependent variables respectively. Mindset scores formed the independent variable Fig. 2. Distribution of mindset scores observed in Study 1, with mean score plotted, along with the mean mindset score for Blackwell et al. (2007), study 1.

Y. Li and T.C. Bates
Intelligence 81 (2020) 101471 and, again, we controlled for entry grades. The results are shown graphically in Fig. 3.
For second year the effect of mindset was non-significant (β = 0.02 CI 95 [−0.10, 0.13], t = 0.30, p = .763). Likewise at third year no significant effect of mindset was found (β = −0.04 CI 95 [−0.19, 0.11], t = −0.54, p = .591), and lack of effect repeated at fourth year, where the effect of mindset was again estimated as negative (β = −0.04 CI 95 [−0.22, 0.14]) and non-significant (t = −0.48, p = .634). The results held after controlling student's age and sex (ps = 0.530, 0.861 and 0.697 respectively). In summary, no support for any significant association of growth mindset and grade was found either across the most challenging transition year from high school to university, nor at any subsequent year in university. Indeed, growth mindset was negatively associated with grades in year 1, year 3 and year 4 (results were not statistically significant; also see Fig. 3).
Finally, we tested hypothesis six, that mindset would predict grades for those students encountering the greatest challenge (those joining the university with the lowest entry grades). This was tested using a linear regression with average grade at the end of first year in university as the dependent variable, mindset and the interaction of mindset and entry grades as the independent variables, and entry grade as covariate. Contrary to prediction, the interaction of mindset and entry grades was not significant (β = −0.03 CI 95 [−0.12, 0.07], t = −0.54, p = .592).

Study 2 Discussion
Study 2 yielded four main findings. First, we again found no evidence for any association of mindset with initial grades (in our case entry grades). Second, and contrary to Blackwell et al. (2007), we found no evidence of a predicted association of growth mindset with grades across the challenging transition from high school to university. Instead, the association observed was in the reverse direction to that predicted by mindset theory. Third, we found no support for association of mindset with change in grades at any subsequent transition through a university degree. Fourth, contrary to the prediction that mindset would be especially effective in participants with low initial grades (Paunesku et al., 2015), we found no interaction of mindset × low entry grades on improvement in grades across the challenging transition to the first year of university. Our results thus did not support any of the predicted associations of growth mindset with educational attainment. We discuss these four findings briefly before concluding with a joint discussion of the impact of both studies for mindset theory.
Similar to subjects in study 1 in Blackwell et al. (2007), our subjects in study 2 were tested across a challenging transition. Their entry grades were controlled, and their grades across a university degree were known. Our subjects were thus a suitable sample to detect a significant positive effect of growth mindset on grades if present, but no such effect emerged. Instead, we found a non-significant negative association of mindset with grades across the most challenging transition from high school to university, which is consistent with our null finding in study 1, and previous work (Li & Bates, 2019). The subsequent years in university are also challenging, but the associations of mindset with grades across those transitions repeated the null result across the most challenging transition. These findings are incompatible with mindset theory and could not support the prediction that growth mindset activates behaviours that causes better grades even across a challenging transition. The lack of interaction (mindset x low entry grades) effect on year 1 grades is consistent with our null main effect. The strongest benefit of growth mindset should emerge in this interaction effect, but it did not. We next discuss the overall findings.

Joint discussion
Mindset theory, like intelligence, has been claimed to play a critical role and having a powerful impact on educational attainment (see Blackwell et al., 2007;Claro et al., 2016;Costa & Faria, 2018). Contrary to intelligence theories, mindset theory predicts that it is beliefs about the malleability of intelligence that can raise fluid and crystalized ability (assessed in academic grades) independently of intelligence. Here, we tested these predictions but found no support for any main effects or interactions of mindset with grades, either at baseline or across challenging transitions.
In two studies, we tested the association between mindset and educational attainment (total 832 undergraduate students). Study 1 used the standard mindset scale to test whether having a growth mindset was associated with better grades. No support was found for this prediction. Study 2 tested whether growth mindset predicted higher university grades across a series of challenging transitions (high school to university, and transitions within university), also examined any effects were apparent in those who were disadvantaged at entry. Mindset, however, was not significantly associated with grades at any point. In addition, all effect sizes (except the second year) we obtained Fig. 3. Grade trajectories across year of university in study 2 separately for students with fixed, mixed, or growth mindset. Note: Mindset was binned into three quantile groups: fixed (mindset ≤ 30); mixed (30 > mindset < 36); and growth (mindset ≥ 36). The sample sizes for fixed, mixed and growth mindset groups were n = 223, 190, and 169 respectively. Error-bars show the standard error of measurement at each time.
were in the reversed direction to the predicted associations in mindset theory. Likewise, growth mindset did not significantly predict higher grades even among students who were disadvantaged at entry.
In study 1, we showed that self-reported grades were not associated with mindset. In study 2 we had a larger sample, with transcript documentation across the challenging transition (from high school to university), and a series of transitions subsequent to this. Thus, our subjects in study 2 are well suited for a strong test of the proposed association between mindset and educational attainment. Despite one of the most important predictions of mindset theory for real-life outcomes being that growth mindset promotes educational attainment, we found no evidence for substantial (or significant) effects of growth mindset on better educational attainment. The lack of relationship between mindset and educational attainment is in keeping with Sisk et al. (2018) and those of others (Bahník & Vranka, 2017;Bazelais et al., 2018;Li & Bates, 2019;Sriram, 2014).
How can we reconcile these null findings with other claims presented as supporting the theory (e.g. Blackwell et al., 2007;Gunderson et al., 2018;Park, Gunderson, Tsukayama, Levine, & Beilock, 2016)? One might posit extremely rapid fade-out of growth mindset effects, such that learning might occur, but the effects would be absent on most measurement occasions. This explanation, however, is inconsistent with claims that mindset has enduring (at least one academic year) effects on educational attainment (Blackwell et al., 2007;Gunderson et al., 2018;Park et al., 2016). Perhaps more relevant, previous results supporting mindset effects have confounded other motivational factors such as achievement goals and attributions, e.g. Gunderson et al. (2018) and Park et al. (2016). These have both been previously shown to have significant effects on educational attainment (e.g., Elliot, Shell, Henry, & Maier, 2005;Houston, 2016). Thus, although these studies have been presented as supporting enduring effects of mindset on educational attainment, confounders may have been responsible for the positive results. A simpler explanation for our results is that mindset is not reliably associated with grades even during a challenging transition.

Internal consistency of mindset theory and implications for mindset theory
Both studies 1 and 2 that, in addition to not being related to grades across a challenge, mindset was unrelated to grades at baseline. Previous researchers have accepted a lack of association of subject's mindsets with performance at baseline as compatible with mindset theory (Blackwell et al., 2007). However, taken together with the lack of any effects across a challenge, we suggest that the lack of association cross-sectionally should itself be cause for caution regarding the foundational claims of mindset theory. At any given time, many students struggle with learning, and encounter significant challenges. Thus, mindset ought to have a significant main effect on grades. Otherwise we are left with the counter intuitive claim that no individuals in these studies were experiencing a learning challenge at baseline, nor had they experienced any challenge previously which would activate the enduring gains predicted by mindset theory. Thus, even if it interacts with challenge, mindset should improve the outcomes of some or most of those who have a growth mindset, leading to a main effect of mindset on grades (see also Munafo et al., 2014 for a similar argument in psychiatry). The lack of a main effect at baseline in studies like ours or those of Blackwell et al. (2007) is thus counter to the statistical expectation of main effects in the presence of an unmeasured interaction. The lack of any support for an interaction (indeed the interaction was estimated in the wrong direction in our study 2) is further evidence against mindset working to raise grades, either in general or for a more limited time.
A second concern regarding the internal consistency of mindset theory involves its predicted relationship to intelligence. Mindset theory is intimately linked to beliefs regarding intelligence as being greatly malleable and predicts that individuals who hold this belief will attain high education grades. Mindsets, however have no documented association with IQ (Li & Bates, 2019), and, other than in one location that we could find (Dweck (2006) states that "Since this was a kind of IQ test, you might say that praising ability lowered the student's IQs. And that praising their efforts raised them" p.73), mindset theorists accept that no such association should emerge. This leads to the curious situation that the belief required by the theorythat intelligence can be raised greatly by adopting a growth mindsetis predicted by the same theory to be false. Like the lack of any enduring effects on grades, this prediction likewise seems to lack coherence. At best, students are being taught a belief which is predicted to be false.
A third concern regards claims for the enduring value of a mindset. Dweck (2019, p. 21) claimed that "a growth mindset could be taught and could have relatively enduring effects, such that the interventions influenced later grades (Aronson, Fried, & Good, 2002;Blackwell et al., 2007) and achievement test scores (Good, Aronson, & Inzlicht, 2003)". The enduring effects of mindset were explained as occurring "because [growth mindset] can trigger enduring changes in the way students perceive their ongoing school experience, which then feed on themselves to produce compounding benefits." (Dweck, Walton, & Cohen, 2014, p. 14). Following this logic, if growth mindset enhances learning on a dynamic life-long basis, as is claimed, the prediction of no difference in attainment at the baseline of any study seems incompatible with such a view of compound interest accumulating to a growth mindset. If mindset is triggered across challenges and failures, and these occur for all of us, and have enduring effects on academic achievement, it is not logical to claim that growth mindset improves grades only during interventions and not at the base line of such studies. A simpler explanation of the lack of differences in grades and ability at baseline is that mindset is ineffectual.
A final concern regarding mindset theory is its status as scientifically refutable. As documented by Burgoyne et al. (2020); Macnamara and Rupani (2017); and Sisk et al. (2018), proponents of mindset have for decades made bold claims for the effects of mindset, and this is to be commended: Dweck in particular made specific claims capable of refutation, clearly linking effects on learning and grades to implicit theories of intelligence. Given that the relationship of mindset to academic achievement is central to the theory, particularly when students undergo challenges, our new data showing longitudinal null and reverseeffects, and previous studies failing to find support for the basic premises of mindset theory (e.g., Burgoyne et al., 2020;Li & Bates, 2019) provide a test case: can mindset theory be refuted? For mindset to be a scientific theory (Popper, 1963) and for the scientific community to function programmatically, mindset theory must be able to be refuted, and the community to reject, rather than protect, the theory when this has happened. Post-hoc revisions discounting effects of subject's own mindsets on core outcomes in favour of variable outcomes, relaxation of the nature of interventions to include wide ranging complex and items chosen because they appear to have worked, caveating potential replications with the proviso that effects of any given intervention are not warranted to work outside the exact environment in which they were initially observed, severely reducing claimed effect sizes, dropping claims of effect durability, and a non-delimited set of auxiliary moderators such as culture, cohort, age, school-district, SES, classroom learning climate etc. do not serve to allow researchers to refine where the theory works. Rather, they render the theory unfalsifiable.

Summary
In two near-replication studies, we found that mindset does not appear to influence educational attainment. In Study 1, we found a near-zero association between mindset and self-reported university grades. In study 2, we found that mindset did not predict official grades upon entry to university, and did not predict grades in the transition, nor across time as coursework became more challenging. Each of these findings runs counter to predictions of mindset theory. While mindset theorists have claimed that mindset should be particularly beneficial for Y. Li and T.C. Bates Intelligence 81 (2020) 101471 low-achieving students, we found that for low achieving students who encountered the greatest challenge when entering university, growth mindset did not increase their educational attainment. Evidence was also found that adopting a growth mindset might harm student's educational attainment. We identified logical problems in mindset theory, challenging the internal coherence of its predictions. Taken together, the studies present compelling evidence calling into question core predictions and assumptions of mindset theory.

Declaration of Competing Interest
None.