Can Goal-Setting and Incentives Improve Student Performance?

have made great progress in increasing primary school enrollment and completion rates, few students make the transition to secondary school, and even fewer complete secondary school. Results-based financing (RBF) has been used in many developing countries in an attempt to incentivize students, parents, teachers, and others to achieve better results. RBF mechanisms work by linking financial (or in some cases non-financial) incentives to measurable results, for example school attendance, dropout rates, student test scores, or other education outcomes. Providing financial incentives to students has been shown to be effective in increasing school attendance and achievement and, in some cases, more effective than providing incentives to parents or teachers. However, these programs are expensive and difficult to implement for countries with limited capacity. On the other hand, the effect of other types of incentives such as personal goal-setting or non-financial incentives has been less extensively studied in developing countries. Can Goal-Setting and Incentives Improve Student Performance?

Furthermore, as a precondition to establishing financial incentives for students, it is first necessary to ensure that students know how to act on these incentives to improve their performance.
The Results in Education for All Children (REACH) Trust Fund at the World Bank funded an evaluation that measured how two different incentive approaches affected the academic performance of grade nine students in Zanzibar (Tanzania). The first approach allowed students to set personal goals at the beginning of the school year regarding their performance by the end of the year. The second approach combined this goal-setting exercise with non-financial rewards such as certificates or in-kind prizes for students who met their goals. These approaches were designed to answer the following questions: (a) whether students setting goals for themselves has any effect on their performance in school and (b) whether this effect is strengthened when combined with non-financial incentives.
The REACH evaluation found that personal goal-setting-with or without non-financial incentives-had no discernible impact on student achievement. However, goal-setting did have a significant positive impact on students' effort in school, and this impact was not affected by the existence of non-financial incentives. The students' inability to translate their increased effort into improved performance may stem from significant over-estimation of their own baseline performance and over-optimistic goal-setting, both of which were done after baseline testing but before the students had received their results. These findings suggest that another precondition of establishing effective RBF incentives at the student level is providing students with timely feedback about their own performance.

CONTEXT
Zanzibar is a semi-autonomous region of the United Republic of Tanzania, with a population of 1.4 million-nearly half of whom live below the poverty line. A key driver of poverty in Zanzibar is the limited economic prospects for young people, due in part to high rates of student dropouts before secondary school completion. Almost half of all students entering secondary school drop out before completion, and only 8.4 percent of students transition from lower secondary to upper secondary in grade 11, which occurs at roughly 16 years of age. 1 The majority of these student dropouts are due to their poor performance in the lower secondary school exit examinations. Therefore, to reduce student dropouts, the Ministry of Education and Vocational Training (MoEVT) of Zanzibar intends to implement programs that will improve student performance in the short run and increase students' aspirations, access to information, and motivation in the long run.

WHY WAS THE INTERVENTION CHOSEN?
After making considerable inputdriven investments in the education system, the MoEVT is keen to shift its focus away from "more of the same" school inputs and towards results. In particular, the government is interested in undertaking an evidence-based scale-up of a student incentive program as part of its US$50 million World Bank-financed education project. To improve student performance, several studies in Kenya, 2 Cambodia, 3 Benin, 4 and the United States 5 have found that incentivizing students directly is a promising option. Furthermore, studies in Mexico 6 and Tanzania 7 have shown that incentives aimed at students could potentially be more effective at improving student performance than incentives aimed at teachers. Similarly, studies in Malawi, 8 India, 9 and Mozambique 10 have found that providing incentives directly to students is at least as effective as providing them to parents.
However, as a precondition to a student-targeted RBF program, it is first necessary to establish whether students know how to convert extrinsic incentives into actions that improve their achievement. Intuitively, performance-based incentives will be effective only to the extent that actors (whether students, teachers, or parents) have control over achieving the intended results, have the ability to achieve the intended results, and understand how much effort is needed to achieve the intended results. The act of proactively setting goals and working towards those goals can help students to motivate themselves to increase their effort, particularly when self-control is a constraint. 11/12 Goal-setting can therefore increase the effectiveness of financial or non-financial incentives by helping students to build the tools to effectively respond to these incentives. From a policy perspective, goal-setting is a low-cost, scalable option that also has intrinsic merit beyond its instrumental value in promoting student achievement. Similarly, it is important to assess how the intrinsic incentives of goalsetting can be enhanced by extrinsic incentives, which may be particularly effective for children who are more motivated by short-run rewards than less tangible long-run rewards.
In assessing how well students set goals for themselves and respond to non-financial incentives, the REACH evaluation was designed to build an evidence base to inform the future scale-up of a student-targeted RBF approach to improving secondary student performance in Zanzibar.

HOW DID THE INTERVENTION WORK?
In February 2016 at the beginning of the 2016 school year, 187 secondary schools with a total of 18,281 grade nine students were randomly assigned to the following three groups. The students in the first group completed an interactive exercise with enumerators in which they estimated their expectations for their own performance on a baseline test and set a goal for themselves on a similar test to be taken in nine months' time at the end of the school year. The students made these estimations of both their baseline and goal-setting scores after the baseline testing had been conducted but before the students had received their results. The students in the second group completed the same goal-setting exercise and, in addition, received non-financial awards including certificates and in-kind prizes at a public ceremony at the end of the school year if they achieved their goals. The team running the intervention chose the type of in-kind prize to be given based on focus group discussions with the students but did not disclose the type of prize to them until the day of the ceremony, so this had no effect on the results. The third group of students constituted the control group.
Teachers and head teachers of the students in both of the treatment groups were asked to give them periodic reminders of the goals they had set throughout the school year (February to November 2016). In addition, the students were given a systematic reminder of their goals in August 2016, and the endline tests were administered in November 2016. The main outcome of interest for comparing the three groups was the endline test score for each student, after controlling for the baseline test score. In addition, the effect of the two approaches on student effort was also assessed as a secondary outcome. Effort was measured mainly using students' self-reports of how much time they spent studying but also using principal components analysis to construct an "effort index" based on multiple survey questions.

Treatment Group 1
Students estimated their performance on a test and set goals for a future test.

Treatment Group 2
Students estimated their performance on a test, set goals for a future test, and received awards if goals were met.

Control Group 3
No interventions used

WHAT WERE THE RESULTS?
The evaluation found that personal goal-setting had no discernible impact on student achievementwith or without non-financial incentives. Neither goal-setting alone nor goal-setting combined with non-financial incentives had a significant impact on student achievement. The differences between the test scores of students in the two treatment groups and the scores of those in the control group were positive, ranging from 0.03 to 0.07 standard deviations in Math and English, but they were small and insignificant (Table 1).
However, goal-setting had significant positive effects on students' effort in school, regardless of the existence of non-financial incentives. Students who did this goal-setting exercise were 4 percentage points more likely to spend at least 30 minutes studying per day outside school, which was the primary outcome related to student effort ( Table 2). Adding the non-financial incentives to the goal-setting exercise did not lead to any additional student effort. The extra 30 minutes of studying was concentrated among students with low-mid baseline test scores and those who aspired to go to college or university. However, the goal-setting exercise had no discernible impact on the composite "effort index," either with or without non-financial incentives. Furthermore, it is possible that the slight increase in self-reported effort was a result of the students being included in the two treatment groups and feeling the need to improve their school achievement or to report having made high levels of effort to the study observers. However, the REACH evaluation did not disentangle these factors from the effect of the goal-setting and non-financial incentives themselves.
The students significantly overestimated their own baseline performance and set over-optimistic goals, both of which may have contributed to the intervention's lack of impact.Only 11 percent of students were able to provide a reasonable estimate (within a 20 percent range) of their actual baseline performance immediately after taking the baseline test. Almost half of the students overestimated their baseline performance by 50 percent or more. It is possible that students' overestimates of their own baseline ability reduced the amount of effort that they thought they needed to make in order to improve (Figure 1).
Partly because of these inflated baseline estimates, the students also set overly ambitious improvement goals for themselves relative to their actual endline performance, and these goals were unrealistic even after controlling for the initial overestimation on the baseline tests ( Figure 2). Goals were defined in three different ways: unchallenging goals, challenging and realistic goals, and challenging but unrealistic goals. Thirty-nine percent of treated students set unchallenging goals which are lower than their expected test scores at the baseline (Group 1). Thirty-two percent of students set challenging and realistic goals which is within 25 percent of their expected scores on baseline test (Group 2). Lastly, 29 percent of students set challenging but unrealistic goals which are more than 25 percent above  their expected scores at the baseline test (Group 3). It is possible that the students' inability to accurately predict how much improvement was realistic contributed to their failure to regulate their own study habits and convert their increased effort into improved learning outcomes. Lastly, the interviews conducted with the students at the midpoint of the study suggest that more than 85 percent of them had difficulty understanding or even recalling their goals from earlier in the school year, which may have reduced the intervention's impact.   20  19  18  17  16  15  14  13  12  11  10  9  8  7  6  5  4  3 2 1 0

WHAT WERE THE LESSONS LEARNED?
A key insight gained from this evaluation study is that one of the preconditions of an effective performance-based incentives program is students' capacity to know how much effort is needed to achieve a goal and to self-regulate their own effort. Most students set highly unrealistic goals, which was a function of both their significant over-estimation of their baseline performance and their overoptimistic endline goals, even after controlling for their inflated baseline expectations. The results of this study suggest that having more accurate information about their own performance may help students to set and achieve more realistic goals. Students' over-estimations of their own level of achievement and their inability to self-regulate their own effort appear to be binding constraints to improving student achievement. It is therefore critical for schools to provide timely feedback to students about their performance and advice on how they can improve over time. New interventions focused on overcoming this information constraint should be developed and tested before RBF mechanisms aimed at students are scaled up. Other interventions may also be needed to address gaps in teacher skills or motivation or other aspects of the learning environment that may be preventing student effort from becoming improved learning.
This evaluation may also help to shed light on what types of non-financial incentives are most effective. While the impact of financial incentives may depend heavily on the size of the incentive, the impact of non-financial incentives may be highly sensitive to the choice of whether to reward students with recognition or in-kind prizes and what type of prize is offered. In this intervention, the students were told only that the non-financial incentives would be "a certificate and a prize," and therefore it is unlikely that the type of prize had any effect on the results. However, in cases where students will know the type of prize ahead of time, careful attention should be paid to assessing student preferences and selecting incentives that are likely to be useful to students and provide them with sufficient motivation.
It is also possible that the increased effort made by the goal-setting students in this intervention simply did not have time to improve their academic achievement in the short time period of this study, which was just one school year. Therefore, designers of future financial or non-financial incentives and other supporting interventions should be careful to ensure that the timelines that they set for measuring outcomes reflect a realistic theory of change regarding how student effort translates into improved performance. Zanzibar is a place where many students are unfamiliar and uncomfortable with the act of test-taking and receive very little feedback from teachers, so simply getting students to understand their own academic performance was a challenge. Future research may show that goal-setting increases student performance in the long run, not only by increasing effort but also by giving students a better understanding of their own ability and the link between effort and performance.
One of the preconditions of an effective performance-based incentives program is students' capacity to know how much effort is needed to achieve a goal and to self-regulate their own effort.

30%
Teacher feedback is crucial.

CONCLUSION
The Government of Zanzibar (Tanzania) has an ambitious agenda for implementing a performancebased incentives program aimed directly at students with the goals of improving their academic performance and increasing rates of secondary school completion. However, before the government embarks on this incentives program, it will be necessary to establish several preconditions for RBF mechanisms to work, which include ensuring that students have the required tools to respond to financial or non-financial incentives to improve their own performance. This study tested the impact of personal student goal-setting with and without non-financial performance incentives and found that, while setting goals increased students' academic effort in school, it had no impact on their performance. However, goalsetting may have value in and of itself and may yield other benefits to students that were not captured in this evaluation, such as higher test scores in other subjects that were not covered by this intervention. In addition, the study found that non-financial incentives did not enhance the effectiveness of goalsetting for students. This suggests that further interventions, such as providing regular performance feedback to students or increasing students' ability to convert effort into learning, are needed to establish the preconditions for effective RBF mechanisms before performancebased incentives at the student level are scaled up in Zanzibar.