Can Understanding How Middle Managers Make Decisions Help Design Effective Results-Based Financing Mechanisms in Education?

is of central importance to the quality of teaching and learning, accounting for around one-quarter of the total impact that schools have on student learning according to some studies.1 School principals and district education officials (DEOs) are some of the most important actors in the education management system. They act as “middle managers” who provide the link between policymaking at the central level and implementation at the school level. However, there is little information on how “middle managers” make decisions and how the context and prevailing norms they work within affect their attitudes towards ‘results-based management’. To better understand decision making processes and how they are aligned with results-based financing principles, the Results for Education for All Children (REACH) Trust Fund funded research to investigate how principals and DEOs from Nepal and Bangladesh understand their roles, set priorities, and make decisions. Can Understanding How Middle Managers Make Decisions Help Design Effective Results-Based Financing Mechanisms in Education?

The quality of school management is of central importance to the quality of teaching and learning, accounting for around one-quarter of the total impact that schools have on student learning according to some studies. 1 School principals and district education officials (DEOs) are some of the most important actors in the education management system. They act as "middle managers" who provide the link between policymaking at the central level and implementation at the school level.
However, there is little information on how "middle managers" make decisions and how the context and prevailing norms they work within affect their attitudes towards 'results-based management'. To better understand decision making processes and how they are aligned with results-based financing principles, the Results for Education for All Children (REACH) Trust Fund funded research to investigate how principals and DEOs from Nepal and Bangladesh understand their roles, set priorities, and make decisions.

Can Understanding How Middle Managers Make Decisions Help Design Effective Results-Based Financing Mechanisms in Education?
The quality of school management is of central importance to the quality of teaching and learning.
School principals and DEOs are some of the most important actors in the education management system.
REACH funded research to investigate how principals and district education officials (DEOs) from Nepal and Bangladesh understand their roles, set priorities, and make decisions.
The findings suggest that middle managers are aligned with principles of results-based financing in three ways: (a) they prioritize objective data for decision-making; (b) they prioritize student learning as their main goal, and (c) they favor results based accountability for teachers. At the same time, they exhibit preferences that may hinder the application of RBF. First, they prioritize the demands of vocal parents over the needs of disadvantaged students. Second, they show low ownership over the learning outcomes of lagging students. Third, they appear unwilling to sanction low-performing teachers.
Fourth, these agents are divided in terms of prioritizing equity in inputs vs. equity in outcomes. These results suggest potential entry points in how to build the capacity, mindsets, and effectiveness of middle-management for increased effectiveness, especially around RBF.

CONTEXT
Middle managers in the education system-school principals and DEOs-exert considerable influence on how schools perform. 2 Not only do they play an important role in education service delivery, they have several characteristics that affect their decision-making role. First, they hold a position of power, but given the tightly circumscribed systems they operate within it can be hard to determine how much autonomy and discretion they truly exercise. 3 Second, partly because of the first point, this layer can also be hard to influence or incentivize. Third, based on the management and governance structures, they operate with little day to day supervision. Fourth, being middle managers, they simultaneously face policy and information demands from above; and resource, information, and personnel demands from below. This gives them both hard-toquantify challenges and influence. 4 Finally, they have been identified as 'boundary spanners'-actors within a system who perform the role of linking the organization's internal networks with external sources of infomation. Given these characteristics and their importance in determining education outcomes, it is important to understand their attitudes and beliefs, the way in which they make decisions, and how they respond to incentives.

WHY WAS THE INTERVENTION CHOSEN?
There is a growing amount of empirical research on school principals and their impact on student outcomes. 5/6/7 However, there is little reliable empirical data on district and other mid-level government officials. Studies of these kinds of officials have been conducted in India and Ghana among other countries. Their findings have shown that middle managers are motivated mainly by a range of non-monetary incentives including increased autonomy and the chance to be promoted to top-level roles and that monetary incentives can crowd out this intrinsic motivation to do good. 8/9/10 However, little of this research has focused on education, which is technically complex and particularly liable to be subject to political influence. 11 The research outlined in this evidence note helps to fill this gap by using an innovative and low-cost approach to study the attitudes, beliefs, decision-making, and motivation of both principals and DEOs. The findings reveal how these officials filter and process available information and how they are likely to respond to incentives such as those associated with RBF. It also sheds light on how they might respond to other initiatives targeted at them, such as capacity building programs.

HOW DID THE INTERVENTION WORK?
To understand the decision-making process of middle managers, a game was developed that could be played on a mobile phone and that used hypothetical real-life scenarios to reveal respondents' attitudes. The game was administered to primary school principals and district education officials in Bangladesh and Nepal. The game presented each principal and official with a series of scenarios and asked them to choose one option each time. For example, the study was interested in whether principals were more There is little reliable empirical data on district and other mid-level government officials. This evidence note helps to fill the gap by using an innovative and lowcost approach to study the attitudes, beliefs, decision-making, and motivation of those individuals.
concerned about investing money in a way that seemed equitable or in a way that maximized the gains to student learning. The respondents were asked to imagine that they had TK80,000 (US$940) to spend on one classroom. They could choose either a larger class that had achieved good examination results or a smaller class with worse results. If the principal chose to invest in the larger class, this suggested that their priority was to equalize inputs per student, indicating a desire to increase equity, whereas if they chose the smaller class with lower scores, this suggested that their priority was increasing learning.
The game was designed to avoid common biases contained in traditional surveys, which, although widely used, are not always a reliable way to collect information on people's opinions and preferences. For example, when people selfrespond, they tend to provide answers that they think are likely to be popular or well received by the interviewer (social desirability bias). Respondents may also be biased towards giving answers in which they take credit for successes but blame others for failures (self-serving bias) or that justify actions that they have taken (rationalization bias). These biases are hard to prevent and can distort evidence gathered from surveys on people's opinions. The idea behind the game was that respondents' choices would reveal their preferences and attitudes more accurately than if they were asked more direct questions.
In addition to the scenarios, the game also contained more traditional multiple-choice questions, as well as asking respondents to the extent to which they agreed with a series of statements such as, "Schools deserve more resources if the students come from poor households." The questions and scenarios measured the attitudes and beliefs of principals and DEOs about a wide range of topics including school management, the allocation of school resources, teacher accountability, student learning, and their perception about leadership. All the scenarios and questions were administered in the local language used in each country. The game and questions were administered in Bangladesh and Nepal in 2018. Altogether, 234 school principals and 220 meso-level administrators took part in the game.

WHAT WERE THE RESULTS?
How are decisions being made at the meso and school level? How likely are these decisions to follow principles of RBF? The study answered these questions in two parts. First, it identified factors that may facilitate the application of RBF principles at the meso-and school levels. Second, the study identified factors that may constrain the use of RBF.

Factors that facilitate the application of RBF
Principals and DEOs were supportive of results-based accountability for teachers (Figure 1). A majority of respondents from Bangladesh supported taking student test scores into account when evaluating, promoting, and rewarding teachers. Around 71 percent felt that student test scores should be taken into account when assessing teacher performance, 61 percent said they should be considered when choosing teachers for promotion, and 91 percent supported providing teachers with bonuses if their students performed well on official examinations.
Principals and DEOs valued objective data over opinion when making decisions (Figure 2). For RBF to be effective, results must be evaluated using factual evidence rather than subjective opinion before assigning any rewards. However, private and public sector institutions in countries at all levels of development often resist using objective measures to evaluate performance for a range of reasons including personal incentives, political influence, and a lack of familiarity with the principles of evidence-based policymaking and scientific methods. 12/13 However, the findings of this study suggest that middle managers in education value objective data. When asked how they would choose the best school or teacher of the year, 87 percent of principals and DEOs in Bangladesh and Nepal claimed that they would rely on student test scores rather than their personal opinions.  Middle managers were able to distinguish between higher-order results (like foundational skills for all) and lower-order results (like completion of curriculum).
Respondents were asked to choose one of two teachers to receive a reward. The first teacher's students could recognize all the letters of the alphabet but had completed only 60 percent of the curriculum, while the second teacher's students had completed the curriculum, but only one in five could recognize the entire alphabet. Ninety-one percent of the respondents said that the first teacher should be chosen to receive the reward, which suggests that principals and DEOs prioritize skills development over finishing the curriculum.
Principals and DEOs saw themselves as leaders with the responsibility to ensure that students learn. The researchers asked respondents in Bangladesh to choose terms from a list that best described themselves. One-third (35 percent) of them chose words that related to qualities of leadership, such as '"committed," "resilient," and "visionary" as opposed to less leader-related terms like "well-liked" or "good writer." Furthermore, they valued student learning and inclusion over other, less pedagogical parts of their jobs. Around 93 percent of respondents saw their main goal as maximizing student learning as opposed to being a leader and inspiration for the community. However, principals and district officers also saw their leadership as primarily administrative. When asked to choose the priority tasks of a good principal or DEO from a list, only 15 percent of respondents chose more leadership tasks than administrative tasks. The most cited priority task was "keep good records," and only 10 percent of respondents thought that "provide teachers with goals" was a high priority.

Factors that constrain the application of RBF
The respondents were focused on children with vocal parents (Figure 3). More than 80 percent of DEOs and school principals in Bangladesh said that they would listen directly to a parent's complaint about a test report rather than referring them to the teacher. However, this can have a negative impact if they focus on the students of the most vocal and engaged parents to the exclusion of other, typically less well-off students. The findings suggest that this could be an area of concern as 75 percent of DEOs and 59 percent of school principals said that there was little a school could do to help students learn if their parents did not seek feedback on student performance. It may be that principals and DEOs are responding to parental demands rather than to objective measures of school needs. More than 90 percent of both principals and DEOs said that a school deserved more resources if parents were demanding better education, while only 52 percent of principals and 38 percent of DEOs said that a school deserved more resources if learning levels were low. In response to a hypothetical scenario, 54 percent of DEOs and 58 percent of principals chose to spend resources on a school with a higher pass rate where parents frequently complained 76% 71% 74%

83%
Parents do not seek feedback from the teacher on student performance Parents do not have the necessary education to help their child be more successful at school Parents have too many personal/financial problems to be concerned with their child's performance at school

DEOs
School principals "There is little a school can do to help students learn if…" over one with a lower pass rate but no complaints from parents.
Principals and DEOs prioritized students with the most potential and felt they had limited ability to help all students learn. A majority of respondents in Bangladesh said that there was little a school could do to help a student learn if their parents were not educated, if the parents had too many financial or personal problems, or if the student was not well prepared in previous grades. Responding to another hypothetical reward scenario, respondents in Bangladesh and Nepal were asked to choose between two teachers to receive an award. In one teacher's class, more students passed the exam but no students received full marks, while in the other, fewer students passed but one in ten received full marks. A majority of the respondents (53 percent) chose to reward the teacher with fewer students passing but a small number getting full marks, which suggests that they put a higher priority on maximizing the achievement of a few high-performing students than on ensuring broad and inclusive learning.
Principals and DEOs were reluctant to hold teachers to high standards. Only 21 percent of respondents in Nepal and Bangladesh claimed that they would give teachers a formal warning if they had high absentee rates. More than 50 percent of DEOs in Bangladesh believed that absenteeism was acceptable if the students were given work to do while the teacher is away or if the assigned curriculum had been completed.
Principals and DEOs had inconsistent views on equalizing inputs and outcomes. The respondents in Nepal and Bangladesh were asked whether they would prefer to spend extra money on a classroom with fewer students and lower test scores or on one with more students and higher scores. If they put a high priority on ensuring equitable outcomes, they would select the first class even though it had fewer students. In fact, 54 percent chose the class with more students and better outcomes. However, when respondents were asked what advice they would give to another principal or DEO who was in the same situation, 65 percent said the reverse, that they should invest in the smaller class with the lower scores. This suggests inconsistencies in the reasoning that principals and district officials use to make decisions.
In general, principals and DEOs gave similar answers, but in a few key areas they disagreed. DEOs were approximately five times more likely than principals to choose test scores as the way to evaluate the best teacher of the year rather than their own opinion. Principals in Bangladesh were 83 percent less likely than DEOs to choose to invest in a classroom with fewer students and a lower pass rate over one with more students and a higher pass rate, suggesting that they put a higher priority on equalizing inputs than on equalizing outcomes. Principals were also much more likely than DEOs to support giving extra resources to classrooms with inadequate infrastructure over those with low learning outcomes (93 percent of principals versus 52 percent of DEOs). "I think it is acceptable for teachers to be absent as long as…"

CONCLUSIONS
The study findings highlight the importance of directly targeting middle-managers as important agents of change to promote principles of results-based financing at the front lines of education service delivery. Important decisions about how schools function are made by meso-level and school-level managers. As such, their actions can have big implications for the quality of education service delivery.
The study points to potential entry points for the use of results-based financing to strengthen incentives and improve education outcomes. The findings of the study highlight areas where decision-making by these managers are already aligned with principles of results-based financing including the use of objective information, and a focus on higherorder results. These findings point to areas where RBF mechanisms would be able to strengthen incentives for better information and improved learning outcomes. However, the study also points to issues that could impede a broader use for resultsbased financing. For example, the study finds that school principals are unwilling to sanction low-performing teachers or support students in need of remediation because they recognize that these actions fall outside of their direct control. Overall the study provides useful insights into how decisions are made by these agents that can provide a useful starting point to consider how RBF can improve decision-making and ultimately improve education outcomes.