Learning Poverty at the Local Level in Colombia

The Policy Research Working Paper Series disseminates the findings of work in progress to encourage the exchange of ideas about development issues. An objective of the series is to get the findings out quickly, even if the presentations are less than fully polished. The papers carry the names of the authors and should be cited accordingly. The findings, interpretations, and conclusions expressed in this paper are entirely those of the authors. They do not necessarily represent the views of the International Bank for Reconstruction and Development/World Bank and its affiliated organizations, or those of the Executive Directors of the World Bank or the governments they represent.


Policy Research Working Paper 10498
This paper extends the concept of learning poverty to provide local-level estimates of the share of children at age 10 who can read and understand a simple text in Colombia. The learning poverty indicator combines the share of children who are out of school and thus schooling deprived with the share of those in school who are learning deprived based on reading tests. Local-level estimates illustrate the immense gaps in learning poverty across municipalities in Colombia in a readily interpretable form. Learning poverty rates in some Colombian municipalities are below 20 percent-the average among high-income countries-while in others, rates exceed 90 percent-the average in Sub-Saharan Africa. High learning poverty rates at the local level are associated with high levels of multidimensional poverty, a large population share of ethnic minorities, and a history of conflict. The paper also shows that the rate of learning deprivation is 60 percent is public schools versus 30 percent in private schools and that reports from school principals identify large gaps between public and private schools in educational inputs. These results highlight the need to enhance foundational skills in public schools in Colombia.
This paper is a product of the Human Development Global Practice. It is part of a larger effort by the World Bank to provide open access to its research and make a contribution to development policy discussions around the world. Policy Research Working Papers are also posted on the Web at http://www.worldbank.org/prwp. The author may be contacted at gdemombynes@worldbank.org.

Introduction
Measures of "learning poverty" at the global and national levels were first introduced by the World Bank and UNESCO in 2019 and updated in 2022 (João Pedro Azevedo 2020). As of 2019, 57 percent of 10year-olds in low-and middle-income countries (LMICs) were unable to read and understand a simple text. Although comprehensive data post-COVID is not yet available, a wide variety of studies have found profound learning losses as a consequence of school closures during the pandemic (see review in Schady et al. 2023). Based on these studies, learning poverty globally is projected to have reached 68-71 percent in LMICs in 2022 (João Pedro Azevedo et al. 2022).
Learning poverty is a hybrid measure which incorporates data on both school enrollment and assessments. This makes it superior to simple enrollment measures, as well as test data, which only reflect the performance of those who are enrolled. The learning poverty concept and estimates have proven highly useful to draw global attention to the deep learning crisis that afflicted much of the world even before the pandemic as well as the extent of learning losses. Reducing learning poverty has become a clarion call of the World Bank, UNESCO, UNICEF, and the rest of the global education community.
National-level learning poverty estimates have the limitation that they conceal within-country variation. The dispersion of education outcomes is likely to be large in some countries, particularly those like Colombia with high levels of inequality, territorial dispersion, and decentralized education systems. Colombia has among the highest levels of income inequality in the world with a Gini of 51.5 and is characterized by a center-periphery dynamic, with economic activity and national policy focused substantially on Bogotá and a few other large cities. Rural parts of the country and the Pacific Coast region in particular have much higher poverty rates than the urban center. While levels of violence in most major urban centers have plummeted from levels of the 1990s and early 2000s, remote rural areas still are affected by conflict, which has limited the reach of the state and education in those areas. The education system is decentralized, with decision-making principally in the hands of state-level authorities.
To better understand the distribution in education outcomes across Colombia, we extend the learning poverty measure to estimate local-level learning poverty. Uniquely among countries in Latin America and the Caribbean, Colombia has conducted national level learning assessments which include all students. This data, combined with primary enrollment data from national census data makes it possible to produce learning poverty for all 1,122 municipalities in Colombia and consider what local characteristics are associated with learning poverty.

Definition of Learning Poverty
Following Azevedo (2020), learning poverty as the percentage of 10-year-olds who cannot read and understand a short passage of age-appropriate material-in other words, those who are below a "minimum proficiency" threshold for reading. This measure can be defined as the union of schooling deprivation and learning deprivation. A child is considered schooling-deprived (SD) if he or she is of primary school age and out of school. The dimension of learning deprivation (LD) applies only for children in school. It identifies those pupils who are below this minimum proficiency level for reading, as measured in standard learning assessments. The learning poverty measure combines the two dimensions in a single indicator.
The learning poverty measure can be expressed with the following measure: Where: = Learning poverty.
= the schooling deprivation dimension, which captures the share of children of primary-school age who are out of school.
= the learning deprivation dimension, which captures the share of children at the end of primary who are below the minimum proficiency level for reading. For the global country-level estimates of learning poverty, schooling deprivation is measured using the Adjusted Net Enrollment Rate (ANER) for primary school, taken from the UNESCO Institute of Statistics (UIS) database. The adjusted net enrollment is defined as the number of pupils of the school-age group for primary education, enrolled either in primary or secondary education, expressed as a percentage of the total population in that age group. (The ANER differs from the Net Enrollment Rate, which excludes from the numerator children who are of primary age but have advanced to secondary school.) The learning deprivation measure is based on the Global Alliance to Monitor Learning, which defines the following minimum proficiency level in reading at the end of primary: "Students independently and fluently read simple, short narrative and expository texts. They locate explicitly stated information. They interpret and give some explanations about the key ideas in these texts. They provide simple, personal opinions or judgements about the information, events and characters in a text." (UIS and GAML 2019) This definition is operationalized in terms of specific minimum standards on various international assessments administered to primary-age students. These include PIRLS, TIMSS, LLECE, PASEC, and SACMEQ, which are all administered on a sample basis in each participating country. For a small number of countries national assessment data was used. Data from these various assessments was aggregated to assemble a set of global country-level learning poverty estimates.

Data and Application to the Local Level in Colombia
National-level learning poverty estimates were produced for Colombia as part of the global work. Table 1 shows the results for learning poverty and its two components in the two rounds of estimates. These are based on the Laboratorio Latinoamericano de Evaluación de la Calidad de Educación (LLECE), a regional assessment administered in 16 countries in Latin America that was carried out in 2013 and 2019. These estimates show a decline in schooling deprivation coupled with an increase among those in school who are learning deprived. In other words, both enrollment and the share of students failing to meet basic standards in reading increased. Estimation of local-level learning poverty for Colombia in this paper follows the methodology of the global estimates but using different datasets that have complete coverage within the country.

Schooling Deprivation: Identifying out-of-school children
The schooling deprivation component was calculated using 2018 Colombian population census data. Specifically, the adjusted net enrollment rate was calculated for the primary age population (ages 6-12) for each municipality using the publicly available microdata. The adjusted net enrollment rate is the share of children in this age group who are enrolled in either primary or secondary.

Learning Deprivation: Identifying reading proficiency
The learning deprivation component was calculated using data from the language portion of the Saber 5 assessment administered to all 5 th grade students in 2017. Saber is the system of standardized national exams administered at 3 rd , 5 th , 7 th , 9 th , and 11 th grade. Coverage for the exams has changed over time, and only in 2017 were the exams administered at all levels to all students nationally. (Currently only the 11 th grade exam is administered to all students, while exams for other grades are conducted on a sample basis.) Individual-level results from the Saber 5 language exams are categorized into four different levels of performance (ICFES, n.d.). The lowest level is "insufficient", defined as follows (all definitions are translated from the Spanish original): The average student placed at this level does not pass the least complex questions of the test.
A "minimal" performance is described as follows: The average student at this level manages to do a non-fragmented reading of everyday texts and habitual; recognizes their surface structure and achieves a specific understanding of parts of the text (sentences, paragraphs). In familiar situations of communication, he or she foresees textual plans attending to the demands of topic, purpose, intention and type of text; identifies the possible interlocutor, reviews and corrects short and simple writings, following basic rules of sentence cohesion.
The next highest level, "satisfactory" is as follows: In addition to achieving what was defined in the previous level, the average student at this level overcomes the superficial comprehension of short and simple texts of a daily nature, understands their global content; accurately recognizes the subject; categorizes, deduces, and infers information; manages to identify global functions and relationships and characterize the characters.
He uses language that is not exclusively familiar. In daily communication situations that require a certain formality and precision in the message, he is able to identify statements that are not adequate to fulfill a purpose, the sequences that ideas should have, rhetorical resources or relevant speech acts, and repeated ideas in a text.
Finally, "advanced" performance is defined in the following terms: In addition to achieving what is defined in the two preceding levels, the average student of this level achieves a broad understanding of short and simple texts of an everyday nature and relates its content with information from other sources; makes inferences of medium complexity about a part or all of the text; infers implicit information from parts of the content; defines words from the content; explains the relationships between parts, the purpose and intent of the text.
She can judge the content, the use of rhetorical resources and the form of the texts. Faced with situations of unusual argumentative communication, makes use of semantic, syntactic strategies and pragmatic to think or review the writing of a text looking for unity and cohesion.
For purposes of the local-level learning poverty estimates, a minimum proficiency level in reading was defined to be equivalent to be the "satisfactory" category described above. This was done because the "satisfactory" definition corresponds most closely to the global standard. Table 2 shows a summary of the municipal-level results for learning poverty as well as the two components. The school deprivation percentage is low-7.1 percent using the weighted mean-reflecting the country's success in expanding primary school enrollment. However, the learning deprivation measure is high, at 57.7 percent using the weighted mean. Overall learning poverty at the national level using this definition is 60.5 percent, higher than the 2019 estimate of 51 percent based on the LLECE data. This difference could reflect differences in sampling coverage of the LLECE as compared to the Saber 5 census coverage and/or differences in the precise definition of learning poverty.  Figure 1 shows the distribution of school deprivation, learning deprivation, and learning poverty. The figures show that while very few municipalities have rates of school deprivation above 10 percent, there is a wide distribution of learning deprivation, which translates into a similar distribution of learning poverty.

Figure 1: Distribution of Learning Poverty and Components
Source: Author's estimates based on 2018 Colombia census and 2017 Saber 5 data.

Learning Poverty
Source: Authors' estimates based on 2018 Colombia census and 2017 Saber 5 data.
Figures 2 shows maps of the three measures. Overall, the maps display the gaps in quality education between the center of the country and the periphery. Notably, the distribution of the two subcomponents of learning poverty is quite different across geography. In many places including much of the Caribbean coast area, levels of school deprivation are low but learning deprivation is high. In other words, in those places nearly all children are in school, but learning among those in school is low. The Pacific Coast and the eastern part of the country are high in levels of both school and learning deprivation.
Next we consider what factors at the municipality level are associated with learning poverty. Figure 3 shows a scatter plot of learning poverty versus multidimensional poverty, as calculated by the country's national statistical office using 2018 population census data, with a quadratic line fit. The correlation between the two measures is 0.59. Notably nearly all municipalities with very high (over 75%) multidimensional poverty also suffer from extreme levels of learning poverty. However, among places in the middle in terms of multidimensional poverty, there is wide variation in learning poverty outcomes. Table 3 displays results from a simple OLS regression of learning poverty rates on municipal characteristics. In the regression context, learning poverty is higher in places with greater multidimensional poverty and larger population shares of Afrocolombians and members of indigenous groups. Learning poverty is also greater in municipalities designated as PDETs (Programas de Desarrollo con Enfoque Territorial) due to a history of poverty and conflict. Controlling for these variables, learning poverty is greater in places with higher levels of GDP per capita and lower in rural areas and has no significant association with a measure of how well managed the municipal government is.

Figure 3: Learning Poverty vs Multidimensional Poverty by Municipality
Source: Authors' estimates based on 2018 Colombia census and 2017 Saber 5 data. We also consider how learning poverty and its components compare among the country's 13 largest cities and between those cities and the rest of the country. Results in Table 4 show that while there is little difference in schooling deprivation between the cities and the rest of Colombia, learning deprivation and learning poverty are both much higher in places other than those cities. Table 5 shows the learning poverty index, the multidimensional poverty index, and the population of primary age children in those cities. Cartagena has the highest learning poverty rate of the cities-67 percent. But even Bogotá-the wealthiest city and the national capital-has a learning poverty rate of 45 percent.  Finally, we consider how learning deprivation varies between public and private schools. (It is not possible to calculate learning poverty by school, since the measure incorporates the fraction of students who are out of school.) Overall, learning deprivation rates are 60 percent in public schools and 30 percent in private schools. Figure 4 shows that these differences are remarkably consistent in each of the major cities, where the bulk of private schools are found. Even in the cities with the best-performing public schools, close to half of students at those schools are learning deprived.  12 The gap in learning between public and private schools has a number of causes, depending on the point of perspective. In the big picture, the gap reflects the historical failure of the country to develop a highquality public school system, and the parallel growth of private schools which cater largely to the country's elite (Cárdenas, Fergusson, and García-Villegas 2021). More directly, low levels of learning in public schools are a consequence of many factors, including a lack of resources, weak administration at the local level, the lack of a national curriculum and textbooks, and ineffective teaching methods (World Bank 2022a). Figure 5 supplies one new perspective on this comparison from secondary school principal reports of factors they view as hindering instruction. Over 90 percent of public-school principals pointed to deficiencies in educational material and physical infrastructure as hindrances. Far fewer private school principals made such reports. Figure 6 compares these reports for public schools across Latin America and with the OECD (not including Latin American countries.) By each of these measures, public schools in Colombia are the weakest or second weakest in the region.  Inadequate or poor quality physical infrastructur The public-private school learning deprivation comparison is correlational. As such, the public-private school differences should be understood as an upper bound on the causal impact of attending private school on learning. Students who attend private schools on average have other advantages that may affect learning besides greater school resources. They are more likely to come from households with higher socioeconomic status, have more educated parents, have educational resources at home, and live in areas not affected by conflict. Because the Saber assessment data is available only at the school level and does not include data on individual students, it is not possible to use that data to examine how learning poverty varies with socioeconomic status at the individual level. However, the municipal-level estimates show the strong correlation between multidimensional poverty and learning poverty at the municipal level. Separate analysis with student-level data shows that socioeconomic status explains 13.7 percent of the variation in performance on the 2018 PISA reading assessment in Colombia (administered to 15-yearolds). This percentage is similar to other PISA countries; the average across all OECD countries is 12.0 percent. †

Conclusion
Global estimates of learning poverty at the national level have helped highlight the shocking fact that more than half of children at age 10 in low-and middle-income countries around the world cannot read and understand a simple text. These learning poverty estimates are now a staple of international conversations on the need to accelerate learning, particularly in the light of the massive learning losses generated by the COVID-19 pandemic.
The analysis in this paper shows that local-level learning poverty estimates, where adequate data exists to produce them, can similarly reveal the immense disparities in learning within a country. For Colombia these results show that some municipalities have learning by this measure at the level of high-income countries, while others have levels similar to those of the worst performers, with nearly all children learning poor at age 10.
Reflecting the general center-periphery economic and political dynamics of the country, learning poverty is high in areas that have high levels of multidimensional poverty, large populations of ethnic minorities, and a history of conflict. Among municipalities with multidimensional poverty rates in the middle of the distribution, learning poverty rates are highly varied. This indicates that even in non-wealthy areas achieving high levels of learning is possible. Future research should examine the correlates of success in areas of comparable socioeconomic status.
The learning deprivation analysis also illustrates the large divide between public and private schools. In part because Colombian elites send their children to private school, there is surprisingly little public discourse about learning in public schools (Joao Pedro Azevedo, Demombynes, and Wong 2023). The analysis in this paper may help to cast more of a spotlight on the urgent need to provide opportunity for learning to all Colombian children.