Investigating the effects of community-based interventions on mathematics achievement of girls from low-income households in Kenya

Abstract This paper uses difference-in-difference and multivariate analyses procedures to examine the effects of two community-based intervention packages on mathematics achievement of primary school girls from low-income urban households in Kenya. The data involved in this study were collected between 2013 and 2015 from 748 12–19 years old primary school girls residing in two major Nairobi slums of Viwandani and Korogocho. These data were part of a larger intervention study that sought to improve schooling outcomes among girls from disadvantaged home backgrounds by providing them with after-school homework support, life skill mentoring and parental counselling. Results reveal intervention effects of between 23 and 26 mathematics score points on a Rasch scale with a mean of 400 a standard deviation of 100. These effects translate to at least one school term of learning for girls in the comparison group. Effect size for one of the intervention packages was small (0.24), while that of the second package was moderate (0.40).

ABOUT THE AUTHOR Njora Hungi is a researcher specializing in education measurement and assessment. He has over 12 years work experience in research in sub-Saharan Africa. His current research interest includes understanding how learning achievement among primary school children from low-income households in developing countries can be improved. This is accomplished through generating research evidence that can be used to inform education policies in sub-Saharan Africa.

PUBLIC INTEREST STATEMENT
Existing evidence indicate that, in the last two decades many developing countries in Africa, Kenya included, succeeded in enhancing education access and improving school enrolment among all children. However, in spite of remarkable improvement in school enrolment, learning achievement remain poor in Kenya and more so among girls and children from low-income households. This study examines the effects of two community-based intervention packages on mathematics achievement among 12-19 years primary school girls from low-income household living in Nairobi slums. Results showed that the two packages-which provided the girls with after-school homework support, life skill mentoring and parental counselling-were useful in boosting mathematics achievement of the girls. The learning gains were substantial and were estimated to be in the tune of at least one school term of the girls in the intervention groups over those in the comparison group.

Introduction and literature review
The main aim of this paper is to investigate the effects of two community-based intervention packages on mathematics achievement of girls from low-income households in Kenya. Both intervention packages hoped to improve girls' schooling outcomes by providing them with after-school homework support, life skill mentoring coupled with a promise for subsidy to join secondary school Grade 1. Specific details about the two packages are provided later in the article but first some Kenyan education background is provided.
In 2003, like a number of other developing countries in Africa, Kenya introduced free primary education (FPE) policy to enhance access to education for all in the country, and more so among children from low-income families who were previously kept out of the system by schooling costs. The FPE policy quickly led to a tremendous increase in primary school enrolment in the country, from 5.9 millions in 2002 to 7.2 millions in 2003 and 9.4 million in 2010 (Ngware, Oketch, Ezeh, & Mudege, 2009;Njoka, Riechi, Obiero, Kemunto, & Muraya, 2011;Oketch, Mutisya, Ngware, & Ezeh, 2010). Even though the country has succeeded in bringing many children to school, there have been concerns that the FPE policy has exerted pressure on existing resources and negatively affected the quality of education. This is because the increase in school enrolment was not matched by increase in human and physical resources. For instance, soon after the introduction of the FPE policy, class sizes in some public schools reached three-digit numbers (Oketch & Somerset, 2010). Results from the Southern and Eastern Africa Consortium for Monitoring Education Quality (SACMEQ) studies revealed that, the quality of basic school inputs such as pupil-teacher and textbook-pupil ratios declined considerably between 2000 and 2007. In the 2007 SACMEQ study, the country scored poorly against its own benchmarks on provision of basic learning materials (exercise books, pencils and rulers), textbooks and on supply of teachers (Hungi, 2011). Others have reported findings that are consistent with the SACMEQ findings (Ngware, Oketch, & Ezeh, 2011).
Apart from the quality of physical school inputs, some studies have indicated that the quality of teachers in Kenya could also be questionable. In a 2010 study of primary schools in six districts in Kenya, Grade 6 teachers scored 61% on average in a mathematics tests covering what they are supposed to teach in Grade 6 (Ngware, Oketch, Mutisya, & Abuya, 2010). Surprisingly, some teachers in the same study scored as low as 17% and no teacher scored 100% on the mathematics test-meaning no teacher had complete mastery of what they taught (Global Monitoring Report [GMR], 2014). The results from the same study also indicated that, overall, teaching styles in primary schools were largely teacher-centred and did not promote critical or creative thinking among pupils. Another study by Ngware and his colleagues, but this time covering six major towns in Kenya, found that over one-third of primary school pupils were taught mathematics by teachers who could not sufficiently demonstrate that they understood the content and teaching knowledge of the subject (Ngware et al., 2013). Moreover, the same study showed that teacher experience was not matched by better pupil scores-with long serving teachers tending to post poorer pupil scores than newly employed teachers. GMR (2015) argued that these findings were indications that, overtime, long-serving teachers in Kenya lost teaching skills (perhaps due lack of in-service training), or lost interest in teaching altogether.
Consequently, scholars have expressed concerns over declining quality of education under the FPE policy and debatable quality of inputs, which has resulted into many children going through the school system without acquiring basic skills needed for day-to-day living or to pursue secondary education (Glennerster, Kremer, Mbiti, & Takavarasha, 2011). For example, the 2013 study by Ngware et al. reported that Grade 3 pupils scored only an average of 40% on a mathematics test based on the official primary school curriculum for this grade. Likewise, only about one-half of Grade 3 pupils were reported to have reached the desired proficiency levels in both literacy and mathematics in a study carried out by the Kenya National Examination Council (KNEC) in 2009 involving a national representative sample of 7931 Grade 3 pupils (KNEC, 2010). Moreover, Uwezo East Africa (2013) found that only 29% of the pupils enrolled in Grade 3 could read and understand an English story for Grade 2; and only 32% passed a Grade 2-level mathematics test.
In the 2007 SACMEQ study that involved 4,436 Grade 6 pupils drawn across all the then eight provinces in Kenya, only 20.1% were ranked in the top two competency levels in reading literacy. Mathematics results from the same SACMEQ study revealed a more worrying situation with only 4% of the Grade 6 pupils being ranked in the top two competency levels (SACMEQ, 2010).
In terms of comparison by pupil sex, most studies carried out at the primary school level in Kenya have generally reported boys outperforming girls in mathematics but have found little or no difference between the performance of boys and girls in literacy. For instance, in the 2000 and 2007 SACMEQ studies which involved Grade 6 pupils in Kenya, boys were reported to significantly outperform girls in mathematics but the gender gap in reading literacy was not significant (Hungi, 2011;Hungi & Thuku, 2010a, 2010bOnsomu, Nzomo, & Obiero, 2005). In addition, the study by Ngware et al. (2013) found that Grade 6 male pupils slightly outperformed their female counterparts in mathematics but found the reverse was the case for literacy. However, the differences between the performance of the male and female pupils in that study were not significant for both subjects, implying no gender gap in academic performance among Grade 6 primary school pupils (Ngware et al., 2013).
Interestingly, some studies carried out in developed economies have also reported gender gaps in mathematics performance in favour of boys, while others have reported gaps in reading performance in favour of girls (see, for example, Chester, 2011;Sturman, Twist, et al., 2012;Thomson et al., 2012a). Arguably, gender gaps in performance could arise from home if parents support boys differently from girls. These gaps could also arise in school if teachers treat boys differently from girls. The gaps could also arise from society and culture that expect boys to perform well in mathematics and girls to performance well in languages. Societal expectation can negatively (or positively) impact on the boys' (or girls') attitudes towards certain subjects resulting in variation in performance between boys and girls in those subjects (Clark & Burke, 2012;Lloyd, Mensch, & Clark, 2000;Ngware, Ciera, Abuya, Oketch, & Mutisya, 2012).
Comparing results by socio-economic status background, studies in Kenya have found pupils from poor households being significantly outperformed by pupils from well-off households in school subjects and this is consistent with what is reported in other developing countries under similar context (see, for example, Hungi & Thuku, 2010a, 2010bKNEC, 2010;Ngware et al., 2013;SACMEQ, 2010). In the 2007 SACMEQ study, for instance, over three times as many pupils from the top socio-economic status (SES) quartile were ranked in the top two competency levels in reading literacy in Kenya (40.6%) than pupils from bottom SES quartile (11.9%).
From the available evidence, it is clear that learning outcomes are wanting in Kenya, and this could be more so in mathematics, especially among girls from poor households where children have fewer opportunities to learn and less is expected of girls. It is in this regards that the two community-based intervention packages, whose effects are examined in this paper, were piloted in two slums in Nairobi with a hope of improving girls' schooling outcomes-namely, learning achievement and transition to secondary school.
Examining the effects of the packages in specific mathematics content areas, and Bloom's cognitive domains, will provide information on which specific areas (if any) are the packages effective. Research evidence regarding variations of pupil performance by specific subject areas and specific cognitive domains in low-income countries is scarce. However, some evidence from developed countries has indicated that pupil performance could vary substantially by specific subject area as well as by specific cognitive domain (See, for example, Sturman, Twist, et al., 2012;Surgenor, Shiel, Close, & Millar, 2006;Thomson et al., 2012bThomson et al., , 2012c. For instance, an Irish study by Surgenor et al. (2006) revealed that mean score of Grade 4 pupils on mathematics items related to data (69%) was higher than their mean score on mathematics items related to measurement (49%). The same study found that pupils performed better on mathematics items involving less challenging cognitive skill of understanding and recalling (62%) when compared to their performance in more challenging cognitive skill of applying and problem-solving (48%). It is likely that the intervention packages examined in the current study could have differential effects across different mathematics content areas, and across various mathematics cognitive domains.
The rest of this article is structured as follows. Section 2, the setting of the study is described. This is followed by two Sections 3 and 4, in which the intervention packages involved and the methods used in this study are covered including the sampling procedures, data collection and the analytical techniques. The final two Sections 5 and 6 included cover the results of the analyses, discussions of the results and conclusions.

Setting of the study
The study was carried out in two informal settlements (slums) in Nairobi, namely Korogocho and Viwandani. Korogocho slum occupies an area of 0.9 km 2 and it is located 11 km from Nairobi's central business district. This informal settlement has 12,909 households. Most residents operate small informal income generating activities as wage employment is difficult to come by. Viwandani occupies an area measuring 5.7 km 2 , and has 17,926 households. It is located within the industrial area part of Nairobi, about 7 km from Nairobi city centre (Kenya National Bureau of Statistics, 2010).
These two slums are characterized by overcrowding, high levels of insecurity, poor and inadequate housing, poor accessibility, poor sanitary and water quality and inadequate social amenities. In addition, communities living in these slums have low access to basic services like health care and education (Ochako, Wawire, & Fotso, 2011). Furthermore, education levels of adults living in these two slums are generally low. Analysis of the distribution of the population aged 15 years and above shows that 6% have no education at all, and only about 35% have attended at least secondary school. Put in other words, nearly two-thirds (65%) of the adults never attended secondary school. Schools serving communities living in the two slums are characterized by low levels of learning resources and poor infrastructure such as classrooms, toilets and drainage systems (Ngware et al., 2013).

The intervention
This article uses data from a larger intervention study targeting girls aged from 12 to 19 years living in Korogocho and Viwandani slums in Nairobi, Kenya. The larger intervention study sought to improve girls' learning achievement, and eventually transition to secondary school. The intervention consisted of three components, namely (a) parental counselling to sensitize parents about the importance of their girls' education, (b) subsidy to join secondary Grade 1 (known as "Form 1" in Kenya) to motivate girls to score at least 250 (out of a possible maximum of 500) points in the national end of primary school cycle examination-known as Kenya Certificate of Primary Education, or KCPE examination, (c) girls' after-school homework support as well as life skills mentoring. The afterschool homework support and life skill mentoring was carried out by girls from the same communities who had already experienced academic success. Parental counselling was carried out by experienced counsellors.
Part of the rationale for the intervention study was that addressing the weak mathematics achievement among girls from low-income urban requires use of community-based methods that goes beyond what happens in schools. The intervention study hoped to address some of the challenges that face girls living in urban slums that make them less interested in learning. For instance, most slum dwellings lack physical space for girls to complete their homework; a vast majority of these girls are involved in household chores, income-generating activities or behaviours that distract them from schooling. To make matters worse, parents of such girls are more often than not uneducated and may lack skills to motivate or support their children's education.
The study adopted a quasi-experimental design, with two treatment groups and one comparison group. Specifically, the intervention was implemented in two treatment packages-to be referred to as "T1" and "T2". Girls in T1 group received all the three intervention components, while those in T2 group received these components but minus the parental component. Girls in the comparison (C) group received the subsidy to join Form 1 at the end of the program in early 2016. By the time the data for this article were collected, a total of 67 after-school homework support sessions in mathematics, 20 life skill mentoring sessions and 9 parental counselling sessions had been implemented. The homework and life skill sessions were conducted by 19 mentors, while the parental sessions were conducted by two counsellors.

Methods
This section covers the sampling procedures, data collection, assessment tool and analyses techniques used in this article.

Sampling procedures
During the design phase of this study, each slum (Korogocho and Viwandani) was divided into three comparable zones based on density and distribution of households. In each slum, the design was such that a middle zone physically separated two zones on each side of the slum area. The middle zone was automatically allocated to the comparison group, while the other two zones were randomly allocated to either of the two treatment groups, T1 or T2.
Thereafter, all targeted girls in the two slum sites were listed. Because of limitation of funds, a girl was included in this study if she (a) belonged to a household in the bottom 40% wealth quartiles, (b) was in primary Grades 6, 7 and 8 and (c) was at least 12 years old and not more than 19 years old. A total of 1,271 girls across the two slums met these inclusion criteria and all these girls were involved in the baseline study in 2013. By design, some Grades 7 and 8 girls exited the study after they completed their primary school education at the end of 2013 and 2014, respectively, and therefore were not involved in the end-term survey in 2015.
The final sample used in this article consisted of 748 girls (404 from Korogocho and 344 from Viwandani) tracked between baseline in mid-2013 and end-term in mid-2015. In terms of treatment groups, 216 of these girls were T1 group, 303 in T2 and 229 were in comparison group.

Data collection
As mentioned elsewhere, baseline and end-term data were collected in mid-2013 and mid-2015, respectively. For each data collection round, data collectors were recruited and trained on the survey tools and the best practices as well as ethical issues during data collection. As part of their training, the data collectors were exposed to hands on experience in the use of the survey tools and procedures through role plays and pre-testing of the tools in other Nairobi slum sites not involved in this study. After pre-testing, debriefing sessions were held to ensure that the all the data collectors had common understanding of the tools and the procedures. By the end of the training, all the data collectors were confident in administration of the tools and were well prepared to collect data.
During each data collection round, the data collectors invited the targeted girls to a central place within each slum and administered the survey tools. Several measures were taken to ensure that quality data were collected throughout the data collection exercise. For example, immediately after administering the tools, the data collectors were required to check and correct their work after which they were required to submit the completed instruments to their team leaders. On their part, the team leaders were required to re-check all the instruments for errors and inconsistencies. If team leaders found doubtful information while re-checking, they were required to go back to the household to confirm the information with the person who responded to the questionnaire. In addition, members of the core research team did spot checks in some household to countercheck the accuracy of data collected and to ensure that ethical protocols were being observed by the data collectors.

Ethical issues
During this study, strict precautions were followed in order to protect the mental and physical welfare, rights, dignity and safety of all participants. For instance, all the members of the key research team were required to have undertaken and passed a research ethics course. In addition, before commencement of the study, ethical and study approvals were obtained from the relevant authorities in Kenya.
During the actual data collection, all respondents were informed about the purpose of the study, their rights to decline to participate in the study, their right to refuse to respond to items that they may find uncomfortable to answer and their rights to withdrawal from the study at any point altogether without victimization. Importantly, the research team was under strict instructions not to pursue girls who already expressed their desire not to participate in the study.
Consents were sought from all the participating girls, while signed proxy informed consent was sought from their parents. The interviews were conducted in privacy and confidentiality was upheld at all times. Moreover, only member of the core research team were allowed access to the collected information.

Mathematics test
The outcome scores presented in this article were derived from a mathematics test that was developed from the official primary school curriculum for Grades 6, 7 and 8 in Kenya. The same test was used across the three grade levels at baseline and also at end-term. Baseline test data for Grades 6, 7 and 8 were analysed concurrently using Rasch measurement techniques, and the scores were placed on a common scale with a mean of 400 and a standard deviation of 100. In addition, baseline and end-term test data were equated using Rasch measurement techniques. This means that valid comparisons of pupils' scores can be made across grades as well as between baseline and end-term. The numbers of items in the mathematics test was 44. The reliability (Cronbach α) of this test was 0.88, meaning it was within acceptable range.
Test data were also analysed using mathematics content domains (specific subject areas tested) as well as using Bloom's cognitive domains (Anderson & Krathwohl, 2001;Bloom, Englehart, Furst, Hill, & Krathwohl, 1956). In this regard, three content domains (numbers, measurement and space and data) and four cognitive domains (knowledge, comprehension, application and analysis) were considered.

Analyses
Difference-in-difference (DID) technique was used to compare the change in test scores between baseline and end-term rounds. DID technique was used because it allowed us to "difference" out unmeasured family-related or individual-related factors that may influence learning achievement such as family support, motivation, cognitive ability and attitudes towards education. Comparisons were made between each of the three groups-T1, T2 and comparison group in each test score of interest (i.e. overall, specific content areas and specific cognitive domains). The DID approach computes the difference between baseline and end-term test scores for each groups and compare the magnitudes of the changes (Shadish, Cook, & Campbell, 2002). Intervention effect sizes, which are measures of effectiveness of the intervention packages that can be compared with other alternative interventions, were calculated by dividing the difference computed using DID approach by the pooled standard deviation of the groups being compared.
In addition to the DID results, the effects of the intervention packages on the overall mathematics scores were examined using multiple regression models, taking into account the key potential intervening factors that might not have been perfectly balanced across the groups being compared, and are known or hypothesized to be predictors of learning outcomes. Three separate regression models (to be referred to as "Model 1", "Model 2" and "Model 3") were run to make comparisons across three groups of girls-C, T1 and T2. In Model 1, C and T1 groups were compared. In Model 2, C and T2 groups were compared, and T1 and T2 groups were compared in Model 3. In each model, controls were made for pupil baseline score, pupil age, grade level, site, household characteristics and household poverty index.

Results and discussions
A preliminary task before the DID and regression analyses was to estimate the average change in mathematics test scores per year between Grades 6 and 8 among the sampled girls, assuming no intervention and that the growth in mathematics learning across the three grades is linear. This was carried out in order to facilitate in the interpretation of mathematics scores in actual learning timefor instance, it would be possible to state what change in test scores among the girls in intervention groups is equivalent to say one year of learning among the girls in the comparison group.
The mean scores for mathematics across grade levels have been depicted in Figure 1 together with their linear trend line. This graph was plotted using baseline results-that is, before implementation of any intervention. As it would be expected, it is evident that the mathematics scores increased moving up the grade levels-an indication of better mastery of mathematics skills among pupils across grade levels. It is also evident that the trend in mathematics scores across these three grades was roughly linear-with about 62 score points on the scale used in this study being equivalent to one year of learning. Since primary school year in Kenya is split into three school terms of about 13 weeks each, this translates to an increase of 20.7 score points per school term or an increase of about 1.6 score points per week of schooling assuming a linear increment.

Difference-in-difference results
DID and effect size results are displayed in Table 1 for the overall mathematics test scores as well as for the sub-test scores as defined by specific mathematics content areas and cognitive domains.

Effects of the intervention packages on the overall mathematics achievement
For the overall mathematics scores, results indicate that the DID between T1 and C groups (34.7 score points) was positive and statistically significant at the 5% level, while that between T2 and C groups (24.4) was also positive and significant at the 5% level. This means that the two intervention packages were useful in improving the overall mathematics achievement of the girls. However, the DID between T1 and T2 (10.2), though positive, was not statistically significant-meaning that the two treatment were of about the same effectiveness. Put in other words, in terms of improving mathematics achievement, it did not matter which intervention package a girl received.
In terms of actually learning time, for the overall mathematics scores, DID of 34.7 (T1) and 24.4 (T2) score points translates to about 22 and 15 school weeks of learning for girls in the comparison group, respectively. These are indeed substantial effects because they translates to actually learning Grade 7 Grade 8 Mathematics score (mean=400; stdev.=100)

Mean
Linear (Mean) time of about 1½ school terms for T1, and at least one school term for T2. The effect size for T1 was 0.40, while that for T2 was 0.28, meaning that the two intervention packages had small to moderate impact on mathematics achievement (Cohen, 1969). The 95% confidence intervals presented in Table 1 indicate that the effect sizes for both T1 and T2 are statistically significant at the 5% level.

Effects of the intervention packages on the specific mathematics content areas and cognitive domains
The DID results in Table 1 further indicate that, both intervention packages were also generally effective in improving achievement in specific mathematics content areas, especially in the content areas of measurement (T1) and, space and data (T1 and T2). Effect sizes for specific mathematics content areas for T1 and T2 generally ranged from small to moderate. Notably, the effect sizes in the content area of space and data were quite large, especially for T1 (0.58).
In regards to the four mathematics cognitive domains, there were clear advantages associated with being in T1 group when compared to being in the C group-even for challenging cognitive skill such as application and analysis. For instance, T1 effects on analysis skills was 39.6 score points, and this is substantial because it equates to a gain of 25 school weeks (or at least half year of schooling) over the girls in the comparison group.
Likewise, there were learning benefits associated with receiving the second treatment package compared to not receiving any intervention, especially in the cognitive domains of comprehension and application. Moreover, results show that girls in the T1 group significantly outperformed those in the T2 group in the knowledge (at the 10% level) and application domains (at the 5% level)-meaning that there were some learning advantages associated with receiving the T1 package over the T2 package in these specific cognitive domains.

Multiple regression results
The results for the regression analyses are displayed in Table 2. Thus, after taking into account pupil achievement in mathematics at baseline and other key factors hypothesized to influence learning achievement in this study, there were significant differences between each of the two treatment groups and the comparison group-with the girls in T1 group outperforming those in C group by about 26 score points, and those in T2 outperforming those in C group by about 23 score points. These differences are substantial because they equate to at least 15 weeks of gain for girls in the comparison group. However, the performance of the girls in the two treatments was about the same. In general, these regression results are consistent with the results reported using the DID approach. Thus, even after taking into consideration potential intervening factors in a multivariate environment, it is clear that the two packages had substantial positive impacts on mathematics achievement.

Summary and conclusions
The objective of this article was to investigate the effects of two community-based intervention packages on mathematics achievement among primary school girls from low-income households residing in urban slums in Kenya. Data involved in the article were collected as part of a larger intervention study that sought to improve schooling outcome of 12-19 years old primary school girls living in Korogocho and Viwandani slums in Nairobi. The study adopted a quasi-experimental design, with two treatment groups and one comparison group. Girls in the first treatment (T1) group received a complete intervention package consisting of after-school homework support in mathematics, life skill mentoring, parental counselling and a promise of subsidy to join secondary Grade 1 if they score at least 250 points in their KCPE examination. Girls in the second treatment (T2) group received the intervention package but without the parental component. Girls in the comparison (C) group received nothing during the implementation. However, at the end of the implementation, the girls in the C group who scored at least 250 points in their KCPE examination were given subsidy to offset costs for joining secondary school.
The results showed that the two community-based intervention packages were effective in boosting mathematics achievement among girls living in urban slums. However, the effects of the two packages on mathematics achievement were about the same-meaning that there was no substantial learning gains associated with the parental component per se. Put in other words, this imply that those wishing to scale-up the larger study could consider dropping the parental component from the implementation.
On overall, the average effect sizes for the two intervention packages ranged from small (0.24) to moderate (0.40). These effects equate to learning gains of at least one school term of the girls in the intervention groups over those in the comparison group, which imply that the effects are quite substantial. The results further showed that the two interventions (and more so T1) were useful in boosting girls' performances in specific mathematics content areas and cognitive domains. Importantly, results revealed that the intervention packages were in general useful in boosting achievement in both less challenging cognitive skills (such as knowledge and comprehension) and in more challenging cognitive skills (such as application and analysis).
All in all, this study has demonstrated that community-based interventions can be useful in improving mathematics achievement among primary school girls from urban low-income households. Thus, in their efforts to reduce the gender inequities in learning achievement that are often reported in Kenya, the education authorities in the country could consider introducing simple communitybased interventions such as the ones examined in this article.