< BACK TO E&L HOMEPAGE
Learning from an Evaluation of Aga Khan Foundation’s Early Childhood Program in Uganda
June 1, 2021
June Lee
Director, Evidence & Learning
The ELMA Philanthropies
Photo: Aga Khan Foundation / Jjumba Martin
In 2017, The ELMA Foundation provided support to the Aga Khan Foundation (AKF) to improve the provision of early childhood services in three districts in Uganda, and to strengthen the ability of government district officials to coordinate and monitor the delivery of ECD services. The ELMA Philanthropies commissioned Ichuli Consulting to conduct an evaluation of the impact of the program on children’s school readiness over a two-year period (early 2018 – late 2020).
Findings
1. Children in AKF-supported schools did better in assessment of school readiness than those in comparable ECD centers.
The evaluation found that four to six-year-old children in 40 AKF-supported schools did better on assessments of school readiness overall (especially in emergent numeracy and socio-emotional skills) than those in matched ECD centers. The effect size of the program is 0.18, which is similar to effects found in other high-quality ECD interventions in low and middle-income countries. Children in both groups had similar competencies in the motor function, early literacy, and executive function domains. There were no notable gender, district, or geographical (urban, peri-urban, or rural) differences in children’s performance.
2. Teachers in AKF-supported and non-AKF supported schools were largely similar in background and teaching practices.
There were many similarities between teachers in the AKF-supported and comparison schools in both background and teaching practices. Teachers in both groups were similar in how they managed their classroom, engaged children, and used teaching materials. A few differences emerged: Even though teachers in AKF-supported centers had less teaching experience, they had more training, were more likely to receive mentorship from someone other than their head teacher, had more teaching and learning resources (i.e., readers and toys, and lesson plans and child assessment records), and were likely to use educational toys during their lessons. Teachers in AKF-supported schools were more likely to encourage equal participation of boys and girls, and more likely to have activities that promote free play. They were, however, less likely to report having their class observed by a head teacher or someone from their school management.
3. The links between AKF support and training, teaching practices, and children’s school readiness are not well understood.
Differences in training and resources between the treatment and comparison teachers may partially explain the differences in child learning outcomes, although there is not a clear pattern of better teaching practices in one group compared to the other. Because the evaluation did not include an explicit examination of AKF-supported training, it is difficult to establish links between AKF-supported training and teaching practices. In addition, further analyses are needed to explicitly examine the links between teaching practice and child outcomes.
What else we learned
In addition to understanding the impact of AKF’s program on school readiness, other lessons emerged from this evaluation that were as important as the primary findings.
1. Probe the assumptions that underlie a program and monitor for unintended consequences.
AKF largely implemented its training and mentorship model as designed, but less obvious risks can undermine sustained impact. For example, while teachers in AKF-supported schools were mentored by AKF, they were less likely to be mentored by head teachers, suggesting that AKF my have inadvertently displaced school support. The evaluation also found high mobility among teachers between classrooms and schools due to routine transfer by government. Because AKF only trained two teachers per school across multiple ECD classes, benefits derived from the training were lost once teachers were transferred. These findings surfaced as an indirect result of the evaluation, and it is an important reminder for ELMA and partners to specifically monitor for unintended consequences and probe assumptions to ensure that they continue to hold up.
2. Child attrition and teacher absenteeism could weaken the gains made through teacher training and support and should be further investigated.
The evaluation revealed an attrition rate of 38% for children in both AKF and control programs. The reasons are unclear, though anecdotal accounts suggest that parents may un-enroll and re-enroll their children in ECD programs within the same or nearby communities. Nearly 1 in 5 teachers reported missing school in the past week, suggesting significant absenteeism. These factors attenuate the effect of investments made in training and mentorship in AKF’s program and the ECD sector in general, and their reasons and impact should be better understood.
3. Future evaluations must aim to better understand mechanisms of change.
The evaluation focused on the impact of AKF on early learning and classroom practice as distinct outcomes. It did not assess which aspects of classroom practice (if any) are most predictive of learning outcomes. Such analyses are more complex but critical for understanding the mechanisms by which a partner’s model results in positive child outcomes.
For further evaluation findings and recommendations:
