The Scaling Challenge

Designing an effective program is only half the battle. Keeping it effective as it scales is the next. Yet a common finding is that programs often lose effectiveness as they move beyond the tightly controlled conditions of an original trial.  

Our experience with phone tutoring in a new paper published in Nature shows another possibility: a learning curve  – a virtuous cycle where implementers learn from experience. Across successive rounds, we showed how implementation can become better over time by identifying what matters, measuring it, and helping partners deliver it. Rather than watching implementation weaken, we saw a learning curve emerge.

An effective approach to mobile phone targeted tutoring goes global 

What began as a response to COVID-era school closures in Botswana soon faced a demanding test: could the program work across different countries, education systems, and implementing partners?

An initial proof of concept in Botswana, published in Nature Human Behaviour in 2022, showed remote phone-based tutoring improved learning by 0.12 standard deviation. Within two years, five additional trials testing the replicability and scalability of the approach reached an additional 16,000 students across Kenya, Nepal, India, the Philippines, and Uganda. The core model focused on a simple platform – mobile phones – and an effective pedagogy – targeted instruction using weekly phone tutorials in foundational mathematics, with instruction matched to each child’s learning level (see Image 1).

Image 1: Components of phone tutoring: mobile platform and targeted tutoring pedagogy

Across the five countries, phone call tutorials worked everywhere, which was especially striking given five diverse country contexts and diverse delivery models, including delivery by government teachers. Youth Impact, the NGO that pioneered the program in Botswana, connected the trials as they scaled across each country, translating lessons on training, monitoring, and instruction from one setting to the next. 

Results: A Learning Curve

Results showed that effects improved over time across countries and trials, rather than decreasing. Learning rose steadily across replications. Across the five-country replication, gains increased from the earlier trials in Kenya and Nepal to 0.56 learning levels in the Philippines and 1.19 learning levels in Uganda, the final trial (see Figure 1).

Figure 1: Improvement in learning outcomes across replications. 

Source: Angrist et al. (2026), “Mobile education builds resilience during shocks in five countries”.

One of the key implementation fidelity measures was accurate targeting of instruction — the share of lessons in which tutors taught at the level indicated by a child’s previous performance. This measure improved in the same direction: from 41.1 percent in Botswana, to 50.9 percent in Nepal, 64.9 percent in the Philippines, and 81.5 percent in Uganda (see Figure 2). These trends are consistent with implementers learning from experience, rather than the usual “voltage drop” associated with scale.

Figure 2: Improvement in Targeting Across Countries

Source: Angrist et al. (2023), “Learning Curve: Progress in the Replication Crisis” 

We saw the same learning curves over time within countries, not just across countries. Accurate targeting in the Philippines rose from 58 to 81 percent as implementers gained experience. In Uganda, we saw a similar improvement from 81 to 95 percent accuracy (see Figure 3). We also see improvements over time when the program was delivered by the government, not only when delivered by NGOs.

Figure 3: Improvement in targeting within settings

Source: Angrist et al. (2023), “Learning Curve: Progress in the Replication Crisis” 

Key Drivers Behind the Learning Curve

How did we achieve a learning curve? Below we list several success factors.

Clear Core Components. First, we developed a clear understanding of what program components mattered most. While contextual factors were adapted across contexts – including language, implementing partners, and delivery systems -- several core principles were consistently adopted: a high-access, low-cost platform – a mobile phone – and an effective pedagogy – targeting instruction – assessing where a child is, and then teaching from that level. 

High Frequency Monitoring Loops. Next, we established monitoring systems that were built into delivery rather than treated as one-off evaluation exercises. Across all sites and partners, we tracked two simple indicators: whether instruction was delivered and whether it was accurately targeted. Monitoring was built into the mechanics of the program through high-frequency, low-cost teacher phone surveys conducted weekly, helping tutors decide what to teach next and managers identify where support was needed. The result was a tight feedback loop, with implementation fidelity improving over time within and across study sites. 

Cross-Site Connections: These insights didn't remain within the original site that generated them. Too often replications are disconnected from the source trial, leaving lessons on the table. Because Youth Impact played a connective role across sites, we were able to capture what was learned in one context and carry it through to the next. 

What’s Next: Continuing to Learn & Scale

The learning curve did not stop when the trials ended. Since the study, the program has continued to evolve at a greater scale in Botswana, the Philippines, Ethiopia, and India.

India illustrates a particularly large-scale rollout called Ganitha Ganaka, a government-led expansion statewide across Karnataka. The program currently reaches more than 137,000 learners with the government planning statewide expansion across Karnataka, including committing resources for teacher training and monitoring tools. Two additional states have since adopted the program.

These experiences reinforce the central lesson of the five-country study: scale and fidelity need not be in tension. With sustained monitoring and adaptation, implementation can strengthen over time, at scale, and across settings, including when delivered by governments. Scaling need not produce a voltage drop. It can, in fact, generate a learning curve.

References

Angrist, N., Ainomugisha, M., Bathena, S. P., Bergman, P., Crossley, C., Cullen, C., Letsomo, T., Matsheng, M., Panti, R. M., Sabarwal, S. & Sullivan, T. (2026). Mobile education builds resilience during shocks in five countries. Nature, 657, 689–696. https://doi.org/10.1038/s41586-026-10990-x

Angrist, N., Cullen, C., Ainomugisha, M., Bathena, S. P., Bergman, P., Crossley, C., Letsomo, T., Matsheng, M., Panti, R. M., Sabarwal, S. & Sullivan, T. (2023). Learning curve: Progress in the replication crisis. AEA Papers and Proceedings, 113, 482–488. https://doi.org/10.1257/pandp.20231009

Angrist, N., Bergman, P. & Matsheng, M. (2022). Experimental evidence on learning using low-tech when school is out. Nature Human Behaviour, 6(7), 941–950. https://doi.org/10.1038/s41562-022-01381-z