Blog
Showing Life Opportunities: Improving outcomes in online learning at scale in High School – focus on the system
17 July 2023
Igor Asanov, Anastasiya-Mariya Asanov, Thomas Åstebro, Guido Buenstorf, Bruno Crépon, Francisco Pablo Flores T., David McKenzie, Mona Mensmann, Mathis Schulte
How to organise education online when students cannot go to school? While massive online open courses struggle from low completion rates among voluntary learners, compulsory education at school demands new approaches. Our international research team had to search for these new approaches as we decided to help students in their final years of high school in Ecuador to finish school during the COVID-19 pandemic. When the pandemic broke out in Spring 2020, we were finalising a randomised controlled trial of educational online materials from our programme “Showing Life Opportunities”, which aims to boost high-growth entrepreneurship and science, technology, engineering, and mathematics (STEM) careers in Ecuador. Under emergency conditions, we successfully scaled up our programme to cover more than 45,000 students in 1,151 schools across Ecuador. In our study we tested a set of light-touch interventions to improve students’ educational process and knowledge outcomes. A recent paper published in the Proceedings of the National Academy of Sciences (PNAS) describes what we learned from experimenting with light-touch interventions.
Working closely with the Ministry of Education of Ecuador, we recognised that, unlike the massive online open courses focusing on students, we can manage the educational process on different levels – students, teachers, and system. Thus, we decided to experiment on these levels.
- At the student level, we tested various reminders to students, but found that these had only limited impact on educational outcomes. We also sought to incentivise students with the help of a lottery for lesson completion and performance. This increased study time on the educational online platform, but did not improve performance on knowledge tests.
- At the teacher level, we had already learned from our pre-pandemic intervention that benchmarking emails – informing teachers how well their classes perform compared to others – do not seem to have an impact on educational outcomes on average. Thus, we experimented with encouragement emails that included a video about the experiences of students and teachers who had previously finished the programme, and tested increasing the salience of monitoring by sending SMS messages to teachers that their performance was being monitored, instructing them to make sure they finished the programme on time. Again, we did not observe an improvement in the educational process and knowledge outcomes.
- At the system level, listening to the feedback from ministry personnel, we understood that real-time data about students’ performance could be a key to organising the educational process remotely. We already had an online learning management system that allowed teachers to see various real-time performance metrics aggregated on classes or for individual students, including study time, lessons completed, and performance in the online exercises. The question was whether embedding the central ministry personnel (central office and heads of the educational zones) into the system would be beneficial and cost-effective, or whether allowing teachers to self-manage the educational process is preferable. Thus, we introduced an online learning management system (with individual information and weekly take-up reports) for the centralised monitoring of teachers by central ministry personnel for randomly selected schools. The remaining schools were under decentralised management, where teachers but not ministry personnel had access to the online learning management system. We found that an online learning management system for centralised monitoring drastically increased participation and performance relative to decentralised management on various program-related subject knowledge tests e.g. English, Entrepreneurial Education, Statistics and Scientific Thinking. Based on students’ study patterns recorded automatically on the educational platform, we infer that the centralised management system allowed ministry personnel to organise regular study patterns, avoiding the need for students to rush studying before the deadline or final knowledge tests.
To sum up, we iteratively experimented on different levels of organising the online educational process in schools across Ecuador. Whereas student- or teacher-level interventions showed limited impact, we found that an inexpensive (below 60 cents per student) online learning management system for centralised monitoring improves students’ performance on the knowledge tests. The size of the improvement is similar to large educational interventions in low- and middle-income countries and comparable to how much a student might learn in 71 percent of a year of business-as-usual schooling in grade 12. These findings encourage moving beyond the student level in online education interventions, focusing on a system that unites people to educate and build a better future – Juntos por la educación! (Together for education).
***
The results presented build on the more extensive “Showing Life Opportunities” programme funded by the Innovation Growth Lab, Innovation Poverty Action, Labex-Ecodec, INCHER-Kassel, and World Bank, provided under the support of the Ministry of Education of Ecuador. There is more to come. Stay tuned!