Abstract
School closures due to teacher strikes or political unrest in low-resource contexts can adversely affect children’s educational outcomes and career opportunities. Phone-based educational technologies could help bridge these gaps in formal schooling, but it is unclear whether or how children and their families will use such systems during periods of disruption. We investigate two mobile learning technologies deployed in sub-Saharan Africa: a text-message-based application with lessons and quizzes adhering to the national curriculum in Kenya (N = 1.3 million), and a voice-based platform for supporting early literacy in Côte d’Ivoire (N = 236). We examine the usage and beliefs surrounding unexpected school closures in each context via system log data and interviews with families about their motivations and methods for learning during the disruption. We find that mobile learning is used as a supplement for formal and informal schooling during disruptions with equivalent or higher intensity, as parents feel responsible to ensure continuity in schooling.
Keywords
Around the world, school children and their families rely on governments to provide access to education. The official academic calendar organizes the year into periods of schooling and school breaks to recognize major holidays, provide students with time to prepare for exams, and account for agricultural needs that are still relevant in some societies. However, unplanned disruptions to schooling also arise for a variety of extenuating circumstances, such as labor disputes resulting in teacher strikes (Wills, 2014), natural disasters like hurricanes (Force, 2013), and public health crises like the COVID-19 (coronavirus disease 2019) pandemic, which has disrupted in-person instruction around the world (Lee, 2020). The frequency of these unplanned disruptions and the availability of resources to overcome them systematically vary across regions of the world. In particular, while sub-Saharan Africa is increasing educational access to students at an incredible rate (UNESCO Institute for Statistics, 2016), it has also experienced many large-scale unplanned disruptions in schooling in recent history, such as civil unrest that can last for weeks at a time (Abadzi, 2009; Gleditsch et al., 2002; Leithead, 2017; Verwimp & Van Bavel, 2014). These frequent disruptions to schooling are detrimental for student learning outcomes and for building a highly skilled workforce to spur economic development in the region.
Recognizing the vital role of education, students and their families across sub-Saharan Africa have started using educational technologies to supplement formal schooling during times of disruption. Although physical resources like classroom space are scarce, it is projected that mobile connectivity will reach more than half of sub-Saharan Africa by 2025 (GSM Association, 2018; Silver & Johnson, 2018). As prices drop, mobile devices are expanding access to learning opportunities in a region where desktop computers or laptops are economically less feasible than in the Global North (Wagner, 2014). In this research, we consider the use of mobile learning in two countries in sub-Saharan Africa, Kenya and Côte d’Ivoire, where mobile phone ownership has increased to 80% and 86% in 2018, respectively (GSM Association, 2018; Silver & Johnson, 2018). A larger share of men (86% and 90%) own mobile phones than women (82% and 82%) in Kenya and Côte d’Ivoire, respectively, and affordability and literacy remain barriers to mobile phone ownership in both contexts (GSM Association, 2018).
Despite significant increases in recent decades, current rates of educational access in sub-Saharan Africa are among the lowest in the world due in part to a shortage of physical resources (UNESCO Institute for Statistics, 2018). In 2010, Kenya had the ninth highest rate of children without education in the world (Clark, 2015). Even among students in school, only 30% of third-grade students achieved second-grade competencies; importantly, this reflects a substantial discrepancy in achievement between students of higher and lower socioeconomic status (Uwezo, 2016). In Côte d’Ivoire, by 2014, 94% of Ivorian primary school-age students were enrolled in school, but only 61% completed primary school and only 50% reached the expected literacy level (PASEC, 2014). The causes of these gaps in educational attainment and literacy are complex, include economic conditions that require family participation in agricultural labor, and are compounded by disruptions in instructional time due to school closures.
Technology is being increasingly adopted to support education in low-infrastructure contexts by providing resources to students and their families (Poon et al., 2019; Pouezevara & King, 2014; Valderrama Bahamóndez et al., 2011; West & Chew, 2014), as well as teachers (Cannanure et al., 2020; Konagai, 2020; Varanasi et al., 2019, 2020). While some devices are used exclusively in schools (Warschauer & Ames, 2010), others are designed for learning outside school to provide students the opportunity to continue learning at home and in their community (Kumar et al., 2012; Valderrama Bahamóndez et al., 2014). We study how such mobile educational technology is being used to supplement education in sub-Saharan Africa in the face of disruptions to in-person learning. We investigate this through two case studies. The first case is situated in Kenya in 2017 when schooling was disrupted by civil unrest due to a series of contested elections. We examine students’ use of a popular SMS-based study tool, Shupavu291, to continue learning outside school by analyzing 25 million logged records from more than 1.3 million students between 2016 and 2018. The second case is situated in Côte d’Ivoire, following the deployment of a new voice-based application designed to improve childhood literacy, called Allô Alphabet. During the period of deployment with students in 16 schools, an unexpected teacher strike disrupted schooling, leaving 236 students who retained access to the provided phones for accessing Allô Alphabet. We examined log data from the application and conducted interviews with families to understand their beliefs and practices around educational technology during the disruption. Our two case studies address the following research questions:
Research Question 1: How do students use educational technology during periods of disruption compared with (a) periods of normal schooling and (b) planned break periods such as school holidays?
Research Question 2: What are families’ beliefs and involvement around educational technology usage during periods of disruption in schooling compared with normal schooling and break periods?
In a complex ecosystem of formal and informal schooling, our findings across the two case studies show that students use mobile learning as a supplement for schooling during periods of disruption. We also find that students and their families engage in mobile learning during planned school breaks, which may have prepared them to use mobile learning during sudden disruptions. Parents feel responsible to ensure continuity in schooling during times of disruption but have little control due to work obligations. When family members encounter difficulties providing hands-on support during periods of school disruption, educational technology provides informal learning opportunities that are relatively accessible. By combining big data analytics with in-depth interviews, our research contributes insight across contexts into how families and students use educational technology, in general, and during periods of disruption, specifically.
Related Work
Impact of School Disruption on Learning
Significant public and philanthropic funding have been invested in ensuring that formal schooling is widely available in the Global South. According to the World Bank, global primary school enrollment has increased from 72% to 91% since 1970, but 57% of today’s out-of-school primary-aged children are in sub-Saharan Africa. 1 Beyond enrollment, global access to consistent instruction in schools may be disrupted by systemic shocks such as violent civil conflict in Burundi (Verwimp & Van Bavel, 2014), Tajikstan (Shemyakina, 2011), and Perú (Leon, 2012); school-level impacts such as teacher absenteeism or strikes (Abadzi, 2009); regional factors such as natural disasters (Baytiyeh, 2019; Rush et al., 2016); or public health crises such as AIDS (Benavot & Gad, 2004), Ebola (United Nations Development Programme, 2017), or COVID-19 (Hallgarten, 2020).
In our first case study, we discuss educational technology usage during school disruption due to political violence. In sub-Saharan Africa, nearly three fourths of countries have experienced some form of civil conflict in the past 40 years (Gleditsch et al., 2002). Among other significant economic, agricultural, and health impacts, children in regions with civil conflict are less likely to complete primary school (Verwimp & Van Bavel, 2014), with girls often more severely affected than boys (Shemyakina, 2011). In some cases, these impacts are due to family migration or displacement (Chamarbagwala & Morán, 2011), families requiring their children to leave school in order to help with unexpected labor shortages (Justino, 2011; Shemyakina, 2011), or schools voluntarily closing or reducing formal instructional time to avoid local violence (Abadzi, 2009; Benavot & Gad, 2004).
In our second case study, we discuss educational technology usage during school closures due to teacher strikes. Teachers’ industrial actions, of which strikes are a part (also including go-slows, overtime bans, and other actions (Wills, 2014)), are a feature of public education around the world, with motivations and methods situated in the unique political context in which they occur (Abadzi, 2009; Benavot & Gad, 2004; Jaume & Willén, 2019; Wills, 2014; Wong et al., 2014). These strikes, while critical for teachers to advocate for better pay and improved working conditions, often result in unplanned closures of schools comparable with the effects of natural disasters (Wong et al., 2014). Such closures due to strikes have been shown to adversely affect students’ learning outcomes (Wills, 2014) and parents’ labor as they take on increased child care responsibilities during extended school closures. Jaume and Willén (2018) report this impact in the context of teacher strikes in Argentina; in a subsequent cross-cohort evaluation of more than 1,500 teacher strikes over 30 years (Jaume & Willén, 2019), they found significant impacts on children’s future labor outcomes due to strike school closures.
Beyond unexpected disruptions, planned schooling gaps are also known to affect student learning. This phenomenon has been studied most extensively in the United States and is known as the “summer slide,” the tendency for children to lose ground in their educational journey when there is a long break in schooling (Gershenson, 2013). The faucet theory proposed by Entwisle et al. (2000) explains this slide by hypothesizing a “resource faucet” that is turned on by the government during the school year that enables all students to make learning gains. Over school breaks, however, the flow of resources stops and only those with access to alternative educational activities can maintain (or increase) those gains. We therefore explore educational technology usage during planned breaks to provide a comparison point to unplanned disruptions and to better understand whether and under what circumstances educational technologies can turn the resource faucet back on for students.
Families’ Beliefs and Practices for Out-of-School Learning
Although formal schooling is typically organized by governments, parents and families play a crucial role in making choices for how educational resources are acquired and utilized for their children, and parental investment is an important predictor of children’s learning outcomes. To understand whether and how families use educational technologies to supplement schooling, we must review family involvement in supporting education without technology. Beliefs around the role of families in learning, the practices they engage in to further their child’s learning, and other aspects of these decisions have tended to be wrapped up in a theoretical construct labeled “parental involvement.” While this research typically refers to parents as primary caregivers and educational deciders, we note that families are complex and multidimensional, and choices about a child’s education often involve a wide variety of community members beyond biological parents (Gregory, 2001; Madaio, Tanoh, et al., 2019; Maynard, 2002; Tudge & Hogan, 2005).
Prior research has outlined several dimensions of parental involvement in education, such as expectations about children’s school achievement, direct contact with schools, and limits on nonschooling activities in the home (see Fan, 2001, for a compilation of indicators). The importance placed on specific dimensions has been shown to vary across cultural groups. For instance, three types of parent involvement—communicating, volunteering at school, and learning at home—were explored in two cultures within the United States, with significant differences across groups observed in which types were valued by most parents in that group (Huntsinger & Jose, 2009).
The history of parental involvement in rural, low-income, postcolonial African communities in particular is complex (Hamunyela, 2008; Matshe, 2014; Simweleba & Serpell, 2020). One study in South Africa found that teachers perceived low-income parents to be less involved in their child’s education (Bridgemohan et al., 2005). In fact, governments have frequently denied parents opportunities to be involved in the decision-making process affecting their children’s lives, as “schools were simply not open to most of our parents” (Samoff, 1993), resulting in low adult literacy rates and limiting parents’ beliefs about their own self-efficacy in supporting their children’s learning (Lareau, 2003; Putnam, 2015). For example, parents may believe that their involvement during their child’s adolescence is more important than their early involvement in childhood, or underestimate the role of technology as an important input in children’s learning (Attanasio et al., 2019). In one study in Ghana, higher levels of parent education were associated with children’s learning outcomes; much of this association was explained by higher parental involvement among higher educated parents (Wolf & McCoy, 2019).
Despite these challenges, parents place a high value on education (e.g., Bidwell & Watine, 2014). In Côte d’Ivoire, one in three children report reading with their parents at home (Gulemetova et al., 2016), and children with parents who can read were more likely to be reading at grade level and completing primary education (PASEC, 2014). Interviews find that parents support their children’s literacy development at home in a variety of other ways, via explicit instruction of preliteracy concepts, providing educational advice, or connecting students with older family members who could provide instructional support (Madaio, Kamath, et al., 2019). Moreover, many parents chose to support their children by “providing resources for learning,” which often included pooling their resources with nearby families and paying for private tutoring at home.
Parents do not always believe that what children receive in school is sufficient. In Kenya today, many families have taken to sending their children to remedial courses outside school and during holidays, some of which have even been organized by teachers themselves (Fleshman, 2005). Despite government efforts to ban such remedial courses (Omondi, 2018), these beliefs are so ingrained in society that parents and teachers have “colluded” to continue organizing remedial courses in secret (Maina & Matara, 2018). Similarly, despite private tutoring receiving negative publicity due to inherent inequality of access between members of different socioeconomic classes, parents continue to send their children to private tutoring out-of-school, citing that “it is less expensive to pay for private supplementary tuition than to pay the costs of repeating a year” (Chui, 2016).
Educational Technology in the Global South
Technology is being increasingly investigated in efforts to bridge the gaps in formal schooling in the Global South. Two meta-analyses of educational interventions in sub-Saharan Africa demonstrate how investments in instructional technology, specifically adaptive systems, improve student learning outcomes more than funding nutritional and health interventions, reducing class sizes, or providing financial incentives for attendance (Conn, 2017; McEwan, 2015). Increasingly, such systems leverage ubiquitous mobile devices to supplement in-school instruction (Porter et al., 2016), such as in after-school programs (Kam et al., 2009), or on mobile devices used in both in-school and out-of-school contexts (Kizilcec & Chen, 2020; Kumar et al., 2012; Poon et al., 2019; Valderrama Bahamóndez et al., 2014). For instance, researchers have studied how apps on e-readers (Rhodes & Walsh, 2016) or tablets are used in schools (Phiri et al., 2014) and in homes (Uchidiuno et al., 2018).
Such educational technologies have been proposed as ways of supporting continuity of learning during crises. Baytiyeh (2019) proposed using digital tools to maintain access to learning materials and communication with instructors and peers following natural disasters, highlighting the importance of maintaining social relationships with teachers and peers and the role of parents in mitigating the mental stress of learning during a crisis. Prior work has discussed the role of technology in maintaining continuity of learning during school closures due to epidemics—for example, severe acute respiratory syndrome (Fox, 2004) and COVID-19, (Hall et al., 2020; Huang et al., 2020; Teräs et al., 2020), although with few exceptions (Angrist et al., 2020; Mhlanga & Moloi, 2020; Teräs et al., 2020) these studies focus on high-income countries with more widespread access to broadband internet and technology for online learning (Andrew et al., 2020; Fox, 2004; Hall et al., 2020; Hammons, 2017; Huang et al., 2020). Others, such as Rush et al. (2016), highlight the critical role that infrastructure plays in emergency online learning in a crisis, including the importance of using low-cost, widely accessible technologies.
Our study is unique in that it investigates the actual usage of two educational technologies across three contexts: regular school period, planned breaks, and unplanned closures. We inform the interpretation of these data with family interviews to explain our findings.
Case Study 1: Studying via Text Message During Election Violence in Kenya
In this case study, we examine how students used a popular text-message-based mobile learning platform called Shupavu291 during planned and unplanned disruptions to schooling in Kenya. In 2017, just as it did in 2008, the Kenyan presidential election caused large-scale violent protesting and major social disruption, including preventing students from safely attending school (Datoo & Johnson, 2013). This civil unrest lasted about 3 months from early August to late October in 2017. Following the victory of the incumbent party in the presidential election on August 8, 2017, the losing party claimed that there was corruption in the voting process, raising tension between the two opposing political parties. The election result was eventually nullified and a reelection slated for October 26, 2017 (Leithead, 2017).
The initial election was held during a school break so that school buildings could serve as polling stations. However, civil unrest continued for months and affected access to formal schooling and especially cram schools (commercial after-school programs for test preparation) that are widely used. Specifically, schools would adjust class times to let students go during safer hours of the day and many students stopped attending cram school during the period of unrest (J. M. Ishimwe, personal communication, July 15, 2020). The disruption occurred leading up to the standardized exams in November that mark a major milestone in students’ academic progress. Thus, with reduced access to formal schooling and cram schools, it is likely that students and families sought an alternative mode of study. This case study addresses the first research question about how students use educational technology during periods of disruption, normal schooling, and break periods.
The Kenyan education system is structured into 8 years of primary school education and 4 additional years of secondary school education. Schooling has become free of charge, but it is not compulsory. The end of primary school is marked by a standardized test administered nationwide called the “Kenya Certificate of Primary Education” (KCPE) examination. Student performance on the KCPE determines whether they are placed into state-funded, private, or “harambee” (partially state-funded) secondary schools. The end of secondary school is marked by another standardized test administered nationwide called the “Kenya Certificate of Secondary Education” (KCSE; Clark, 2015). Passing this test is required to complete secondary school education. The KCPE and KCSE exams test students’ mastery of the national curriculum. The school year is organized into three terms separated by holiday break periods; secondary school students get a half-term break in the middle of the first two terms. 2
The Shupavu291 platform 3 is widely adopted among students in Kenya, and as an SMS-based mobile learning platform, it is highly accessible even to families who possess only a basic phone. The platform and all its learning materials were designed by a group of certified Kenyan teachers who sought to create a study tool that could supplement learning in areas where educational resources are scarce. The content aligns with the stated learning outcomes of the Kenyan national curriculum for primary and secondary education. Students access Shupavu291 by dialing *291\# on their mobile phone. Shupavu291 is primarily marketed via billboards or radio ads, and through word of mouth from friends, family, or teachers. All interactions are via text message (SMS), as illustrated in Figure 1. Students navigate through menus by sending a text message with a number corresponding to a menu item from the options provided in the message they received. After registering for a specific grade level and choosing a grade-specific subject and topic, students receive messages containing compact lecture notes and an accompanying quiz consisting of five multiple-choice questions. Students answer questions sequentially and receive instant feedback on correctness with an explanation. Students can retake any quiz as many times as they like and use the “Ask a teacher” feature to get help from a teacher working with the platform.
Screenshots of text-message interactions with Shupavu291: options available from the main menu (left) and example quiz question on English grammar with automatic feedback (right).
Method
We analyze the de-identified Shupavu291 platform log data collected between January 1, 2016, and December 31, 2018. It contains 24,800,309 quiz attempts made by 1,325,666 primary and secondary school students in Grades 4 to 12 in Kenya. Demographic information beyond students’ grade level is not collected by the platform; Table 1 compares our large but nonrepresentative sample with the Kenyan student population. We first aggregate log data for each student and each week. To identify the effect of the disruption in 2017, we fit a regression model to all 3 years with fixed effects for student, week (1–52), and year (regression equations in the online Supplemental Appendix). We acknowledge that this approach cannot account for changes in the sample characteristics over time because no personal data are collected and users are identified by their mobile phone number, which may be shared among family and friends, and updated over time.
Case Study Sample Characteristics and National Student Population Statistics
Source. Kenyan student population statistics (Wills, 2014; https://www.statista.com/statistics/1135862/primary-school-enrollment-in-kenya/) and mobile carrier SMS market share (https://ca.go.ke/wp-content/uploads/2019/09/Sector-Statistics-Report-Q4-2018-19.pdf). Ivorian population statistics (https://www.worldometers.info/demographics/cote-d-ivoire-demographics/), language data (http://www.axl.cefan.ulaval.ca/afrique/cotiv.htm), socioeconomic status proxy design (RTI International, 2009), and telecom provider (https://oxfordbusinessgroup.com/overview/stronger-connection-telecoms-regulator-looking-enhance-quality-and-security-market-dominated-mobile).
Note. L2 = Language 2; SES = socioeconomic status; NA = not applicable; CM1 = Cours Moyen 1.
As predictor variables, we code each week based on whether it is during the period of disruption, examination period, or holiday period, considering variation between primary and secondary school schedules. For outcomes, we compute the number of days a student was active during a week (0–7), the number of quiz attempts, the number of courses accessed (e.g., mathematics and English represent two courses), and the average quiz performance in terms of the percentage of correctly answered questions (60% is the passing score for most quizzes). In a follow-up analysis, we examine how much the time of day that students are active on Shupavu291 changed during the period of disruption relative to other times in 2016–2018.
Findings
We first examine how students used the Shupavu291 mobile learning platform in aggregate over the year. Figure 2 shows the number of active primary and secondary school students each week for 2016–2018. Spikes in activity coincide with school breaks, especially for secondary school students. Full-term holidays for all students (18 days in April, 18 in August, and 30 or 50 in November/December depending on the grade level) and half-term holidays for secondary school students (8 days in February and 8 in June) are marked by gray areas. The 3-day KCPE and 3.5-week KCSE exam periods are marked by blue areas in November. Student activity increases in the weeks leading up to the exam period. Activity then declines sharply for primary school students after the KCPE period and remains high during most of the KCSE period for secondary school students. The period of civil unrest in 2017 is marked by two vertical dotted lines (one in early August and another in late October), which indicate the original election date and the rescheduled election date. Student activity spikes during the period of disruption in 2017 but not 2016 or 2018, which suggests that students increasingly sought out mobile learning during the period of disruption. However, only 21.73% of students who were active during the disruption were also active earlier in 2017, suggesting a large influx of new users. In contrast, for the same time frame in 2018, twice as many students (42.48%) had been active earlier in 2018.
Number of weekly active primary and secondary school students in 2016, 2017, and 2018. Gray bars: half-term and full-term holidays. Blue bar: dates of KCPE or KCSE examination. Dotted vertical lines: dates of Kenya’s eventually annulled general election (left) and reelection (right).
In Kenya, parents and teachers encourage children to attend after-school remedial coursework and private tutoring when affordable, but during the period of civil unrest between elections, schoolteachers reportedly encouraged students to return home as early as possible in order to avoid violence (J. M. Ishimwe, personal communication, July 15, 2020). An increase in Shupavu291 usage during this period, compared with the same period in 2016 or 2018, suggests that educational technology acted as a supplement for educational access during the disruption of schooling. The adoption of mobile learning may come naturally to students who already use it during holiday or exam periods when activity tends to be higher than during periods of normal schooling. Moreover, the disruption occurred leading up to high-stakes exams, which creates competitive pressure for students to find ways to continue their studies.
We use individual-level regressions to formally analyze how students used mobile learning during periods of disruption, break periods, and exam periods relative to normal schooling (Table 2). By comparing across years, weeks, and individuals using fixed effects, we can examine the impact of the 2017 disruption on mobile learning behaviors. We find that students used the platform on significantly more days during the disruption relative to normal periods of schooling (Model 1a). Active students also accessed significantly more courses during the disruption (2a), but they attempted fewer quizzes (3a) and achieved lower quiz scores on average (4a). This may reflect an influx of new or previously inactive students who explore a variety of subjects on the platform but do not engage as deeply as regular users with the study materials. However, the disruption affected the learning patterns of secondary school students differently than those of primary school students, as evidenced by the significant interaction terms in the secondary specifications: The increase in days active on the platform was twice as large for primary as for secondary school students (1b), and only primary school students accessed more courses than usual (2b). The number of weekly quiz attempts dropped less among active primary than secondary school students (3b), and while quiz performance dropped for secondary school students, it rose for primary school students (4b).
Fixed-Effects Regressions for Weekly Student Outcomes Between 2016 and 2018 to Identify the Impact of the Disruption in 2017 Overall and for Different Grade Levels
Note. Indicators for exam and holiday periods are included. For Models 1a and 1b, students with activity in a given year have zero days imputed for inactive weeks in that year; all other models are conditional on weekly activity. Standard errors in parentheses are clustered by student. FE = fixed effects; M, million..
p < 0.01. **p < .001.
To compare student activity during exam and holiday periods with periods of normal schooling, we fitted fixed-effects regressions to the same outcomes but without week fixed effects and analyzing primary and secondary school students separately (Table 3). We omitted week fixed effects because holiday and exam periods take place in the same weeks across years, and we fitted separate models for each school type because of their distinct exam and holiday schedules (see the online Supplemental Appendix for details). Relative to normal periods of schooling, we find increased activity during the exam period for primary and secondary school students in terms of days active (Model 5ab), courses (6ab), and quizzes attempted (7ab). Quiz performance drops during exam periods for primary school students (8a) but rises for secondary school students (8b). During holiday periods (including half-term breaks in secondary school), primary school students are active on fewer days (5a) while secondary school students are active on more days (5b). Both primary and secondary school students who are active during holiday weeks attempt more courses (6ab), attempt more quizzes (7ab), and score higher on them (8ab), compared with normal periods of schooling. Overall, there are strong similarities in students’ mobile learning behavior between holiday and exam periods and the period of disruption, especially for secondary school students.
Fixed-Effects Regressions for Weekly Student Outcomes Between 2016 and 2018 to Compare Periods of Normal Schooling With Holiday Periods, Exam Periods, and the Period of Disruption in 2017
Note. For models 5a-b, students with activity in a given year have 0 days imputed for inactive weeks in that year; all other models are conditional on weekly activity. Standard errors in parentheses are clustered by student. FE = fixed effects. All coefficients are statistically significant at p < .001.
The disruption may affect when students spend time studying in school or cram school compared with using mobile learning. We investigate how the disruption may have affected students’ daily mobile study schedules, based on what time of day students attempt quizzes. Figure 3 shows the distribution of median activity times during days in 2016, 2017, and 2018 for all students (distributions for primary and secondary school students look identical). Whereas student activity is concentrated after school hours in 2017 and 2018, we also see substantial activity during school hours in 2016. A Shupavu291 employee suggested that the usage pattern in 2016 likely reflects their initial marketing strategy of in-school demonstrations. The finding that most organic activity occurs after school hours in recent years is consistent with the idea that students use Shupavu291 as a low-cost alternative to cram schools. We use fixed effects regressions to identify variation in study schedules related to the disruption (Table 4; details in the online Supplemental Appendix). We find that during the period of disruption, primary and secondary school students use mobile learning a little later in the day than normal (Model 9ab), but this shift is notably smaller compared with how much later they use it during exam periods. During school breaks, secondary school students used Shupavu291 a little earlier than normal (9b). The slight change in the median time of activity, combined with the general increase in activity during the disruption, suggests that students studied more and for longer after school using mobile learning.
Distribution of median activity time in 2016, 2017, and 2018.
Fixed-Effects Regressions for Daily Student Activity Between 2016 and 2018 to Identify the Impact of the Disruption in 2017 on Study Schedules for Primary and Secondary School Students
Note. Standard errors in parentheses are clustered by student. FE = fixed effects.
p < .001.
Case Study 2: Early Literacy Learning During Teacher Strikes in Côte d’Ivoire
We examine how families in Côte d’Ivoire used a mobile learning technology called Allô Alphabet during a period of school closures due to teacher strikes. National teachers’ associations have wielded significant political influence in postcolonial Ivorian society (Woods, 1996). Throughout the 1980s, teachers mobilized to protest the national government’s rollout of an expensive educational TV program, seen by teachers and parents as an effort to defund and disempower teachers (Woods, 1996). Teachers’ resistance to investment in educational television led to the first nationwide teachers’ strikes in Côte d’Ivoire, a precursor of future organized industrial actions from Ivorian teachers.
Beginning on January 22, after a series of local teacher strikes, a coalition of teachers’ unions launched a strike in elementary and secondary schools across the country, demanding better living and teaching conditions. 4 It is difficult to get an accurate estimate of the number of institutions closed during the strike due to disputed sources of data. The teacher coalition claimed that 98% of primary and secondary schools were closed, while the Ivorian Ministry of Education claimed that “only a few pockets” were closed. 5 In March, the teacher coalition reached an agreement with the Ministry of Education. Schools were officially reopened on March 25, 2019, after 2 months of striking.
Before the strikes, Eneza Education implemented a mobile learning technology, Allô Alphabet, designed by a team of researchers specialized in human-computer interaction and reading development, to target younger children with gaps in fundamental literacy skills (Madaio, Kamath, et al., 2019) who would be unlikely to be able to read SMS messages, unlike the users of Shupavu291 (Figure 4). A 4-month study of the efficacy of Allô Alphabet in 16 schools in eight villages (Madaio et al., 2020) coincided with the 2-month-long teacher strike. While Allô Alphabet was designed to be an at-home learning intervention, it was not clear how sustained school closures would affect families’ adherence to a learning intervention. Thus, to understand how educational technologies may be used during periods of school disruption, compared with normal schooling and planned breaks, we analyze quantitative call log data from children’s use of Allô Alphabet during the teacher strike and periods of normal schooling, as well as a 2-week-long Easter holiday. We supplement these analyses of log data with qualitative data from interviews with families during and after the strike.
A parent helps his child use the voice response literacy system.
Method
Allô Alphabet uses an interactive voice response (IVR) system to deliver voice-based lessons and quizzes in order to foster early literacy skills such as phonological awareness, and later introduce letters via SMS accessible on low-cost mobile devices (Madaio, Kamath, et al., 2019). The system provides instructions, quiz questions, and feedback primarily via voice messages recorded by an Ivorian speaker, with answers input via touchtone. Here, we analyze log data from the 236 children who called to access the system, including data on the frequency, length, and timing of calls, and their performance on the lessons. We tested the data for normality with a Shapiro-Wilk test and, given the nonnormality of the data, use Kolmogorov-Smirnov tests for the analysis. We supplement these analyses with semistructured interviews with caregivers to understand their perceptions of the strike, whether and how children learn during a strike, and how and why their children would and did use Allô Alphabet during the strike. This was a purposive sample conducted with families whose children used the system at different rates (e.g., nonusage, low-usage, high-usage) from which we randomly selected participants to interview. Our final set comprised a convenience sample of those who responded to our interview request within the timeline and physical location of the study. As such, it is not intended to be a representative sample. To obtain the qualitative data, one of the authors (a human-computer interaction researcher), together with a linguistics graduate student at an Ivorian university who spoke several local mother tongues, visited 37 participants at their homes: 15 in the first month of the study and 25 in the final month of the study (three were interviewed twice). More data on the participants are included in Table 1 and online Supplemental Table A1. To understand the most salient themes in our qualitative data, we adopt an inductive thematic analysis approach (Braun & Clarke, 2006). As this is designed to be an iterative process of sense-making from data, we discussed the emerging themes and synthesized the emerging codes to arrive at theoretical saturation, or the point at which the data are fully described by the codes (Strauss & Corbin, 1990). Throughout data analysis, we conducted regular debrief sessions with collaborators from the region to resolve questions and validate emerging themes.
Findings
Educational Technology Usage During the Strike
First, we used log data to examine how students’ use of Allô Alphabet differed in the 6 weeks of the teacher strike compared with the 7 weeks of the study when school was in session, and the 2 weeks of holiday. We investigated whether there were differences in the number of times that students accessed the IVR platform (Figure 5). Although we expected to see lower usage during the strike, given the disruption to schooling (cf. Abadzi, 2009), we did not find a significant difference in the number of students calling in per week during the strike compared with the school period (Kolmogorov-Smirnov D [KSD] = 0.19, p = .99). We also did not find a significant difference in the number of calls each student made per week to the IVR during the strike than during the school period (KSD = 0.11, p = .26). Thus, roughly the same number of students accessed the IVR, and they called roughly the same number of times during strike and school periods. We also found no significant difference in the number of students who called Allô Alphabet during the 2-week holiday relative to the rest of the study (KSD = 0.8, p = .15), and we found no significant difference in the number of calls each student made per week over the holiday relative to the rest of the study (KSD = 0.11, p = .95).
Number of unique users calling the IVR (interactive voice response) during the study, during the strike, holiday, and normal schooling periods.
Next, we focus on the length of time students spent on calls, the total number of questions they attempted, and the average correctness of those questions. We find that, on average, students spent more time using Allô Alphabet during the strike compared with the school period (median = 12 s vs. 10 s; KSD = 0.04, p < .001). In addition, perhaps due to the greater amount of time spent using the system, students attempted more questions during the strike than normal schooling (median = 50 vs. 36; KSD = 0.12, p < .001). However, there was no difference in the average correctness of these questions (KSD = 0.01, p = .90). Alternatively, students spent less time on the lessons during planned holidays than when school was in session (mean = 75 s vs. 99 s; KSD = 0.04, p < .005), and perhaps as a result, students also attempted fewer questions (median = 25 vs. 36; KSD = 0.17, p < .001). Conversely, the average correctness during the holiday was higher than during the period of normal schooling (median = 1 vs. 0; KSD = 0.05, p < .01). As in Case Study 1, we wanted to understand how disruptions to schooling related to the time of day that students accessed lessons. We found significant differences in timing: students attempted questions later in the day during normal schooling compared with during the strike (median = 5:00 p.m. vs. 4:00 p.m.; KSD = 0.13, p < .001) and holiday periods (median = 5:00 p.m. vs. 3:00 p.m., KSD = 0.16, p < .001). Figure 6 shows the distribution of call times during each of the three periods. In sum, during the strike, children accessed the system more evenly throughout the day, used the system for longer durations, and attempted more questions than during the normal schooling period. In contrast, during the holiday break, children spent less time using Allô Alphabet and answered fewer questions.
Proportion of hourly IVR (interactive voice response) calls during the strike, holiday, and normal school periods.
Family Beliefs and Involvement in Learning With Technology During a Strike
In the interviews with families participating in the study, we find three major themes describing the beliefs of parents and other caregiving adults: (1) teacher strikes prompt a shift in responsibility for children’s education from the state to the family; and that (2) children should continue studying and learning during the strikes, with technology if possible; but that (3) parents’ work travel during the strike meant that the ultimate responsibility for learning (with technology or otherwise) was left to the child.
Parents strongly believed that the national teacher strikes—of which this was just the latest instance (Woods, 1996)—had negative repercussions for children’s education, with one parent saying, “I pray to God the strike must stop. My wish is that it will end, and afterwards, the children will go to school again” (P26). The most common concern that parents had was that the strike would “make children late” (P7) in their educational development. This language was echoed almost verbatim by five families (P1, P7, P18, P22, and P36). One parent elaborated, saying that during the strike, the child will “forget everything they have done” (P1)—essentially, describing the “summer slide” phenomenon. This was in contrast to some parents’ understanding of the importance of official school holidays, indicating their trust in the government plan and perhaps explaining the dip in technology use during the break period: They say you have two months of rest. The two months of rest, he doesn’t rest. He keeps doing homework. When the car rolls up there is nothing left in it, kwam kwam kwam kwam, it’s spoiled . . . The holiday is to rest. We go, we play, all of that. (P36)
Parents suggested that the strikes precipitated a shift away from the state’s responsibility for children’s learning, saying “[The strike is] a handicap for Côte d’Ivoire. Now the State of Côte d’Ivoire is turning away from the conditions of the institutions here” (P13). This parent, who was simultaneously a teacher and the president of the school-community association, believed that the state was “turning away” from the local conditions in their village. Another family member explicitly described this shift in responsibility for education from the state to the family, saying that “it is now up to the parents to be vigilant so children don’t abandon their studies” (P34).
Parents thus felt that children should continue learning during the strike. However, parents felt unable to support children’s learning at home due to their work or travel (e.g., to the cocoa fields or nearby villages). Parents noted that, before the strike, older siblings who are still in school would help the child when parents were working, but “once the strike started, the older brothers have gone to the field” (P11), removing access to the family literacy support described in Madaio, Tanoh, et al. (2019).
Some parents told us how they hoped that Allô Alphabet would allow their child to continue doing exercises even when the parents were not home. We were told how Allô Alphabet would “allow many students to study, because with the phones, when the parents are traveling, [the child] can always call and do lessons by message to study. It will allow him to progress” (P10). Others told us that they told their child to use Allô Alphabet “if you have some time to call during the day” (P28). After several weeks of using Allô Alphabet during the strike, one parent told us, She must continue to exercise when she is not at school. Normally, during summer vacation the child goes to summer lessons. Now, when we go to the field, the phone works like that . . . like she’s at the summer lessons. (P20)
According to this parent, interviewed after the strike, Allô Alphabet served the same purpose during the strike as summer lessons with a tutor.
During longer vacations, younger children accompanied their parents to the fields, to help in various ways (P11), but due to the unpredictability of the duration of the strike, parents felt that children who were still of school age needed to remain in the village (while parents were away) in case school starts again. One shared, If the teachers are on strike today, tomorrow it can stop. If I send them to the field, and there is school tomorrow, how am I going to know? Sometimes when you are in the bush the phone calls do not go through. (P19)
For some parents, when their children remained in the village without them during the strike, they hired tutors to help continue their children’s learning. Some families already paid for these tutors during the school year, while others hired them just for the strike period, if they could afford it, sharing costs with others (sometimes in groups as large as their original class size). “[The tutor] is normally his teacher at the school, and he is now the one who teaches them during the strike. My son goes to the gentleman’s house. He takes four or five students at a time for lessons” (P22). In some cases, we observed evidence that these private tutors also supervised children’s use of Allô Alphabet. In other cases, older siblings would supervise younger children’s learning with the mobile device. One older sibling told us that, during the strike, “most days, when I’m babysitting, she comes to ask me [to use Allô Alphabet]. I was surprised that she came to ask me for it” (P34). During the strike, however, many families brought older children out to the fields to help with agricultural work (P11), precluding them from filling this supervisory role.
Faced with children living on their own or with other family back in the village during the strike, many parents felt that learning during the strike—whether with Allô Alphabet or not—was the child’s responsibility. One parent told us, “It depends first of all on the child. If he is self-aware, and if he wants to study” (P17). They went on to explain this with an example of the child being responsible for their own hygiene, saying, “I’m not going to force you to wash, be clean, stuff like that” (P17). During the strike, this parent told us, “Since now there is the strike, you do not see [my son] anymore” (P17). Other parents described how children felt empowered to do what they wanted during a strike: When there is a strike, the children are all happy. They say: “Yes, we won! We can go play and do anything.” Dad cannot do otherwise. I can try to say, “Come and read.” But he will say, “There is a strike, we have been left alone, what do you want?” (P23)
When parents such as this left the village to work at the field, sometimes for days, they did not feel that they could properly supervise their children learning during a strike. In fact, at the end of the study, when we asked parents whether their children had used Allô Alphabet to learn during the strike, most parents were unable to say for sure. Although parents were unable to tell us whether or how their children were using educational technology at home during the strike, the usage data described in the above section revealed that the extent of system use was roughly equivalent during the strike and the normal school period, although the nature of that use differed.
These results suggest that, despite challenges for adults’ support for learning with educational technology during disruptions to schooling, technology could allow for continuity of learning. Given that many adults in our study reported that children are (whether the adults liked it or not) more autonomous during sustained periods of disruption, educational technology systems could provide support for children’s self-directed or self-regulated learning. These findings also suggest that there may be opportunities for educational technology to foster asynchronous support from family members who may be working while the child is learning with the system, but who still want to be involved. Finally, given the historical legacy of teacher strikes motivated by educational technology deployment (e.g., educational TV in Côte d’Ivoire), care should be taken to further understand teachers’ desires for educational technology, including its use during strikes, to ensure that teachers’ well-being and political advocacy are not subverted in service of student learning.
General Discussion
Educational technology is often touted as a vehicle for expanding access to education and overcoming barriers to schooling in low-resource contexts. The past two decades have shown that, in practice, it is a more complex enterprise to support learning with technology than to simply make it available to students (Ames, 2019; Cuban, 2018; Reich, 2020). Creating the conditions under which technology can be an effective study aid can be costly and time-consuming: procuring technology, training students, training teachers, to name but a few. In contrast, this study focused on the role of an already ubiquitous technology (i.e., feature phones) during unexpected disruptions of formal schooling. Specifically, we presented two new case studies of how mobile educational technology was used during disruptions in schooling, compared with normal schooling and break periods (Research Question 1). We further explored families’ beliefs and practices around technology during such periods (Research Question 2).
The two case studies answer these research questions from complementary perspectives. The first study took place in East Africa with 1.3 million mostly urban and suburban primary and secondary school students. The second study took place in West Africa with 236 rural primary school students. The educational technology in the first study relies entirely on text messaging, while the second is based on an IVR with text messaging. The unplanned disruption is due to an outbreak of election violence in the first study and an organized teacher strike in the second. Both case studies allowed us to draw comparisons to holidays as planned periods of school disruption, such as in the summer slide literature, and to examine the role of technology with respect to informal schooling like cram schools and tutoring at home. While the Kenyan case study lacks a qualitative understanding of student experiences with technology during the disruption, it features one of the largest samples in the mobile learning literature. The Ivorian case study involves a smaller sample, which does not permit a formal econometric analysis, but its on-the-ground interviews provide a rich understanding of the student and family experience. Taken together, the case studies therefore complement each other to strengthen the empirical support for our conclusions.
We found strong evidence that educational technology provided opportunities for continuity of learning by supporting formal schooling in both case studies; in other words, educational technology appears to have turned the “resource faucet” back on for students (Entwisle et al., 2000). At least two contextual factors are likely to have contributed to this outcome. First, the barrier to using the technology was extremely low: Text messaging is affordable in Kenya, and families incurred no cost with the IVR in the Côte d’Ivoire study. Second, students and their families had reason to trust the application: Shupavu291 was already widely used, and Allô Alphabet was provided by a team of Ivorian and American researchers, in partnership with local village leaders and the Ivorian Ministry of Education. The ease with which students and their families were found to adopt mobile learning during the disruptions speaks to its accessibility. Students increasingly used mobile learning during the disruption and school breaks in Kenya, while students in Côte d’Ivoire continued using mobile learning during the disruption and to a lesser extent the school break. It demonstrates the willingness of students and their families to use educational technology in order to supplement schooling when needed.
Our research questions focus on the disruptions to formal schooling, but in the process of conducting this research, it became apparent that educational technology interacts with a more complex educational ecosystem. Many families in Kenya and Côte d’Ivoire hire private tutors, send their children to cram schools, or provide their own familial instruction (Chui, 2016; Fleshman, 2005; Madaio, Tanoh, et al., 2019; Maina & Matara, 2018; Omondi, 2018). This system of informal schooling was disrupted during unplanned school closures due to either the risk of violence (in Kenya) or ongoing agricultural demands (in Côte d’Ivoire). The case studies provide suggestive evidence that educational technology can supplement existing systems for formal and informal schooling, depending on the type of disruption and prevailing norms around informal learning.
Educational technology use during break periods, such as mid-term and holiday breaks, offered a reference point for interpreting use during periods of disruption. Families who were already using learning technologies during planned breaks may be better positioned to leverage those technologies during sudden disruptions. Meanwhile, such technologies may be better able to support learning during the unpredictable conditions and duration of periods of disruption than existing informal systems such as cram schools, private tutors, and family instruction. Prior research on the summer slide indicates that school disruptions correspond to larger learning losses among students in upper grades (i.e., secondary) than lower grades (i.e., primary), in addition to variation by school subject and student demographics (Quinn & Polikoff, 2017). Findings from the case study in Kenya show that students in upper grades had a smaller increase in engagement during the disruption than students in lower grades, which suggest that educational technologies may have a different role in mitigating learning loss between primary and secondary students. Thus, future research should specifically examine different usage patterns and impacts on learning outcomes across primary and secondary school students.
Educational technology assumes a particular role in the ecosystem not only because it tends to be more affordable than private tutoring or cram school, but also, as we saw in the Ivorian case, because it enables more autonomous learning in situations where children may not have access to a tutor or a family member who could provide instruction. Our interviews with Ivorian parents highlight that families wish to be involved with their children’s mobile learning, but they may not have the time or the digital literacy to achieve this. We therefore identify the need for engaging families in their children’s educational technology use as an area ripe for further research.
We present two case studies to complement the limitations of one with the strengths of the other. Both studies are observational accounts centered on exogenous shocks to the education system; we thus cannot make unbiased causal claims, nor do we have evidence of learning gains after using the systems (details about Case Study 2’s original design are included in the online Supplemental Appendix). These exogenous shocks are a common feature of many educational ecosystems around the world. While they are obstructive to randomized controlled studies designed to assess the efficacy of educational technology, they offer a window to study how families ensure continuity of student learning. To assess the impact of educational technology on student outcomes, future research could explore the possibility of linking mobile learning records to standardized test results, such as the KCPE or KCSE in Kenya.
Conclusion
In a world where schools are well-resourced and instruction is not subject to disruption, families would receive sufficient academic support from formal schooling. However, the reality in many low-infrastructure contexts, and especially in sub-Saharan Africa, is a different one. Schools are frequently disrupted, there is a shortage of teachers, and families are already resorting to a wide range of informal schooling options, some of which teachers and families may find in conflict with their own educational and professional goals. Educational technology appears to be entering into a complex ecosystem of informal learning that exists as a resource to families. Our findings show that educational technology offers them much-needed support during times of school disruption, but when, where, and for whom it is effective compared with formal schooling or other types of informal schooling remains an open question.
Supplemental Material
sj-docx-1-ero-10.1177_23328584211014860 – Supplemental material for Mobile Learning During School Disruptions in Sub-Saharan Africa
Supplemental material, sj-docx-1-ero-10.1177_23328584211014860 for Mobile Learning During School Disruptions in Sub-Saharan Africa by René F. Kizilcec, Maximillian Chen, Kaja K. Jasińska, Michael Madaio and Amy Ogan in AERA Open
Footnotes
Acknowledgements
We thank our Ivorian research partners, Fabrice Tanoh, Hermann Akpe, and Anicet N’Goran, for their assistance in the data collection and interpretation process. We also thank Eneza Education for providing archival data access to enable the Kenyan case study.
Notes
Authors
RENÉ F. KIZILCEC is an assistant professor of information science at Cornell University. He studies technology in education, online learning, and scalable interventions to support students. Contact email:
MAXIMILLIAN CHEN is a computer science and statistics major at Cornell University. He studies patterns of student engagement with mobile learning.
KAJA K. JASIŃSKA is an assistant professor of applied psychology and human development at University of Toronto. She studies the neural mechanisms that support language learning.
MICHAEL MADAIO is a PhD graduate from the Human-Computer Interaction Institute at Carnegie Mellon University and a postdoctoral researcher at Microsoft Research. He studies how codesign can yield more equitable data-driven technologies.
AMY OGAN is the Thomas and Lydia Moran Associate Professor of Learning Science at Carnegie Mellon University. She studies ways to make learning experiences more engaging, effective, and enjoyable with educational technologies.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
