Abstract
Academic behaviors such as attendance are highly associated with academic outcomes. High schools are also increasingly turning to online courses to educate their most marginalized students. In this study, I explored the extent to which enrollment in an online course improved engagement and allowed students to make course progress online outside the traditional school day by examining within-student changes in academic behaviors. Students completed their online course in fewer class periods than required to complete a comparable course in a traditional, face-to-face instructional setting. At the same time, students attended, on average, three additional days of school when enrolled in an online course as when enrolled in solely face-to-face courses, indicating a potentially positive spillover effect. Results have implications for practitioners and policy makers interested in online learning and understanding what programs might be most effective in reengaging students at risk of course failure or dropping out of high school.
U.s. school districts are increasingly turning to online courses to educate students, with lower achieving and historically underserved student populations often assigned to online versus traditional, face-to-face instruction for purposes such as credit recovery (Ahn, 2011; Heinrich, Darling-Aduana, Good, & Cheng, 2019; Watson & Gemin, 2008). The use of digital tools has the potential to improve educational outcomes by broadening access, engaging students in active learning, facilitating individualized educational experiences, and providing access to authentic, relevant learning opportunities (Bakia, Shear, Toyama, & Lasseter, 2012; Darling-Aduana & Heinrich, 2018; Selwyn, 2016). Yet, technology often does not live up to this promise, particularly for students from low-socioeconomic backgrounds (Darling-Aduana, Good, & Heinrich, 2019; Heinrich et al., 2019; Heppen et al., 2017; Hohlfeld, Ritzhaupt, Dawson, & Wilson, 2017; Xu & Jaggars, 2014).
Instead, the primary benefit to online courses for students may be access to anytime, anywhere learning (Jaggars, 2014; Levy, 2011; Watson & Gemin, 2008). Online courses offer the flexibility to earn course credit based on work completed outside of the school day or building (Collins & Halverson, 2009; Levy, 2011). This has the potential to support students in balancing school and life responsibilities by allowing students to make course progress on their own schedule (Jacob, Berger, Hart, & Loeb, 2016; Powell, Roberts, & Patrick, 2015). In postsecondary settings, there is evidence that online courses allowed students who might not otherwise pursue education to enroll in and earn degrees (Goodman, Melkers, & Pallais, 2016). Other attempts to quantify how students respond to access to anytime, anywhere learning are rare, with generalizability to secondary school populations and in settings with relatively low rates of Internet access not yet established.
Termed academic behaviors or behavioral engagement, the extent to which students go to class, do homework, and participate in learning is highly correlated with learning outcomes (Farrington et al., 2012; Fredricks, Blumenfeld, & Paris, 2004). Among academic behaviors, attendance and out-of-school studying are particularly important predictors of grades, assessments scores, and high school completion (Allensworth & Easton, 2007; Balfanz & Byrnes, 2006; Gershenson, Jacknowitz, & Brannegan, 2017; Henry, Knight, & Thornberry, 2012; Lamdin, 1996; Nichols, 2003; Rumberger, 1995; Rumberger & Thomas, 2000). Online courses also expand definitions of attendance for students with Internet-enabled devices by allowing students to earn course credit for any time spent logged into an online course at home, adding an incentive for out-of-class learning. Furthermore, 37 states, the District of Columbia, and Puerto Rico all incorporated some measure of attendance in their Every Student Succeeds Act plan (Bauer, Liu, Schanzenbach, & Shambaugh, 2018), yet relatively little is known about how changes in students’ instructional environment due to online course taking may affect attendance. The specific hybrid blended model implemented by the district examined in this study that combines in-school computer lab time with remote access outside the school day merits attention, as it may be particularly desirable for districts in light of this recent emphasis on attendance in accountability measures.
Additional work is required to document the prevalence and impact of the out-of-school use of digital resources, which may not be captured by traditional measures of effort and attendance (Darling-Aduana et al., 2019; Heinrich et al., 2019). Furthermore, prior research in the school setting examined in this study identified mixed achievement outcomes associated with online course enrollment depending on factors such as student attendance and out-of-school online course-taking behaviors (Heinrich et al., 2019). Understanding more about these mechanisms can help identify when, why, and under what circumstances online courses may facilitate improved student learning. Jackson (2018) also established that the use of measures beyond test scores, including attendance, improves the predictive power of value-added scores compared with using test score information alone. These findings indicate the need for researchers to examine nontest score as well as test score outcomes when determining program effectiveness.
This study employed a student fixed effect strategy to explore within-student differences over time in academic behaviors when enrolled in online versus fully face-to-face courses. I conducted this analysis using 6 years of longitudinal data from a large, urban school district that enrolled students in online courses primarily for credit recovery. Results have implications for school districts and policy makers interested in the use of online courses by students at risk of dropping out of high school due to poor academic performance and more generally for those interested in designing programs to engage or reengage lower performing students in school. Understanding factors associated with attendance also has practical implications for school districts in states where measures of attendance are used to assess school performance or allocate district funding (Picciano & Seaman, 2009).
Prior Research on School Absences and Online Course Taking
School Absences: A Barrier to Educational Access
School absences are associated with lower assessment scores (Balfanz & Byrnes, 2006; Lamdin, 1996; Nichols, 2003) and a higher risk of dropping out of high school (Henry et al., 2012; Rumberger, 1995; Rumberger & Thomas, 2000). To place these implications in context, students entering high school in Chicago Public Schools with eighth-grade test scores in the lowest national quartile passed more courses than students with test scores in the highest national quartile who attended only 1 week less of school per semester (Allensworth & Easton, 2007). Gottfried (2011) exploited within-family differences in attendance patterns among siblings to provide evidence of a negative association between assessment scores and school absences in Philadelphia School District. After accounting for a family fixed effect, students achieved −0.08 standard deviations lower test scores in reading and −0.10 standard deviations lower test scores in math for each additional day of school absence (Gottfried, 2011). Using value-added models, Gershenson et al. (2017) identified effects of 0.04 and 0.02 of a standard deviation increase in math and reading test scores for each standard deviation decrease in absences.
Researchers have suggested that improving students’ educational access through improved attendance may be a powerful lever to reduce current income and race-based achievement gaps, with Gershenson et al. (2017) estimating that reducing low-income student absences by 10 days a school year could reduce the income-based achievement gap by 5% to 10%. Furthermore, the negative ramifications of school absences were amplified (effect size [ES] = 0.23) among the approximately 10% to 15% of students nationally demonstrating chronic absenteeism, which is defined as missing 18 days or more of school (out of 180) in a single year (Gottfried, 2014). As 48% of the full sample and 63% of students enrolled in at least one online course missed 10% or more days in a given school year, the potential benefits of increased attendance might be even higher for the students in this study.
Behavioral Engagement in Online Courses
Behavioral engagement, including attendance and out-of-school learning, is a critical mediator to achievement, particularly in an online course setting where students, versus teachers, dictate how much time students spend logged in and engaged in learning-related activities (Darling-Aduana et al., 2019; Heinrich et al., 2019; Jaggars, 2014; Levy, 2011; Xu & Jaggars, 2014; Yukselturk & Bulut, 2007). Despite evidence that technology use can increase student engagement (Warschauer, 2006; Zhao, Lei, Yan, Lai, & Tan, 2005), the primary study that examined the effects of online course taking on engagement among high school students identified no significant differences in engagement compared with students in a traditional classroom setting (Heppen et al., 2017). In addition to not representing a robust literature, Heppen et al. (2017) relied on self-report measures and focused on cognitive versus behavioral engagement. Furthermore, the study did not specifically examine associations related to anytime, anywhere access, since the online program model was designed to be delivered primarily in a school setting.
Means, Toyama, Murphy, Bakia, and Jones (2009) identified larger learning gains in fully online and hybrid blended instructional settings when access to online content facilitated more time engaged in learning (ES = 0.46) versus replacing the time that would have otherwise been spent in a traditional classroom (ES = 0.19). Online course systems, such as the one examined in this study, might provide a mechanism for students to increase the amount of “seat time” but rely on students to initiate any out-of-school learning. Furthermore, individualizing features offered through online courses, such as self-pacing, generally do not cater to the academic and motivational realities of students who have struggled academically in traditional classroom settings (Bambara, Harbour, Davies, & Athey, 2009; Xu & Jaggars, 2014; Yukselturk & Bulut, 2007). The belief that self-pacing will contribute to improved learning assumes student self-regulation and engagement in the online learning processes despite often providing less oversight and accountability to ensure that level of commitment (Heissel, 2016; Jacob et al., 2016; Yukselturk & Bulut, 2007).
Theories of Behavioral Engagement: Relevance for Online Courses
Academic behaviors reflect the amount of effort a student decides to invest in their education (Fredricks et al., 2004). This decision is influenced by a need for competence and the desire for a sense of belonging (Marks, 2000; Newmann, Wehlage, & Lamborn, 1992; Rumberger & Lim, 2008). In the frustration–self-esteem model, low prior achievement decreases student self-esteem by not meeting a students’ need for competence, resulting in subsequent disengagement from school (Finn, 1989; Rumberger & Lim, 2008). Research demonstrates that academic achievement is the single strongest predictor of dropping out (Battin-Pearson et al., 2000). Student achievement also serves as a mediator for the effects of contextual factors, such as behavioral concerns, friendship with antisocial peers, and socioeconomic status (Battin-Pearson et al., 2000). These findings indicate that academic achievement is not just an outcome but also an important predictor of academic behaviors (Battin-Pearson et al., 2000). Based on this model, participation in online courses may improve student academic behaviors due to early, regular feedback from progress monitoring reports and the lower time commitment required to complete course content.
Students also engage more in activities that provide a sense of belonging, as detailed in the participation-identification model (Finn, 1989; Rumberger & Lim, 2008). This model asserts that when students do not participate in school activities, including classroom instruction, they identify less with school and perform at lower levels. Alternatively, positive patterns of participation in school, and with achievement minded peers, may lead to an increased sense of belonging and higher performance (Finn, 1989; Ream & Rumberger, 2008). This theory is consistent with research demonstrating that students with higher absence rates also experience increased school disengagement and alienation (Corville-Smith, Ryan, Adams, & Dalicandro, 1998; Finn, 1989; Newmann, 1981) and that students with more negative attitudes toward school are more likely to be chronically absent (Gottfried & Gee, 2017). This model suggests that segregating students from the general high school community into computer labs to complete online courses may result in less favorable academic behaviors (Ream & Rumberger, 2008).
Present Study and Research Questions
This study is part of a larger research project examining the use of online course taking in a large, urban district across multiple years. Prior project findings highlighted disparate educational outcomes by student course-taking behaviors, with students from historically disadvantaged groups more likely to interact with the online course system in a manner that resulted in less desirable academic outcomes (Heinrich et al., 2019). This study was designed to explore an important mediator associated with the differential outcomes observed in previous research. I isolated measures of behavioral engagement that students could control that did not require a minimum level of academic competency. I also prioritized the examination of metrics that incorporated time spent engaging with content outside of school to more explicitly study any changes in behavioral engagement patterns associated with options for anytime, anywhere access.
Many early studies of online learning focused on high-performing student populations (Heissel, 2016) or postsecondary students (Alpert, Couch, & Harmon, 2016; Bettinger, Fox, Loeb, & Taylor, 2017; Joyce, Crockett, Jaeger, Altindag, & O’Connell, 2015; Xu & Jaggars, 2013, 2014). Yet, as technology access has become more prevalent, high school students with academic, behavioral, or social concerns are increasingly assigned to online courses (Ahn, 2011; Heinrich et al., 2019). Students with various levels of academic preparedness and academic engagement require different tools and resources to succeed. Additional research on how student assignment to online courses shapes the educational experiences of these students has important equity implications. Furthermore, results have potential implications for early intervention systems designed to prevent students from dropping out of school (Henry et al., 2012). To address these gaps, I examine the following research questions. To what extent do students who completed more coursework online demonstrate differential rates of behavioral engagement, and by how much do students belonging to marginalized groups benefit differentially from online course enrollment?
Method
I employed a student fixed effect strategy to identify changes in within-student attendance patterns when enrolled in at least one online course. A description of the online course program, data and sample, measures, and empirical strategy follows.
Program Description
The online course vendor studied provides online courses to over 16,000 schools nationwide, including 8 of the 10 largest school districts in the United States. The district that administered the online courses examined in this study uses a hybrid blended learning model (see Christensen, Horn, & Staker, 2013) where students were assigned to complete at least some of their courses during the school day in their regularly assigned school. Students were also able to log in and complete course content at any other time from any Internet-enabled digital device. This type of hybrid blended model is common in traditional schools looking to offer online courses, as the infrastructure to support more radical forms of technology-based learning is often not available (Christensen et al., 2013).
Approximately 300 observations of the physical classrooms and computer labs where students accessed online course content during the school day provided context into the instructional settings. Students had one-to-one access to devices and at least one in-person lab monitor per classroom. In observations, I saw lab monitors encourage students to make progress in the evenings and on weekends by logging into the online course interface from home or a local library. However, teachers also reported in interviews that many students had limited access to out-of-school technology. While some students without a computer and Internet at home completed lessons using their phones or at a library or community center, teachers reported access to both was often limited by mobile data plans and time limit restrictions. These reports are consistent with regional statistics, which indicate as few as half of the students enrolled in online courses may have had access to the Internet at home (Ryan & Lewis, 2017).
Data and Sample
The study relied on administrative data provided by a large, urban school district in the Midwest. Around one quarter of high school students enrolled in at least one online course in a given school year. This rate of online course taking is higher than the national average. Nationally, around 14% of secondary students enroll in at least one online course each year (Gemin, Pape, Vashaw, &Watson, 2015). Data were provided from the 2010–2011 through the 2015–2016 school years for all 9th- through 12th-grade students. For each student in a given year, there were data on enrollment in online courses, attendance, and sociodemographic variables. Among students enrolled in online courses, I also had information on course-taking behaviors.
In the administrative data provided by the school district, 123,833 student-year cases contained sufficient data for inclusion in the analysis, which represented the approximately 20,000 high school students enrolled in the district each year. However, only 40,910 (33%) of those cases contained data on students who switched to or from online enrollment during the study period and thus were included in the main student fixed effect analysis. This restricted sample contained information on 12,853 unique students compared with the 52,838 unique students represented within the full sample. District administrators indicated that the vast majority of students in the district enrolled in online courses for credit recovery, which provided students a second chance to earn course credit required for high school graduation after previously failing the course. Their assertion is supported by the high rate of prior course failure among online course takers. Within the analytic sample, 89% of switchers (students in the restricted sample) failed at least one course before online enrollment, as shown in Table 1.
Sample Characteristics and Dependent Variables by Enrollment in an Online Course
Note. Standard deviations in parentheses.
Students within the restricted sample attended 63 unique school settings. In a year, anywhere from 0% to 93% of all students in a school enrolled in at least one online course. Alternative schools often enrolled a larger proportion of their student population, while schools serving students identified as gifted and talented enrolled a smaller proportion. Within schools, there were changes in the course-taking rate of over 50% and by hundreds of students from year to year. Changes in staffing, school programming, or administrator priorities explained some of this variation. The ease with which students could opt in or out of taking an online course also varied by school. However, the program administrator reported that very few students offered the option to take a course online opted out, as the traditional, face-to-face course required a full semester to complete, whereas students could complete the online course more quickly (Email, February 28, 2018).
Among cases in the restricted sample, 85% represented a student who qualified for free or reduced-price lunch (FRL), 68% of cases represented a student who identified as Black, and 21% of cases represented a student who identified as Hispanic (see Table 1). These demographic characteristics aligned with district averages with the exception of a larger percentage of students who qualified for FRL or who identified as Black enrolled in online courses than in the general student population. When examining descriptive statistics among only students who failed at least one course pretreatment (refer to Appendix Table A1), characteristics for students in the restricted sample remained qualitatively similar, while characteristics for students who never enrolled in an online course became more comparable to students in the restricted sample.
Measures of Behavioral Engagement
To examine by how much behavioral engagement patterns changed when a student enrolled in an online course, I measured student engagement using several methods, including the district-reported days of school attended and the number of class sessions attended. I also provided supplemental analyses examining the number of hours students spent in class (refer to Appendix B for additional information). The equations and assumptions employed in the calculation of each measure are summarized in Table 2. Importantly, in the calculation of sessions logged, only online course-taking information was used for online course takers. The examination of sessions in addition to days attended allowed for the incorporation of the number of sessions attended in online courses outside the traditional school day. In a traditional, face-to-face class the number of class sessions attended was equivalent to the days of school attended. In contrast, a student enrolled in an online course might log into the system in the evenings or on the weekends, meaning more than one session might be logged in a single day and more than five sessions might be logged in a single week. Importantly, students could also log into their online course even if they did not attend school on a given day.
Dependent Variable Calculations
Empirical Strategy
I coded all students enrolled in at least one online course in a given school year as having participated in an online course. This definition does not require students to complete any content in the course, although they must have created (or have created for them) a login. I then compared within-student changes in attendance between years when a student was enrolled in at least one online course versus when the student enrolled solely in traditional, face-to-face courses. By employing this quasi-experimental design, I compared pre- and postattendance and behavioral patterns among students who switched to or from enrollment in at least one online course to the pre- and postpatterns for the same student.
To implement the student fixed effect approach, I estimated the impact of enrollment in an online course on behavioral engagement using equation 1 (labeled Full Model in tables). The student fixed effect (
I estimated the model separately for each dependent variable y for each student i enrolled in school s in grade g during year t. Online represented a binary variable indicating whether the student was enrolled in at least one online course in a given year. The model included a school-by-year-by-grade fixed effect (
In addition to estimating an average treatment effect, I examined shifts in attendance behaviors among students with various levels of exposure. Specifically, I compared the estimates of students enrolled in one versus two online courses and students enrolled in an online course for 1, 2, or 3 years. I also explored whether there was evidence of heterogeneous treatment effects at the student level by race/ethnicity and gender as well as for students identified as chronically absent, qualifying for FRL, and with pretreatment course failure. Next, I examined differences at the school level by school type and achievement group based on average school-level math and reading standardized test scores.
Lastly, I conducted several validity checks, including a comparison of intent-to-treat (ITT) versus treatment-on-the-treated (TOT) estimates. I examined concerns related to generalizability to the larger sample and the extent to which the student fixed effect controlled for relevant fixed and nonfixed student-level characteristics. I also tested the strict exogeneity assumption, examined whether there was evidence of regression to the mean, and examined the influence of outliers. Refer to Appendix C for more information on the methods employed and findings from these supplemental analyses.
Results
On average, students enrolled in at least one online course attended 2 to 3 more days of school a year than in years not enrolled in an online course, as presented in Table 3. Prior to controlling for school-by-year-by-grade fixed effects and student covariates, students attended 2.45 more days of school during the year(s) in which they enrolled in an online course. The inclusion of school-by-year-by-grade fixed effects resulted in a qualitatively similar estimate of 3.14 days, which increased to 3.32 days when student covariates were accounted for. The consistent attendance estimates observed across model specifications demonstrated that it was unlikely that the identified estimates were due to a spurious correlation introduced by a confounding variable.
Attendance Shifts Among Students Enrolled in an Online Course
Note. Each cell summarizes estimates from a separate model. Results are based on 40,910 observations. Standard errors (in parentheses) were clustered at the student level.
p < .10. **p < .05. ***p < .01.
Across all three model specifications, students logged 63 fewer sessions online than face-to-face sessions attended in years that they did not enroll in an online course. This finding was consistent with the goal of online instruction in the district to allow students to complete courses more quickly than possible within a traditional, face-to-face setting. Minimal variation in estimates between models presented in Table 3 indicated that these results were also robust to alternative specifications. As a result, subsequent tables only report estimates from the model that controls for student and school-by-year-by-grade fixed effects as well as vectors for student and school characteristics.
Dosage
Attendance patterns varied by exposure, as shown in Table 4. Specifically, the increase in days of school attended appeared almost entirely realized by students enrolled in two or more courses, although these students also logged fewer sessions. I chose not to examine more than two courses at a time separately, as in some schools and years students were only allowed to enroll in up to two online courses at a time. Therefore, I was concerned that an examination of enrollment of three or more courses separately from enrollment in two courses might inadvertently attribute the success of students who completed a course (and thus were permitted to move on to a third course) as more positive benefits associated with enrolling in more courses. I identified few practical differences by the number of years in which a student enrolled in an online course. There was an inconsistent relationship between sessions logged and the number of years enrolled online, with students enrolled 1 year logging an average of 64 fewer sessions compared with 61 and 69 fewer sessions logged when students enrolled for 2 and 3 years, respectively.
Attendance Shifts by Online Course Exposure
Note. Each row summarizes estimates from a separate model limited to the subsample indicated. Estimates were produced from the full model specification. Standard errors (in parentheses) were clustered at the student level.
p < .10. **p < .05. ***p < .01.
Heterogeneity
Attendance patterns also varied by pretreatment student characteristics, as demonstrated in Table 5. The increase in days of school attended was entirely realized by students identified as chronically absent, students who qualified for FRL, and students who failed more than two courses a year before enrolling online. Additionally, students who repeated one or more grades attended an average of 9.03 more days of school in years when they enrolled in at least one online course. The number of sessions logged by these more at-risk student populations also decreased less when enrolled in an online course than among their relatively more advantaged peers. In particular, the gains observed by students who qualified for FRL indicated that concerns regarding a potential lack of Internet access at home did not appear to limit these students’ progress. In contrast, there was little notable variation in the days of school attended or sessions logged by student gender or race/ethnicity.
Attendance Shifts by Pretreatment Student Characteristics
Note. Grade-level models excluded grade fixed effects. Each row summarizes estimates from a separate model limited to the subsample indicated. Estimates were produced from the full model specification. Standard errors (in parentheses) were clustered at the student level.
p < .10. **p < .05. ***p < .01.
Lastly, I examined differences in student attendance and behavioral engagement by school characteristics (Table 6). Students enrolled in an alternative school attended 3.62 more days of school in years when enrolled online compared with the qualitatively similar 3.10 more days of schools attended among students enrolled in another type of school. Despite similar attendance trends, students enrolled in alternative schools logged more online sessions (−57 vs. −62) than students not enrolled in an alternative school. I also examined patterns by schools in the top, middle, and bottom third of school-level math and reading standardized test scores. Consistent with patterns observed among students from more advantaged backgrounds and with higher levels of prior achievement, students attending schools in the top third of schools logged fewer sessions (−76 vs. −57 to −58).
Attendance Shifts by School Characteristics
Note. Each row summarizes estimates from a separate model limited to the subsample indicated. Estimates were produced from the full model specification. Standard errors (in parentheses) were clustered at the student level.
p < .10. **p < .05. ***p < .01.
Discussion
Limitations
The results presented in this study represent associations between online course taking and attendance. However, the fixed effect strategy employed can produce plausibly causal estimates when key assumptions, such as the strict exogeneity and homogeneity assumptions, are met. I demonstrated in Appendix C that the main models met these assumptions and that there was no evidence of a pretreatment dip or regression to the mean. Furthermore, while prior research linked attendance and out-of-school studying to short- and long-term student outcomes (Allensworth & Easton, 2007; Heinrich et al., 2019), this study did not establish whether the behavioral engagement patterns observed translated into improved academic outcomes.
The measures examined in this study also failed to distinguish between students actively interacting with course content and those, for instance, running a lecture-based video in the background while talking with friends. These challenges in measuring instructional time (vs. time at school) were also present in students enrolled in solely traditional, face-to-face courses, since the measures used only captured time in school versus time spent engaging in learning. For instance, I was unable to distinguish between a student who actively engaged in learning activities in a traditional, face-to-face classroom and a student who attended the same classes but did not listen to the instructor or complete course activities. Thus, in both online and traditional, face-to-face contexts, the behavioral engagement measures represent a precondition to learning but not necessarily that learning occurred.
Similarly, the use of the days of school attended metric did not guarantee that a student attended all class periods during the school day. Furthermore, students enrolled in a traditional, face-to-face course might have engaged in educational activities outside of the school day. However, I was not able to measure time devoted to those activities for students not enrolled in an online course. I was also unable to calculate estimates for students with only 1 year of data or who will in the future enroll in an online course outside the 6 years of data provided by the district. Lastly, I lacked information on the 7% of student-year observations where the student had not graduated and did not appear in subsequent years of data. Two percent of total cases represented instances where students were not enrolled in an online course in the year prior to the attrition, whereas an additional 5% of total cases represented instances where students were enrolled online in the year prior. Likely these students either dropped out or transferred to another school district. The attrition observed in these cases prevented the estimated models from fully accounting for all changes in attendance behaviors associated with online course taking. Despite these measurement limitations, estimates capture an important precondition to learning—school attendance and online sessions logged. Future studies could improve on these estimates by capturing and accounting for the quality of student interactions with course content and information on out-of-school time spent engaging with learning materials for students enrolled in only face-to-face courses.
Lastly, the student fixed effect analysis did not identify the mechanisms through which online course taking might be associated with more days of school attended. Future research should clarify any processes through which the observed behavioral engagement patterns shifted among the high school students enrolled in online instruction. Possible mechanisms include improved self-confidence through regular, formative feedback, clearly communicated expectations, and short, modularized lessons (Finn, 1989; Rumberger & Lim, 2008). Alternatively, students might find the self-contained computer labs provided opportunities to reestablish more positive learner identities (Finn, 1989; Rumberger & Lim, 2008). These hypotheses were consistent with findings that the most marginalized student populations experienced the largest increases in days of school attended when enrolled in an online course. However, the empirical strategy employed in this study did not allow me to draw conclusions toward that end.
Implications for Research and Practice
This study extends current literature on attendance and behavioral engagement patterns among high school students and students enrolled in online courses. Overall, students attended around 3 more days of school a year when enrolled in an online course. This finding is consistent with research showing contemporaneous achievement benefits to online high school course taking in Florida (Hart, Berger, Jacob, Loeb, & Hill, 2019). An additional 3 days of school translates into an approximately 2 percentage point increase in attendance, which is only slightly smaller than the 3 percentage point increase in attendance identified by Tran and Gershenson (2018) as a result of a 10 student decrease in class size. Online course takers also logged significantly fewer sessions per course, consistent with the district goal of allowing students to earn credit more quickly through online courses. Results were robust to alternative model specifications and met the assumptions required for the use of a student fixed effect strategy.
The one prior study that examined student engagement in online courses at the high school level used self-reported engagement measures and a program model designed for students to complete instruction during the school day. That study identified no significant difference in engagement between students randomly assigned to an online versus face-to-face course (Heppen et al., 2017). Specific to attendance, there appeared to be few negative ramifications to student enrollment in online courses in this study. Concerns regarding limited access for some students to Internet-enabled devices for out-of-school work on online courses did not appear to limit the ability of students who qualified for FRL to benefit from online course taking. Considering that engagement becomes increasingly stable as students progress through their education (Gottfried, Fleming, & Gottfried, 2001), the moderate shifts in attendance patterns among high school students enrolled in online courses identified in this study have practical significance and implications. Based on ES estimates identified in prior studies, reducing absences by around 3 days of school a year might translate into small increases in reading and math test scores (Gershenson et al., 2017).
Consistent with the frustration-self-esteem model, students who enrolled in more courses online in a given year and students who previously failed more courses demonstrated larger increases in days of school attended (Finn, 1989; Rumberger & Lim, 2008). These shifts might be due to the structure of the online courses that chunked course content into small sections completable during a single class period and regularly communicated progress toward completion (Newmann et al., 1992; Wang & Holcombe, 2010). Furthermore, in contrast to my hypothesis that students with greater isolation from general education classes through the completion of more coursework online would demonstrate lower rates of behavioral engagement, the opposite appeared true for most students. Based on these findings and classroom observations, it is possible that instead, depending on the learning environment and interactions with lab monitors and peers, the more contained computer lab might have provided the sense of community a student needed to reengage in learning (Brion-Meisels, 2016; Darling-Aduana et al., 2019; Newmann et al., 1992).
Positively, theory as well as descriptive analyses in the sample studied lends plausibility to assertions that students’ improved behaviors (where observed) would likely lead to improved cognitive and emotional engagement, as well as subsequent achievement, by fulfilling students’ need for competency and a sense of belonging (Heinrich et al., 2019; Newmann et al., 1992; Rumberger & Lim, 2008). Prior research identifying improved contemporaneous and subsequent achievement outcomes for students completing online credit recovery courses lends additional credibility to this possibility (Hart et al., 2019). For students at risk of dropping out of high school, the possibility to earn course credit quicker than feasible through traditional, face-to-face instruction was likely a motivator and advantage to online instruction independent of whether online instruction encouraged the development of more positive learner identities or subsequent academic success. However, the increased days of school attended observed among marginalized student populations when enrolled in online courses suggested that access to online courses also encouraged students to attend more school, representing a potentially beneficial spillover effect. Whether students had no other access to the technology required to complete online course content or online enrollment provided an alternative, explicit structure to demonstrate competency, increased school attendance represents a step in the right direction.
With additional research, many of the same strategies associated with increased behavioral engagement among students enrolled in online courses could potentially be applied to traditional or blended classrooms. For instance, students might benefit from the option to complete face-to-face as well as online courses over a shortened period. Similarly, providing incentives, such as progress toward course completion, for out-of-school time spent engaged with educational material might encourage increased time engaged. Providing regular formative assessment and progress monitoring as well as allowing students to demonstrate competency early and often might also assist in the development of more positive learner identities (Newmann et al., 1992; Wang & Holcombe, 2010). Continued research is needed to confirm the effectiveness of these mechanisms in online, traditional, and blended classroom settings.
From a larger policy perspective, the hybrid blended model of online course taking enacted in this district did not dampen student attendance. This is important as schools are increasingly held accountable for student attendance and often receive funding based on days of school attended (Bauer et al., 2018; Picciano & Seaman, 2009). Toward that end, this study identified positive associations between online course taking and student attendance among students with prior records of academic course failure and demonstrated that the online courses administered in this district allowed students, on average, to regain credit faster than feasible in a semester-long, face-to-face setting.
Footnotes
Appendix A
Appendix B
Appendix C
Acknowledgements
I would like to thank Carolyn Heinrich, Seth Gershenson, and peer reviewer for their instrumental feedback; the William T. Grant Foundation for generous funding of this research; researchers employed by the ed-tech vendor for their support in accessing data from the online instructional system; school district staff for their assistance in providing student record data and support of the project data analysis and fieldwork; and Vanderbilt University and the Wisconsin Evaluation Collaborative at the Wisconsin Center for Education Research, University of Wisconsin Madison, for ongoing support of this initiative.
Author
JENNIFER DARLING-ADUANA is a PhD candidate in education leadership and policy studies at Vanderbilt University. Her research focuses on mechanisms to reduce educational inequities in the United States, with particular attention to how digital learning transforms teaching and learning processes in schools.
