Sage Journals: Discover world-class research

Abstract

Learning to read is an essential skill for later academic success, positive self-esteem, and gainful employment. Students who display reading difficulties/disabilities at the end of third grade are less likely to succeed in content areas and graduate from high school. Recent data suggests that many students in today’s schools do not become skilled readers, and the reading loss widens during summer months due to skill regression. Regression of reading skills is greater for students from low Socioeconomic status (SES) families and for students with disabilities. This study examined the effects of two computer-assisted reading programs on the reading skills of 21 students at-risk for reading failure during a summer break. All students were pre- and post-tested after 8 weeks of intervention. Furthermore, tutors’ and students’ perceptions regarding the effectiveness and desirability of the programs were measured. A description of the computer programs, results, implications, and limitations of the study are discussed.

Keywords

computer-assisted reading programs early literacy skills summer reading loss

Introduction

Reading competency involves mastering the essential skills of word recognition and comprehension. Successfully mastering basic literacy skills in the earlier grades is fundamental for later academic success, positive self-esteem, and employment opportunities (Hammond, 2012; Mather et al., 2001; Slavin et al., 2009; Pindiprolu & Forbush, 2021). However, the National Assessment of Educational Progress (NAEP) data from the past decade suggests that a large percentage of students in today's schools do not become skilled readers. For example, only 37% of fourth-grade students and 36% of eighth-grade students performed at or above the “proficient” level in the area of reading according to the 2017 NAEP data (National Center for Education Statistics, 2018). Lyon (2001) estimated that 70% of the students identified as at-risk for reading failure at the start of first grade would continue to experience reading difficulties in their adult lives. Thus, the prevention of reading failure in the earlier grades is fundamental to overcoming reading skill deficits and a subsequent decrease in motivation to read.

Students who struggle in reading in their early years tend to make fewer gains across grades than students who do not (Sandberg-Patton & Reschly, 2013). Parallel to these weaker gains in reading is skill loss during the summer breaks. According to Sandberg-Patton and Reschly's (2013) research, students in the lower elementary grades (i.e., second and third grades) demonstrated greater summer learning loss in oral reading fluency than students in the upper elementary grades. The researchers suggested that this discrepancy is because students in earlier elementary grades do not have the essential skills to read independently during the summer break. Furthermore, the study results indicated that the learning loss was greater for students receiving special education services and/or for students from lower socioeconomic backgrounds (Sandberg-Patton & Reschly, 2013). Hence, it is essential to support the development of basic literacy skills in the early grades because the lack of basic reading skills can create large skill deficits over the years. For example, the loss in reading skills when compounded over multiple years creates a 2–3 year reading deficit in children (Allington & McGill-Franzen, 2003).

Computer-assisted reading programs (CRPs) offer an alternative for supporting the literacy instructional needs of students at risk for reading failure during summer break. CRPs offer multiple advantages over print-based reading programs. First, they require minimal training. Second, they can provide highly specialized instruction in the area of reading with high fidelity and relatively low costs (Torgensen et al., 2010). Third, the use of games, graphics, and multimodal presentations can facilitate the motivation of students who struggle to read (Regtvoort & Leiji, 2007). Fourth, recent literature suggests that CRPs are effective in facilitating basic early literacy skills.

Literature on CRPs

In 2002, Blok et al. examined the literature to evaluate the effectiveness of computer-assisted programs in facilitating beginning reading instruction. Based on their analysis of the literature, the authors concluded that computer-assisted programs were positive in facilitating early literacy skills. The overall effect size of the computer-assisted programs was small but positive (Blok et al., 2002). Given the poor-quality design of many of the studies, the authors called for a cautious interpretation of the findings. In 2011, Chambers et al. evaluated the effectiveness of a small group Tier II computer-assisted intervention on the reading skills of struggling readers. The study was conducted in 33 high-poverty Success for All (SFA) schools over a 1-year period. The schools were randomly assigned to experimental and control conditions. The intervention, Team Alphie, consisted of computer-assisted intervention, embedded multimedia, cooperative learning, and tutoring. The intervention was implemented four times a week. The control group students received individualized tutoring conducted in SFA schools. The results of the study indicated that the experimental group students in the first grade had higher reading achievement than students in the control group. The reading achievement for second graders was the same for students in experimental and control groups (Chambers et al., 2011).

In 2008, Macaruso and Walker examined the effects of phonics-based computer-assisted program, Early Reading, on the preliteracy skills of kindergartners. Intervention sessions lasted for 15–20 min, and 2–3 sessions were conducted weekly over a 6-month period. Both the intervention and control group students also received daily reading instruction. The results indicated that the students in the computer-assisted program group significantly outperformed students in the control group on the phonological awareness skill measure. The authors concluded that treatment group students, especially the low-performing students, greatly benefited from the Early Reading program when implemented as a supplemental program to the daily reading instruction (Macaruso & Walker, 2008). In 2011, Macaruso and Rodman conducted two studies to evaluate the effectiveness of computer-assisted instruction. In study 1, they examined the effectiveness of Early Reading program with pre-schoolers. In study 2, the authors examined the effectiveness of Early Reading and Primary Reading with kindergartners. In study 1, the experimental group computer-assisted instruction (CAI) students performed significantly better on phonological awareness measure and the control group made greater gains on naming upper case letters. In the second study with kindergartners, the results indicated that the students in the experimental group (CAI) made significantly greater gains on the Total Test scores and on the Word Reading subtest. Based on the findings, the authors concluded the CRPs provided intensive practice and benefits for pre-schoolers and low-performing kindergartners (Macaruso & Rodman, 2011).

Watson and Hempenstall (2008) examined the effects of Funnix CRP on the reading skills of kindergarten (KG) and first-grade students. The results indicated that KG students in the Funnix group had significant gains on phonemic awareness, letter-sound fluency, oral reading fluency, and non-word decoding measures. For first-grade students, participants in the Funnix group achieved significant pre–post gains on letter-name knowledge, letter-sound fluency, oral reading fluency, and non-word decoding measures. The researchers concluded that KG students made the most gains and that future studies examine the effectiveness of Funnix with at-risk beginning readers to measure the extent to which they benefit from the Funnix program. Similarly, Huffstetter et al. (2010), examined the effects of Headsprout program on the basic literacy skills of at-risk pre-school children. The program was implemented in 30 min every day for 8 weeks during the school year. Based on the results, the authors concluded that Headsprout CRP was effective in facilitating early reading skills of at-risk pre-school children. Social validity data indicated that the teachers and teacher assistants were positive regarding program effectiveness with one teacher reporting “feeling burned out” due to the daily implementation of the program. The authors called for additional studies, with a literacy program condition, to evaluate the effectiveness of the Headsprout program.

In 2011, Saine et al. conducted a study in Finland to examine the effectiveness of computer-assisted intervention on the letter knowledge, reading accuracy, fluency, and spelling of children at-risk for reading disabilities. The 7-year-olds were assigned to either a regular remedial reading intervention program, a computer-assisted reading program, or a mainstream reading program. The results indicated that the computer-assisted reading group made greater gains on the letter knowledge, decoding, accuracy, fluency, and spelling measures than the students in the regular remedial program or the mainstream reading program (Saine et al., 2011). Furthermore, the students in the computer-assisted program maintained their gains at 12- and 16-month maintenance checks after the intervention. The authors concluded that computer-assisted remedial reading intervention programs are highly beneficial for at-risk students.

In 2014, Regan et al. examined the effects of a computer-assisted instruction program, Lexia Strategies for Older Students (SOS)™ CRP, on the word recognition skills of four elementary students with mild disabilities. Almost all the study participants required additional direct instruction to master some of their target skills (Regan, et al., 2014). Based on the results, the authors concluded that Lexia SOS CRP with additional direct instruction could facilitate word recognition skills of upper elementary-grade students. In 2018, Messer and Nash evaluated the effectiveness of a computer-assisted reading intervention, Trainertext, on at-risk students’ performance on six reading-related measures in six UK schools. Each tutorial session lasted 10–15 min and was implemented two to three times a week. Initially, the intervention was provided to the students in the experimental group and the students in the control group received intervention after post-test 1. The results of post-test 1 indicated that students in the experimental group had significantly higher standard scores than the students in the control group on decoding, phonological awareness, naming speed, phonological short-term memory, and executive working memory measures and maintained the gains at post-test 2 (Messer & Nash, 2018). The students in the control group showed improvements on most of the measures on post-test 2. Based on the results, the authors concluded that computer-assisted intervention had a significant effect on the reading-related abilities of English-speaking children and that longer intervention duration resulted in higher gains for the students. In 2018, Rogowsky et al. examined the effects of computer-assisted instruction on the reading and math skills of pre-school children. Children in the computer-assisted group engaged with the program for 10 min each day for 11 consecutive weeks. Results indicated that students in the computer-assisted group had significant improvement in the scores on literacy skills than students in the comparison group. Based on the results the authors concluded that computer-assisted learning may enhance the literacy skills of pre-schoolers (Rogowsky et al., 2018).

Given the debilitating consequences of early reading failure, there is a need to study the effectiveness of educational technology in ameliorating the reading deficits of at-risk students in early grades (Agodini et al., 2003; Pindiprolu & Forbush, 2021). Furthermore, there is a significant dearth of rigorous evaluations of the effectiveness of CRPs (Rouse & Krueger, 2004) and the effectiveness of the CRPs in reducing summer learning loss. It is in this context the present study was undertaken. The purpose of the study was to evaluate the effectiveness of two CRPs, when implemented by university students as tutors during a summer break, on the reading skills of elementary-grade students with reading difficulties. Furthermore, the tutor's perceptions regarding the effectiveness, ease of implementation, and desirability of the program were assessed, along with the students’ perceptions of the effectiveness and desirability of the programs.

Method

Participants and Setting

Participants for the study consisted of students with reading difficulties in grades KG through second grade. To recruit students, the following steps were undertaken. First, focus group meetings were conducted with district supervisors of a school district located in Northern Utah. Participants reviewed the objectives of the study and offered feedback on the procedures, measures, and computer-based programs to be used in the study. Second, elementary-grade teachers working in KG through second grades in an elementary school were contacted to identify students scoring below the 15th percentile on a local or state test of reading achievement. Third, study flyers were developed and sent home with student candidates through their teachers. Twenty-one parents gave consent for their students to participate in the study. All students were randomly assigned to one of the two CRPs.

CRPs

Two CRPs, Funnix and Headsprout, were chosen for the study based on the following three criteria. One, they addressed the critical literacy skills of phonemic awareness, phonics, fluency, vocabulary, and reading comprehension. Two, they were designed for use with students in kindergarten to second grade. Third, they required similar lengths of time to complete a lesson/episode.

Funnix CRP. Funnix is a CD-based, early literacy program for teaching literacy skills to beginning readers. It provides explicit instruction in the areas of phonological awareness, phonics, fluency, vocabulary, and comprehension. Funnix Beginning Reading consists of 120 lessons and a workbook for students to complete after each lesson (Engelmann et al., 2001). Funnix 2 consists of 100 CD-based lessons and a book containing reading passages (Engelmann & Engelmann, 2002). The program comes with a print-based placement test, for use to determine the entry points for each student into the program (see http://www.funnix.com/ for more details). Each lesson begins with a narrated introduction followed by questions for the student to produce oral responses to stimuli presented on screen, complete workbook exercises, or read from the reader. The program requires an adult to navigate the program. The adult chooses whether to continue or repeat an exercise if the student is making multiple errors. Also, the adult decides when to provide positive feedback to the student (Engelmann & Engelmann, 2003). Instructions for the adult in effectively correcting sound pronunciations, navigating through an exercise and praising the student's correct responses are provided in a training CD that comes with the program.

Headsprout CRP. Headsprout Early Reading (HS) is a web-based early literacy program consisting of 80 lessons/episodes, collections of Sprout stories, and benchmark assessments. The program employs explicit instruction and immediate, cumulative practice to teach phonemic awareness, phonics, vocabulary, oral reading fluency, and reading comprehension (Layng et al., 2003). Unlike Funnix, there is no placement test to determine the entry point of a student into the HS program. All students start from the first lesson/episode. Furthermore, unlike Funnix, it is primarily student-navigated (i.e., the student manipulates a mouse to select a response) and requires students to be able to independently use a mouse. Students learn literacy skills through cartoon-based episode environments of Space World, Dinosaur World, Undersea World, and Jungle World (Layng et al., 2003). HS requires multimodal responses such as oral production responses and motor selection responses. For example, the narrator asks the student to say a word aloud. It is then followed by a presentation of three characters saying the word and the student is asked to click on the character s/he thinks is saying the correct response (see https://www.headsprout.com/main/ViewPage/name/product-features/ for more information). The program also prompts students to read aloud paper-based Sprout Stories. Headsprout recommends that adults supervise the students to ensure that students are producing oral responses to program-provided prompts and that they provide corrective feedback for reading errors produced during the reading of Sprout stories.

Tutor training and feedback. The computer-based programs were implemented during a summer break by tutors at students’ homes in the Cache Valley region of Northern Utah. Project staff trained four university students to work with 21 school-age students. Two of the tutors implemented the HS program, one implemented the Funnix program, and one student implemented both programs. The training of tutors included program descriptions of key program features, modeling of key instructional behaviors by project staff, and opportunities for tutors to practice using program tools under the direction of the project staff. Furthermore, four project coordinators were trained to collect treatment fidelity data. This training consisted of a review of program-required instructional delivery procedures, correct use of the observation tool developed for the study, and scoring program implementation using video footage of students tutored using Funnix and Headsprout. The coordinators were required to demonstrate 95% interobserver agreement (IOA) before their observations. The coordinators randomly observed the tutors implementing the program and collected treatment fidelity data. Tutors kept daily logs of the times required to complete lessons/episodes and the lesson/episode number completed.

Dependent Measures

DIBELS and training of administrators. The students were pre- and post-tested using the Dynamic Indicators of Basic Early Literacy Skills (DIBELS) progress monitoring probes to evaluate the effectiveness of the CRPs. DIBELS probes are sensitive to small changes in skill acquisition over a short period of time and for this reason, they were used to measure students’ skill gains. DIBELS consists of seven subtests. These subtests are Letter Naming Fluency (LNF), Initial Sound Fluency (ISF), Word Use Fluency (WUF), Phoneme Segmentation Fluency (PSF), Nonsense Word Fluency (NWF), Oral Reading Fluency (ORF), and Retell Fluency (RF), (Good et al., 2001). For this study, progress monitoring (PM) probe 19 (except for LNF) was used as the pre-test, and PM probe 20 was used as the post-test for six of the seven measures. For the LNF measure, the KG through third-grade benchmark assessment was used as the pre-test and the first benchmark at first grade was used as the post-test as PM probes are not available for this subtest. Because ORF PM probes for KG do not exist, the first-grade PM probes were used with KG students. A video resource, modeling of probe administration, practice with administration, and feedback were employed to train the four project coordinators with the administration of DIBELS probes until they demonstrated the probe administration skills specified in DIBELS training materials. Furthermore, the IOA among coordinators was collected on at least 20% of all pre- and post-tests.

Social validity measures. A social validity questionnaire for tutors and one for the students was developed before the study commenced. At the completion of the study, the social validity questionnaire was administered to elicit participants’ perceptions. Students were orally asked about their satisfaction with the program, their perception of specific elements (e.g., stories, graphics, activities, etc.), and their perception of the effectiveness of the CRP in teaching them to read. Project coordinators recorded the responses to the questionnaire. Tutors also responded to questions such as the ease of using the program, their perception of the effectiveness of the program in teaching reading skills, and their overall perception of the program.

Results

Four tutors implemented the CRP programs with 21 KG through second-grade students. Of the 21 students, 11 were assigned to the Headsprout (HS) group and 10 were assigned to the Funnix group. Six of the 11 students in the HS group were also classified as English as a Second Language (ESL) students and three of the 10 students in the Funnix group were classified as ESL students. For the HS group, five students were in kindergarten, five were in first grade, and one was in second grade. Of the 10 students in the Funnix group, three were in kindergarten, four were in first grade, and three were attending second grade. Approximately 55% of the students in the HS program and 70% of the students in the Funnix program completed 30 or more episodes/lessons. The average number of episodes completed by students in the HS group was 28, and the range was 20–34 episodes. The average number of lessons completed by the students in the Funnix group was 32 and the range was 6–34 lessons.

Interobserver Agreement

During the course of the study, IOA on the DIBELS measures was undertaken for at least 20% of the pre- and post-tests. IOA was calculated using an item-by-item agreement. The average pre-test IOA was 99.3% on the LNF measure, 99% on the ORF measure, 97% on the ISF measure, 95.5% on the RF measure, 95.8% on the NWF measure, 91% on the WUF measure, and 89.3% on the PSF measure. The average post-IOA was 100% for the ISF measure, 97% for the LNF measure, 90.4% for the PSF measure, 89.8% for the ORF measure, 85% for the NWF measure, 81.3% for the WUF measure, and 79.45% on the RF measure.

Treatment Fidelity

Tutors were observed a minimum of two times during their program's implementation. A treatment fidelity form with instructions was developed for each computer-assisted program. Project coordinators collected data using a treatment fidelity recording form. For Funnix, the observations on the implementation of the CRP included recording the number of opportunities available and tutors’ correct responses to those opportunities on five components: (a) pausing the program after the child made an error; (b) modeling a correct response; (c) prompting the student to repeat incorrect sounds and words; (d) doing entire exercises “over” if frequent errors occurred; and (e) praising for correct student's responses during the lesson and workbook activities. Approximately 265 feedback and/or error correction opportunities occurred during the six observation sessions for tutors implementing the Funnix program. The students correctly responded to 98% of those response opportunities and received positive feedback/praise. The average correct response rate for tutors was 100% on repeating an exercise, 86.6% for pausing the program and prompting the student to repeat incorrect sounds, and 66.6% for modeling correct responses. The average praise rate was two praises per minute. For Headsprout, the observations on the implementation of the CRP included the number of opportunities tutors had and the correct responses of the tutors on the following two components: (a) modeling correct responses when the student mispronounced a word while reading or responding to stimuli and (b) prompting the student to repeat incorrect sounds and words. The average correct rate across tutoring sessions was 70% for modeling correct responses and 45% for prompting students to repeat incorrect sounds and the lesson. The average praise rate was 2.1 praises per minute.

CRP Effectiveness

To evaluate the effects of the programs on the basic early reading skills of the students, two types of analyses were conducted. First, a one-way analysis of covariance was used to compare the relative effectiveness of the two CRPs on the seven DIBELS measures. The one-way analysis of covariance was conducted with the computer-assisted program as the independent variable, post-tests as the dependent variable, and the pre-tests as the covariate. Second, paired-sample t-tests were undertaken to measure the pre–post gains on the DIBELS probes for each group. All null hypotheses were tested at the .05 significance level (two-tailed tests).

Before conducting the one-way analysis of covariance (ANCOVA) on the seven DIBELS measures, the homogeneity of slopes assumption test was evaluated. The null hypothesis for the homogeneity of slopes test was rejected (i.e., the homogeneity of slopes assumption was not met) for the ORF measure, F(1, 17) = 7.6, p = .01. Furthermore, the partial eta square was large, .31, indicating that ANCOVA results of a significant finding on the ORF measure should be re-examined using other statistical procedures. In addition, the null hypothesis for the LNF measure was not significant, F(1, 17) = 3.76, p = .07. The partial eta square was large, .18, indicating that ANCOVA results of significant finding on the LNF measure should be re-examined using other statistical procedures. The ANCOVA main analyses for effects indicated a statistically significant difference only on the WUF measure, F(1, 18) = 10.83, p = .004, and the effect size was large (partial ŋ² = .38) in favor of the Funnix group. The estimated marginal means for the Funnix group on the WUF measure was 42.08 compared to 26.2 for the Headsprout group. The difference in the PSF measure was not significant, F(1, 18) = 3.48, p = .08, and the effect size was large (partial ŋ² = .16) with the students in the Funnix group performing better than the students in the Headsprout group. The estimated marginal mean on the PSF measure for the Funnix group was 53.88 compared to 43.02 for the Headsprout group (see Table 1).

Table 1.

Means, Standard Deviation, and Adjusted Means on the Post-tests for the Two Groups.

Post-test	Group	Mean	SD	Adjusted means
LNF	Headsprout	49.81	23.96	50.18
	Funnix	55.60	14.36	55.20
ISF	Headsprout	19.89	10.35	19.42
	Funnix	23.90	10.61	24.41
PSF	Headsprout	43.81	17.41	43.02
	Funnix	53.00	14.40	53.87
NWF	Headsprout	56.90	35.31	56.75
	Funnix	55.80	25.67	55.96
WUF	Headsprout	26.18	11.28	26.200
	Funnix	42.10	9.93	42.07
ORF	Headsprout	31.09	23.71	36.15
	Funnix	46.40	33.07	40.83
RF	Headsprout	10.27	12.49	12.26
	Funnix	17.00	17.02	14.80

A paired-sample t-test was conducted for each group on each of the seven DIBELS measures to evaluate pre–post gains. For the Funnix group, a statistically significant difference was found between the pre- and post-test scores on the WUF measure, t (9), = 3.24, p = .01. Further, the value on the PSF measure was t (9), = 2.02, p = .07. The Funnix group had higher post-test mean scores on the PSF, NWF, WUF, and RF measures. The Headsprout group had higher post-test mean scores on the NWF measure. As the sample size was small, standardized effect size index d was computed. For the Funnix group, the effect size was large for the WUF measure (1.03), medium for the PSF measure (.64), and small for the RF measures (.38). No significant positive effect sizes were found for the HS group.

Tutors’ Perceptions

The results from the social validity questionnaires administered to the tutors indicated that 100% of the tutors (n = 3) implementing the HS program and 100% of the tutors (n = 2) implementing the Funnix program had previous experiences implementing reading instruction. Further, 100% of the tutors in the Funnix program indicated that they would recommend the program to others helping students with reading and would use the program to help other students. None of the tutors in the HS program indicated that they would recommend the program or indicated that they would use the program to help another student. Regarding the ease of using the Funnix program, 100% of the tutors in the Funnix program agreed that starting the lesson, moving between exercises, repeating the lesson, and pausing the program were easy. Similarly, 100% of the Funnix tutors agreed that (a) students easily followed the instructions of the narrator; (b) they were comfortable using the program; (c) the program was useful in teaching reading skills to students; and (d) Funnix was a good program. Regarding the ease of using the HS program, 67% of the tutors in the program agreed that starting the lesson and 33% agreed that students could easily follow the instructions. On the question of students’ comfortableness using the HS program, 33% disagreed (67% were neutral). All tutors indicated “neutral” to the statements regarding (a) the usefulness of the HS program in teaching reading skills to students and (b) that HS was a good program.

Regarding the appropriateness of teaching activities in the Funnix program, 100% of the Funnix tutors agreed that (a) activities for teaching sounds of letters and comprehension were appropriate and 50% indicated that activities for teaching words were appropriate. Similarly, 100% of the Funnix tutors agreed that (a) computer graphics on the program helped maintain students’ attention; (b) workbook activities for practicing reading skills were useful; (c) their feedback helped their students learning of reading skills; and (d) the instructional time was sufficient for teaching reading skills (see Table 2). Regarding the appropriateness of teaching activities in the Headsprout program, 33% of the HS tutors agreed that activities for teaching sounds of letters and words were appropriate. Similarly, 33% of the HS tutors agreed that (a) computer graphics on the program helped maintain students’ attention, and (b) their feedback helped their students learning of reading skills. On the usefulness of the Sprout stories, sufficiency of instructional time, and appropriateness of the comprehension activities, 67% of the HS tutors were neutral (33% disagreed).

Table 2.

Tutor's Perceptions of Computer-Assisted Reading Programs.

Questions	Funnix	Headsprout
Questions	% Agree	% Agree
1. Starting a lesson was easy.	100	67
2. Students could easily follow the narrator's instructions.	100	33
3. Overall quality of the program was very good.	100	0
4. The program was useful in teaching reading skills.	100	0
5. Would recommend the program	100	0
6. Activities for teaching letter sounds were appropriate.	100	33
7. Activities for teaching comprehension were appropriate.	100	0
8. Workbook activities (Funnix)/Sprout stories (HS) were useful for practicing reading skills.	100	0
9. Computer graphics helped maintain child's attention	100	33
10. Would use the program to help another student	100	0

Students’ Perceptions

The results from the social validity questionnaire administered to the students in the Funnix group (n = 10) indicated that (a) 100% of the students liked having an adult help them learn to read, (b) 90% (n = 10) of the students liked the Funnix program, (c) 100% of the students liked the visuals in the program, (d) 90% liked the workbook activities, (e) 90% liked the stories, (f) 90% of the students indicated that Funnix helped them learn to read (10% indicated “I don’t know”), and (g) 70% indicated that their friends might like the program (20% indicated that they don’t know). The results from the social validity questionnaire administered to the students in the Headsprout group (n = 11) indicated that (a) 91% liked the Headsprout program, (b) 73% liked having an adult help them read, (c) 91% liked the visuals, (d) all students (100%) liked the activities/games, (e) 82% liked the stories, (f) 91% of the students indicated that HS helped them learn to read, and (g) 73% indicated that their friends might like the program (18% indicated that they don’t know/did not answer; see Table 3).

Table 3.

Students’ Perceptions of Computer-Assisted Reading Programs.

Questions	Funnix	Headsprout
Questions	% Agree	% Agree
1. Do you like computer programs that teach reading?	100	100
2. Did you like the CRP?	90	91
3. Did the program help you read better?	90	91
4. Did you like the stories?	90	82
5. Did you like the graphic/visuals	100	91
6. Did you like the workbook activities (Funnix) and computer activities (HS)	90	100
7. Do you think your friends will like the CRP?	70	73

Discussion

This study examined the impact of a summer implementation of two CRPs on the basic early literacy skills of KG through second-grade at-risk students. The results indicated that (a) there was a statistically significant difference between the two groups on the WUF measure in favor of the Funnix program and (b) the Funnix group showed pre–post gains on four of the seven measures (PSF, WUF, NWF, and RF). The HS group showed gains (higher post-test) scores only on the NWF measure. The social validity data indicated that (a) the tutors were very positive regarding the effectiveness, ease of use, and appropriateness of the activities in the Funnix program than the tutors who implemented the HS program and (b) the students in both programs were equally positive regarding the program activities and effectiveness.

A significant finding of this study is that the Funnix CRP program facilitated gains in the areas of phonemic awareness (as measured by PSF), phonics (as measured by NWF), vocabulary (as measured by WUF), and comprehension (RF measures). Of these four, the effects were significant on three of the measures (i.e., the effect size was large for the WUF, medium for the PSF, and small for the RF), which are critical early reading skills. These findings support Watson and Hempenstall (2008) findings that the Funnix program facilitates the growth of basic early literacy skills of elementary-grade students and extends the findings to the at-risk student population. The results from the study provide additional evidence that the Funnix program can be used to provide focused, explicit intervention in the areas of phonemic awareness, phonics, and fluency during the summer break and thereby prevent the compounding of reading deficits across years.

The results from the study indicated that the students in the HS group did not show any significant gains and in fact exhibited low post-test scores on five of the DIBELS measures. The decreases were significant (medium to small) on the LNF, ISF, PSF, and WUF measures. There could be multiple reasons or a confluence of factors for the lack of growth of skills of students in the HS group. First, (a) the summer intervention occurred for over a month (the other month was spent on pre- and post-testing) and (b) all students in the HS group started from episode 1 in the program irrespective of their abilities, as there was no program pre-assessment to determine entry points for students based on their abilities. The HS program is designed as such, as it is a performance guarantee program that promises mastery of basic literacy skills at the completion of the program, which is 80 episodes. The combination of the duration of the intervention and the program placement structure could have prevented the students from learning new skills that were appropriate to their skill gaps, but were targeted later in the program. For example, Huffstetter et al. (2010) also hypothesized that students completing 40 episodes or less might not have been exposed to the skills measured by a standardized test. Second, fewer students in the HS program than in the Funnix program completed 30 or more episodes/lessons. The average number of episodes completed by students in the HS group was lower than that of episodes completed by students in the Funnix group. This could have influenced the results. The average number of episodes completed by the HS students was 28, which was less than the episodes completed by the students in the Huffstetter et al. (2010) study and could have been a factor for the lack of positive findings for this group in this study. Third, six of the 11 students in the HS group were also classified as ESL students compared to three students in the Funnix group and this could have differentially influenced the results. Fourth, a majority of the students in the HS group were KG students and the Funnix group had a relatively higher number of students in the second grade. Given that students in Funnix were placed based on the results on the program pre-assessments could have resulted in more gains on the post-tests.

The results from the social validity assessment also indicated that none of the tutors in the HS program indicated that they would recommend the program nor indicated that they would use the program to help another student. Similarly, the tutors’ responses were not positive on statements regarding students’ comfortableness using the HS program, the usefulness of the HS program in teaching reading skills, and that HS was a good program. This lack of affirmative ratings for the HS program could be due to the tutors’ perceptions that the tasks in the beginning episodes were not appropriate for their students. Furthermore, because HS is a student-centered program requiring the students with ESL to understand the directions and make choices could have influenced tutors’ perceptions. It should be noted that only one of the three HS tutors agreed that students were able to follow directions easily. The factors discussed above of starting every student from episode 1, the ability of the ESL students to follow directions, and the short duration of the intervention did not allow tutors to a complete exposure to activities and skills taught in the program and could have contributed to the lack of positive tutors’ perceptions.

Regarding the appropriateness of teaching activities, there were more positive responses from tutors regarding Funnix than HS. This could be due to students being placed at their instructional level. This appropriate placement, despite the short duration of the intervention, could have also influenced positive responses from both tutors that Funnix was useful in teaching reading skills to the students and that it was a good program. Furthermore, given that Funnix is an adult-navigated program, the tutors were comfortable using the program, and a fewer number of ESL students in the Funnix group could have influenced the tutor's positive responses. Also, all tutors agreed that students were able to easily follow instructions of the narrator and this along with the above factors could have influenced their positive perception of the program and their positive recommendation of the program to others attempting to help students with reading difficulties.

Limitations

There are multiple limitations to this summer study that need to be considered when examining the results. These include (a) the absence of a control group, (b) a small sample size, and (c) the short duration of the intervention. Each limitation is discussed below.

Absence of a control group. The results of the study indicated skills gains on four DIBELS measures for students in the Funnix group and decreases in four areas for students in the HS group. The presence of a control group could have helped understand if the decreases in skills for the students in the HS group were a natural summer learning loss phenomenon or a result of the incompatibility among the student abilities/characteristics, program placement, and short duration of the study. Furthermore, due to the random assignment of the students to the two CRP groups, more students with ESL ended up being assigned to the HS program. Also, the use of raw scores, which are biased by student grade level, could have affected the results (due to the presence of more kindergarteners in the HS group). Additionally, having a control group is essential to understand the unique contributions of CRPs. Thus, additional research with a control condition is essential to understand the role of CRPs in facilitating the reading skills of at-risk students when implemented as a summer intervention program.

Small sample size. The sample for the study consisted of 21 students across three grades and the sample size was small given the number of tests conducted. When sample sizes are small, the power of statistical tests in rejecting the null hypotheses is weak. For this study, despite conducting multiple tests, the null hypothesis was tested at the .05 significance level (two-tailed test) to increase the power of the test. Further, effect sizes were computed to overcome the issue of limited power of the statistical tests. The effect sizes computed did indicate promising results for the Funnix CRP. However, the results should be considered as exploratory as (a) the students were not equally distributed across grades and this could have resulted in some variance in the effect sizes computed for each group and (b) ESL students were not equally distributed across groups. Hence, future studies are necessary to replicate the findings with a larger sample.

Duration of intervention. None of the students in the two groups completed their entire programs. The average number of lessons/episodes was 32 for Funnix and 28 for the HS program. Furthermore, there was a variance in attendance rates and completion of episodes. Some of this is natural to field investigations as families go on a vacation and other competing activities might cause students to miss summer sessions. These factors could have differentially influenced the results of the study. Future studies with longer duration of interventions are essential to attain an understanding of the true effects of these CRPs.

Significant Contributions of the Study

It is essential to support the development of basic literacy skills in the early grades as the failure to do so has debilitating life consequences for an individual. Research suggests reading interventions with at-risk students and students with reading disabilities in early grades is essential as the loss of reading skills in early grades compounded over multiple years can create large skill deficits over the years. Hence, interventions during summer breaks are essential to prevent the decline of reading loss and to promote reading skill acquisition. This study adds to the small literature base on the effectiveness of computer-assisted reading programs in promoting early literacy skills of at-risk readers when implemented by tutors during a summer break. Evaluation of CRPs is indispensable as they can provide support and help overcome the limited knowledge of early literacy instruction of providers and are very cost-effective. More importantly, they can provide explicit instruction, which is essential for improving the literacy skills of students (Kim et al., 2011) and can be implemented with high fidelity. The present study also measured the perceptions of tutors and students regarding ease of use, likeability, and effectiveness of the programs. The information on the ease to use and desirableness is essential for understanding the sustainability of program implementation in practice. Such information facilitates the translation of research to practice by providing teachers, parents, and other practitioners with a complete picture of the programs, aid in their selection, and the effective use of CRP programs.

Footnotes

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The research was supported by Grant H327A040103 from the Office of Special Education Programs of the U.S. Department of Education. We thank the families and children who participated in the study.

ORCID iD

Sekhar S. Pindiprolu

Author Biographies

Sekhar S. Pindiprolu is a professor in the Department of Teacher Education at The University of Toledo. He teaches undergraduate, graduate, and doctoral courses in special education. His research interests and publications focus on computer assisted instruction, teaching and learning strategies, functional behavioral assessments and interventions, and program evaluations. He has authored a number of successful grant proposals and presents regularly at national and international conferences.

David Forbush is the associate director of the Center for the School of the Future at Utah State University. He has applied and research interests in reading interventions, evidence-based school instructional interventions, the science of implementation of evidence-based practices, the use of technology to benefit students in public schools, and the delivery of pre-service and in-service instruction. Drs. Pindiprolu and Forbush co-planned the study. Dr. Pindiprolu conducted data analysis and took the lead in writing the manuscript. Dr. Forbush supervised the implementation of the study, data collection, and provided feedback on the manuscript.

References

Agodini

Dynarski

Honey

Levin

(2003). The effectiveness of educational technology: Issues and recommendations for the national study (Report No. ED498071). Mathematica Policy Research. Retrieved from https://files.eric.ed.gov/fulltext/ED498071.pdf.

Allington

R. L.

McGill-Frazen

(2003). Use students’ summer-setback months to raise minority achievement. Education Digest, 69(2), 19–24.

Blok

Oostdam

Otter

M. E.

Overmaat

(2002). Computer-assisted instruction in support of beginning reading instruction: A review. Review of Educational Research, 72(1), 101–130. https://doi.org/10.3102/00346543072001101

Chambers

Slavin

R. E.

Madden

N. A.

Abrami

Logan

Gifford

(2011). Small-group, computer-assisted tutoring to improve reading outcomes for struggling first and second graders. The Elementary School Journal, 111(4), 625–640. https://doi.org/10.1086/659035

Engelmann

(2002). Funnix 2: Reading. Royal Limited Partnership.

Engelmann

(2003). Funnix reviewers’ guide: A consumer’s guide to evaluating supplemental and intervention reading programs. Retrieved from www.funnix.com/ funnix2012/PDFs/Reviewers_Guide.pdf.

Engelmann

Seitz-Davis

(2001). Funnix: A tutor in your computer. Royal Limited Partnership.

Good

R. H.

Simmons

D. C.

Kaméenui

E. J.

(2001). The importance and decision-making utility of a continuum of fluency-based indicators of foundational reading skills for third-grade high-stakes outcomes. Scientific Studies in Reading, 5(3), 257–288.

Hammond

S. S.

(2012). The effects of the Headsprout early reading program on the literacy skills and on-task behavior of at-risk urban kindergarten students [Doctoral dissertation]. Retrieved from https://etd.ohiolink.edu/!etd.send_file?accession=osu1343103239& disposition=inline.

10.

Huffstetter

King

J. R.

Onwuegbuzie

A. J.

Schneider

J. J.

Powell-Smith

K. A.

(2010). Effects of a computer-based early reading program on the early reading and oral language skills of at-risk preschool children. Journal of Education for Students Placed at Risk, 15(4), 279–298. https://doi.org/10.1080/10824669.2010.532415

11.

Kim

J. S.

Capotosto

Hartry

Fitzgerald

(2011). Can a mixed-method literacy intervention improve the reading achievement of low-performing elementary school students in an after-school program? Results from a randomized controlled trial of READ 180 enterprise. Education Evaluation and Policy Analysis, 33(2), 183–201. https://doi.org/10.3102/0162373711399148

12.

Layng

Twyman

J. S.

Stikeleather

(2003). Headsprout early reading: Reliably teaching children to read. Behavioral Technology Today, 3, 7–20.

13.

Lyon

G. R.

(2001). Measuring success: Using assessments and accountability to raise student achievement. Subcommittee on Education Reform, Committee on Education and the Workforce, U.S. House of Representatives, Washington, D.C. Retrieved from http://www.reidlyon.com/edpolicy/10-measuring-success-using-assessments-and-accountability-to-raise-student-achievement.pdf.

14.

Macaruso

Rodman

(2011). Efficacy of computer-assisted instruction for the development of early literacy skills in young children. Reading Psychology, 32(2), 172–196. https://doi.org/10.1080/02702711003608071

15.

Macaruso

Walker

(2008). The efficacy of computer-assisted instruction for advancing literacy skills in kindergarten children. Reading Psychology, 29(3), 266–287. https://doi.org/10.1080/02702710801982019

16.

Mather

Bos

Babur

(2001). Perceptions and knowledge of preservice and in-service teachers about early literacy instruction. Journal of Learning Disabilities, 34(50), 472–482. https://doi.org/10.1177/002221940103400508

17.

Messer

Nash

(2018). An evaluation of the effectiveness of a computer-assisted reading intervention. Journal of Research in Reading, 41(1), 140–158. https://doi.org/10.1111/1467-9817.12107

18.

National Center for Education Statistics (2018). 2017 Reading Results. Retrieved from https://www.nationsreportcard.gov/reading_math_2017_highlights/files/infographic_2018_reading.pdf.

19.

Pindiprolu

S. S.

Forbush

D. E.

(2021). Comparative effects of computer-based reading programs on the early literacy skills of at-risk students. Journal of Educational Technology Systems, 50(2), 255–272. https://doi.org/10.1177/00472395211040048

20.

Regan

Berkeley

Hughes

Kirby

(2014). Effects of computer-assisted instruction for struggling elementary readers with disabilities. The Journal of Special Education, 48(2), 106–119. https://doi.org/10.1177.0022466913497261

21.

Regtvoort

A. G. F. M.

Leij

A. V. D.

(2007). Early intervention with children of dyslexic parents: Effects of computer-based reading instruction at home on literacy acquisition. Learning and Individual Differences, 17(1), 35–53. https://doi.org/10.1016/j.lindif.2007.01.005

22.

Rogowsky

B. A.

Terwilliger

C. C.

Young

C. A.

Kribbs

E. E.

(2018). Playful learning with technology: The effect of computer-assisted instruction on literacy and numeracy skills of preschoolers. International Journal of Play, 7(1), 60–80. https://doi.org/10.1080/21594937.2017.1348324

23.

Rouse

C. E.

Krueger

A. B.

(2004). Putting computerized instruction to the test: A randomized evaluation of a “scientifically based” reading program. Economics of Education Review, 23(4), 323–338. https://doi.org/doi 10.1016/j.econedurev.2003.10.005

24.

Saine

N. L.

Lerkkanen

Ahonen

Tolvanen

Lyytinen

(2011). Computer-assisted remedial reading intervention for school beginners at risk for reading disability. Child Development, 82(3), 1013–1028. https://doi.org/10.1111/j.1467-8624.2011.01580.x

25.

Sandberg-Patton

K. L.

Reschly

A. L.

(2013). Using curriculum-based measurement to examine summer learning loss. Psychology in Schools, 50(7), 738–753. https://doi.org/10.1002/pits.21704

26.

Slavin

R. E.

Lake

Chambers

Cheung

Davis

(2009). Effective beginning reading programs: A best-evidence synthesis. Review of Educational Research, 79(4), 1391–1465. https://doi.org/10.3102/0034654309341374

27.

Torgesen

J. K.

Wagner

R. K.

Rashotte

C. A.

Herron

Lindamood

(2010). Computer-assisted instruction to prevent early reading difficulties in students at risk for dyslexia: Outcomes from two instructional approaches. Annals of Dyslexia, 60(1), 40–56. https://doi.org/10.1007/s11881-009-0032-y

28.

Watson

Hempenstall

(2008). Effects of a computer based beginning reading program on young children. Australasian Journal of Educational Technology, 24(3), 258–274. https://doi.org/10.14742/ajet.1208

Effects of Computer-Assisted Reading Interventions on Literacy Skill Gains of At-Risk Students During a Summer Break

Abstract

Keywords

Introduction

Literature on CRPs

Method

Participants and Setting

CRPs

Dependent Measures

Results

Interobserver Agreement

Treatment Fidelity

CRP Effectiveness

Tutors’ Perceptions

Students’ Perceptions

Discussion

Limitations

Significant Contributions of the Study

Footnotes

Declaration of Conflicting Interests

Funding

ORCID iD

Author Biographies

References