Sage Journals: Discover world-class research

Abstract

Early literacy development is crucial for academic success, yet the COVID-19 pandemic has widened pre-existing educational gaps, particularly affecting students from low-income households through uneven access to quality remote instruction and learning technology. While educational technology offers promising solutions for personalized learning, many digital tools lack robust evidence of effectiveness. This paper reports a quasi-experimental study investigating the impact of My Reading Academy, an adaptive, game-based literacy program, on early reading skills in pre-kindergarten and kindergarten students. The study involved 1,092 students (402 treatment, 690 comparison) across 14 schools, using propensity score weighting and hierarchical linear models to analyze program impact. Results showed significant gains in literacy skills among kindergarten students using My Reading Academy, particularly in alphabet knowledge. Pre-kindergarten students who mastered more than half of the alphabet knowledge content in the program showed enhanced performance compared to peers. Teachers reported the program effectively supported personalized instruction and classroom equity.

Keywords

personalized learning adaptive games early childhood reading evaluation research

Introduction

Theoretical frameworks in cognitive psychology and educational science underscore the essential role of early literacy skills as harbingers for future academic achievement and life success (Hernandez, 2011; Snow et al., 1998). Reading literacy encompasses multiple interconnected competencies, including letter knowledge, phonological awareness, word recognition, vocabulary, and comprehension (National Institute of Child Health and Human Development (NICHD), 2000). These foundational skills develop sequentially and work in concert to enable fluent reading with understanding. However, recent data from the National Assessment of Educational Progress (NAEP) reveal that merely one-third of 4th graders achieved proficiency in reading in 2022 (National Center for Education Statistics, 2022). Socio-economic (SES) disparities further widen this educational gap. Children from low SES backgrounds not only enter school with comparatively lower literacy skills, but also show larger literacy development gaps compared to their more affluent peers (Reardon, 2013). Educators are tasked with catering to this wide spectrum of reading readiness within the same classroom, often with limited resources and support (Dixon et al., 2014; Reich et al., 2020). The pandemic-induced educational disruption has further intensified these disparities and challenges.

Given the significance of early literacy for children's academic trajectory and the challenges educators face in leveling the literacy playing field, this article focuses on My Reading Academy, an adaptive, game-based digital literacy program designed for children in pre-kindergarten through 2nd grade. A primary objective of this study is to understand how My Reading Academy's adaptive approach impacts overall early literacy development in authentic pre-kindergarten and kindergarten classrooms. A secondary objective is to investigate how teachers leverage My Reading Academy to enhance their ability to provide effective instruction for all learners and address the increased variability in school readiness. This investigation builds on previous correlational studies on My Reading Academy and aims to deepen our understanding of how digital game-based learning (DGBL) can support young learners in learning to read (Bang & Thai, 2022). As the first quasi-experimental evaluation comparing My Reading Academy users to a business-as-usual comparison group, this investigation is the first to establish causal evidence by contrasting outcomes between students using the personalized, adaptive game-based learning program and those engaged in business-as-usual instruction.

Literature

Technology and Early Childhood Education

Concerns about children's media consumption and the optimal use of their time have led groups such as the American Academy of Pediatrics to advocate for limiting young children's screen time. Nevertheless, decades of research have shown that children ages three and older can learn from developmentally appropriate, well-designed, curriculum-based television (Fisch, 2004), suggesting that young learners can benefit from carefully designed digital learning games. Behnamnia et al. (2023), for example, provide an extensive review of 37 studies, and their analysis reveals that digital learning games positively affect young learners by enhancing thinking skills, learning outcomes, and intrinsic motivation. These games often employ pedagogical strategies that focus on cognitive processes, thereby enhancing children's critical thinking, problem-solving, and creativity (Karime et al., 2012; Karwowski & Beghetto, 2019). Therefore, when used thoughtfully, in line with developmentally appropriate practices (NAEYC, 2012, 2022), incorporating technology into early childhood instruction can support young learners.

Game-Based Learning in Early Literacy

Research on DGBL with young children is relatively limited, and few large-scale studies have been conducted with children under the age of 5. Nevertheless, the existing body of evidence suggests that young children enjoy DGBL resources and that DGBL generally helps increase children's motivation to learn, which, in turn, increases their learning (Behnamnia et al., 2023; Schmitt et al., 2018). In a review of literature on the effects of interactive e-book interventions on young children's literacy development, López-Escribano et al. (2021) demonstrated that across 14 randomized controlled trials involving 1,138 children, e-books that are appropriately selected and used can be more beneficial for children's development of phonological awareness and vocabulary than print books. They also showed that e-books, with the enhanced conditions in the software (e.g., animation, audio, video content), along with the systematic adult planning of intervention sessions, led to greater literacy skills for young children than conditions without the technology enhancements.

Evaluations of digital games have also shown the games’ positive effects on children's development of foundational skills identified by the National Reading Panel (NICHD, 2000) as essential skills for later literacy development. Digital games can promote children's alphabet knowledge (Cornito, 2023), phonemic awareness (Jamshidifarsani et al., 2018), phonics (Savage et al., 2013), vocabulary (Dore et al., 2019), fluency (Giacomo Dina et al., 2016), as well as comprehension (Castro & Sevillano, 2022). Additionally, in a meta-analysis of educational apps with high-quality activities grounded in research-based principles, aimed at improving literacy and math outcomes among children in preschool through third grade, Kim et al. (2021) showed that the magnitude of the impact of these programs are similar to the effect of tutoring interventions (ES = 0.37) and early elementary literacy interventions (ES = 0.39).

Leveraging Digital Game-Based Learning for Individualized Instruction

As technology has made it possible for learning to happen anywhere, at any time, digital learning games need to increasingly account for learners’ diverse backgrounds and needs and variability of learning opportunities in time and space (Roll et al., 2018). Faced with the diverse learning needs and varying levels of knowledge and skills in the classroom – exacerbated by the pandemic-related learning loss – educators have the challenging task of appropriately addressing individual student needs (Goddard et al., 2015).

Research comparing the learning gains of students on assessments administered pre- and post-pandemic indicates that there is greater variability in students’ learning post-pandemic (Kuhfeld et al., 2022). In the wake of the COVID-19 pandemic, educational institutions are seeking effective strategies and tools that can help accelerate learning while ensuring that each learner receives personalized instruction that genuinely enhances their learning. Digital game-based learning programs can be especially helpful when customized to suit different learning needs, thereby allowing teachers to offer differentiated instruction (e.g., Haataja et al., 2019).

While previous research demonstrates the potential of digital games for specific literacy skills, several questions remain unaddressed. Individual studies have demonstrated positive effects on discrete skills such as phonemic awareness (Jamshidifarsani et al., 2018) and vocabulary (Dore et al., 2019), but limited evidence exists on how comprehensive, adaptive game-based learning platforms support the full spectrum of early literacy development in authentic classroom settings. Also, while Kim et al. (2021) established the general effectiveness of reading educational apps, there is a need to better understand how teachers can leverage these tools to enhance their instructional capacity, especially given the increased variability in student preparedness post-pandemic (Kuhfeld et al., 2022).

My Reading Academy Intervention

The goal of this study was to test the efficacy of My Reading Academy as an intervention for elementary-age students. My Reading Academy is an adaptive, game-based curriculum designed to help young children develop strong foundational reading skills. It operates on a patented Personalized Mastery Learning System^TM and provides individually tailored paths to learning to read through games, books, and videos (Dohring et al., 2019). Grounded in the science of reading and cognitive development research, the program delivers explicit, systematic phonemic awareness and phonics instruction paired with rich reading and language experiences that help learners develop vocabulary, fluency, and comprehension. The student-facing program consists of game-based activities with adaptive learning trajectories; it is accompanied by an educator center with dashboards and resources that help teachers support students’ learning, as well as offline activities that caregivers can use to extend in-game learning experiences.

Using research about how children learn and an understanding of playful engagement and educational games, the developers built My Reading Academy to provide more than 20 games with over 650 levels, more than 150 books that address 500+ learning objectives, and 900+ connections between the learning objectives. My Reading Academy uses a mastery-based learning approach to deliver differentiated instruction, appropriate scaffolding, and feedback to ensure that every learner masters each skill as they advance through the program. In My Reading Academy, students play through engaging learning experiences that provide explicit, systematic foundational skills instruction paired with rich reading and language comprehension experiences to build reading proficiency (see Figure 1).

Figure 1.

Example reading activities from My Reading Academy.

Theoretical Foundations Underlying My Reading Academy

My Reading Academy's curriculum and activities were informed by an extensive analysis of state and national standards frameworks (e.g., Common Core State Standards) and literature on reading interventions, including The Big Five and the pillars of early reading development identified by the National Reading Panel (National Institute of Child Health and Development (NICHD), 2000). The Big Five are: phonemic awareness (identifying sounds and their articulatory features), phonics (identifying letter-sound correspondences), vocabulary (understanding words and meanings), fluency (reading with speed, accuracy, and expression), and comprehension (understanding a text; shown in Figure 2). Research indicates that phonemic awareness is the strongest predictor of early reading success (Goodman et al., 2010); comprehension is considered the ultimate goal of reading (Petscher et al., 2020); and phonics, fluency, and vocabulary are essential to achieving phonemic awareness and comprehension. The National Reading Panel (NICHD, 2000) concluded that each component should be incorporated into instructional practices and suggested several techniques for effective instruction, including using computerized activities to teach reading. Research has also shown that effective instruction must include explicit and clear instructions (Archer & Hughes, 2011; Pearson & Gallagher, 1983) and be systematic in the scope and sequence of activities (NICHD, 2000), with focus on mapping letters and spellings to the sounds of spoken language represented by letters (Snow et al., 1998).

Figure 2.

My Reading Academy's curriculum is thoughtfully designed to scaffold students’ learning. Skills progress in a sequential manner, beginning with letter and sound recognition, advancing through decoding and encoding words, and culminating in the dictation of sentences before empowering students to read independently in connected texts using decodable readers.

Based on this research, a proprietary knowledge map was created, consisting of granular, measurable learning objectives and pathways toward learning to read. Each learning objective was mapped to create a connected model of reading knowledge and skills, including phonics, phonological awareness, alphabet knowledge, vocabulary, and others. The My Reading Academy knowledge map represents all possible learning trajectories, and this nonlinearity provides the blueprint for creating flexible learning paths that are responsive to each child's individual strengths and needs. This mapping allows My Reading Academy to accommodate learner variability by determining what children know and deciding what they are most ready to learn next, based on this knowledge. Figure 3 below provides a visual representation of a section of the proprietary knowledge map that underlies My Reading Academy's adaptive approach. This structure allows the program to identify multiple entry points based on a child's prior knowledge and create personalized learning trajectories. For example, if the initial placement assessment shows that a student has mastered letter recognition but has some difficulty with letter-sound correspondence, the system will prioritize activities designed to strengthen phoneme-grapheme relationships before advancing to more complex decoding tasks. The knowledge map's granularity enables the system to detect specific skill gaps and adapt content difficulty in real-time based on individual performance patterns.

Figure 3.

My Reading Academy Knowledge map depicting different intertwining topics and a zoomed-in view of a particular set of learning objectives addressing how children can learn to pronounce, read, and understand consonant digraphs.

Mastery-Based Learning and Evidence-Centered Design

The program is grounded in Bloom's (1968) Mastery Learning theory, which posits that all students can learn given needed time and appropriate instruction and advocates for a mastery-based personalized learning approach (see Bingham et al., 2018; Plass & Pawar, 2020). This approach works because it respects learner variability by differentiating instruction, offering appropriate feedback, and ensuring that children master each topic before moving on. The program also integrates evidence-centered design – an assessment framework that enables the estimation of students’ competency levels via in-game learning evidence (Mislevy et al., 2003; Shute, 2011).

Figure 4 shows My Reading Academy's personalized mastery-based learning model, whose key components are aligned with Bloom's Mastery Learning model (1968), including placement assessments; instruction; feedback and correctives; evaluation; and alignment to a hierarchy of learning goals and objectives. It is based on the evidence-centered design framework which provides a principled alignment of the concepts, skills, and abilities that a game is designed to teach with evidence of learning and task design (DiCerbo et al., 2015). My Reading Academy differs from other products in its ability to determine gaps in each child's learning through engaging, interactive, embedded formative and summative assessments (Shute, 2011).

Figure 4.

Personalized mastery-based learning model.

Engaging Children in Reading

My Reading Academy's approach to reading instruction fosters learning through play, where children interact with dynamic learning materials to master learning objectives. Play is essential in children's learning and development (Dietze & Kashin, 2011). It is the mechanism for learning in games, allowing children the opportunity to explore action and meaning in liberating ways (Barab et al., 2005). My Reading Academy leverages games to promote playful engagement, contextual learning, and embedded assessment. The games engage children in activities and stories contextualizing phonics learning in meaningful situations. Children then play the game and receive immediate, specific, and understandable feedback, correctives, and scaffolding based on their decisions in the activities, which help reinforce learning and address misunderstandings immediately (Guskey, 1997).

Learner Experience

When students use My Reading Academy for the first time, they complete in-game placement assessments that determine their prior knowledge and place them into proficiency-appropriate games. My Reading Academy builds on the rope model of reading (Scarborough, 2001) that represents skilled reading as a rope through which many foundational skills are woven. Children learn and practice word recognition and comprehension through activities that offer structured, repeated practice and corrective feedback, leading to accuracy and automaticity with phonemic awareness and phonics skills. Rich reading and language experiences provide modeling, direct instruction, and guided practice to build fluency, vocabulary, and comprehension strategies. My Reading Academy follows the research-based progression for effective literacy instruction (International Literacy Association, 2019). (see Figure 5 for examples of games in My Reading Academy).

Figure 5.

My Reading Academy includes 20 games and 150 books that help students learn skills in phonological awareness, phonics, vocabulary, comprehension, and fluency.

Developmentally Appropriate Tasks

Research in human cognition and development informed various aspects of the interaction design of My Reading Academy. Frequent user testing and application of design research practices (Design-Based Research Collective, 2003) informed insights that drove design iterations. For example, with the user interface design, developers considered cognitive load, i.e., the amount of working memory used in a given context. Designers made decisions to include minimal elements on screen required for each learning game. Furthermore, motor skill considerations shaped specific interaction design, iterating with user testing data to support tap, drag, and drop interactions calibrated to the target age group for each game. Executive function capability was also a consideration in designing the complexity of games, to keep the layers of instruction, visual and verbal cues, and problem-solving steps at appropriate levels for early learners.

Current Study

This study builds on an earlier correlational study which showed a positive relationship between My Reading Academy usage and literacy achievement (Bang & Thai, 2022). The initial study did not include a comparison group, so the current quasi-experimental study aims to understand if students who use the program demonstrate higher learning than comparison students. The main research question guiding the study is: Do pre-kindergartners and kindergarteners who use My Reading Academy demonstrate higher literacy skills than comparison students controlling for pretest, demographic, and classroom characteristics?

Methods

Design and Participants

The study took place between October 2021 and May 2022 and used a quasi-experimental design. The authors partnered with an independent nonprofit research organization with nearly 80 years of history working in research and development for government and industry and collaborated on a recruitment approach that leveraged the teacher network of the authors’ organization and district-level contacts of the nonprofit research organization. A recruitment email was distributed to district leaders, principals, and program directors, which was accompanied by a flyer with information about the study, a screening survey, and an invitation to schedule a call to learn more about the study. Additionally, a screening survey was distributed to about 200,000 early childhood educators, in which they were asked about: their level of interest in using My Reading Academy in pre-kindergarten or kindergarten; whether they anticipate having a class size of 10 or more students; degree of classroom internet access; whether they administer Star Early Literacy, Phonological Awareness Literacy Screener, Texas Kindergarten Entry Assessment, or CIRCLE Progress Monitoring Assessment in pre-kindergarten or kindergarten at least two times a year; and whether they anticipate that their students will be able to use My Reading Academy for at least 60 min per week during the school year. Educators also provided the contact information of their principals. The study team met with interested district leaders, principals, and program directors to confirm participation. This process resulted in the recruitment of four school districts across two states (2 in Virginia – districts A and B, 2 in Texas – districts C and D). Table 1 summarizes the characteristics of the participating districts, including the demographic information used in propensity score weighting.

Table 1.

Characteristics of Participating School Districts.

District	Locale	White	Hispanic	African American	Asian	Other Ethnicity	Female
A	Small City	55%	6%	34%	2%	3%	55%
B	Rural	27%	46%	21%	5%	1%	49%
C	Remote Town	28%	1%	70%	0%	1%	47%
D	Large Suburb	43%	19%	27%	2%	9%	38%

Ethics Approval and Consent to Participate

The study and all protocols were approved by Health Media Lap IRB (https://www.healthmedialabirb.com), an independent fee-paying institutional review board, which is fully accredited by the non-profit Consortium of Independent Institutional Review Boards.¹ Informed consent was obtained from teachers for their participation in the study. Notification letters were sent to parents, informing them of the research study and their child's participation in the study. Parents were offered the opportunity to have their children opt out of the study with no penalty. All research activities were carried out in accordance with relevant guidelines and regulations.

The study sample included pre-kindergarten and kindergarten classrooms. All teachers, schools, and districts were offered a stipend to participate in the study. Students were excluded from the baseline sample if they were in the control group but had My Reading Academy usage, were English Language Learners, had Special Education status, or were under four years old. After identifying the sample of students who met the initial eligibility criteria, 368 students were excluded from the baseline sample if they were missing data on any of the variables used for propensity score weighting (district, age, grade level, race/ethnicity, gender, phonemic awareness pre-test score, and alphabet knowledge pre-test score). Across the districts, 570 students who used My Reading Academy and 890 students who did not use the program were matched and weighted based on their context, demographics, and pre-test scores.

The attrition rate for the treatment (10.9%) and comparison sample (6.96%) gave an overall attrition rate of 17.86% and a differential attrition rate of 3.94%. These rates meet What Works Clearinghouse (WWC) attrition standards (WWC, 2020).

Across the two grades, the sample included 30 treatment and 46 comparison classrooms. A total of 1,092 students were included in the analytic sample (402 students (222 pre-kindergartners and 180 kindergarteners) who used My Reading Academy and 690 students (251 pre-kindergartners and 439 kindergarteners) who used other reading programs in the comparison group, which included Fountas and Pinnell, Frog Street, Handwriting without Tears, Heggerty, and/or Orton Gillingham). On average, the pre-kindergarten students were 4.37 years old (SD = 0.38) and kindergartners were 5.44 years old (SD = 0.35). This sample included 541 females and 551 males, of whom 37% were White, 32% African American, 26% Hispanic, 3% Asian, and 2% identified as Native American, mixed race, or other. Table 2 displays the numbers of pre-kindergarten and kindergarten classrooms and students in each district.

Table 2.

Number of Students and Classes by Grade and District (Total n = 1,092, Total Classrooms = 76).

	Pre-kindergarten		Kindergarten
District	Students	Classes	Students	Classes
A	75	5	250	16
B	159	11	369	26
C	132	11	0	0
D	107	7	0	0
Total	473	34	619	42

Measures and Data Sources

Student Reading Assessment

The research partner collected district records of students’ scores on standardized literacy assessments in January and May 2022 for both treatment and comparison classrooms. As part of their standard practice, participating pre-kindergarten teachers administered the Phonological Awareness Literacy Screening-PreK (PALS-PreK; Invernizzi et al., 2015) or the CIRCLE Progress Monitoring System (Children's Learning Institute at UTHealth Houston, 2023). Participating kindergarten teachers administered the Phonological Awareness Literacy Screening-Kindergarten (PALS-K) at least twice during the school year.

The PALS PreK, the PALS-K, and the CIRCLE were selected as early literacy outcome measures because they assess the foundational reading skills targeted in My Reading Academy, particularly phonemic awareness and phonics skills, including letter naming. All assessments have strong established psychometric properties (see Table 3).

Table 3.

Subtests Included in Analyses by Literacy Assessment.

			Subtest
Assessment	Grade	District	Alphabet Knowledge	Phonics	Phonemic Awareness
PALS PreK	PreK	A, B	Alphabet knowledge	Letter sounds	Beginning sound awareness
PALS K	K	A, B	Alphabet knowledge	Letter sounds	Beginning sound awareness
CIRCLE	PreK	C, D	Rapid letter naming	Letter-sound correspondence	Phonological awareness

My Reading Academy Usage Data

Treatment teachers were asked to use My Reading Academy for at least 60 min per week (20-min sessions, 3 times/week) as a supplement to their core curricula. The usage data include time, activities completed, performance on the in-game preassessment, games accessed, skills and learning objectives targeted in the games, and all user interactions within the app.

Teacher Surveys

Online teacher post surveys (one for the treatment and one for the control group) were delivered via email and text messages. All treatment and comparison group teachers received the survey, and follow-ups were conducted with non-responsive teachers approximately twice per week for two weeks. Teachers provided information about their literacy curricula and the implementation, integration, enjoyment, and challenges of their reading educational technologies (My Reading Academy or another technology). The completion rate was 93% for treatment teachers (n = 28) and 43% for comparison teachers (n = 19).

Teacher Interviews

All 30 treatment teachers were invited to participate in an end-of-study interview, and 15 interviews were conducted in May and June 2022, each lasting for about 45 minutes. All interviews were conducted virtually using Zoom, accommodating to teachers’ schedules and preferences. Semi-structured interviews were chosen as the data collection method as they provide a systematic approach while allowing flexibility to probe deeper into teachers’ unique experiences and perspectives (Galletta, 2013). The interview protocol included open-ended questions organized around four domains: a) existing reading curriculum and instructional practices; b) My Reading Academy implementation processes and challenges; c) perceived impact on student reading skills; and d) recommendations for program improvement. This format enabled the study team to gather consistent data across participants while allowing teachers to elaborate on topics they deemed most relevant to their experience.

Analysis

Z-Score Transformation of the Student Reading Assessments

Participating districts administered different standardized literacy assessments; therefore, the research team combined scores on different assessments of the same early literacy skills (see Table 1) into a consistent outcome measure by converting scores into z-scores. This approach is based on guidance from the What Works Clearinghouse (WWC) standards developers, WWC study reviews, and published guidance documents (May et al., 2009; Lipsey et al., 2012; What Works Clearinghouse, 2020).

Propensity Score Weighting

Propensity score weighting was used to minimize potential biases introduced by baseline differences in the treatment and comparison group and to approximate findings obtained from randomized controlled trials (Becker & Ichino, 2002; Moons, 2020). This method attempts to equalize the mean values of potentially confounding observed covariates in the treatment and comparison groups. Inverse propensity weighting was used so that the weight of the treated students was 1; weight of the comparison students was $\frac{p_{i}}{(1 - p_{i})}$ , where $p_{i}$ is the propensity score for the i-th comparison student. The following characteristics were used to compute the propensity scores: district, age, grade level, race/ethnicity, gender, phonemic awareness pre-test score, and alphabet knowledge pre-test score. Teacher characteristics were not included in computing propensity weights due to the limited comparison group teacher survey data, which would have significantly restricted the sample size. Several key assumptions were tested to inspect the appropriateness and fit of the propensity scores. After weighting, the treatment and comparison scores were more evenly balanced (e.g., drastically lowered standardized mean differences), and other tests indicated a good balance of covariates.

Baseline equivalence of students in the treatment and comparison groups was evaluated using propensity score weights as recommended by the WWC Standards v. 4.1 (What Works Clearinghouse, 2020). Hedge's g and Cox's Index were used to evaluate the effect size difference of continuous and dichotomous variables, respectively. All covariates met the recommended guidelines for baseline equivalence. The propensity score weighted results, therefore, mimic a randomized controlled trial design. All domains reported met baseline equivalence requirements set by the WWC.

Hierarchical Linear Models

A two-level hierarchical linear model (HLM) was used to account for the nesting of students (level 1) within classrooms (level 2) and estimate the impact of My Reading Academy on student outcomes (Raudenbush & Bryk, 2002).

In the model below, subscripts i and j denote students and teachers, respectively. Reading represents student achievement in reading, and Treatment is a dichotomous variable indicating student enrollment in a classroom assigned to the treatment condition. Pre_Reading represents the baseline measure of reading performance (alphabet knowledge and phonemic awareness scores). Age, Gender, and Ethnicity correspond to student-level demographic variables. The intervention effect is represented by $β_{1},$ which captures the treatment/comparison differences in changes in the outcome variable between the pretest and posttest. $r_{0 j}$ is a random effect term, the variance component of which accounts for the nesting of students within classes; $e_{i j}$ represents the student-level random error term.

R e a d i n g_{i j} = β_{0} + β_{1} (T r e a t m e n t)_{j} + β_{2} (P r e_R e a d i n g)_{i j} + β_{3} (A g e)_{i j} + β_{4} (G e n d e r)_{i j} + β_{5} (E t h n i c i t y)_{i j} + T e a c h e r_{0 j} + r_{0 j} + e_{i j}

Separate regression models were used to estimate the effect of My Reading Academy for Alphabet Knowledge, Phonics, and Phonemic Awareness, as well as by grade. Hedge's g was calculated using the HLM estimated difference in outcome between treatment and comparison groups, divided by the pooled standard deviation on that outcome measure. In this sample, the intraclass correlation across teachers was 0.09 for Alphabet Knowledge, 0.15 for Phonemic Awareness, and 0.13 for Phonics.

Qualitative Data Analysis

Interviews were audio-recorded with participant consent and transcribed verbatim. A hybrid approach to coding was employed, which combines deductive and inductive methods (Fereday & Muir-Cochrane, 2006). Initial codes were developed deductively based on the research questions and interview protocol domains. This was followed by inductive coding to capture emerging themes from the data. The coding process included three stages: 1) structural coding to organize responses according to interview protocol domains, followed by descriptive coding to identify basic topics and in vivo coding to capture participants’ voices and experiences (Saldaña, 2021); 2) pattern coding to identify emerging themes and relationships between codes; 3) thematic analysis to synthesize the coded data to overarching themes that addressed the research questions. The final coding scheme included implementation context / environment, student engagement with the program, perceived effectiveness of the program, and suggestions for program refinement. Two coders independently coded all transcripts and met to resolve any discrepancies until consensus was achieved.

Results

My Reading Academy Usage

Pre-kindergarten students used My Reading Academy (n = 222) on average for 44.2 minutes per active week (SD = 15.9) over 16.9 active weeks (SD = 6.4). They spent, on average, 13.3 hours (SD = 8.15) using My Reading Academy and completed an average of 50.7 Learning Activities (SD = 23.7).

Kindergarten students used My Reading Academy (n = 180) on average for 39.9 minutes per active week (SD = 18.1) over 21.2 active weeks (SD = 4.91). They spent, on average, 14.8 hours (SD = 7.2) using My Reading Academy and completed an average of 67.9 Learning Activities (SD = 26.5).

Overall Impact

Kindergarten

Kindergarteners who used My Reading Academy made significantly greater gains on the standardized end-of-year literacy assessment than the comparison group, especially on Alphabet Knowledge (see Figure 6).

Figure 6.

Change in raw Alphabet Knowledge score from fall to spring for kindergarteners who used My Reading Academy (n = 180) compared with those who did not (n = 439).

Comparing Kindergarten students who used My Reading Academy and those who did not on the literacy subskills assessed showed that those who used the program outperformed the non-users on alphabet knowledge (β = 0.26, SE = 0.08, p < .01, g = 0.32). Kindergarten students who used My Reading Academy also had higher phonics scores (β = 0.22, SE = 0.11, p < .05, g = 0.25); however, this difference is not significant because the p-value does not meet requirements for multiple comparisons. There were no significant differences in phonemic awareness (see Table 4 for all models).

Table 4.

Propensity Score Weighted Impact of Treatment on Posttest Score for Kindergarteners.

Grade	n	$β$	SE	p	g
Alphabet Knowledge	619	0.26	0.08	p < .01	0.32
Phonics	619	0.22	0.11	.0495	0.25
Phonemic Awareness	619	−0.04	0.09	.64	−0.05

Using My Reading Academy was especially beneficial for kindergarteners who completed more than 16 alphabet knowledge activities (i.e., mastered half of the alphabet knowledge content in the program). Comparing Kindergarten students who used My Reading Academy and those who did not on the literacy subskills assessed showed that those who used the program outperformed the non-users on alphabet knowledge (β = 0.31, SE = 0.08, p < .001, g = 0.37; Figure 7) and phonics (β = 0.27, SE = 0.10, p < .01, g = 0.32; Figure 8). There were no significant differences in phonemic awareness.

Figure 7.

Change in raw Alphabet Knowledge score from fall to spring for comparison group kindergarteners (n = 439), all students who used My Reading Academy (n = 180), and those who mastered 16 + alphabet knowledge skills in My Reading Academy (n = 143).

Figure 8.

Change in raw Phonics score from fall to spring for comparison group kindergarteners (n = 439), all students who used My Reading Academy (n = 180), and those who mastered 16 + alphabet knowledge skills in My Reading Academy (n = 143).

Pre-Kindergarten

No significant differences were observed between pre-kindergartners who used the program compared with those who did not on each of the three scores (see Table 5).

Table 5.

Propensity Score Weighted Impact of Treatment on Posttest Score for Pre-Kindergarteners.

Grade	n	$β$	SE	p	g
Alphabet Knowledge	473	−0.08	0.10	.45	−0.08
Phonics	458	−0.23	0.14	.1	−0.23
Phonemic Awareness	473	−0.16	0.16	.32	−0.16

However, among pre-kindergarteners who used My Reading Academy, those who mastered at least 16 alphabet knowledge skills in the program (n = 125) (i.e., half of the alphabet knowledge content in the program) had a greater change in Alphabet Knowledge score than comparison students, although this difference is not significant (Figure 9).

Figure 9.

Change in raw Alphabet Knowledge score from fall to spring for pre-kindergarteners in the comparison group (n = 251), all pre-kindergarteners who used My Reading Academy (n = 222), and those who mastered at least 16 alphabet knowledge skills in My Reading Academy (n = 125).

Teachers’ Perceptions of Impact

On the surveys, teachers were asked which skills they covered in their literacy curriculum this school year. Responses showed that more treatment group teachers were able to cover grade level skills (19 of 22 skills presented). The top 6 most covered concepts are shown in Figure 10.

Figure 10.

Percent of teachers reporting coverage of reading skills during literacy instruction.

Teachers were also asked to rate their confidence in planning, instruction, and assessment on a scale of 1 to 5 (1 = not confident to 5 = extremely confident). Results indicated that treatment teachers had higher rates of confidence across 9 of the 11 skills than comparison group teachers. Figure 11 shows the top four skills and the percentage of teachers who rated themselves as being “very” or “extremely” confident in that category.

Figure 11.

Teacher confidence in specific skills.

Teachers also rated the extent to which they agreed with statements regarding My Reading Academy. Of the 28 treatment teachers who completed the survey:

96% agreed that “I want to continue using My Reading Academy in my class.”

93% agreed that “I understand how My Reading Academy personalizes learning for my students.”

89% agreed that “My Reading Academy was easy for me to use as a teacher.”

85% agreed that “My students enjoyed using My Reading Academy.”

While these survey results are subjective and retrospective, they are a meaningful source of data about the implementation and usefulness of the My Reading Academy.

Teacher Interviews

Interviews were conducted to learn more about how teachers used My Reading Academy, and in response to semi-structured questions, teachers shared anecdotes and feedback about using the program during the 2021–2022 school year. Overall, teachers shared their perceptions of My Reading Academy as a valuable learning resource that is effective, engaging, empowering, and equitable. The following section summarizes and unpacks exemplar quotes from teachers that help define, explain, and contextualize each of the four themes identified across the interviews.

Effective

Teachers widely reported that My Reading Academy effectively assisted students in developing foundational literacy skills, particularly for those who found early reading concepts challenging. When explaining how My Reading Academy proved effective in aiding students to build these foundational skills, teachers often mentioned improvements in standardized test scores:

“I think it's been very effective. We can tell from their fall PALS [Phonological Awareness Literacy Screener] and then using My Reading Academy to work on those skills that needed a little bit more love. All but one kid passed the PALS” (PreK, District A).

“I’ve done PALS over the past nine years …and this year you can see a growth that wasn’t there before, especially not having to pull out that individual rhyme book is great. … I think the ones that did not have an involved parent, or they don’t have books at home, I think they benefited the most” (K, District A).

Other teachers further described how they saw growth in skills and test scores for struggling learners:

“I think my struggling learners benefited more because they were the ones that needed that extra love and support. We had one kid gain 74 points from the beginning of the year to the end of the year. He didn’t have a good foundation of academics and rhyming. But now that he's got the skills, he was more engaged at the end of the year doing the robot [referring to one of the characters in My Reading Academy], mastering the skills because he had confidence and letter knowledge that made it easier for him to be more successful” (PreK, District A).

Across interviews, teachers explained how the program helped those struggling with foundational literacy skills. Teachers observed this growth during instruction, and it was also measurable on external assessments. Many teachers cited measurable improvements in standardized assessments such as PALS, noting that students who used the app demonstrated significant growth. This effectiveness was especially pronounced among students without strong home literacy support, as the structured learning environment of My Reading Academy provided them with consistent exposure to literacy skills.

Engaging

Across classrooms, teachers observed that My Reading Academy was highly engaging for students, that is, helping them stay focused on learning to read. Teachers knew their students were engaged because they were able to focus on the activities for long periods of time without needing to ask others for help. One teacher described:

“For my lower students, it really engaged them… when they were on their tablet, they were not coming over and asking me any questions. They were focused on what they were doing” (K, District A).

Teachers also observed engagement as students wanted to tell their teacher about successes (indicated through in-game celebrations for skill mastery) and new activities. For example:

“They love the robot, and when the robot danced, they stood up and started dancing and you can see them focus on the stories or they come up and tell me, I got a new book or if it was something really was interesting, they would come to let me know. I thought it was pretty exciting when they would come to show me something that they were excited about (PreK, District C).

Students were not only excited to finish activities, but also proud of their progress, wanting share and their accomplishments with their peers and teachers.

Some students were so engaged with My Reading Academy that they wanted to continue playing at home.

“I have one little girl who really was engaged with it most of the time, and it worked for her. Her Mom sent me messages asking for the at-home codes because she would ask to play at home. … I think that having the reinforcement that came through My Reading Academy was definitely one of the factors that helped springboard her into knowing a lot more letters and sounds, being able to put them together” (K, District B).

Students remained focused on activities for extended periods, worked independently, and were eager to share their progress with peers and teachers. Teachers described how students reacted enthusiastically to in-game celebrations and actively sought opportunities to continue using the app at home. This sustained engagement contributed to a positive learning experience and increased student motivation.

Empowering

Interviews further revealed that My Reading Academy was empowering for not only students, but also for teachers, as it provided information that enabled them to plan lessons, provide tailored instruction, support specific students, or communicate information with others. The teacher dashboard in My Reading Academy helped teachers gain a sense of classroom progress and identify common struggles. A kindergarten teacher recounted:

“I can use the information [on dashboard] especially if it seems to coincide with what I’m seeing in small groups. I can see okay well this one is struggling with rhyme in small group also, and this confirms that. So it helps when it confirms to know what is a struggle across the board. I move kids depending on what the needs are so it's helpful to use that as a grouping tool along with my own data” (K, District B).

Other teachers also noted how the program was also empowering for students:

“My lower kids, they really benefited from it because it pushed them to be independent and not rely on their peers. For my higher kids, I definitely saw that it was a nice push for them because they were able to get through the beginning stages really quickly and move on further from there. It was really nice that it catered to their individual needs” (PreK, District D).

The program's personalization in the program allowed students with lower skills to be challenged to learn independently without relying on their peers. Students, particularly those who struggle with literacy, gained confidence and independence in their learning. The app's adaptive nature also permitted higher-level students to progress at their own pace, ensuring all learners faced appropriate challenges.

Equitable

A final theme from interviews was how My Reading Academy promoted equitable classrooms by addressing the needs of diverse learners. Teachers appreciated the app's ability to personalize instruction based on individual student needs, ensuring that all learners had access to appropriate learning experiences. They recognized that it offered equitable learning opportunities to students of diverse needs by placing students in activities personalized to their skill level.

“I think it's equitable. If you had exposure to letters and reading, then it puts you where you need to be. If you’ve never seen anything, then it puts you exactly where you needed to be. If you were ready to move on and start that reading and rhyming and stuff, then it put you there. Everybody had choices no matter what level they were on” (PreK, District A).

Students were able to start at their skill level and then progress as they were ready. Additionally, the personalized learning experience fostered student agency:

“I think [for] my kids with sensory and behavioral issues it was really good for them because it blocked out the rest of the activity in the room. And I think for my lower-level readers, the reinforcement of recognizing letters and recognizing sounds was very beneficial for them. … My higher end kids, the kids who already knew how to read before they got to kindergarten, I think they enjoyed it.” (K, District B).

Additionally, teachers noted that students with sensory or behavioral challenges benefited from the app's structured, distraction-free learning environment. As illustrated in these examples, My Reading Academy was able to address learner variability and promote more equitable classrooms and serve students with diverse needs.

Analyses across all 15 teacher interviews revealed several insights specific to classroom-level implementation. These themes emerged consistently across multiple districts and were supported by teachers’ direct observations, student progress data, and classroom experiences.

Teachers consistently described how the program supported student independence, noting particularly how struggling learners benefited from the scaffolding and adaptive feedback offered in the program. A recurring observation was how the program's engagement features – such as the robot animations and skill mastery celebrations – helped sustain student motivation during independent work time. This engagement aspect proved especially valuable for younger learners who typically struggled with sustained attention. Teachers also noted shifts in student behavior, describing how successful experiences with the program translated into increased confidence during other literacy activities (e.g., greater willingness to participate in class, more persistence when faced with challenges in reading).

The interviews also revealed how teachers used program data to inform their understanding of student progress. Teachers described using dashboard information to confirm their observations from small group instruction and identify areas where students needed additional support.

These classroom-level observations suggest that beyond its direct impact on literacy skills, the program provided teachers with valuable insights for supporting individual student learning needs.

Discussion

Overview of Key Findings

This study provides evidence that a personalized, mastery-based game can meet individual student's literacy learning needs. Results showed that overall, kindergarteners who used My Reading Academy outperformed their comparison group peers on standardized literacy tests, especially in alphabet knowledge and phonics, with effects most pronounced among students who engaged consistently with the program. These results align with López-Escribano and colleagues’ meta-analysis (2021) of interactive literacy interventions, while extending their findings to show how adaptive games can effectively support foundational skills development. The effect sizes observed in this study (g = 0.32 for alphabet knowledge) are comparable to those in Kim and colleagues’ meta-analysis of educational apps, suggesting that well-designed digital interventions can contribute meaningfully to early literacy instruction.

While kindergarteners demonstrated significant gains, particularly in alphabet knowledge, pre-kindergarten results require careful interpretation. A significant ransomware attack in one district prevented technology access for approximately 4 months, primarily affecting pre-kindergarten classrooms and reducing their exposure to the intervention by one-third of the intended duration. This disruption, combined with developmental factors typical of pre-kindergarten students such as shorter attention spans (Bassok et al., 2016) and varying school readiness levels (Blair & Raver, 2015), likely contributed to the lack of significant positive outcomes in this age group. These implementation challenges highlight the importance of consistent access and developmentally appropriate support when implementing digital learning tools with very young children.

Additionally, the relationship between measured outcomes and classroom implementation reveals important nuances in how such tools can best support teaching and learning. While quantitative results showed modest overall effects, teacher interviews and surveys revealed broader impacts on classroom dynamics and instructional practices. Teachers reported increased confidence in literacy instruction and valued the program's ability to support differentiated instruction – findings which extend Goddard and colleagues’ work (2015) on teacher efficacy in addressing diverse learning needs. This is particularly significant given the increasing variability in student preparedness in the post-pandemic context (Kuhfeld et al., 2022).

Finally, the apparent disconnect between quantitative outcomes (especially for pre-kindergarteners) and teacher perceptions warrants careful consideration in implementation planning. While teachers consistently reported strong positive impacts, the quantitative results suggest a need for realistic expectations and strategic integration. Success appears to depend on viewing digital tools as complementary to, rather than replacements for, effective teacher-lead instruction.

Pedagogical Implications and Recommendations

Analyses of teacher interviews, classroom implementation data, and educator survey responses revealed implications for substantial pedagogical practices that vary across educational contexts. These data sources provided complementary perspectives: interviews offered insights into day-to-day practices, surveys captured broader patterns across classrooms, and usage data helped validate reported implementation approaches.

Our findings suggest that successful integration of digital learning tools requires thoughtful attention to both implementation strategies and instructional design. Teachers in the study found particular success when using My Reading Academy as part of a comprehensive literacy approach, rather than as a standalone intervention. They leveraged the program's adaptive capabilities to create more flexible instructional groupings, using dashboard data to inform small-group composition and targeted instruction. This approach allowed them to simultaneously support struggling readers through scaffolded, self-paced learning while enabling more advanced students to progress at their own pace.

The study also offers several guidelines for effective implementation of personalized learning through digital tools. First, consistent engagement is crucial, as students who completed the majority of the content in a given skill showed greater gains, suggesting the importance of establishing regular usage routines to enable students to make progress in the program. Second, the program's effectiveness was enhanced when teachers used progress monitoring data to adjust their instruction. Teachers reported using the data reported on their dashboard not only to track progress but also to identify common skill gaps and inform their whole-class instruction.

Implementation success varied across different educational contexts based on available resources and infrastructure. Schools need to carefully consider device availability and internet reliability. Teachers developed structured schedules to maximize limited technology access while maintaining instructional continuity. The implementation data showed that establishing clear routines and procedures was particularly important in early childhood settings, where shorter, more frequent sessions of 15–20 minutes proved more effective than longer periods.

Teacher Perceptions and Implementation Success

Analyses of teacher survey and interview data revealed that treatment group teachers reported higher confidence in literacy instruction and valued My Reading Academy as a powerful tool for personalized learning. They particularly appreciated the program's ability to differentiate instruction and meet students at their current reading levels. This adaptability created opportunities for teachers to engage in other instructional activities or provide targeted instruction to smaller groups of students. Teachers consistently recognized My Reading Academy as a valuable learning resource that effectively improved test scores, fostered independent learning, empowered them to track student progress, and equitably addressed classroom learning variability.

Supporting Diverse Learners Through Adaptive Technology

These findings underscore the critical need to continue developing adaptive support for children and creating instructional materials that help teachers effectively support diverse learners. The study also demonstrates the value of personalized learning and the potential of digital learning games to successfully provide appropriately leveled content across the learning spectrum – from basic content for early learners to more challenging content for advanced students.

Professional Development Implications

The study further reveals important considerations for supporting teacher professional development around technology integration. Teachers require comprehensive support, including initial training on program features, integrating digital tools with existing curricula, interpreting and acting on student data, opportunities to collaborate with colleagues, and maintaining appropriate balance between computer-based and teacher-led instruction. Professional learning communities may be valuable for sharing successful integration strategies and troubleshooting challenges.

Limitations and Future Research

This research, conducted during a school year disrupted by the pandemic, encountered numerous challenges, including prolonged student and teacher absences and school closures. Additionally, one of the districts where most of the pre-kindergarten classrooms were located was targeted with a ransomware attack that prevented the use of My Reading Academy and other technology-based instructional materials for about 4 months. These implementation challenges may partly explain the disconnect between strong teacher perceptions and more modest quantitative results, especially among pre-kindergarteners.

An additional limitation relates to the comparison group design. As noted in the Methods section, students in the comparison group used various reading applications and programs. This variation in instructional approaches within the comparison group may have introduced variability in the comparison data, as different applications likely produced different learning outcomes. While our analysis treated the comparison group as a unified condition, a more granular analysis examining the differential effects of specific applications would have provided valuable insights. However, in the current study, we did not have access to the usage data of students using the other applications, making such comparative analyses impossible. Ideally, future research should consider designs that allow for comparison between My Reading Academy and specific alternative digital literacy programs, with detailed implementation and usage data collected across all conditions.

The study's timeframe presents another significant limitation, particularly regarding our ability to assess the long-term impacts of the intervention. While immediate gains were observed among kindergarten students, important questions remain about the sustainability of these improvements and their influence on later reading development. Research indicates that the long-term effects of early reading interventions can vary (e.g., Hurry & Sylva, 2007), and sometimes, effects might show several years after the implementation of the intervention (van der Weijden et al., 2024).

Future studies should address these limitations through carefully designed longitudinal research that ideally tracks students through upper elementary grades. Such studies would help illuminate how early digital literacy experiences shape students’ later engagement with technology-enhanced learning environments. Additionally, extended observation periods would enable more detailed analyses of the relationship between implementation fidelity and long-term outcomes, helping to identify which aspects of program implementation are most crucial for sustained literacy development.

Despite the limitations, this study offers evidence that a supplemental, mastery-based, personalized reading program, grounded in research and data-driven learning engineering, can significantly enhance foundational reading skills in young learners while keeping them engaged in learning.

Future studies can build on this research by incorporating more data collected from comparison group classrooms, which could provide detailed insights into implementing My Reading Academy and other literacy curricula. Additionally, future investigations can focus on the use of the program outside of the classroom, which could contribute to the body of knowledge about the role of digital learning resources in young learners’ ecosystem.

Conclusion

The results of this study support the potential of digital learning games to enhance early literacy instruction, but they also underscore the complexity of effective implementation. Success depends on thoughtful integration within comprehensive literacy programs, consistent student engagement, and active teacher use of progress monitoring data to inform instruction. As schools continue to seek ways to address growing learner variability, these insights can guide more effective implementation of personalized learning technologies.

The growing interest in digital game-based learning and the evaluation of independent learning tools by educators, for use both in and outside the classroom, underlines the importance of evidence presented in this study. The insights gained from this study also guide current enhancements to My Reading Academy, particularly in developing activities that build reading readiness among very young learners by providing them with regular, diverse exposure to basic literacy concepts like the alphabet, letter-sound correspondence, and concepts of print.

Teachers’ recognition of My Reading Academy as a valuable educational tool, especially for its ability to adapt to individual student needs, has prompted the development of additional resources to help educators customize instruction beyond the app itself. This evolution from a standalone application to a comprehensive instructional support system reflects the broader understanding that effective educational technology must serve both student and teacher needs. Future research will evaluate the effectiveness of these new features while expanding our understanding of successful My Reading Academy integration in typical classroom environments.

These developments, coupled with the study findings, suggest that well-designed digital learning tools can play a crucial role in supporting early literacy development when implemented thoughtfully and systematically. As the educational landscape continues to evolve, the evidence presented here provides valuable guidance for both the development and implementation of educational technology that serves the needs of diverse learners and educators.

Footnotes

Declaration of Conflicting Interests

The authors declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: Both authors are full time employees of Age of Learning, the organization that developed My Reading Academy, the program that is the focus of this study.

Funding

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The research was funded by Age of Learning, Inc., which commissioned and paid an independent nonprofit research organization to plan and carry out the research.

ORCID iDs

Hee Jin Bang

Amanda Siebert-Evenstone

Notes

Author Biographies

Hee Jin Bang, PhD, serves as vice president of Efficacy Research and Evaluation at Age of Learning, Inc., leading the development and management of the organization's efficacy research strategy. Her area of research includes education technology, evaluation of educational programs, early childhood development, and education of English language learners.

Amanda Siebert-Evenstone, PhD, is a learning scientist and efficacy researcher. Her work focuses on the design and evaluation of innovative educational technologies, with an emphasis on mixed methods research, learning analytics, and discourse analysis in virtual learning environments.

References

Archer

Hughes

(2011). Explicit instruction: Effective and efficient teaching. Guilford.

Bang, H. J. & Thai, K. P. (2022). My Reading Academy^{T
M} helps prekindergarten and kindergarten students strengthen early reading skills and become more interested in learning to read: A case study of implementation during the pandemic. Research Brief. Age of Learning, Inc. https://www.ageoflearning.com/wp-content/uploads/2025/02/MRA-2022-TX-RB_RGB_28.pdf

Barab

Thomas

Dodge

Carteaux

Tuzun

(2005). Making learning fun: Quest Atlantis, a game without guns. Educational Technology Research and Development, 53(1), 86–107. https://doi.org/10.1007/BF02504859

Bassok

Latham

Rorem

(2016). Is kindergarten the new first grade? AERA Open, 2(1), 1–31. https://doi.org/10.1177/2332858415616358

Becker

S. O.

Ichino

(2002). Estimation of average treatment effects based on propensity scores. The Stata Journal, 2(4), 358–377. https://doi.org/10.1177/1536867X0200200403

Behnamnia

Kasmin

Ismail

Hayati

(2023). A review of using digital game-based learning for preschoolers. Journal of Computing in Education, 10(4), 603–636. https://doi.org/10.1007/s40692-022-00240-0

Betts, A. (2019). Mastery learning in early childhood mathematics through adaptive technologies. In The IAFOR International Conference on Education–Hawaii 2019 Official Conference Proceedings (pp. 51–63).

Bingham

A. J.

Pane

J. F.

Steiner

E. D.

Hamilton

L. S.

(2018). Ahead of the curve: Implementation challenges in personalized learning school models. Educational Policy, 32(3), 454–489. https://doi.org/10.1177/0895904816637688

Blair

Raver

C. C.

(2015). School readiness and self-regulation: A developmental psychobiological approach. Annual Review of Psychology, 66, 711–731. https://doi.org/10.1146/annurev-psych-010814-015221

10.

Bloom

(1968). Learning for mastery. In Block

J. H.

(Ed.), Mastery learning: Theory and practice (pp. 47–63). Holt, Rinehart, & Winston.

11.

Castro

Sevillano

(2022). Effectiveness of a digital serious game in improving readingcomprehension and academic performance. Investigaciones Sobre Lectura, 17, 40–66. https://doi.org/10.24310/isl.vi17.14325

12.

Children’s Learning Institute at UTHealth Houston. (2023) CIRCLE progress monitoring pre-K user guide. Available at: https://cliengage.org/clirep/CPM/CPM-User-Guide.pdf

13.

Cornito

C. M.

(2023). The effectiveness of contextualized digital game-based learning resource in improving kindergarten pupil’s alphabet knowledge level. International Journal on Studies in Education (IJonSE, 5(2), 130–140. https://doi.org/10.46328/ijonse.103

14.

Design-Based Research Collective. (2003). Design-based research: An emerging paradigm for educational inquiry. Educational Researcher, 32(1), 5–8. https://doi.org/10.3102/0013189X032001005

15.

DiCerbo

K. E.

Bertling

Stephenson

Jia

Mislevy

Bauer

Jackson

G. T.

(2015). An application of exploratory data analysis in the development of game-based assessments. Serious Games Analytics, 319–342. https://doi.org/10.1007/978-3-319-05834-4_14

16.

Dietze

Kashin

(2011). Playing and Learning in Early Childhood. Pearson Education Canada.

17.

Dixon

F. A.

Yssel

McConnell

J. M.

Hardin

(2014). Differentiated instruction, professional development, and teacher efficacy. Journal for the Education of the Gifted, 37(2), 111–127. https://doi.org/10.1177/0162353214529042

18.

Dohring, D., Hendry, D., Gunderia, S., Hughes, D., Owen, V. E., Jacobs, D. E., Betts, A., & Salak, W. (2019). U.S. Patent No. 20190236967 A1. U.S. Patent and Trademark Office.

19.

Dore

R. A.

Shirilla

Hopkins

Collins

Scott

, et al. (2019). Education in the app store: Using a mobile game to support U.S. preschoolers’ vocabulary learning. Journal of Children and Media, 13(4), 452–471. https://doi.org/10.1080/17482798.2019.1650788

20.

Fereday

Muir-Cochrane

(2006). Demonstrating rigor using thematic analysis: A hybrid approach of inductive and deductive coding and theme development. International Journal of Qualitative Methods, 5(1), 80–92. https://doi.org/10.1177/160940690600500107

21.

Fisch

S. M.

(2004). Children's learning from educational television: Sesame Street and beyond. Lawrence Erlbaum.

22.

Galletta

(2013). Mastering the semi-structured interview and beyond: From research design to analysis and publication. New York University Press.

23.

Giacomo Dina

Vincenza

Mascio Tania

Maria Rosita

Daniela

Rosella

, et al. (2016). The silent Reading supported by adaptive learning technology: Influence in the children outcomes. Computers in Human Behavior, 55(Part B), 1125–1130. https://doi.org/10.1016/j.chb.2014.09.053

24.

Goddard

Kim

(2015). School instructional climate and student achievement: An examination of group norms for differentiated instruction. American Journal of Education, 122(1), 111–131. https://doi.org/10.1086/683293

25.

Goodman

Libenson

Wade-Woolley

(2010). Sensitivity to linguistic stress, phonological awareness, and early Reading ability in preschoolers. Journal of Research in Reading, 33(2), 113–127. https://doi.org/10.1111/j.1467-9817.2009.01423.x

26.

Guskey

T. R.

(1997). Implementing Mastery Learning (2nd ed.). Wadsworth.

27.

Haataja

Moreno-Esteva

Salonen

Laine

Toivanen

Hannula

(2019). Teacher’s visual attention when scaffolding collaborative mathematical problem solving. Teaching and Teacher Education, 86, 102877. https://doi.org/10.1016/j.tate.2019.102877

28.

Hernandez

D. J.

(2011). Double jeopardy: How third-grade Reading skills and poverty influence high school graduation. Annie E. Casey Foundation.

29.

Hurry

Sylva

(2007). Long-term outcomes of early reading intervention. Journal of Research in Reading, 30, 227–248. https://doi.org/10.1111/j.1467-9817.2007.00338.x

30.

International Literacy Association. (2019). Phonological awareness in early childhood literacy development Position statement and research brief]. Author.

31.

Invernizzi

Juel

Swank

Meier

(2015). PALS K technical reference. University of Virginia. https://literacy.virginia.edu/sites/g/files/jsddwu1006/files/2022-03/K_Tech_Ref_2015.pdf

32.

Jamshidifarsani

Garbaya

Lim

Blazevic

Ritchie

(2018). Technology-based reading intervention programs for elementary grades: An analytical review. Computers & Education, 128, 427–451. https://doi.org/10.1016/j.compedu.2018.10.003

33.

Karime

Hossain

Rahman

Gueaieb

Alja’Am

El Saddik

(2012). RFID-based interactive multimedia system for the children. Multimedia Tools and Applications, 59(3), 749–774. https://doi.org/10.1007/s11042-011-0768-3

34.

Karwowski

Beghetto

R. A.

(2019). Creative behavior as agentic action. Psychology of Aesthetics. Creativity, and the Arts, 13(4), 402–415. https://doi.org/10.1037/aca0000190

35.

Kim

Gilbert

Gale

(2021). Measures Matter: A Meta-Analysis of the Effects of Educational Apps on Preschool to Grade 3 Children’s Literacy and Math Skills. AERA Open, 7, https://doi.org/10.1177/23328584211004183

36.

Kuhfeld

Soland

Tarasawa

Johnson

Ruzek

Liu

(2022). The COVID-19 School Year: Learning and Recovery Across 2020-2021. AERA Open, 8, https://doi.org/10.1177/23328584221099306

37.

Lipsey

M. W.

Puzio

Yun

Hebert

M. A.

Steinka-Fry

Cole

M. W.

Roberts

Anthony

K. S.

Busick

M. D.

(2012). Translating the statistical representation of the effects of education interventions into more readily interpretable forms (NCSER 2013-3000). U.S. Department of Education, Institute of Education Sciences. National Center for Special Education Research.

38.

López-Escribano

Valverde-Montesino

García-Ortga

(2021). The impact of e-book reading on young children’s emergent literacy skills: An analytical review. International Journal of Environmental Resources and Public Health, 18(12), 6510. https://doi.org/10.3390/ijerph18126510

39.

May

Perez-Johnson

Haimson

Sattar

Gleason

(2009). Using state tests in education experiments: A discussion of the issues (NCEE 2009-013). U.S. Department of Education, Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance.

40.

Mislevy

R. J.

Almond

R. G.

Lukas

J. F.

(2003). A brief Introduction to evidence-centered design (p. 37). Educational Testing Service. https://www.ets.org/Media/Research/pdf/RR-03-16.pdf

41.

Moons

(2020). Propensity weighting: How to minimise comparative bias in non-randomised studies? European Journal of Cardiovascular Nursing, 19(1), 83–88. https://doi.org/10.1177/1474515119888972

42.

NAEYC. (2012). Technology and interactive media as tools in early childhood programs serving children from birth through age 8. NAEYC.

43.

NAEYC. (2022). Position statement: Technology and young children. NAEYC.

44.

National Center for Education Statistics. (2022). The nation's report card. https://www.nationsreportcard.gov/

45.

National Institute of Child Health and Development (NICHD). (2000). Report of the National Reading Panel: Teaching children to read: An evidence-based assessment of the scientific research literature on reading and its implications for reading instruction (NIH Publication No. 00-4769). Washington, DC: U.S. Government Printing Office.

46.

Pearson

P. D.

Gallagher

M. C.

(1983). The instruction of reading comprehension. Contemporary Educational Psychology, 8(3), 317–344. https://doi.org/10.1016/0361-476X(83)90019-X

47.

Petscher

Cabell

S. Q.

Catts

H. W.

Compton

D. L.

Foorman

B. R.

Hart

S. A.

Lonigan

C. J.

Phillips

B. M.

Schatschneider

Steacy

L. M.

Terry

N. P.

Wagner

R. K.

(2020). How the science of reading informs 21st-century education. Reading Research Quarterly, 55(S1), S267–S282. https://doi.org/10.1002/rrq.352

48.

Plass

J. L.

Pawar

(2020). Toward a taxonomy of adaptivity for learning. Journal of Research on Technology in Education, 52(3), 275–300. https://doi.org/10.1080/15391523.2020.1719943

49.

Raudenbush

S. W.

Bryk

A. S.

(2002). Hierarchical linear models: Applications and data analysis methods (Vol. 1). Sage Publishing.

50.

Reardon

S. F.

(2013). The widening income achievement gap. Educational Leadership, 70(8), 10–16.

51.

Reich

Buttimer

C. J.

Fang

Hillaire

Hirsch

Larke

L. R.

Littenberg-Tobias

Moussapour

R. M.

Napier

Thompson

Slama

(2020). Remote learning guidance from state education agencies during the COVID-19 pandemic: A first look. EdArXiv.

52.

Roll

Russell

D. M.

Gašević

(2018). Learning at scale. International Journal of Artificial Intelligence in Education, 28(4), 471–477. https://doi.org/10.1007/s40593-018-0170-7

53.

Saldaña

(2021). The coding manual for qualitative researchers (4th ed.). Sage Publications.

54.

Savage

R. S.

Abrami

P. C.

Piquette

Wood

Deleveaux

Sanghera-Sidhu

, et al. (2013). A (Pan-Canadian) cluster randomized control effectiveness trial of the ABRACADABRA web-based literacy program. Journal of Educational Psychology, 105, 310–328. https://doi.org/10.1037/a0031025

55.

Scarborough

H. S.

(2001). Connecting early language and literacy to later Reading (dis)abilities: Evidence, theory, and practice. In Neuman

S. B.

Dickinson

D. K.

(Eds.), Handbook of early literacy research (pp. 97–110). Guilford.

56.

Schmitt

Hurwitz

Duel

Linebarger

(2018). Learning through play: The impact of web-based games on early literacy development. Computers in Human Behavior, 81, 378–389. https://doi.org/10.1016/j.chb.2017.12.036

57.

Shute

V. J.

(2011). Stealth assessment in computer-based games to support learning. In Tobias

Fletcher

J. D.

(Eds.), Computer games and instruction (pp. 503–524). Information Age Publishers.

58.

Snow

C. E.

Burns

M. S.

Griffin

(1998). Preventing reading difficulties in young children. National Academy Press.

59.

van der Weijden

F. A.

van den Boer

Zijlstra

B. J. H.

de Jong

P. F.

(2024). Implementation takes time: Reduction of literacy problems in schools implementing an early-literacy intervention. Journal of Research on Educational Effectiveness, 1–33. https://doi.org/10.1080/19345747.2024.2384365

60.

What Works Clearinghouse. (2020). What Works Clearinghouse standards handbook, version 4.1. U.S. Department of Education, Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance. https://ies.ed.gov/ncee/wwc/handbooks

Evaluation of a Personalized Game-Based Learning app for Developing Young Children's Reading Skills

Abstract

Keywords

Introduction

Literature

Technology and Early Childhood Education

Game-Based Learning in Early Literacy

Leveraging Digital Game-Based Learning for Individualized Instruction

My Reading Academy Intervention

Theoretical Foundations Underlying My Reading Academy

Mastery-Based Learning and Evidence-Centered Design

Engaging Children in Reading

Learner Experience

Developmentally Appropriate Tasks

Current Study

Methods

Design and Participants

Ethics Approval and Consent to Participate

Measures and Data Sources

Student Reading Assessment

My Reading Academy Usage Data

Teacher Surveys

Teacher Interviews

Analysis

Z-Score Transformation of the Student Reading Assessments

Propensity Score Weighting

Hierarchical Linear Models

Qualitative Data Analysis

Results

My Reading Academy Usage

Overall Impact

Kindergarten

Pre-Kindergarten

Teachers’ Perceptions of Impact

Teacher Interviews

Effective

Engaging

Empowering

Equitable

Discussion

Overview of Key Findings

Pedagogical Implications and Recommendations

Teacher Perceptions and Implementation Success

Supporting Diverse Learners Through Adaptive Technology

Professional Development Implications

Limitations and Future Research

Conclusion

Footnotes

Declaration of Conflicting Interests

Funding

ORCID iDs

Notes

Author Biographies

References