Restricted accessResearch articleFirst published online 2025
Effects of a Math Gaming Intervention With Teacher Implementation Support: A Single Case Experimental Design Study With Generalized Linear Mixed Models
Technology-based intervention programs have become an increasingly popular approach to providing intensive intervention in elementary schools given the prevalence of classroom technology and the ability to provide simultaneous, individualized instruction to a range of users. While technology holds great potential for helping to close achievement gaps for students with or at risk for learning disabilities, carefully designed strategies and methods to assist with the implementation of evidence-based technology tools for teacher’s day-to-day practice are of critical importance. Our research team has iteratively developed a technology-based gaming intervention called NumberShire Level 1 (NS1) and supplemental tools to improve teacher implementation and students’ learning outcomes. The purpose of this study was to examine the effects of the NS1 instructional package with implementation support on teachers’ data-based decision making (DBDM) and differentiated instructional practice utilizing a series of complementary data analyses in a single-case experimental design. Our findings revealed that the NS1 intervention with supplemental tools and implementation support (e.g., online resources, data dashboard, professional development with instructional coaching) resulted in positive outcomes for some teachers. Our study suggests that other factors (e.g., awareness, perception, user buy-in) are critical to increase the likelihood of teachers’ use of data-based intensified instruction and make it practical and accessible for teachers to maximize intended benefits.
AlonzoD.QuimnoV.TownendG.OoC. Z. (2024). Using information and communication technology (ICT)-based data systems to support teacher data-driven decision-making: Insights from the literature (2013–2023). Educational Assessment, Evaluation and Accountability, 36(4), 433–451. https://doi.org/10.1007/s11092-024-09443-8
2.
Al OtaibaS.FuchsD. (2006). Who are the young children for whom best practices in reading are ineffective? An experimental and longitudinal study. Journal of Learning Disabilities, 39(5), 414–431. https://doi.org/10.1177/00222194060390050401
3.
BećirovićS. (2023). Challenges and barriers for effective integration of technologies into teaching and learning. In Digital pedagogy: The use of digital technologies in conteporary education. SpringerBriefs in education. Springer. https://doi.org/10.1007/978-981-99-0444-0_10
4.
Benavides-VarelaS.CallegherC. Z.FagioliniB.LeoI.AltoèG.LucangeliD. (2020). Effectiveness of digital-based interventions for children with mathematical learning difficulties: A meta-analysis. Computers & Education, 157, Article 103953. https://doi.org/10.1016/j.compedu.2020.103953
5.
BrodC. (1984). Technostress: The human cost of the computer revolution.Addison-Wesley.
6.
BrooksM. E.KristensenK.Van BenthemK. J.MagnussonA.BergC. W.NielsenA.SkaugH. J.MachlerM.BolkerB. M. (2017). glmmTMB balances speed and flexibility among packages for zero-inflated generalized linear mixed modeling. The R Journal, 9(2), 378–400. https://doi.org/10.3929/ethz-b-000240890
7.
BryantD. P.BryantB. R.RobertsG.VaughnS.PfannenstielK. H.PorterfieldJ.GerstenR. (2011). Early numeracy intervention program for first-grade students with mathematics difficulties. Exceptional Children, 78(1), 7–23. https://doi.org/10.1177/001440291107800101
8.
ButlerD. L.WinneP. H. (1995). Feedback and self-regulated learning: A theoretical synthesis. Review of Educational Research, 65(3), 245–281. https://doi.org/10.3102/00346543065003245
9.
CarrJ. E. (2005). Recommendations for reporting multiple-baseline designs across participants. Behavioral Interventions, 20(3), 219–224. https://doi.org/10.1002/bin.191
10.
ClarkeB.DoablerC. T.KostyD.Kurtz NelsonE.SmolkowskiK.FienH.TurturaJ. (2017). Testing the efficacy of a kindergarten mathematics intervention by small group size. AERA Open, 3(2), Article 2332858417706899. https://doi.org/10.1177/2332858417706899
11.
ClarkeB.DoablerC. T.TurturaJ.SmolkowskiK.KostyD. B.SutherlandM.FienH.BakerS. K. (2020). Examining the efficacy of a kindergarten mathematics intervention by group size and initial skill: Implications for practice and policy. The Elementary School Journal, 121(1), 125–153. https://doi.org/10.1086/710041
12.
CoddingR. S.LaneK. L. (2015). A spotlight on treatment intensity: An important and often overlooked component of intervention inquiry. Journal of Behavioral Education, 24(1), 1–10. https://doi.org/10.1007/s10864-014-9210-z
13.
CoddingR. S.VanDerHeydenA. M.MartinR. J.DesaiS.AllardN.PerraultL. (2016). Manipulating treatment dose: Evaluating the frequency of a small group intervention targeting whole number operations. Learning Disabilities Research & Practice, 31(4), 208–220. https://doi.org/10.1111/ldrp.12120
14.
ComptonD. L.FuchsL. S.FuchsD.LambertW.HamlettC. (2012). The cognitive and academic profiles of reading and mathematics learning disabilities. Journal of Learning Disabilities, 45(1), 79–95. https://doi.org/10.1177/0022219410393012
15.
DoablerC. T.BakerS. K.KostyD. B.SmolkowskiK.ClarkeB.MillerS. J.FienH. (2015). Examining the association between explicit mathematics instruction and student mathematics achievement. The Elementary School Journal, 115(3), 303–333. https://doi.org/10.1086/679969
16.
DoablerC. T.ClarkeB.KostyD.Kurtz-NelsonE.FienH.SmolkowskiK.BakerS. K. (2019). Examining the impact of group size on the treatment intensity of a tier 2 mathematics intervention within a systematic framework of replication. Journal of Learning Disabilities, 52(2), 168–180. https://doi.org/10.1177/0022219418789376
17.
DoablerC. T.ClarkeB.KostyD.TurturaJ. E.SutherlandM.MaddoxS. A.SmolkowskiK. (2020). Using direct observation to document “practice-based evidence” of evidence-based mathematics instruction. Journal of Learning Disabilities, 54(1), 20–35. https://doi.org/10.1177/0022219420911375
18.
ElbaumB.VaughnS.HughesM. T.MoodyS. W.SchummJ. S. (2000). How reading outcomes of students with disabilities are related to instructional grouping formats: A meta-analytic review. In Contemporary special education research (pp. 105–135). Routledge.
19.
EvmenovaA. S.ReganK. S.SchladantM.HallT. E.BuzhardtJ.EricksonK. A.AiJ.SudduthC.JacksonT. (2023). Stepping-up technology implementation—How does it happen?Journal of Special Education Technology, 38(1), 61–74. https://doi.org/10.1177/01626434221074357
20.
EvmenovaA. S.VermeerA.CooneyL.WashburnJ.JacksonT.SulaimonT. (2024). Evidence-based technology tools to support diverse learners, educators, and service providers across instructional settings. Focus on Exceptional Children, 48(2). https://doi.org/10.17161/foec.v48i2.22951
21.
FienH.DoablerC. T.NelsonN. J.KostyD. B.ClarkeB.BakerS. K. (2016). An examination of the promise of the NumberShire level 1 gaming intervention for improving student mathematics outcomes. Journal of Research on Educational Effectiveness, 9(4), 635–661. https://doi.org/10.1080/19345747.2015.1119229
22.
FosterM. E. (2023). Evaluating the impact of supplemental computer-assisted math instruction in elementary school: A conceptual replication. Journal of Research on Educational Effectiveness, 17(1), 94–118. https://doi.org/10.1080/19345747.2023.2174919
23.
FuchsD.FuchsL. S.VaughnS. (2014). What is intensive instruction and why is it important?Teaching Exceptional Children, 46(4), 13–18. https://doi.org/10.1177/0040059914522966
24.
FuchsL. S.Newman-GoncharR.SchumacherR.DoughertyB.BuckaN.KarpK. S.WoodwardJ.ClarkeB.JordanN. C.GerstenR.JayanthiM.KeatingB.MorganS. (2021). Assisting students struggling with mathematics: Intervention in the elementary grades (WWC 2021006). National Center for Education Evaluation and Regional Assistance (NCEE), Institute of Education Sciences, U.S. Department of Education. https://ies.ed.gov/ncee/wwc/Docs/PracticeGuide/WWC2021006-Math-PG.pdf
25.
GerstenR.BeckmannS.ClarkeB.FoegenA.MarshL.StarJ. R.WitzelB. (2009). Assisting students struggling with mathematics: Response to Intervention (RtI) for elementary and middle schools (NCEE 2009-4060). National Center for Education Evaluation and Regional Assistance, Institute of Education Sciences, U.S. Department of Education. https://files.eric.ed.gov/fulltext/ED504995.pdf
26.
GunnB.SmolkowskiK.StryckerL. A.DennisC. (2021). Measuring explicit instruction using classroom observations of student-teacher interactions (COSTI). Perspectives on behavior science, 44(2-3), 267–283. https://doi.org/10.1007/s40614-021-00291-1
27.
IrwinV.WangK.JungJ.KesslerE.TezilT.AlhassaniS.FilbeyA.DiligR.Bullock MannF. (2024). Report on the condition of education 2024 (NCES 2024-144). U.S. Department of Education, National Center for Education Statistics. https://nces.ed.gov/pubsearch/pubsinfo.asp?pubid=2024144
28.
JimersonS. R.AroraP.BlakeJ. J.CanivezG. L.EspelageD. L.GonzalezJ. E.GravesS. L.HuangF. L.JanuaryS.-A. A.RenshawT. L.SongS. Y.SullivanA. L.WangC.WorrellF. C. (2021). Advancing diversity, equity, and inclusion in school psychology: Be the change. School Psychology Review, 50(1), 1–7. https://doi.org/10.1080/2372966X.2020.1827682
29.
JitendraA. K.AlghamdiA.EdmundsR.McKevettN. M.MouanoutouaJ.RoessleinR. (2021). The effects of tier 2 mathematics interventions for students with mathematics difficulties: A meta-analysis. Exceptional Children, 87(3), 307–325. https://doi.org/10.1177/0014402920969187
30.
JitendraA. K.LeinA. E.ImS.AlghamdiA. A.HefteS. B.MouanoutouaJ. (2017). Mathematical interventions for secondary students with learning disabilities and mathematics difficulties: A meta-analysis. Exceptional Children, 84(2), 177–196. https://doi.org/10.1177/0014402917737467
31.
KhlaifZ. N.SanmugamM.AyyoubA. (2023a). Impact of technostress on continuance intentions to use mobile technology. The Asia-Pacific Education Researcher, 32(2), 151–162. https://doi.org/10.1007/s40299-021-00638-x
32.
KhlaifZ. N.SanmugamM.JomaA. I.OdehA.BarhamK. (2023b). Factors influencing teacher’s technostress experienced in using emerging technology: A qualitative study. Technology, Knowledge and Learning, 28(2), 865–899. https://doi.org/10.1007/s10758-022-09607-9
LiH.AvendañoS. M.BakM. Y. S. (2025a). Analyzing the effects of a repeated reading intervention on reading fluency with generalized linear mixed models. Evaluation & the Health Professions, 48(1), 112–128. https://doi.org/10.1177/01632787241257450
35.
LiH.LuoW.BaekE.ThompsonC. G.LamK. H. (2025b). Multilevel modeling in single-case studies with count and proportion data: A demonstration and evaluation. Psychological Methods, 30(4), 815–842. https://doi.org/10.1037/met0000607
36.
LuoJ.WangM.YuS. (2022). Exploring the factors influencing teachers’ instructional data use with electronic data systems. Computers & Education, 191, Article 104631. https://doi.org/10.1016/j.compedu.2022.104631
37.
MagginD. M.CookB. G.CookL. (2018). Using single–case research designs to examine the effects of interventions in special education. Learning Disabilities Research & Practice, 33(4), 182–191. https://doi.org/10.1111/ldrp.12184
38.
MartensB. K.WittJ. C.ElliottS. N.DarveauxD. X. (1985). Teacher judgements concerning the acceptability of school-based interventions. Professional Psychology: Research and Practice, 16, 191–198. https://doi.org/10.1037/0735-7028.16.2.191
NelsonG.McMasterK. L. (2019). The effects of early numeracy interventions for students in preschool and early elementary: A meta-analysis. Journal of Educational Psychology, 111(6), 1001–1022. https://doi.org/10.1037/edu0000334
41.
NelsonG.PowellS. R. (2018). A systematic review of longitudinal studies of mathematics difficulty. Journal of Learning Disabilities, 51(6), 523–539. https://doi.org/10.1177/0022219417714773
42.
NelsonN. J.FienH.DoablerC. T.ClarkeB. (2016). Considerations for realizing the promise of educational gaming technology. Teaching Exceptional Children, 48(6), 293–300. https://doi.org/10.1177/0040059916650639
43.
ParkerR. I.VannestK. J.DavisJ. L. (2011). Effect size in single-case research: A review of nine nonoverlap techniques. Behavior Modification, 35(4), 303–322. https://doi.org/10.1177/0145445511399147
44.
PerlmanC. L.ReddingS. (2011). Handbook on effective implementation of school improvement grants. Center on Innovation & Improvement.
45.
PowellS. R.FuchsL. S. (2015). Intensive intervention in mathematics. Learning Disabilities Research & Practice, 30(4), 182–192. https://doi.org/10.1111/ldrp.12087
46.
PustejovskyJ. E.SwanD. M.EnglishK. W. (2023). An examination of measurement procedures and characteristics of baseline outcome data in single-case research. Behavior Modification, 47(6), 1423–1454. https://doi.org/10.1177/0145445519864264
47.
SiddiquiS.ArifI.HindujaP. (2022). Technostress: A catalyst to leave the teaching profession - a survey designed to measure technostress among teachers in Pakistan during COVID-19 pandemic. E-learning and Digital Media, 20(1), 53–79. https://doi.org/10.1177/20427530221107506
48.
SlocumT. A.PinkelmanS. E.JoslynP. R.NicholsB. (2022). Threats to internal validity in multiple-baseline design variations. Perspectives on behavior science, 45(3), 619–638. https://doi.org/10.1007/s40614-022-00326-1
49.
TuL.RaoZ.JiangH.DaiL. (2025). Technostress, burnout, and job satisfaction: An empirical study of STEM teachers’ well-being and performance. Behavioral Sciences, 15(7), 992. https://doi.org/10.3390/bs15070992
U.S. Department of Education. (2025). The Nation’s report card: Results from the 2024 national assessment of educational progress (NAEP). Institute of Education Sciences, National Center for Education Statistics. https://www.nationsreportcard.gov/
52.
van GeelM.KeuningT.VisscherA.FoxJ. P. (2017). Changes in educators’ data literacy during a data-based decision-making intervention. Teaching and Teacher Education, 64, 187–198. https://doi.org/10.1016/j.tate.2017.02.015
53.
VanlommelK.SchildkampK. (2018). How do teachers make sense of data in the context of high-stakes decision making?American Educational Research Journal, 56(3), 792–821. https://doi.org/10.3102/0002831218803891
WarrenS. F.FeyM. E.YoderP. J. (2007). Differential treatment intensity research: A missing link to creating optimally effective communication interventions. Mental Retardation and Developmental Disabilities Research Reviews, 13(1), 70–77. https://doi.org/10.1002/mrdd.20139
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
