To inform the development of gamified assessments, this study explored how students with or at risk for reading difficulties in Grades 6–8 (N = 202) perceived and interacted with a decoding assessment designed with gamification characteristics. Three data sources enhanced the methodological triangulation: observations and scores from testing, surveys of students’ perceptions, and focus group discussions with a stratified random sample of students (n = 25). Findings suggest students became immersed in the gamified reading assessment and were motivated by tasks that were challenging but not frustratingly difficult. However, they were dissatisfied with some design features and reported focusing on identifying patterns and gaming strategies rather than on the reading skills being assessed. This suggests students’ expectations of gamified assessments might contribute construct irrelevant variance to the instruments.
AbramovichS.SchunnC.HigashiR. M. (2013). Are badges useful in education? It depends upon the type of badge and expertise of learner. Educational Technology Research and Development, 61, 217–232. doi:10.1007/s1142-013-9289-2
2.
AlmondP.WinterP.CametoR.RussellM.SatoE.Clarke-MiduraJ.LazarusS. (2010). Technology-enabled and universally designed assessment: Considering access in measuring the achievement of students with disabilities—A foundation for research. Journal of Technology, Learning, and Assessment, 10, 1–52. Retrieved fromhttp://ejournals.bc.edu/ojs/index.php/jtla/article/view/1605
3.
ArmstrongM. B.FerrellJ. Z.CollmusA. B.LandersR. N. (2016). Correcting misconceptions about gamification of assessment: More than SJTs and badges. Industrial and Organizational Psychology, 9, 671–677. doi:10.1017/iop.2016.69
4.
ArmstrongM. B.LandersR. N.CollmusA. B. (2015). Gamifying recruitment, selection, training, and performance management: Game-thinking in human resource management. In DavisD.GangadharbatlaH. (Eds.), Emerging research and trends in gamification (pp. 140–165). Hershey, PA: IGI Global.
5.
AttaliY.Arieli-AttaliM. (2015). Gamification in assessment: Do points affect test performance?Computers & Education, 83, 57–63. doi:10.1016/j.compedu.2014.12.012
6.
BachmanL. F. (1990). Fundamental considerations in language testing. Oxford, England: Oxford University Press.
7.
BooJ.VispoelW. (2012). Computer versus paper-and-pencil assessment of educational development: A comparison of psychometric features and examinee preferences. Psychological Reports, 111, 443–460. doi:10.2466/10.03.11.pr0.111.5.443-460
8.
BorgonoviF. (2016). Video gaming and gender differences in digital and printed reading performance among 15-year-old students in 26 countries. Journal of Adolescence, 48, 45–61. doi:10.1016/j.adolescence.2016.01.004
9.
BouckE. C.FlanaganS. (2009). Assistive technology and mathematics: What is there and where can we go in special education. Journal of Special Education Technology, 24, 17–29. doi:10.1177/016264340902400202
10.
BoyleE. A.HaineyT.ConnollyT. M.GrayG.EarpJ.OttM.PereiraJ. (2016). An update to the systematic literature review of empirical evidence of the impacts and outcomes of computer games and serious games. Computers & Education, 94, 178–192. doi:10.1016/j.compedu.2015.11.003
11.
BrantlingerE.JimenezR.KlingnerJ.PugachM.RichardsonV. (2005). Qualitative studies in special education. Exceptional Children, 71, 195–207. doi:10.1111/1467-8527.t01-1-00151
12.
CirinoP. T.RomainM. A.BarthA. E.TolarT. D.FletcherJ. M.VaughnS. (2013). Reading skill components and impairments in middle school struggling readers. Reading and Writing, 26, 1059–1086. doi:10.1007/11145-012-9406-3
13.
ClarkD. B.Tanner-SmithE. E.KillingsworthS. S. (2016). Digital games, design, and learning: A systematic review and meta-analysis. Review of Educational Research, 86, 79–122. doi:10.3102/0034654315582065
14.
CuevasJ.RussellR.IrvingM. (2012). An examination of the effect of customized reading modules on diverse secondary students’ reading comprehension and motivation. Educational Technology Research and Development, 60, 445–467. doi:10.1007/s11423-012-9244-7
15.
DeciE. L.KoestnerR.RyanR. M. (2001). Extrinsic rewards and intrinsic motivation in education: Reconsidered once again. Review of Educational Research, 71, 1–27. doi:10.3102/00346543071001001
16.
DenzinN. K. (1970). The research act in sociology. Chicago, IL: Aldine.
17.
Federation of American Scientists. (2006). Summit on educational games: Harnessing the power of video games for learning. Washington, DC: Author. Retrieved fromhttps://fas.org
18.
FerrellJ. Z.CarpenterJ. E.VaughnE. D.DudleyN. M.GoodmanS. A. (2015). Gamification of human resource processes. In DavisD.GangadharbatlaH. (Eds.), Emerging research and trends in gamification (pp. 108–139). Hershey, PA: IGI Global.
19.
FinkA. (1995). How to measure survey reliability and validity (Vol. 7). Thousand Oaks, CA: Sage.
20.
FlowersC.KimD. H.LewisP.DavisV. C. (2011). A comparison of computer-based testing and pencil-and-paper testing for students with a read-aloud accommodation. Journal of Special Education Technology, 26, 1–12. doi:10.1177/016264341102600102
21.
FoleyB. P. (2016). Getting lucky: How guessing threatens the validity of performance classifications. Practical Assessment, Research & Evaluation, 21, 1–23.
22.
GrayL.ThomasN.LewisL. (2010). Teachers’ use of educational technology in U.S. public schools: 2009(NCES 2010-040). Washington, DC: National Center for Education Statistics, Institute of Education Sciences, U.S. Department of Education. Retrieved fromhttp://nces.ed.gov/pubs2010/2010040.pdf
23.
HinesP. J.JasnyB. R.MervisJ. (2009). Adding a T to the three R’s. Science, 323, 53. doi:10.1126/science.323.5910.53a
24.
KeF.AbrasT. (2013). Games for engaged learning of middle school children with special learning needs. British Journal of Educational Technology, 44, 225–242. doi:10.1111/j.1467-8535.2012.01326.x
25.
KiddP. S.ParshallM. B. (2000). Getting the focus and the group: Enhancing analytical rigor in focus group research. Qualitative Health Research, 10, 293–308. doi:10.1177/104973200129118453
26.
KiefferM. J. (2010). Socioeconomic status, English proficiency, and late-emerging reading difficulties. Educational Researcher, 39, 484–486. doi:10.3102/0013189X10378400
27.
KitzingerJ. (1995, July29). Qualitative research: Introducing focus groups. British Medical Journal, 311, 299–302. PMCID: PMC2550365
28.
KocadereS. A.CaglarS. (2015). The design and implementation of a gamified assessment. Journal of e-Learning and Knowledge Society, 11, 85–99.
29.
LandersR. N. (2014). Developing a theory of gamified learning: Linking serious games and gamification of learning. Simulation & Gaming, 45, 752–768. doi:10.1177/1046878114563660
30.
LarmanC.BasiliV. R. (2003). Iterative and incremental development: A brief history. Computer, 36, 47–56. doi:10.1109/mc.2003.1204375
31.
LumsdenJ.EdwardsE. A.LawrenceN. S.CoyleD.MunafoM. R. (2016). Gamification of cognitive assessment and cognitive training: A systematic review of application and efficacy. JMIR Serious Games, 4, e11. doi:10.2196/games.5888 Retrieved fromhttp://games.jmir.org/2016/2/e11
32.
MessickS. (1989). Validity. In LinnR. L. (Ed.), Educational measurement (3rd ed., pp. 13–103). New York, NY: Macmillan.
33.
MeyenE. (2015). Significant advancements in technology to improve instruction for all students: Including those with disabilities. Remedial and Special Education, 36, 67–71. doi:10.1177/0741932514554103
34.
MilesM. B.HubermanA. M. (1994). Qualitative data analysis (2nd ed.). Thousand Oaks, CA: Sage.
35.
MillmanJ.BishopH. I.EbelR. (1965). An analysis of test wiseness. Educational and Psychological Measurement, 25, 707–726. doi:10.1177/001316446502500304
36.
MorfordZ. H.WittsB. N.KillingsworthK. J.AlavosiusM. P. (2014). Gamification: The intersection between behavior analysis and game design technologies. The Behavior Analyst, 37, 25–40. doi:10.1007/s40614-014-0006-1
37.
MurrayB.Silver-PacuillaHHelselI. F. (2007). Improving basic mathematics instruction promising technology resources for students with special needs. Technology in Action, 2, 1–8.
38.
National Center for Education Statistics. (2018). The nations report card. 2017 reading results. Washington, DC: Institute of Education Sciences, U.S. Department of Education. Retrieved fromwww.nationsreportcard.org
39.
RoembkeT. C.HazeltineE.ReedD. K.McMurrayB. (2019). Automaticity of word recognition is a unique predictor of reading fluency in middle-school students. Journal of Educational Psychology, 111, 314–330.
StraussA.CorbinJ. (1998). Basics of qualitative research: Techniques and procedures for developing grounded theory (2nd ed.). Thousand Oaks, CA: Sage.
42.
VaughnS.WexlerJ.LerouxA.RobertsG.DentonC. A.BarthA.FletcherJ. (2012). Effects of intensive reading intervention for eighth-grade students with persistently inadequate response to intervention. Journal of Learning Disabilities, 45, 515–525. doi:10.1177/0022219411402692
43.
WangS.JiaoH.YoungM.J.BrooksT.OlsonJ. (2008). Comparability of computer-based and paper-and-pencil testing in K-12 reading assessments: A meta-analysis of testing mode effects. Educational and Psychological Measurement, 68, 5–24. doi:10.1177/0013164407305592
44.
WatsonW. R.MongC. J.HarrisC. A. (2011). A case study of the in-class use of a video game for teaching high school history. Computers & Education, 56, 466–474. doi:10.1016/j.compedu.2010.09.007
45.
WerbachK.HunterD. (2012). For the win: How game thinking can revolutionize your business. Philadelphia, PA: Wharton Digital Press.
46.
ZimmermanB.SchunkD. H. (2006). Competence and control beliefs: Distinguishing the means and ends. In AlexanderP. A.WinneP. H. (Eds.), Handbook of educational psychology (pp. 349–367). Mahwah, NJ: Lawrence Erlbaum Associates.
