Students with complex communication needs have increasingly been using non-dedicated communication systems, such as mobile devices, to support their communication needs. This in turn, has led to an increased used of augmentative and alternative communication apps. The main challenge currently faced is the lack of empirically validated apps and evaluation systems to assess the features of the apps. As a result, this study attempted to determine the reliability of an app evaluation tool that was grounded in the components of the feature matching model. The goal was also to identify if the app evaluation tool could be used to evaluate various types of augmentative and alternative communication apps. Participants evaluated apps across the dimensions of usability, output, and display. Results suggest that expert raters were more reliability than novice raters across the various types of apps. Practical implications and future research directions are discussed.
AbdelmoulaM.ChakrounW.AkroutF. (2015). The effect of sample size and the number of items on reliability coefficients: Alpha and Rhô: A meta-analysis. International Journal of Numerical Methods and Applications, 13(1), 1–20. https://doi.org/10.17654/IJNMAMar2015_001_020
2.
BajpaiS.BajpaiR.ChaturvediH. K. (2015). Evaluation of inter-rater agreement and inter-rater reliability for observational data: An overview of concepts and methods. Journal of the Indian Academy of Applied Psychology, 41(3), 20–27.
3.
BarnumC. M. (2020). Usability testing essentials: Ready, set … test! (2nd Edition). Elsevier.
4.
BeukelmanD. R.LightJ. C. (2020). Augmentative and alternative communication: Supporting children and adults with complex communication needs. Paul H. Brookes.
5.
BiasR. G.HoffmanR. (2014). Where’s the rigor in the field of usability analysis?Institute of Electronical and Electronics Engineers, 28(6), 66–72. https://doi.org/10.1109/MIS.2013.150
6.
BoeschM. C.Da FonteM. A.HolmesE. E.NardiK. L.Searching for app evaluation tools: A systematic literature review. Disability and Rehabilitation: Assistive Technology. (in review).
7.
BrookhartS. M. (2013). How to create and use rubrics for formative assessment and grading. Association for Supervision and Curriculum Development.
ChernerT.DixJ.LeeC. (2014). Cleaning up that mess: A framework for classifying educational apps. Contemporary Issues in Technology and Teacher Education, 14(2), 158–193.
10.
CicchettiD. V. (1994). Guidelines, criteria, and rules of thumb for evaluating normed and standardized assessment instrument in psychology. Psychological Assessment, 6(4), 284–290. https://doi.org/10.1037/1040-3590.6.4.284
11.
Da FonteM. A.BoeschM. C. (2019). Implementing effective augmentative and alternative communication practices: A handbook for school-based practitioners. Taylor and Francis.
12.
DychesT. T.DavisA.LucidoB.YoungJ. (2002). Generalization of skills using pictographic and voice output communication devices. Augmentative and Alternative Communication, 18(2), 124–131. https://doi.org/10.1080/07434610212331281211
13.
GliemJ. A.GliemR. R. (2003). Calculating, interpreting, and reporting Cronbach’s alpha reliability coefficient for Likert-type scales[Conference session]. 2003 Midwest Research to Practice Conference in Adult, Continuing, and Community Education, Columbus, OH, United States, October 8–10. https://hdl.handle.net/1805/344
14.
GosnellJ.CostelloJ.ShaneH. (2011). Using a clinical approach to answer “What communication apps should we use?”Perspectives on Augmentative and Alternative Communication, 20(3), 87–96. https://doi.org/10.1044/aac20.3.87
15.
GronlundG.JamesM. (2005). Focus observations: How to observe children for assessment and curriculum planning. Redleaf Press.
16.
HarrisP.TaylorR.ThielkeR.PayneJ.GonzalezN.CondeJ. G. (2009). Research electronic data capture (REDCap): A meta-driven methodology and workflow process for providing translational research informatics support. Journal of Biomedical Informatics, 42(2), 377–381. https://doi.org/10.1016/j.jbi.2008.08.010
17.
HeimannM.StridK.SmithL.TjusT.UlvundS.MeltzoffA. (2006). Exploring the relation between memory, gestural communication, and the emergence of language in infancy: Longitudinal study. Infant Child Development, 15(3), 233–249. https://doi.org/10.1002/icd.462
18.
LeeJ.KimS. (2015). Validation of a tool evaluating educational apps for smart education. Journal of Educational Computing Research, 52(3), 435–450. https://doi.org/10.1177/0735633115571923
19.
LightJ.McNaughtonD. (2014). Communicative competence for individuals who require augmentative and alternative communication: A new definition for a new era of communication?Augmentative and Alternative Communication, 30(1), 1–18. https://doi.org/10.3109/07434618.2014.885080
20.
MaattaJ.JarvinenR.LuostairinenR.LauriL.TairaT.MannerJ. (2011). Analysis and implementation of the virtual network system [Conference session]. Military Communications Conference, Baltimore, MD, United States, November, 2011. https://doi.org/10.1109/MILCOM.2011.6127528
Ohio Center for AutismLow Incidence (2012). Student inventory for technology supports. Retrieved fromhttps://sifts.ocali.org/
23.
O’NeillT.WilkinsonK.M. (2020). Preliminary investigation of the perspectives of parents of children with cerebral palsy on the supports, challenges, and realities of integrating augmentative and alternative communication into everyday life. American Journal of Speech-Language Pathology, 29, 238–254. DOI:10.1044/2019_AJSLP-19-00103
24.
PapadakisS.KalogiannakisM.ZaranisN. (2017). Designing and creating an educational app rubric for preschool teachers. Education and Information Technologies, 22(6), 3147–3165. https://doi.org/10.1007/s10639-017-9579-0
RobinsonS. B.LeonardK. L. (2019). Designing quality survey questions. Sage.
27.
RubinJ.ChisnellD. (2008). Handbook of usability testing: How to plan, design, and conduct effective tests (2nd ed.). John Wiley and Sons.
28.
RussS.ChiangB.RylanceB. J.BongersJ. (2001). Caseload in special education: An integration of research findings. Exceptional Children, 67(2), 161–172. https://doi.org/10.1177%2F001440290106700202
29.
Sanromà-GiménezM.Lázaro CantabranaJ. L.Usart RodríguezM.Gisbert-CerveraM. (2021). Design and validation of an assessment tool for educational mobile applications used with autistic learners. Journal of New Approaches in Educational Research, 10(1), 101–121. https://doi.org/10.7821/naer.2021.1.574
30.
SchlosserR.RaghavendraP. (2004). Evidence-based practice in augmentative and alternative communication. Augmentative and Alternative Communication, 20(1), 1–21. https://doi.org/10.1080/07434610310001621083
31.
SemmelrothC. L.JohnsonE. (2014). Measuring rater reliability on a special education observation tool. Assessment for Effective Instruction, 39(3), 131–145. https://doi.org/10.1177/1534508413511488
32.
SteinbergL.RogersA. (2020). Changing the scale: The effect of modifying response scale labels on the measurement of personality and affect. Multivariate Behavioral Research, 57(1), 79–93. https://doi.org/10.1080/00273171.2020.1807305
WengP.Taber-DoughtyT. (2015). Developing an app evaluation rubric for practitioners in special education. Journal of Special Education Technology, 30(1), 43–58. https://doi.org/10.1177/016264341503000104
35.
WestlingD. L.CarterE. W.Da FonteM. A.KurthJ. A. (2020). Teaching students with severe disabilities (6th ed.). Pearson.
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
