Reliability among School-Based Professionals: Using a Feature-Matching Screening Checklist to Identify Communication Systems

Abstract

Students with complex communication needs may require augmentative and alternative communication systems to supplement or replace their speech abilities. To effectively identify a communication system, a feature-matching process should be implemented as it considers the student’s present levels of performance. Due to the unique communication characteristics of students with complex communication needs, informal assessment tools are often used to help determine the students’ skills in natural contexts. A challenge often faced with informal assessment tools is the reliability among evaluators. As such, this pilot study attempted to evaluate the reliability of a feature-matching screening checklist and its corresponding matrices among potential professionals who would be part of the educational team. Results indicated that (a) pre-service and in-service special education teachers were the most reliable combination when completing the screening checklist, (b) exposure to the screening checklist was an influencing factor on reliability, and (c) in-service speech-language pathologists made the most errors while completing the screening checklist. Implications for practices and future research directions are discussed.

Keywords

augmentative and alternative communication feature matching informal assessment reliability

Get full access to this article

View all access options for this article.

References

Abbot

Bride

(2014). AAC decision making and mobile technology: Points to ponder. Perspectives on Augmentative and Alternative Communication, 23(2), 104–111. https://doi./org/10.1044/aac23.2.104

Achenbach

T. M.

(2018). Multi-informant and multicultural advances in evidence-based assessment of students’ behavioral-emotional-social difficulties. European Journal of Psychological Assessment, 34(2), 127–140. https://doi.org/10.1027/1015-5759/a000448

Achenbach

T. M.

McConaughy

S. H.

Howell

C. T.

(1987). Child-adolescent behavioral and emotional problems: Implications of cross-informant correlations for situational specificity. Psychological Bulletin, 101(2), 213–232. https://doi.org/10.1037/0033-2909.101.2.213

Andzik

N. R.

Schaefer

J. M.

Nichols

R. T.

Cannella-Malone

H. I.

(2019). Exploring relationships between teacher training and support strategies for students utilizing AAC. Journal of International Special Needs Education, 22(1), 25–34. https://doi.org/10.9782/16-00044

Baroody

A. E.

(2017). Exploring the contribution of classroom formats on teaching effectiveness and achievement in upper elementary classrooms. School Effectiveness and School Improvement, 28(2), 314–335. https://doi.org/10.1080/09243453.2017.1298629

Beukelman

D. R.

Light

J. C.

(2020). Augmentative and alternative communication: Supporting children and adults with complex communication needs. Baltimore, MD, USA: Paul H. Brookes.

Boesch

M. C.

Da Fonte

M. A.

(2014). Best practices for assessing communication skills prior to preference assessments for students with severe developmental disabilities. Evidence-Based Practice Briefs, 9(3), 1–9.

Brady

L. J.

(n.d.). Quick feature matching checklist. Retrieved from: https://proactivespeech.files.wordpress.com/2014/01/quick_feature_checklist_updated_12_2013_blog.pdf

Brady

N. C.

Bruce

Goldman

Erickson

Mineo

Ogletree

B. T.

Wilkinson

(2016). Communication services and supports for individuals with severe disabilities: Guidance for assessment and intervention. American Journal on Intellectual and Developmental Disabilities, 121(2), 121–138. https://doi.org/10.1352/1944-7558-121.2.121

10.

Costigan

F. A.

Light

(2010). A review of preservice training in augmentative and alternative communication for speech language pathologists, special education teachers, and occupational therapists. Assistive Technology: The Official Journal of Rehabilitation Engineering and Assistive Technology. Society of North America, 22(4), 200–212. https://doi.org/10.1080/10400435.2010.492774

11.

Da Fonte

M. A.

Boesch

M. C.

(2019a). Implementing effective augmentative and alternative communication practices for students with complex communication needs: A handbook for school-based practitioners. New York, NY: Routledge.

12.

Da Fonte

M. A.

Boesch

M. B.

DeLuca

E. R.

Papp

S. K.

Mohler

A. E.

Holmes

E. E.

Clouse

K. A.

Young

R. D.

Urbano

(2022). Current preparation status in augmentative and alternative communication: A perspective of special education teachers in the United States. Augmentative and Alternative Communication.

13.

Da Fonte

M. A.

Boesch

M. C.

Wolfe

N. P.

Young

R. D.

(2019b). Communication system identification for individuals with complex communication needs: The need for effective feature matching. International Review of Research in Developmental Disabilities, 57, 171–228. DOI:10.1016/bs.irrdd.2019.07.002.

14.

DiStefano

Sadhwani

Wheeler

A., C.

(2020). Comprehensive assessment of individuals with significant levels of intellectual disability: Challenges, strategies, and future directions. American Journal on Intellectual and Developmental Disabilities, 125(6), 434–448. https://doi.org/10.1352/1944-7558-125.6.434

15.

Grainger

P. R.

Adie

(2014). How do preservice teacher education students move from novice to expert assessors? Australian Journal of Teacher Education, 39(7), 89–105. https://doi.org/10.14221/ajte.2014v39n7.9

16.

Gronlund

James

(2013). Focused observations: How to observe children for assessment and curriculum planning (2nd ed.). USA :Readleaf Press.

17.

Hodapp

R. M.

Core

R. E.

Burke

M. M.

Mello

M. P.

Urbano

R. C.

(2019). Health issues across adulthood in Down syndrome. International Review of Research in Developmental Disabilities, 57, 229–265. https://doi.org/10.1016/bs.irrdd.2019.07.001

18.

Individuals with Disabilities Education Act (2004). U.S.C. § 1414 et seq. (2004).

19.

Landis

J. R.

Koch

G. G.

(1977). The measurement of observer agreement for categorical data. Biometrics, 33(1), 159–174. https://doi.org/10.2307/2529310

20.

Larriba-Quest

Byiers

B. J.

Beisang

Merbler

A. M.

Symons

F. J.

(2020). Special education supports and services for Rett syndrome: Parent perceptions and satisfaction. Intellectual and Developmental Disabilities, 58(1), 49–64. https://doi.org/10.1352/1934-9556-58.1.49

21.

Light

McNaughton

(2012). The changing face of augmentative and alternative communication: Past, present, and future challenges. Augmentative and Alternative Communication, 28(4), 197–204. https://doi.org/10.3109/07434618.2012.737024

22.

Lund

S. K.

Quach

Weissling

McKelvey

Dietz

(2017). Assessment with children who need augmentative and alternative communication (AAC): Clinical decisions of AAC specialists. Language, Speech, and Hearing Services in Schools, 48, 56–68. https://doi.org/10.1044/2016_LSHSS-15-0086

23.

Marfilius

Fonner

(2012). Feature match checklists. Retrieved from: http://www.spectronics.com.au/conference/2012/pdfs/handouts/kelly-fonner/Feature_Match_Checklists_JAN2012.pdf

24.

Moorcroft

Scarinci

Meyer

(2019). Speech pathologist perspectives on the acceptance versus rejection or abandonment of AAC systems for children with complex communication needs. Augmentative and Alternative Communication, 35(3), 193–204. https://doi.org/10.1080/07434618.2019.1609577

25.

Ohio Center for Autism and Low Incidence (OCALI) (2012). The student inventory for technology supports. https://sifts.ocali.org

26.

Olgetree

B. T.

Price

J. R.

(2017). Nonstandardized evaluation of emergent communication in individuals with severe intellectual disabilities: Exploring existing options and proposing innovations. Advances in Neurodevelopmental Disorders, 2, 38–48. https://doi.org/10.1007/s41252-017-0043-3

27.

Peckham-Hardin

K. D.

Hanreddy

Ogletree

B. T.

(2018). Preparing teachers to facilitate communication skills in students with severe disabilities (Document No. IC-17). Collaboration for Effective Educator Development, Accountability, and Reform Center, University of Florida. Retrieved from: http://ceedar.education.ufl.edu/tools/innovation-configurations/

28.

Quality Indicators for Assistive Technology Services Leadership Team (QIAT) (2008). Plan for evaluation of effectiveness of at use. Retrieved from: https://qiat.org/docs/resources/14QIATEvalofEffPlan.pdf

29.

Rubin

Chisnell

(2008). Handbook of usability testing: How to plan, design and conduct effective tests. NJ, USA: Wiley.

30.

Schlosser

R. W.

Raghavendra

(2004). Evidence-based practice in augmentative and alternative communication. Augmentative and Alternative Communication, 20(1), 1–21. https://doi.org/10.1080/07434610310001621083

31.

Senner

(n.d.). Feature matching checklist. Retrieved from: https://talcaac.com/SGDFeaturesChecklist.pdf

32.

Sievers

S. B.

Trembath

Westerveld

M. F.

(2020). Speech-language pathologists’ knowledge and consideration of factors that may predict, moderate, and mediate AAC outcomes. Journal of Autism and Developmental Disorders, 50(1), 238–249. https://doi.org/10.1007/s10803-019-04217-4

33.

Zapf

Castell

Morawietz

Karch

(2016). Measuring inter-rater reliability for nominal data – which coefficients and confidence intervals are appropriate? BMC Medical Research Methodology, 16(1), 93. https://doi.org/10.1186/s12874-016-0200-9