Establishing Content Validity of the Quality Indicators for Classrooms Serving Students With Autism Spectrum Disorders Instrument

Abstract

The purpose of this study was to provide content validation for quality indicators included in an observation instrument developed to evaluate classrooms serving students with autism spectrum disorders (ASD). A panel of 103 subject-matter experts consisting of a mixture of field personnel (n = 64; 59 classroom teachers, five school administrators) and university faculty (n = 39) provided feedback regarding construct validity. Results were analyzed using two approaches: a simple aggregation of responses in the form of averages, with an a priori threshold for the determination of what will be considered acceptable, and Lawshe’s Content Validity Ratio methodology. All items in the instrument were deemed to have demonstrated content validity as did the overall instrument. Based on the data from this study, the Quality Indicators for Classrooms Serving Students With ASD (QIASD) instrument can be considered to have demonstrated content validity.

Keywords

autism accountability teacher preparation policy/service delivery assessment teacher preparation practices and outcomes

Get full access to this article

View all access options for this article.

References

American Statistical Association. (2014, April). ASA statement on using value-added models for educational assessment. Retrieved from https://www.amstat.org/asa/files/pdfs/POL-ASAVAM-Statement.pdf

Ayre

Scally

A. J.

(2014). Critical values for Lawshe’s Content Validity Ratio: Revisiting the original methods of calculation. Measurement and Evaluation in Counseling and Development, 47, 79-86.

Ballou

Springer

M. G.

(2015). Using student test scores to measure teacher performance: Some problems in the design and implementation of evaluation systems [Special issue]. Educational Researcher, 44, 77-86. doi:10.3102/0013189X15574904

Belfiore

P. J.

Fritts

K. M.

Herman

B. C.

(2008). The role of procedural integrity using self-monitoring to enhance discrete trial instruction (DTI). Focus on Autism and Other Developmental Disabilities, 23, 95-102.

Brock

M. E.

Huber

H. B.

Carter

E. W.

Juarez

A. P.

Warren

Z. E.

(2014). Statewide assessment of professional development needs related to educating students with autism spectrum disorder. Focus on Autism and Other Developmental Disabilities, 29, 67-79.

Browder

Cooper-Duffy

(2003). Evidence-based practices for students with severe disabilities and the requirements for accountability in No Child Left Behind. The Journal of Special Education, 37, 157-163.

Buzick

H. M.

Jones

N. D.

(2015). Using test scores from students with disabilities in teacher evaluation. Educational Measurement, 34(3), 28-38.

Centers for Disease Control and Prevention. (2014). Prevalence of autism spectrum disorder among children ages 8 years—Autism and developmental disabilities monitoring network, 11 sites, United States, 2010. Retrieved from http://www.cdc.gov/mmwr/preview/mmwrhtml/ss6302a1.htm?s_cid=ss6302a1_w

Council for Exceptional Children. (2012). CEC position on special education teacher evaluation. Arlington, VA: Author.

10.

Council for Exceptional Children. (2015). What every special educator must know: Professional ethics and standards (7th ed.). Retrieved from http://www.cec.sped.org/standards/

11.

Darling-Hammond

(2015). Can value added add value to teacher evaluation? Educational Researcher, 44, 132-137. doi:10.3102/0013189X15575346

12.

Darling-Hammond

Wise

A. E.

Pease

S. R.

(1983). Teacher evaluation in the organizational context: A review of the literature. Review of Educational Research, 53, 285-328.

13.

District of Columbia Public Schools. (2014). Impact: The District of Columbia Public Schools Effectiveness Assessment System for School-Based Personnel. Retrieved from http://dcps.dc.gov/node/979742

14.

Doherty

K. M.

Jacobs

(2015, November). State of the states: Evaluating teaching, leading and learning, National Council on Teacher Quality. Retrieved from http://www.nctq.org/dmsView/StateofStates2015#page=3&;zoom=auto,-194,80

15.

Doyle

(1977). Paradigms for research on teacher effectiveness. Review of Research in Education, 5, 163-198.

16.

Every Student Succeeds Act. (2015). Retrieved from https://www.congress.gov/bill/114th-congress/senate-bill/1177/text#toc-HB1FE71BC6C094FA285180CCC608ADADE

17.

Gansle

K. A.

Noell

G. H.

Grandstaff-Beckers

Stringer

Roberts

Burns

J. M.

(2015). Value-added assessment of teacher preparation: Implications for special education. Intervention in School and Clinic, 51, 106-111.

18.

Goe

Bell

Little

(2008). Approaches to Evaluating Teacher Effectiveness: A Research Synthesis. National Comprehensive Center for Teacher Quality.

19.

Goldhaber

(2015). Exploring the potential of value-added performance measures to affect the quality of the teacher workforce. Educational Researcher, 44, 87-95.

20.

Harris

D. N.

Herrington

C. D.

(2015). Editors’ introduction: The use of teacher value-added measures in schools: New evidence, unanswered questions, and future prospects. Educational Researcher, 44, 71-76.

21.

Harris

D. N.

Ingle

W. K.

Rutledge

S. A.

(2014). How teacher evaluation methods matter for accountability: A comparative analysis of teacher effectiveness ratings by principles and teacher value-added measures. American Educational Research Journal, 51, 73-112.

22.

Horner

Carr

Halle

Mcgee

Odom

Wolery

(2005). The use of single-subject research to identify evidence-based practice in special education. Exceptional Children, 71, 165-179.

23.

Johnson

Semmelroth

C. L.

(2014a). Introduction to AEI’s special issue on special education teacher evaluations. Assessment for Effective Intervention, 39, 67-70.

24.

Johnson

Semmelroth

C. L.

(2014b). Special education teacher evaluation: Why it matters, what makes it challenging, and how to address these challenges. Assessment for Effective Intervention, 39, 71-82.

25.

Johnston

Wilkinson

(2009). Enhancing validity of critical tasks selected for college and university program portfolios. National Forum of Teacher Education Journal, 19(3), 1-6.

26.

Jones

N. D.

Brownell

M. T.

(2014). Examining the use of classroom observations in the evaluation of special education teachers. Assessment for Effective Intervention, 39, 112-124.

27.

Jones

N. D.

Buzick

H. M.

Turkan

(2013). Including students with disabilities and English learners in measures of educator effectiveness. Educational Researcher. doi:0013189X12468211.

28.

Kearns

J. F.

Kleinert

H. L.

Thurlow

M. L.

Gong

Quenemoen

(2015). Alternate assessments as one measure of teacher effectiveness: Implications for our field. Research and Practice for Persons With Severe Disabilities, 40, 20-35.

29.

Klinger

Brownell

Bateman

(2011). Special education teacher evaluation. New Times for Division for Learning Disabilities, 29(3), 1-5.

30.

Lawshe

C. H.

(1975). A quantitative approach to content validity. Personality Psychology, 28, 563-575.

31.

Massachusetts Department of Elementary and Secondary Education. (2012). Massachusetts model system for educator evaluation: Part III: Guide to rubrics and model rubrics for superintendent, administrator and teacher. Retrieved from http://www.doe.mass.edu/edeval/model/

32.

Nathan

Gorman

(2002). A guide to treatments that work (2nd ed.). New York, NY: Oxford University Press.

33.

National Autism Center (NAC). (2009). National Standards Re- port. Randolph, Massachusetts: National Autism Center.

34.

National Research Council. (2001). Educating children with autism. Washington, DC: National Academies Press.

35.

No Child Left Behind Act of 2001, Pub. L. No. 107-110, 115 Stat. 1425. (2002).

36.

Odom

Brantlinger

Gersten

Horner

Thompson

Harris

(2005). Research in special education: Scientific methods and evidence-based practices. Exceptional Children, 71, 137-148.

37.

Rogers

Vismara

(2008). Evidence-based comprehensive treatments for early autism. Journal of Clinical Child & Adolescent Psychology, 37, 8-38. doi:10.1080/15374410701817808

38.

Simpson

(2005). Evidence-based practices and students with autism spectrum disorders. Focus on Autism and Other Developmental Disabilities, 20, 140-149.

39.

U.S. Department of Education. (2009, November). Race to the Top executive summary. Retrieved from http://www2.ed.gov/programs/racetothetop/executive-summary.pdf

40.

U.S. Department of Education, Office of Elementary and Secondary Education, & Office of State Support. (2015). Fundamental change: Innovation in America’s schools under Race to the Top. Washington, DC: U.S. Department of Education.

41.

U.S. Department of Education, Office of Planning, Evaluation and Policy Development. (2010). A blueprint for reform: The reauthorization of the Elementary and Secondary Education Act. Retrieved from http://www2.ed.gov/policy/elsec/leg/blueprint/

42.

Wong

Odom

S. L.

Hume

K. A.

Cox

A. W.

Fettig

Kucharczyk

Schultz

T. R.

(2015). Evidence-based practices for children, youth, and young adults with autism spectrum disorder: A comprehensive review. Journal of Autism and Developmental Disorders, 45, 1951-1966.

43.

Woolf

S. B.

(2015). Special education professional standards: How important are they in the context of teacher performance evaluation? Teacher Education and Special Education, 38, 276-290.