The Scales for Assessing Emotional Disturbance–Third Edition: Reliability and Validity of the Screener

Abstract

The Scales for Assessing Emotional Disturbance Screener–Third Edition (SAED-3) is a standardized, norm-referenced measure designed to identify school-age students at risk for emotional and behavioral problems. Four studies are reported to address the psychometric status of the SAED-3 Screener. Study 1 examined the internal consistency of the Screener using a sample of 1,430 students. Study 2 investigated the interrater reliability of the Screener results across 123 pairs of teachers who had worked with the student for at least 2 months. Study 3 assessed the extent to which the results from the Screener are consistent over time by examining test–retest reliability. Study 4 examined convergent validity by comparing the Screener to the Strength and Difficulties Questionnaire (SDQ). Across all studies, samples were drawn from populations of students included in the nationally representative normative sample. The averaged coefficient alpha for the Screener was .88. Interrater reliability coefficient for the composite was .83. Test–retest reliability of the composite was .83. Correlations with the SDQ subscales ranged from .74 to .99, and the correlation of the Screener to the SDQ composite was .99. Limitations and implications for use of the Screener are discussed.

Keywords

emotional disturbance behavior disorder assessment reliability norm-referenced

Get full access to this article

View all access options for this article.

References

Aiken

L. R.

Groth-Marnat

(2006). Psychological testing and assessment (12th ed.). Pearson Educational Group.

Anastasi

Urbina

(1997). Psychological testing (7th ed.). Prentice Hall.

Benner

G. J.

Kutash

Nelson

J. R.

Fisher

M. B.

(2013). Closing the achievement gap of youth with emotional and behavioral disorders through multi-tiered systems of support. Education and Treatment of Children, 36(3), 15–29. https://doi.org/10.1353/etc.2013.0018

Berkeley

Bender

W. N.

Gregg Peaster

Saunders

(2009). Implementation of response to intervention: A snapshot of progress. Journal of Learning Disabilities, 42(1), 85–95. https://doi.org/10.1177/0022219408326214

Beswick

J. F.

Willms

J. D.

Sloat

E. A.

(2005). A comparative study of teacher ratings of emergent literacy skills and students’ performance on a standardized measure. Education, 126(1), 116–137.

Bradley

Doolittle

Bartolotta

(2008). Building on the data and adding to the discussion: The experiences and outcomes of students with emotional disturbance. Journal of Behavioral Education, 17(1), 4–23. https://doi.org/10.1007/s10864-007-9058-6

Bruhn

A. L.

Woods-Groves

Huddle

(2014). A preliminary investigation of emotional and behavioral screening practices in K-12 schools. Education and Treatment of Children, 37(4), 611–634. https://doi.org/10.1353/etc.2014.0039

Center on Positive Behavioral Interventions and Supports. (2019). Tiered framework. https://www.pbis.org/pbis/tiered-framework

Cipriano

Barnes

T. N.

Bertoli

M. C.

Rivers

S. E.

(2018). Applying the classroom assessment scoring system in classrooms serving students with emotional and behavioural disorders. Emotional and Behavioural Difficulties, 23(4), 343–360. https://doi.org/10.1080/13632752.2018.1461454

10.

Cohen

(1968). Weighted kappa: Nominal scale agreement provision for scaled disagreement or partial credit. Psychological Bulletin, 70(4), 213–220. https://doi.org/10.1037/h0026256

11.

Conroy

M. A.

(2016). Moving the dial for students with emotional and behavioral disorders. Journal of Emotional and Behavioral Disorders, 24(3), 191–193. https://doi.org/10.1177/1063426616651602

12.

Cronbach

L. J.

(1951). Coefficient alpha and the internal structure of tests. Psychometrika, 16(3), 297–334. https://doi.org/10.1007/BF02310555

13.

Cullinan

Epstein

M. H.

(2012). Emotional and behavioral screener. PRO-ED.

14.

Epstein

Cullinan

Pierce

Huscroft-D’Angelo

Wery

(2020). Scales for Assessing Emotional Disturbance (3rd ed.). PRO-ED.

15.

Erford

B. T.

Short

Freeman

(2012). The Screening Test for Emotional Problems– Self-Report (STEP-S): Studies of reliability and validity. Assessment for Effective Intervention, 37(4), 251–256. https://doi.org/10.1177/1534508411413759

16.

Federal Register. (2006). Assistance to states for the education of children with disabilities and preschool grants for children with disabilities; Final Rule 34. CFR Parts 300 and 301, 71 Fed. Reg. 46540 (August 14, 2006).

17.

Freeman

Yell

M. L.

Shriner

J. G.

Katsiyannis

(2019). Federal policy on improving outcomes for students with emotional and behavioral disorders: Past, present, and future. Behavioral Disorders, 44(2), 97–106. https://doi.org/10.1177/0198742918814423

18.

Goodman

(2014, August). Scoring the strengths & difficulties questionnaire for age 4–17. https://www.ehcap.co.uk/content/sites/ehcap/uploads/NewsDocuments/236/SDQEnglishUK4-17scoring-1.PDF

19.

Goodman

(2001). Psychometric properties of the Strengths and Difficulties Questionnaire (SDQ). Journal of the American Academy of Child and Adolescent Psychiatry, 40(11), 1337–1345. https://doi.org/10.1097/00004583-200111000-00015

20.

Goodman

(2005). Strengths and Difficulties Questionnaire. http://www.sdqinfo.com/py/sdqinfo/b3.py?language=Englishqz(USA)

21.

Hopkins

W. G.

(2002). A new view of statistics: A scale of magnitudes for effect statistics [Research resource]. http://www.sportsci.org/resource/stats/effectmag.html

22.

Individuals with Disabilities Education Improvement Act, 20 U.S.C. § 1400 (2004).

23.

Kamphaus

R. W.

Reynolds

C. R.

Dever

B. V.

(2014). Behavioral and mental health screening. In Kettler

R. J.

Glover

T. A.

Albers

C. A.

Feeney-Kettler

K. A.

(Eds.), Universal screening in educational settings: Evidence-based decision making for schools (pp. 249–273). American Psychological Association.

24.

Lambert

M. C.

Epstein

M. H.

Cullinan

(2014). The diagnostic quality of the emotional and behavioral screener. Journal of Psychoeducational Assessment, 32(1), 51–61. https://doi.org/10.1177/0734282913485541

25.

Lambert

M. C.

Garcia

A. G.

Epstein

M. H.

Cullinan

Martin

J. D.

(2020). A review of the research on the Scales for Assessing Emotional Disturbance: Screener. Journal of Applied School Psychology, 37(1), 43–68. https://doi.org/10.1080/15377903.2020.1780659

26.

Lewis

T. J.

McIntosh

Simonsen

Mitchell

B. S.

Hatton

H. L.

(2017). Schoolwide systems of positive behavior support: Implications for students at risk and with emotional/behavioral disorders. AERA Open, 3(2), 1–11. https://doi.org/10.1177/2332858417711428

27.

Mellor

(2004). Furthering the use of the strengths and difficulties questionnaire: Reliability with younger child respondents. Psychological Assessment, 16(4), 396–401. https://doi.org/10.1037/1040-3590.16.4.396

28.

Merrell

K. W.

(2017). Behavioral, social, and emotional assessment of children and adolescents. Taylor & Francis.

29.

Mertens

D. M.

(1998). Research methods in education and psychology: Integrating diversity with quantitative and qualitative approaches. SAGE.

30.

Miller

M. D.

Linn

R. L.

Gronlund

N. E.

(2009). Measurement and assessment in teaching (10th ed.). Pearson.

31.

Mitchell

B. S.

Kern

Conroy

M. A.

(2019). Supporting students with emotional or behavioral disorders: State of the field. Behavioral Disorders, 44(2), 70–84. https://doi.org/10.1177/0198742918816518

32.

Mrazek

P. J.

(2005). Prevention of psychiatric disorders in children and adolescents. In Sadock

B. J.

Sadock

V. A.

(Eds.), Kaplan & Sadock’s comprehensive textbook of psychiatry (Vol. II, pp. 3513–3518). Lippincott Williams & Wilkins.

33.

Nordness

P. D.

Epstein

M. H.

Cullinan

Pierce

C. D.

(2014). Emotional and behavioral screener: Test–retest reliability, interrater reliability, and convergent validity. Remedial and Special Education, 35(4), 211–217. https://doi.org/10.1177/0741932513497596

34.

Pierce

C. D.

Lambert

Alamer

(2016). Convergent, criterion and social validity of the emotional and behavioral screener. Journal of Child and Family Studies, 25(1), 77–85. https://doi.org/10.1007/s10826-015-0206-7

35.

ProQuest LLC. (2018). ProQuest statistical abstracts of the United States, 2019 (7th ed.). Bernan.

36.

Reynolds

C. R.

Livingston

R. B.

Willson

(2009). Measurement and assessment in education (2nd ed.). Pearson.

37.

Salvia

Ysseldyke

J. E.

Witmer

(2017). Assessment in special and inclusive education (13th ed.). Cengage Learning.

38.

U.S. Department of Education, Office of Special Education and Rehabilitative Services, Office of Special Education Programs. (2016). IDEA Part B Child Count and Educational Environments for School Year (2015-2016): OSEP Data Documentation. Washington D.C.

39.

U.S. Department of Education, Office of Special Education and Rehabilitative Services, Office of Special Education Programs. (2018). 40th annual report to Congress on the Implementation of the Individuals with Disabilities Education Act.

40.

Viera

A. J.

Garrett

J. M.

(2005). Understanding interobserver agreement: The kappa statistic. Family Medicine, 37(5), 360–363.

41.

Walker

H. M.

Marquez

Yeaton

Pennefather

Forness

S. R.

Vincent

C. G.

(2015). Teacher judgment in assessing students’ social behavior within a response-to-intervention framework: Using what teachers know. Education and Treatment of Children, 38(3), 363–382. https://doi.org/10.1353/etc.2015.0019

42.

Weiss

L. G.

(2016). Standardized assessment for clinical practitioner: A primer. Pearson.

43.

Wery

J. J.

Cullinan

(2011). State definitions of emotional disturbance. Journal of Emotional and Behavioral Disorders, 21(1), 45–52. https://doi.org/10.1177/1063426611418234

44.

Wexler

(2018). School-based multi-tiered systems of support (MTSS): An introduction to MTSS for neuropsychologists. Applied Neuropsychology: Child, 7(4), 306–316. https://doi.org/10.1080/21622965.2017.1331848