The Scales for Assessing Emotional Disturbance–Third Edition: Internal Reliability,Interrater Reliability,and Test

Abstract

The Scales for Assessing Emotional Disturbance–Third Edition Rating Scale (SAED-3 RS; Epstein et al.) is a standardized, norm-referenced measure designed to aid in the identification process by providing useful data to professionals determining eligibility of students with an emotional disturbance (ED). Three studies are reported to address the reliability of the SAED-3 RS. Study 1 investigated the internal reliability of the SAED-3 RS using data from a nationally representative sample of 1,430 students and 441 with ED. Study 2 examined interrater reliability between 123 pairs of educators who had worked with the student for at least 2 months. Study 3 assessed the test–retest reliability over a 2-week period to determine stability of the SAED-3 RS. Across all studies, scores collected from the SAED-3 RS were determined to be a reliable, stable for measuring the emotional and behavioral functioning of students. Specifically, the averaged coefficient alpha for internal consistency ranged from .79 to .92 for each subscale and .96 for the composite score; interrater reliability coefficients ranged from .77 to .89 for each subscale and .89 for the composite score, and test–retest reliability coefficients ranged from .79 to .92 for each subscale and .96 for the composite score. Limitations, future research and implications for use of the SAED-3 RS are discussed.

Keywords

emotional disturbance behavior disorder assessment reliability norm-referenced

Get full access to this article

View all access options for this article.

References

Aiken

L. R.

Groth-Marnat

(2006). Psychological testing and assessment (12th ed.). Pearson.

Altman

D. G.

(1991). Practical statistics for medical research. Chapman & Hall.

Anastasi

Urbina

(1997). Psychological testing (7th ed.). Prentice Hall.

Benner

G. J.

Uhing

B. M.

Pierce

C. D.

Beaudoin

K. M.

Ralston

N. C.

Mooney

(2009). An extension convergent validity study of the Systematic Screening for Behavior Disorders and the Achenbach Teacher’s Report Form. Journal of At-Risk Issues, 15(1), 9–15.

Bracken

B. A.

McCallum

R. S.

(2016). Universal nonverbal intelligence test (2nd ed.). PRO-ED.

Bradley

Doolittle

Bartolotta

(2008). Building on the data and adding to the discussion: The experiences and outcomes of students with emotional disturbance. Journal of Behavioral Education, 17(1), 4–23. https://doi.org/10.1007/s10864-007-9058-6

Breen

M. J.

Fiedler

C. R.

(2003). Behavioral approach to assessment of youth with emotional/behavioral disorders: A handbook for school-based practitioners. PRO-ED.

Center on Positive Behavioral Interventions and Supports. (2019). Tiered framework. https://www.pbis.org/pbis/tiered-framework

Chaturvedi

S. R. B. H.

Shweta

R. C.

(2015). Evaluation of inter-rater agreement and inter-rater reliability for observational data: an overview of concepts and methods. Journal of the Indian Academy of Applied Psychology, 41(3), 20–27.

10.

Cohen

(1968). Weighted kappa: Nominal scale agreement provision for scaled disagreement or partial credit. Psychological Bulletin, 70(4), 213–220. https://doi.org/10.1037/h0026256

11.

Conroy

M. A.

(2016). Moving the dial for students with emotional and behavioral disorders. Journal of Emotional and Behavioral Disorders, 24(3), 191–193. https://doi.org/10.1177/1063426616651602

12.

Cronbach

L. K.

(1951). Coefficient alpha and the internal structure of tests. Psychometrika, 16(3), 297–334. https://doi.org/10.1007/bf02310555

13.

Epstein

M. H.

Cullinan

(1998). Scales for assessing emotional disturbance. PRO-ED.

14.

Epstein

M. H.

Cullinan

(2010). Scales for assessing emotional disturbance (2nd ed.). PRO-ED.

15.

Epstein

M. H.

Cullinan

Pierce

Huscroft-D’Angelo

Wery

(2021). Scales for assessing emotional disturbance (3rd ed.). PRO-ED.

16.

Epstein

M. H.

Nelson

J. R.

Trout

A. L.

Mooney

(2005). Achievement and emotional disturbance: Academic status and intervention research. In Epstein

M. H.

Kutash

Duchnowski

A. J.

(Eds.), Outcomes for children and youth with emotional and behavioral disorders and their families: Programs and evaluation best practices (2nd ed., pp. 451–477). PRO-ED.

17.

Erford

B. T.

Short

Freeman

(2012). The Screening Test for Emotional Problems–Self-Report (STEP-S): Studies of reliability and validity. Assessment for Effective Intervention, 37(4), 251–256. https://doi.org/10.1177/1534508411413759

18.

Federal Register. (2006, August 14). Assistance to states for the education of children with disabilities and preschool grants for children with disabilities; Final Rule 34 (CFR Parts 300 and 301, 71 Fed. Reg. 46540). https://www.federalregister.gov/documents/2006/08/14/06-6656/assistance-to-states-for-the-education-of-children-with-disabilities-and-preschool-grants-for

19.

Flannery

K. B.

McGrath Kato

Horner

R. H.

(2019, September). Using outcome data to implement multi-tiered behavior support (PBIS) in high schools. https://www.pbis.org/resource/using-outcome-data-in-high-schools

20.

Goodman

(2001). Psychometric properties of the Strengths and Difficulties Questionnaire (SDQ). Journal of the American Academy of Child & Adolescent Psychiatry, 40(11), 1337–1345. https://doi.org/10.1097/00004583-200111000-00015

21.

Goodman

(2005). Strengths and Difficulties Questionnaire. http://www.sdqinfo.com/py/sdqinfo/b3.py?language=Englishqz(USA)

22.

Gould

S. J.

(1996). The mismeasure of man (2nd ed.). WW Norton & Company.

23.

Gresham

F. M.

Elliott

S. N.

(2008). Social skills improvement system: Rating scales. Pearson.

24.

Guilford

J. P.

(1954). Psychometric methods (2nd ed.). McGraw-Hill.

25.

Hopkins

W. G.

(2002). A new view of statistics: A scale of magnitudes for the effect statistics. http://www.sportsci.org/resource/stats/effectmag.html

26.

Individuals with Disabilities Education Improvement Act, 20 U.S.C. § 1400 et seq. (2004). https://sites.ed.gov/idea/statute-chapter-33

27.

Kalton

Flores-Cervantes

(2003). Weighting methods. Journal of Official Statistics, 19(2), 81–97.

28.

Landis

J. R.

Koch

G. G.

(1977). An application of hierarchical kappa-type statistics in the assessment of majority agreement among multiple observers. Biometrics, 33(2), 363–374. https://doi.org/10.2307/2529786

29.

Lane

K. L.

Cook

B. G.

Tankersley

(Eds.). (2013). Research-based strategies for improving outcomes in behavior. Pearson.

30.

Maggin

Wehby

Gilmour

(2016). Intensive academic interventions for students with emotional and behavioral disorders: An experimental framework. Journal of Emotional and Behavioral Disorders, 24(3), 138–147. https://doi.org/10.1177/1063426616649162

31.

Miller

M. D.

Linn

R. L.

Gronlund

N. E.

(2013). Measurement and assessment in teaching (11th ed.). Pearson.

32.

National Center on Response to Intervention. (2010, March). Essential components of RTI—A closer look at response to intervention. U.S. Department of Education, Office of Special Education Programs, National Center on Response to Intervention.

33.

National Research Council & Institute of Medicine. (2009). Preventing mental, emotional, and behavioral disorders among young people: Progress and possibilities. Committee on the prevention of mental disorders and substance abuse among children, youth, and young adults: Research advances and promising interventions. In O’Connell

M. E.

Boat

Warner

K. E.

(Eds.), Board on children, youth, and families, division of behavioral and social sciences and education (pp. 1–592). National Academies Press.

34.

Nordness

P. D.

Epstein

M. H.

Cullinan

Pierce

C. D.

(2014). Emotional and behavioral screener: Test-retest reliability, inter-rater reliability, and convergent validity. Remedial & Special Education, 35(4), 211–217. https://doi.org/10.1177/0741932513497596

35.

Pierce

C. D.

Nordness

P. D.

Epstein

M. H.

Cullinan

(2016). Applied examples of screening students at risk of emotional and behavioral disabilities. Intervention in School & Clinic, 52(1), 6–11. https://doi.org/10.1177/1053451216630276

36.

Pike

G. R.

(2008). Using weighting adjustments to compensate for survey nonresponse. Research in Higher Education, 49(2), 153–171. https://doi.org/10.1007/s11162-007-9069-0

37.

ProQuest LLC. (2016a). ProQuest statistical abstracts of the United States, 2017 (5th ed.). Bernan.

38.

ProQuest LLC. (2016b). ProQuest Statistical Abstract of the United States, 2016 (4th ed.). Bernan.

39.

Reynolds

C. R.

Kamphaus

R. W.

(2015). Behavior assessment system for children (3rd ed.). Pearson.

40.

Reynolds

C. R.

Livingston

R. C.

Willson

(2009). Measurement and assessment in education (2nd ed.). Pearson.

41.

Salvia

Ysseldyke

J. E.

Witmer

(2017). Assessment in special and inclusive education (13th ed.). Cengage Learning.

42.

Stoiber

K. C.

(2014). A comprehensive framework for multitiered systems of support in school psychology. In Harrison

P. L.

Thomas

(Eds.), Best practices in school psychology-VI: Data-based and collaborative decision making (pp. 41–70). NASP.

43.

U.S. Department of Education. (2019). Positive behavior and intervention supports. https://www.pbis.org/

44.

U.S. Department of Education, Office of Special Education and Rehabilitative Services, Office of Special Education Programs. (2012). IDEA Part B child count and educational environments. https://www2.ed.gov/programs/osepidea/618-data/collection-documentation/data-documentation-files/part-b/child-count-and-educational-environment/idea-partb-childcountandedenvironment-2017-18.pdf

45.

U.S. Department of Education, Office of Special Education and Rehabilitative Services, Office of Special Education Programs. (2016). 38th annual report to congress on the implementation of the individuals with disabilities education act. https://www2.ed.gov/about/reports/annual/osep/2016/parts-b-c/index.html

46.

U.S. Department of Education, Office of Special Education and Rehabilitative Services, Office of Special Education Programs. (2018). 40th annual report to congress on the implementation of the individuals with disabilities education act. https://www2.ed.gov/about/reports/annual/osep/2018/parts-b-c/40th-arc-for-idea.pdf

47.

Vaske

J. J.

Beaman

Sponarski

C. C.

(2017). Rethinking internal consistency in Cronbach’s Alpha. Leisure Sciences, 39(2), 163–173. https://doi.org/10.1080/01490400.2015.1127189

48.

Wagner

Newman

(2012). Longitudinal transition outcomes of youth with emotional disturbances. Psychiatric Rehabilitation Journal, 35(3), 199–208. https://doi.org/10.2975/35.3.2012.199.208

49.

Weiss

L. G.

(2016). June). Standardized Assessment for Clinical Practitioner: A Primer. Pearson.

50.

Wexler

(2018). School-based multi-tiered systems of support (MTSS): An introduction to MTSS for neuropsychologists. Applied Neuropsychology: Child, 7(4), 306–316. https://doi.org/10.1080/21622965.2017.1331848

51.

Whitcomb

(2018). Behavioral, social, and emotional assessment of children and adolescents (5th ed.). Taylor & Francis.

The Scales for Assessing Emotional Disturbance–Third Edition: Internal Reliability,Interrater Reliability,and Test–Retest Reliability

Abstract

Keywords

Get full access to this article

References