Content Validation of Direct Behavior Rating Multi-Item Scales for Assessing Problem Behaviors

Abstract

Direct Behavior Rating (DBR) is an efficient method for monitoring changes in student behavior in response to intervention. Emerging research on DBR Multi-Item Scales (DBR-MIS) indicates that DBR-MIS has promising characteristics as a progress-monitoring assessment. Specifically, the multiple items within DBR-MIS allow stakeholders to measure specific behaviors at the item level, as well as global constructs at the scale level. In addition, studies have shown that fewer rating occasions are necessary to reach acceptable levels of dependability when using DBR-MIS as opposed to single-item scales (DBR-SIS). The purpose of the study was to develop and validate DBR-MIS problem behavior scales (Disruptive, Oppositional, Interpersonal Conflict, and Conduct Problems) that may be used to evaluate students’ response to social-emotional or behavioral intervention. Within the first phase of development, item content was generated and subjected to evaluation by panels of researchers and school-based consumers. Exploratory factor analysis (EFA) was then used in the second phase to identify items that represented the strongest indicators of each construct. Teachers (N = 307) in Grades K–3, from 35 school districts across 13 states in the Northeastern, Midwestern, Southern, and Southwestern United States each completed ratings for one randomly selected student (N = 307). Results of the EFA using a starting pool of nine to 11 items for each DBR-MIS initially indicated one-factor solutions for the Disruptive and Oppositional scales and a two-factor solution for the Interpersonal Conflict scale. Consequently, a new Conduct Problems scale was created from items loading on the second factor. Implications for progress monitoring and future research are discussed.