This study is a preliminary examination of the reliability of frequent retrospective teacher behavior ratings. Frequent retrospective behavior ratings are an approach for creating scales that can be used to monitor individual behavioral progress. In this study, the approach is used to progress monitor behavioral individualized education plan goals for 7 students with, or at risk of, emotional and behavioral disorders. Reliability was examined for both quantitative-based and qualitative judgment-based retrospective teacher behavior ratings. Quantitative-based behavior ratings focused primarily on frequency and duration scales, whereas qualitative-based behavior ratings focused on topography and intensity scales. Findings indicated that reliability can be adequately achieved, although there is variability depending on the type of scale. Implications are discussed as they relate to progress monitoring individual behavioral goals and examining responsiveness to intervention.
Get full access to this article
View all access options for this article.
References
1.
AgrestiA. (1996). An introduction to categorical data analysis.New York: Wiley.
2.
AryD., & SuenH. K. (1989). Analysing qualitative behavioural observational data.Mahwah, NJ: Lawrence Erlbaum Associates.
3.
BaileyJ. S., WolfM. M., & PhillipsE. L. (1970). Home-based reinforcement and the modification of pre-delinquents’ classroom behavior. Journal of Applied Behavior Analysis, 3, 223–233.
4.
BarnettD. W., ElliottN., WolsingL., BungerC. E., HaskiH., & McKissickC. (2006). Response to intervention for young children with extremely challenging behaviors: What it might look like. School Psychology Review, 35, 568–582.
5.
ChafouleasS. M., ChristT. J., Riley-TillmanT. C., BrieschA. M., & ChaneseJ. A. M. (2007). Generalizability and dependability of direct behavior ratings to assess social behavior of preschoolers. School Psychology Review, 36, 63–79.
6.
ChafouleasS. M., McDougalJ. L., Riley-TillmanT. C., PanahonC. J., & HiltA. M. (2005). What do daily behavior report cards (DBRCs) measure? An initial comparison of DBRCs with direct observation for off-task behavior. Psychology in the Schools, 42, 669–676.
7.
ChafouleasS. M., Riley-TillmanT. C., & McDougalJ. L. (2002). Good, bad, or in-between: How does the daily behavior report card rate?Psychology in the Schools, 39, 157–169.
8.
ChafouleasS. M., Riley-TillmanT. C., & SassuK. A. (2006). Acceptability and reported use of daily behavior report cards among teachers. Journal of Positive Behavior Interventions, 8, 174–182.
9.
ChafouleasS. M., Riley-TillmanT. C., SassuK. A., LaFranceM. J., & PatwaS. S. (2007). Daily behavior report cards: An investigation of the consistency of on-task data across raters and methods. Journal of Positive Behavior Interventions, 9, 30–37.
10.
CheneyD., FlowerA., & TempletonT. (2008). Applying response to intervention metrics in the social domain for students at risk of developing emotional or behavioral disorders. Journal of Special Education, 42, 108–126.
11.
CroneD. A., HornerR. H., & HawkenL. S. (2004). Responding to problem behavior in schools: The behavior education program.New York: Guilford Press.
12.
CullinanD., & SabornieE. J. (2004). Characteristics of emotional disturbance in middle and high school students. Journal of Emotional and Behavioral Disorders, 12, 157–167.
13.
DenoS. L. (1985). Curriculum-based measurement: The emerging alternative. Exceptional Children, 52, 219–232.
14.
GreshamF. M. (2005). Response to intervention: An alterative means of identifying students as emotionally disturbed. Education and Treatment of Children, 28, 328–344.
15.
HawkenL. S., VincentC. G., & SchumannJ. (2008). Response to intervention for social behavior, Journal of Emotional and Behavioral Disorders, 16, 213–225.
16.
HintzeJ. (2001). NCSS and PASS.Kaysville, UT: Number Cruncher Statistical Systems.
17.
HintzeJ. M., & MatthewsW. J. (2004). The generalizability of systematic direct observations across time and setting: A preliminary investigation of the psychometrics of behavioral observation. School Psychology Review, 33, 258–270.
18.
Individuals with Disabilities Education Improvement Act, H.R. 1350, 108th Congress (2004).
19.
LaneK. L. (2007). Identifying and supporting students at risk for emotional and behavioral disorders within multi-level models: Data driven approaches to conducting secondary interventions with an academic emphasis. Education and Treatment of Children, 30, 135–164.
20.
Riley-TillmanT. C., ChafouleasS. M., SassuK. A., ChaneseJ. A. M., & GlazerA. D. (2008). Examining the agreement of direct behavior ratings and systematic direct observation data for on-task and disruptive behavior. Journal of Positive Behavior Interventions, 10, 136–143.
21.
SchumakerJ. B., HovellM. F., & ShermanJ. A. (1977). An analysis of daily behavior report cards and parent-managed privileges in the improvement of adolescents’ classroom performance. Journal of Applied Behavior Analysis, 10, 449–464.
22.
SteegeM. W., DavinT., & HathawayM. (2001). Reliability and accuracy of a performance-based behavioral recording procedure. School Psychology Review, 30, 252–261.
23.
ShinnM. R. (2007). Identifying students at risk, monitoring performance, and determining eligibility within rti: Research on educational need and benefit from academic intervention. School Psychology Review, 36, 601–617.
24.
SuenH. K., & AryD. (1989). Analyzing quantitative behavioral observation data.Hillsdale, NJ: Lawrence Erlbaum & Associates.
25.
WagnerM., KutashK., DuchnowskiA. J., EpsteinM. H., & SumiW. C. (2005). The children and youth we serve: A national picture of the characteristics of students with emotional disturbances receiving special education. Journal of Emotional and Behavioral Disorders, 13, 79–96.
26.
WalkerH. M., RamseyE., & GreshamR. M. (2004). Antisocial behavior in school: Evidence-based practices (2nd ed.). Belmont, CA: Wads-worth.
