Supported by decades of research on praise and its effect on student behaviors, we developed the Behavior-Specific Praise–Observation Tool (BSP-OT) to measure characteristics of effective praise. We evaluated interrater reliability of the BSP-OT to measure praise specificity, contingency, and variety using intraclass correlation (ICC) and Cohen’s kappa statistics. In addition, we assessed usefulness and practicality of the tool with social validity measures. Four raters with experience in praise research completed a survey and coded videos (n = 14) of teachers in authentic situations delivering praise. Overall assessment indicated strong reliability between raters with ICC(2, k) of .80: 95% confidence interval (CI) = [0.77, 0.83], F(269, 19906) = 5.1, p < .001, and mean kappa score of .91. Furthermore, high social validity ratings suggest the BSP-OT is a valuable contribution to the field concerning praise research and teacher development. The process of developing the BSP-OT and study findings are presented, with a discussion of implications and suggestions for future research.
CicchettiD. V. (1994). Guidelines, criteria, and rules of thumb for evaluating normed and standardized assessment instruments in psychology. Psychological Assessment, 6, 284–290. https://doi.org/10.1037/1040-3590.6.4.284
5.
ConroyM. A.SutherlandK. S.SnyderA.Al-HendawiM.VoA. (2009). Creating a positive classroom atmosphere: Teachers’ use of effective praise and feedback. Beyond Behavior, 18, 18–26.
FloressM. T.BeschtaS. L. (2018). An analysis of general education teachers’ use of diverse praise. Psychology in the Schools, 55, 1188–1204. https://doi.org/10.1002/pits.22187
8.
FloressM. T.BeschtaS. L.MeyerK. L.ReinkeW. M. (2017). Praise research trends and future directions: Characteristics and teacher training. Behavioral Disorders, 43, 227–243. https://doi.org/10.1177/0198742917704648
9.
FranklinR. K.MitchellJ. O.WaltersK. S.LivingstonB.LinebergerM. B.PutmanC.YarboroughR.Karges-BoneL. (2018). Using “Swivl” robotic technology in teacher education preparation: A pilot study. TechTrends, 62, 184–189. https://doi.org/10.1007/s11528-017-0246-5
10.
GableR. A.ShoresR. E. (1980). Comparison of procedures for promoting reading proficiency of two children with behavioral and learning disorders. Behavioral Disorders, 5, 102–107. https://doi.org/10.1177/019874298000500204
11.
GageN. A.MacSuga-GageA. S. (2017). Salient classroom management skills: Finding the most effective skills to increase student engagement and decrease disruptions. Report on Emotional & Behavioral Disorders in Youth, 17, 19–24.
12.
GageN. A.PrykanowskiD. A.HirnR. (2014). Increasing reliability of direct observation measurement approaches in emotional and/or behavioral disorders research using generalizability theory. Behavioral Disorders, 39, 228–244. https://doi.org/10.1177/019874291303900407
13.
GisevN.BellJ. S.ChenT. F. (2013). Interrater agreement and interrater reliability: Key concepts, approaches, and applications. Research in Social and Administrative Pharmacy, 9, 330–338.
14.
HagerK. D. (2012). Self-monitoring as a strategy to increase student teachers’ use of effective teaching practices. Rural Special Education Quarterly, 31, 9–17.
15.
HallR.LundD.JacksonD. (1968). Effects of teacher attention on study behavior. Journal of Applied Behavior Analysis, 1, 1–12.
16.
HallgrenK. A. (2012). Computing inter-rater reliability for observational data: An overview and tutorial. Tutorials in Quantitative Methods for Psychology, 8, 23–34. https://doi.org/10.20982/tqmp.08.1.p023
17.
HawkinsS. M.HeflinL. J. (2011). Increasing secondary teachers’ behavior-specific praise using a video self-modeling and visual performance feedback intervention. Journal of Positive Behavior Interventions, 13, 97–108.
18.
HintzeJ. (2005). Psychometrics of direct observation. School Psychology Review, 34, 507–519.
19.
HirnR. G.ScottT. M. (2014). Descriptive analysis of teacher instructional practices and student engagement among adolescents with and without challenging behavior. Education and Treatment of Children, 37, 589–610. https://doi.org/10.1353/etc.2014.0037
20.
KilgusS. P.Riley-TillmanT. C.StichterJ. P.SchoemannA. M.BellesheimK. (2016). Reliability of direct behavior ratings–social competence (DBR-SC) data: How many ratings are necessary?School Psychology Quarterly, 31, 431–442. https://doi.org/10.1037/spq0000128
21.
KirbyF. D.ShieldsF. (1972). Modification of arithmetic response rate and attending behavior in a seventh-grade student. Journal of Applied Behavior Analysis, 5, 79–84. https://doi.org/10.1901/jaba.1972.5-79
MarkelzA. M.TaylorJ. C.KitchenT.RiccominiP. J.ScheelerM. C.McNaughtonD. B. (2019). Effects of tactile prompting and self-monitoring on teachers’ use of behavior specific praise. Exceptional Children, 85(4), 471–489. https://doi.org/10.1177/0014402919846500
24.
MooreT. C.MagginD. M.ThompsonK. M.GordonJ. R.DanielsS.LangL. E. (2019). Evidence review for teacher praise to improve students’ classroom behavior. Journal of Positive Behavior Interventions, 21, 3–18.
25.
RathelJ. M.DrasgowE.BrownW. H.MarshallK. J. (2014). Increasing induction-level teachers’ positive-to-negative communication ratio and use of behavior-specific praise through e-mailed performance feedback and its effect on students’ task engagement. Journal of Positive Behavior Interventions, 16, 219–233. https://doi.org/10.1177/1098300713492856
26.
ReinkeW. M.HermanK. C.StormontM. (2013). Classroom-level positive behavior supports in schools implementing SW-PBIS: Identifying areas for enhancement. Journal of Positive Behavior Interventions, 15, 39–50. https://doi.org/10.1177/1098300712459079
27.
ReinkeW. M.StormontM.HermanK. C.PuriR.GoelN. (2011). Supporting children’s mental health in schools: Teacher perceptions of needs, roles, and barriers. School Psychology Quarterly, 26, 1–13. https://doi.org/10.1037/a0022714
28.
RoyerD. J.LaneK. L.DunlapK. D.EnnisR. P. (2019). A systematic review of teacher-delivered behavior-specific praise on K–12 student performance. Remedial and Special Education, 40, 112–128. https://doi.org/10.1177/0741932517751054
29.
ShoukriM. M. (2010). Measures of interobserver agreement and reliability (2nd ed.). CRC Press.
30.
SimonsenB.FreemanJ.DooleyK.MaddockE.KernL.MyersD. (2017). Effects of targeted professional development on teachers’ specific praise rates. Journal of Positive Behavior Interventions, 19, 37–47.
31.
SkinnerB. F. (1953). Science and human behavior. Free Press.
32.
SutherlandK. S.WehbyJ. H.CopelandS. R. (2000). Effect of varying rates of behavior-specific praise on the on-task behavior of students with EBD. Journal of Emotional and Behavioral Disorders, 8, 2–8. https://doi.org/10.1177/106342660000800101
33.
SutherlandK. S.WehbyJ. H.YoderP. J. (2002). Examination of the relationship between teacher praise and opportunities for students with EBD to respond to academic requests. Journal of Emotional and Behavioral Disorders, 10, 5–13. https://doi.org/10.1177/106342660201000102
34.
ThompsonR. F.SpencerW. A. (1966). Habituation: A model phenomenon for the study of neuronal substrates of behavior. Psychological Review, 73, 16–43. https://doi.org/10.1037/h0022681
35.
WestlingD. L. (2010). Teachers and challenging behavior: Knowledge, views, and practices. Remedial and Special Education, 31, 48–63. https://doi.org/10.1177/0741932508327466
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
