This study examined the generalizability and dependability of observational estimates of middle and high school teachers’ use of praise and reprimands. Frequency of behavior-specific praise, general praise, and total reprimands were collected across 67, 20-min observations that took place during class-wide instruction in general education classrooms. Generalizability theory was used to determine the number of observations needed to obtain dependable estimates of teacher behavior. Behavior-specific praise rates were consistently low. General praise rates were slightly higher and reprimand rates were notably higher and slightly more variable. Total reprimands had the strongest generalizability results and general praise had the weakest. Behavior-specific praise reached an acceptable level of dependability after 15 min, whereas general praise required a 35-min observation, and reprimand only required 5 min. Implications and future directions are discussed.
BrieschA. M.ChafouleasS. M.JohnsonA. J. (2016). Use of generalizability theory within K–12 school-based assessment: A critical review and analysis of the empirical literature. Applied Measurement in Education, 29(2), 83–107. https://doi.org/10.1080/08957347.2016.1138955
2.
BrieschA. M.ChafouleasS. M.Riley-TillmanT. C. (2010). Generalizability and dependability of behavior assessment methods to estimate academic engagement: A comparison of systematic direct observation and direct behavior rating. School Psychology Review, 39(3), 408–421.
3.
BrieschA. M.FergusonT. D.DanielsB.VolpeR. J.FeinbergA. B. (2017). Examining the influence of interval length on the dependability of observational estimates. School Psychology Review, 46, 426–432. https://doi.org/10.17105/SPR-2016-0006.V46-4
4.
BrieschA. M.SwaminathanH.WelshM.ChafouleasS. M. (2014). Generalizability theory: A practical guide to study design, implementation, and interpretation. Journal of School Psychology, 52, 13–35. http://dx.doi.org/10.1016/j.jsp.2013.11.008
5.
BrieschA. M.VolpeR. J.FloydR. G. (2018). A practical guide to classroom observation. Guilford Press.
CaldarellaP.LarsenR. A. A.WilliamsL.DownsK. R.WillsH. P.WehbyJ. H. (2020). Effects of teachers’ praise-to-reprimand ratios on elementary students’ on-task behaviour. Educational Psychology, 40, 1306–1322. https://doi.org/10.1080/01443410.2020.1711872
8.
CaldarellaP.LarsenR. A. A.WilliamsL.WillsH. P.WehbyJ. H. (2019). Teacher praise-to-reprimand ratios: Behavioral of students at risk for EBD compared with typically developing peers. Education and Treatment of Children, 42, 447–468. https://doi.org/10.1353/etc.2019.0021
9.
ChafouleasS. M.BrieschA. M.Riley-TillmanT. C.ChristT. J.BlackA. C.KilgusS. J. (2010). An investigation of the generalizability and dependability of Direct Behavior Rating Single Item Scales (DBR-SIS) to measure academic engagement and disruptive behavior of middle school students. Journal of School Psychology, 48, 219–246.
10.
ChalkK.BizoL. A. (2004). Specific praise improves on-task behaviour and numeracy enjoyment: A study of year four pupils engaged in the numeracy hour. Educational Psychology in Practice, 20(4), 335–352. https://doi.org/10.1080/0266736042000314277
11.
Clunies-RossP.LittleE.KienhuisM. (2008). Self-reported and actual use of proactive and reactive classroom management strategies and their relationship with teacher stress and student behaviour. Educational Psychology, 28(6), 693–710.
12.
CronbachL. J.GleserG. C.NandaH.RajaratnamN. (1972). The dependability of behavioral measurements. John Wiley.
13.
DownsK. R.CaldarellaP.LarsenR. A. A.CharltonC. T.WillisH. P.KampsD. M.WehbyJ. H. (2019). Teacher praise and reprimands: The differential response of students at risk for emotional and behavioral disorders. Journal of Positive Behavior Interventions, 21, 135–147. https://doi.org/10.1177/109830071880082
14.
FergusonT. D.BrieschA. M.VolpeR. J.DonaldsonA. R.FeinbergA. B. (2018). Psychometric considerations for conducting observations using time-sampling procedures. Assessment for Effective Intervention, 44, 45–57.
15.
FisherM.HamptonK.FloressM. T. (2019, January). Teacher praise training using a multi-tiered system of support model [Poster presentation]. 2019 Illinois School Psychology Association Convention, Springfield, IL, United States.
16.
FloraS. R. (2000). Praise’s magic reinforcement ratio: Five to one gets the job done. The Behavior Analyst Today, 1(4), 64–69.
17.
FloressM. T.BeaudoinM. M.BernasR. S. (2021). Exploring secondary teachers’ actual and perceived praise and reprimand use. Journal of Positive Behavior Interventions. Published on-line.
18.
FloressM. T.BeschtaS. (2018). An analysis of general education teachers’ use of diverse praise. Psychology in the Schools, 55, 1188–1204. doi.org/10.1002/pits.22187
19.
FloressM. T.BeschtaS.MeyerK.ReinkeW. (2017). Praise research trends and future directions: Characteristics and teacher training. Behavioral Disorders, 43, 227–243.
20.
FloressM. T.CatesG.PoroitK.EstradaN. (2020). Conceptualizing fixed-interval praise delivery. Intervention in School and Clinic, 56, 84–91. https://doi.org/10.1177/1053451220914889
21.
FloressM. T.JenkinsL. N. (2015). A preliminary investigation of kindergarten teachers’ use of praise in general education classrooms. Preventing School Failure: Alternative Education for Children and Youth, 59, 253–262. https://doi.org/10.1080/1045988X.2014.942834
22.
FloressM. T.JenkinsL. N.ReinkeW.McKownL. (2018). General education teachers’ natural rates of praise: A preliminary investigation. Behavioral Disorders, 43, 411–422.
23.
FloressM. T.RaderR.BerlinghofJ.FanokP. (2018). Externalizing behaviors within general, at-risk, and special education preschool classrooms: A preliminary investigation. Preventing School Failure: Alternative Education for Children and Youth, 62(4), 279–288. https://doi.org/10.1080/1045988X.2018.1443424
24.
FloressM. T.Zoder-MartellK.BeaudoinM.YehlingZ. (2021). Teacher praise to reprimand ratios during small and large group instruction: A video pilot study. Preventing School Failure: Alternative Education for Children and Youth. Published on-line.
25.
FornessS. R.KimJ.WalkerH. M. (2012). Prevelance of students with EBD: Impact on general education. Beyond Behavior, 21, 3–10.
26.
GableR. A.HesterP. H.RockM. L.HughesK. G. (2009). Back to basics: Rules, praise, ignoring, and reprimands revisited. Intervention in School and Clinic, 44, 195–205. https://doi.org/10.1177/1053451208328831
27.
GageN. A.GrasleyB. N. M.MacSugaG. A. S. (2018). Professional development to increase teacher behavior-specific praise: A single-case design replication. Psychology in the Schools, 55(3), 264–277. https://doi.org/10.1002/pits.22106
28.
GageN. A.PrykanowskiD.HirnR. (2014). Increasing reliability of direct observation measurement approaches in emotional and/or behavioral disorders using generalizability theory. Behavioral Disorders, 39(4), 228–244.
29.
GoodT.GrouwsD. (1977). Teaching effects: A process-product study in fourth grade mathematics classrooms. Journal of Teacher Education, 28, 49–54. https://doi.org/10.1177/002248717702800310
30.
HawkinsS. M.HeflinL. J. (2011). Increasing secondary teachers’ behavior-specific praise using a video self-modeling and visual performance feedback intervention. Journal of Positive Behavior Interventions, 13(2), 97–108. doi.org/10.1177/1098300709358110
31.
HintzeJ. M.MatthewsW. J. (2004). The generalizability of systematic direct observations across time and setting: A preliminary investigation of the psychometrics of behavioral observation. School Psychology Review, 33, 258–270.
32.
IngersollR. M. (2001). Teacher turnover and teacher shortages: An organizational analysis. American Educational Research Journal, 28, 499–534. https://doi.org/10.3102/00028312038003499
33.
JenkinsL. N.FloressM. T.ReinkeW. (2015). Rates and types of teacher praise: A review and future directions. Psychology in the Schools, 52(5), 463–476. https://doi.org/10.1002/pits.21835
34.
JohnstonJ. M.PennypackerH. S. (1993). Strategies and tactics of behavioral research (2nd ed.). Lawrence Erlbaum.
35.
KimY.StormontM. (2016). Korean early childhood educators’ self-reported versus observed strategies for addressing challenging behaviors: An exploratory study. International Journal on Disability and Human Development, 15, 43–48.
36.
MadsenC. H.BeckerW. C.ThomasD. R. (1968). Rules, praise, and ignoring: Elements of elementary classroom control. Journal of Applied Behavior Analysis, 1(2), 139–150. https://doi.org/10.1901/jaba.1968.1-139
37.
MarkelzA. M.RidenB. S.Zoder-MartellK.MillerJ. E.BolingerS. J. (2020). Reliability assessment of an observation tool to measure praise characteristics. Journal of Positive Behavior Interventions, 23, 17–29. https://doi.org/10.1177/1098300720907988
38.
MudfordO. C.TaylorS. A.MartinN. T. (2009). Continuous recording and interobserver agreement algorithms reported in the Journal of Applied Behavior Analysis (1995–2005). Journal of Applied Behavior Analysis, 42, 165–169. doi.org/10.1901/jaba.2009.42-165
39.
O’HandleyR. D.DufreneB. A.WhippleH. (2018). Tactile prompting and weekly performance feedback for increasing teachers’ behavior-specific praise. Journal of Behavioral Education, 27(3), 324–342. https://doi.org/10.1007/s10864-017-9283-6
40.
PattersonG. R. (1982). A social learning approach: Coercive family process (Vol. 3). Castalia.
41.
PattersonG. R. (2002). The early development of coercive family process. In ReidJ. B.PattersonG. R.SnyderJ. (Eds.), Antisocial behavior in children and adolescents: A developmental analysis and model for intervention (pp. 25–44). American Psychological Association. https://doi.org/10.1037/10468-002
42.
PfiffnerL. J.RosenL. A.O’LearyS. G. (1985). The efficacy of an all-positive approach to classroom management. Journal of Applied Behavior Analysis, 18, 257–261. https://doi.org/10.1901/jaba.1985.18-257
43.
ReinkeW. M.HermanK. C.StormontM. (2013). Classroom-level positive behavior supports in schools implementing SW-PBIS: Identifying areas of enhancement. Journal of Positive Behavior Interventions, 15(1), 39–50. https://doi.org/10.1177/1098300712459079
44.
ReinkeW. M.StormontM.HermanK. C.WachsmuthS.NewcomerL. (2015). The brief classroom interaction observation-revised: An observation system to inform and increase teacher use of universal classroom management practices. Journal of Positive Behavior Interventions, 17(3), 159–169. https://doi.org/10.1177/1098300715570640
45.
SalviaJ.YsseldykeJ. E.WitmerS. (2016). Assessment in special and inclusive education (13th ed.). Cengage Learning.
46.
ScaramellaL. V.LeveL. D. (2004). Clarifying parent-child reciprocities during early childhood: The early childhood coercion model. Clinical Child and Family Psychology Review, 7(2), 89–107. https://doi.org/10.1023/B:CCFP.0000030287.13160.a3
47.
ShookA. C. (2012). A study of preservice educators’ dispositions to change behavior management strategies. Preventing School Failure: Alternative Education for Children and Youth, 56(2), 129–136. doi.org/10.1080/1045988X.2011.606440
48.
ShoresR. E.GunterP. L.JackS. L. (1993). Classroom management strategies: Are they setting events for coercion?Behavioral Disorders, 18, 92–102. https://doi.org/10.1177/019874299301800207
49.
SimonsenB.FairbanksS.BrieschA.MyersD.SugaiG. (2008). Evidence-based practices in classroom management: Considerations for research to practice. Education and Treatment of Children, 31(3), 351–380. https://doi.org/10.1353/etc.0.0007
50.
SmithP. L. (1981). Gaining accuracy in generalizability theory: Using multiple designs. Journal of Educational Measurement, 18, 147–154.
51.
StichterJ. P.LewisT. J.WhittakerT. A.RichterM.JohnsonN. W.TrussellR. P. (2009). Assessing teacher use of opportunities to respond and effective classroom management strategies: Comparisons among high- and low-risk elementary schools. Journal of Positive Behavior Interventions, 11(2), 68–81. https://doi.org/10.1177/1098300708326597
52.
StormontM. A.SmithS.LewisT. J. (2007). Teacher implementation of prescorrection and praise statements in Head Start classrooms as a component of a program-wide system of positive behavior support. Journal of Behavioral Education, 16, 280–290.
53.
SuenH. K. (1990). Principles of test theories. Lawrence Erlbaum.
54.
TrussellR. P. (2008). Classroom universals to prevent problem behaviors. Intervention in School and Clinic, 43, 179–185. doi.org/10.1177/1053451207311678
WalkerH. M.RamseyE.GreshamF. M. (2004). Antisocial behavior in school: Evidence-based practices (2nd ed.). Wadsworth.
57.
WebbN. M.RowleyG. L.ShavelsonR. J. (1988). Using generalizability theory in counseling and development. Measurement and Evaluation in Counseling and Development, 21(2), 81–90. https://doi.org/10.1080/07481756.1988.12022886
58.
WilsonM. S.ReschlyD. J. (1996). Assessment in school psychology training and practice. School Psychology Review, 25, 9–23.
59.
YoderP.SymonsF. (2010). Observational measurement of behavior. Springer.
