Disaggregating Between- and Within-Classroom Variation in Student Behavior: A Multilevel Factor Analysis of Teacher Ratings of Student Prosociality and Aggression
Restricted accessResearch articleFirst published online August, 2019
Disaggregating Between- and Within-Classroom Variation in Student Behavior: A Multilevel Factor Analysis of Teacher Ratings of Student Prosociality and Aggression
Teacher ratings of student behaviors vary systematically both at the student and teacher/classroom level. Multilevel confirmatory factor analysis (ML-CFA) can disaggregate between- and within-teacher/classroom variance, identify an optimal psychometric model at each level, and test correlates of the resulting dimensions. In this study, 250 teachers (37% males) rated an average of 4.02 students (51% males; aged 10 years at Time 1 and 11 years at Time 2) from a normative sample of Swiss youth. Substantial and unidimensional between-teacher variation in ratings of both prosociality and aggression were identified, and this was stable across time. These dimensions were not associated at the between-teacher/classroom level with teacher gender or teacher–student relationships, although they were associated with teacher-student relationships at the within-teacher/classroom level. There was little between-teacher/classroom variation observed in student self-reports of prosocial and aggressive behavior, and multilevel CFA was not possible for these ratings. Future research should aim to identify sources of between-teacher/classroom variation. This should include factors that influence negative and positive teacher perceptions of and response biases related to student behavior as well as those that influence student behavior itself.
AbikoffH.CourtneyM.PelhamW. E.Jr.KoplewiczH. S. (1993). Teachers’ ratings of disruptive behaviors: The influence of halo effects. Journal of Abnormal Child Psychology, 21, 519-533.
2.
AloeA. M.ShislerS. M.NorrisB. D.NickersonA. B.RinkerT. W. (2014). A multivariate meta-analysis of student misbehavior and teacher burnout. Educational Research Review, 12, 30-44.
3.
ArensA. K.MorinA. J. (2016). Relations between teachers’ emotional exhaustion and students’ educational outcomes. Journal of Educational Psychology, 108, 800-813.
4.
BarthJ. M.DunlapS. T.DaneH.LochmanJ. E.WellsK. C. (2004). Classroom environment influences on aggression, peer relations, and academic focus. Journal of School Psychology, 42, 115-133.
5.
Berg-NielsenT. S.VikaA.DahlA. A. (2003). When adolescents disagree with their mothers: CBCL-YSR discrepancies related to maternal depression and adolescent self-esteem. Child: Care, Health and Development, 29, 207-213.
6.
De Los ReyesA.HenryD. B.TolanP. H.WakschlagL. S. (2009). Linking informant discrepancies to observed variations in young children’s disruptive behavior. Journal of Abnormal Child Psychology, 37, 637-652.
7.
DobbsJ.ArnoldD. H. (2009). Relationship between preschool teachers’ reports of children’s behavior and their behavior towards those children. School Psychology Quarterly, 24, 95-105.
8.
DownerJ. T.StuhlmanM.SchweigJ.MartínezJ. F.RuzekE. (2015). Measuring effective teacher-student interactions from a student perspective: A multi-level analysis. The Journal of Early Adolescence, 35, 722-758.
9.
DunnE. C.MasynK. E.JonesS. M.SubramanianS. V.KoenenK. C. (2015). Measuring psychosocial environments using individual responses: An application of multilevel factor analysis to examining students in schools. Prevention Science, 16, 718-733.
10.
DyerN. G.HangesP. J.HallR. J. (2005). Applying multilevel confirmatory factor analysis techniques to the study of leadership. The Leadership Quarterly, 16, 149-167.
11.
EisenbergN.GuthrieI. K.CumberlandA.MurphyB. C.ShepardS. A.ZhouQ.CarloG. (2002). Prosocial development in early adulthood: A longitudinal study. Journal of Personality and Social Psychology, 82, 993-1006.
12.
EisnerM.RibeaudD. (2007). Conducting a criminological survey in a culturally diverse context lessons from the Zurich project on the social development of children. European Journal of Criminology, 4, 271-298.
13.
EisnerN.MurrayA. L.EisnerM.RibeaudD. (2018). An analysis of non-response and attrition in the Zurich project on social development from childhood to adulthood (z-proso). International Journal of Behavioral Development.
14.
FraserM. W.GalinskyM. J.SmokowskiP. R.DayS. H.TerzianM. A.RoseR. A.GuoS. (2005). Social information-processing skills training to promote social competence and prevent aggressive behavior in the third grades. Journal of Consulting and Clinical Psychology, 73, 1045-1055.
15.
GanzeboomH. B.De GraafP. M.TreimanD. J. (1992). A standard international socio-economic index of occupational status. Social Science Research, 21, 1-56.
16.
GeldhofG. J.PreacherK. J.ZyphurM. J. (2014). Reliability estimation in a multilevel confirmatory factor analysis framework. Psychological Methods, 19, 72-91.
17.
HartyS. C.MillerC. J.NewcornJ. H.HalperinJ. M. (2009). Adolescents with childhood ADHD and comorbid disruptive behavior disorders: Aggression, anger, and hostility. Child Psychiatry and Human Development, 40, 85-97.
18.
HeckR. H.ThomasS. L. (2015). An introduction to multilevel modeling techniques: MLM and SEM approaches using Mplus. Routledge.
19.
HopfD.HatzichristouC. (1999). Teacher gender-related influences in Greek schools. British Journal of Educational Psychology, 69, 1-18.
20.
HuL. T.BentlerP. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal, 6, 1-55.
21.
KellamS. G.RebokG. W. (1992). Building developmental and etiological theory through epidemiologically based preventive intervention trials. In McCordJ.TremblayR. E. (Eds.), Preventing antisocial behavior: Interventions from birth through adolescence (pp. 162-195). New York, NY: Guilford Press.
22.
KrkovicK.GreiffS.KupiainenS.VainikainenM. P.HautamäkiJ. (2014). Teacher evaluation of student ability: What roles do teacher gender, student gender, and their interaction play?Educational Research, 56, 244-257.
23.
Laninga-WijnenL.HarakehZ.DijkstraJ. K.VeenstraR.VolleberghW. (2018). Aggressive and prosocial peer norms: Change, stability, and associations with adolescent aggressive and prosocial behavior development. The Journal of Early Adolescence, 38, 178-203.
24.
LiuY.MillsapR. E.WestS. G.TeinJ. Y.TanakaR.GrimmK. J. (2017). Testing measurement invariance in longitudinal data with ordered-categorical measures. Psychological Methods, 22, 486-506.
25.
LöselF.StemmlerM. (2012). Preventing child behavior problems in the Erlangen-Nuremberg Development and Prevention Study: Results from preschool to secondary school age. International Journal of Conflict and Violence, 6, 214-224.
26.
MaltiT.RibeaudD.EisnerM. P. (2011). The effectiveness of two universal preventive interventions in reducing children’s externalizing behavior: A cluster randomized controlled trial. Journal of Clinical Child & Adolescent Psychology, 40, 677-692.
27.
MarshH. W.LüdtkeO.NagengastB.TrautweinU.MorinA. J.AbduljabbarA. S.KöllerO. (2012). Classroom climate and contextual effects: Conceptual and methodological issues in the evaluation of group-level effects. Educational Psychologist, 47, 106-124.
28.
MashburnA. J.HamreB. K.DownerJ. T.PiantaR. C. (2006). Teacher and classroom characteristics associated with teachers ratings of pre-kindergartners’ relationships and behaviors. Journal of Psychoeducational Assessment, 24, 367–380.
29.
MenesiniE.PalladinoB. E.NocentiniA. (2015). Emotions of moral disengagement, class norms, and bullying in adolescence: A multilevel approach. Merrill-Palmer Quarterly, 61, 124-143.
30.
MorinA. J.MarshH. W.NagengastB.ScalasL. F. (2014). Doubly latent multilevel analyses of classroom climate: An illustration. The Journal of Experimental Education, 82, 143-167.
31.
MurrayA. L.EisnerM.RibeaudD. (2017). Can the Social Behavior Questionnaire help meet the need for dimensional, transdiagnostic measures of childhood and adolescent psychopathology?European Journal of Psychological Assessment. In press.
32.
MurrayA. L.ObsuthI.EisnerM.RibeaudD. (2016). Shaping aggressive personality in adolescence: Exploring cross-lagged relations between aggressive thoughts, aggressive behaviour and self-control. Personality and Individual Differences, 97, 1-7.
33.
MurrayA. L.ObsuthI.EisnerM.RibeaudD. (2017). Evaluating longitudinal invariance in dimensions of mental health across adolescence: An Analysis of the Social Behavior Questionnaire. Assessment. Advance online publication. doi:10.1177/1073191117721741
34.
MurrayA. L.ObsuthI.Zirk-SadowskiJ.RibeaudD.EisnerM. (2016). Developmental relations between ADHD symptoms and reactive versus proactive aggression across childhood and adolescence. Journal of Attention Disorders. Advance online publication. doi:10.1177/1087054716666323
MuthénL. K.MuthénB. O. (2014). Mplus user’s guide (7th ed.). Los Angeles, CA: Author.
37.
O’BrennanL. M.BradshawC. P.FurlongM. J. (2014). Influence of classroom and school climate on teacher perceptions of student problem behavior. School Mental Health, 6, 125-136.
38.
ObsuthI.MurrayA. L.MaltiT.SulgerP.RibeaudD.EisnerM. (2017). Non-bipartite propensity score analysis of the effects of teacher-student relationships on adolescent problem and prosocial behavior. Journal of Youth and Adolescence, 46, 1661-1687.
39.
ObsuthI.SutherlandA.CopeA.PilbeamL.MurrayA. L.EisnerM. (2017). London Education and Inclusion Project (LEIP): Results from a cluster-randomized controlled trial of an intervention to reduce school exclusion and antisocial behavior. Journal of Youth and Adolescence, 46, 538-557.
40.
PakarinenE.SilinskasG.HamreB. K.MetsäpeltoR. L.LerkkanenM. K.PoikkeusA. M.NurmiJ. E. (2017). Cross-lagged associations between problem behaviors and teacher-student relationships in early adolescence. The Journal of Early Adolescence. Advance online publication. doi:10.1177/0272431617714328
41.
PasE. T.BradshawC. P. (2014). What affects teacher ratings of student behaviors? The potential influence of teachers’ perceptions of the school environment and experiences. Prevention Science, 15, 940-950.
42.
PodsakoffP. M.MacKenzieS. B.PodsakoffN. P. (2012). Sources of method bias in social science research and recommendations on how to control it. Annual Review of Psychology, 63, 539-569.
43.
R Core Team. (2016). R: A language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing. Available from from https://www.R-project.org/
44.
SalmivalliC.VoetenM.PoskipartaE. (2011). Bystanders matter: Associations between reinforcing, defending, and the frequency of bullying behavior in classrooms. Journal of Clinical Child & Adolescent Psychology, 40, 668-676.
45.
SpiltJ. L.KoomenH. M.JakS. (2012). Are boys better off with male and girls with female teachers? A multilevel investigation of measurement invariance and gender match in teacher–student relationship quality. Journal of School Psychology, 50, 363-378.
46.
TheimannM. (2016). School as a space of socialization and prevention. European Journal of Criminology, 13, 67-91.
47.
ThomasD. E.BiermanK. L. (2006). The impact of classroom aggression on the development of aggressive behavior problems in children. Development and Psychopathology, 18, 471-487.
48.
ThomasD. E.BiermanK. L.PowersC. J. (2011). The influence of classroom aggression and classroom climate on aggressive–disruptive behavior. Child Development, 82, 751-757.
49.
TremblayR. E.LoeberR.GagnonC.CharleboisP.LariveeS.LeBlancM. (1991). Disruptive boys with stable and unstable high fighting behavior patterns during junior elementary school. Journal of Abnormal Child Psychology, 19, 285-300.
50.
TremblayR. E.VitaroF.GagnonC.PichéC.RoyerN. (1992). A prosocial scale for the Preschool Behaviour Questionnaire: Concurrent and predictive correlates. International Journal of Behavioral Development, 15, 227-245.
51.
TuvbladC.RaineA.ZhengM.BakerL. A. (2009). Genetic and environmental stability differs in reactive and proactive aggression. Aggressive Behavior, 35, 437-452.
52.
VerhulstF. C.Van Der EndeJ. (1995). The eight-year stability of problem behavior in an epidemiologic sample. Pediatric Research, 38, 612-617.
53.
WetzelsP.EnzmannD.MecklenburgE.PfeifferC. (2001). Jugend und Gewalt: Eine repräsentative Dunkelfeldanalyse in München und acht anderen deutschen Städten: Interdisziplinäre Beiträge zur kriminologischen Forschung [Youth and Violence: a representative self-report study in Munich and eight other German cities, in: interdisciplinary contributions to criminological research.] (Vol. 17). Baden-Baden, Germany: Nomos.
54.
YoungstromE.LoeberR.Stouthamer-LoeberM. (2000). Patterns and correlates of agreement between parent, teacher, and male adolescent ratings of externalizing and internalizing problems. Journal of Consulting and Clinical Psychology, 68, 1038-1050.
55.
YuC. Y. (2002). Evaluating cutoff criteria of model fit indices for latent variable models with binary and continuous outcomes (Vol. 30). Los Angeles: University of California, Los Angeles.
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
