We used multitrait-multimethod (MTMM) modeling to examine general factors of psychopathology in three samples of youths (Ns = 2,119, 303, and 592) for whom three informants reported on the youth’s psychopathology (e.g., child, parent, teacher). Empirical support for the p-factor diminished in multi-informant models compared with mono-informant models: The correlation between externalizing and internalizing factors decreased, and the general factor in bifactor models essentially reflected externalizing. Widely used MTMM-informed approaches for modeling multi-informant data cannot distinguish between competing interpretations of the patterns of effects we observed, including that the p factor reflects, in part, evaluative consistency bias or that psychopathology manifests differently across contexts (e.g., home vs. school). Ultimately, support for the p factor may be stronger in mono-informant designs, although it does not entirely vanish in multi-informant models. Instead, the general factor of psychopathology in any given mono-informant model likely reflects a complex mix of variances, some substantive and some methodological.
AchenbachT. M. (2009). The Achenbach System of Empirically Based Assessment (ASEBA): Development, findings, theory and applications. Research Center for Children, Youth, & Families, University of Vermont.
AchenbachT. M.IvanovaM. Y.RescorlaL. A. (2017). Empirically based assessment and taxonomy of psychopathology for ages 1½–90+ years: Developmental, multi-informant, and multicultural findings. Comprehensive Psychiatry, 79, 4–18. https://doi.org/10.1016/j.comppsych.2017.03.006
4.
AchenbachT. M.KrukowskiR. A.DumenciL.IvanovaM. Y. (2005). Assessment of adult psychopathology: Meta-analyses and implications of cross-informant correlations. Psychological Bulletin, 131(3), 361–382. https://doi.org/10.1037/0033-2909.131.3.361
5.
AchenbachT. M.McConaughyS. H.HowellC. T. (1987). Child/adolescent behavioral and emotional problems: Implications of cross-informant correlations for situational specificity. Psychological Bulletin, 101(2), 213–232. https://doi.org/10.1037/0033-2909.101.2.213
6.
AlexanderL. M.EscaleraJ.AiL.AndreottiC.FebreK.MangoneA.Vega-PotlerN.LangerN.AlexanderA.KovacsM.LitkeS.O’HaganB.AndersenJ.BronsteinB.BuiA.BusheyM.ButlerH.. . . MilhamM. P. (2017). An open resource for transdiagnostic research in pediatric mental health and learning disorders. Scientific Data, 4(1), Article 170181. https://doi.org/10.1038/sdata.2017.181
7.
AllegriniA. G.CheesmanR.RimfeldK.SelzamS.PingaultJ.-B.EleyT. C.PlominR. (2020). The p factor: Genetic analyses support a general dimension of psychopathology in childhood and adolescence. Journal of Child Psychology and Psychiatry, 61(1), 30–39. https://doi.org/10.1111/jcpp.13113
8.
AnusicI.SchimmackU.PinkusR. T.LockwoodP. (2009). The nature and structure of correlations among Big Five ratings: The halo-alpha-beta model. Journal of Personality and Social Psychology, 97(6), 1142–1156. https://doi.org/10.1037/a0017159
9.
BäckströmM.BjörklundF. (2021). Is the general factor of personality really related to frequency of agreeable, conscientious, emotionally stable, extraverted, and open behavior? An experience sampling study. Journal of Individual Differences, 42(3). https://doi.org/10.1027/1614-0001/a000341
10.
BarkleyR. A. (2003). Issues in the diagnosis of attention-deficit/hyperactivity disorder in children. Brain and Development, 25(2), 77–83. https://doi.org/10.1016/S0387-7604(02)00152-3
11.
BauerD. J.HowardA. L.BaldasaroR. E.CurranP. J.HussongA. M.ChassinL.ZuckerR. A. (2013). A trifactor model for integrating ratings across multiple informants. Psychological Methods, 18(4), 475–493. https://doi.org/10.1037/a0032475
12.
Becker-HaimesE. M.Jensen-DossA.BirmaherB.KendallP. C.GinsburgG. S. (2018). Parent–youth informant disagreement: Implications for youth anxiety treatment. Clinical Child Psychology and Psychiatry, 23(1), 42–56. https://doi.org/10.1177/1359104516689586
13.
BiesanzJ. C.WestS. G. (2004). Towards understanding assessments of the Big Five: Multitrait-multimethod analyses of convergent and discriminant validity across measurement occasion and type of observer. Journal of Personality, 72(4), 845–876. https://doi.org/10.1111/j.0022-3506.2004.00282.x
14.
BonifayW.LaneS. P.ReiseS. P. (2017). Three concerns with applying a bifactor model as a structure of psychopathology. Clinical Psychological Science, 5(1), 184–186. https://doi.org/10.1177/2167702616657069
15.
BrandesC. M.HerzhoffK.SmackA. J.TackettJ. L. (2019). The p Factor and the n Factor: Associations between the general factors of psychopathology and neuroticism in children. Clinical Psychological Science, 7(6), 1266–1284. https://doi.org/10.1177/2167702619859332
16.
CampbellD. T.FiskeD. W. (1959). Convergent and discriminant validation by the multitrait-multimethod matrix. Psychological Bulletin, 56(2), 81–105. https://doi.org/10.1037/h0046016
17.
CannonC. J.MakolB. A.KeeleyL. M.QasmiehN.OkunoH.RaczS. J.De Los ReyesA. (2020). A paradigm for understanding adolescent social anxiety with unfamiliar peers: Conceptual foundations and directions for future research. Clinical Child and Family Psychology Review, 23(3), 338–364. https://doi.org/10.1007/s10567-020-00314-4
18.
CaspiA.HoutsR. M.BelskyD. W.Goldman-MellorS. J.HarringtonH.IsraelS.MeierM. H.RamrakhaS.ShalevI.PoultonR.MoffittT. E. (2014). The p Factor: One general psychopathology factor in the structure of psychiatric disorders?Clinical Psychological Science, 2(2), 119–137. https://doi.org/10.1177/2167702613497473
19.
CaspiA.MoffittT. E. (2018). All for one and one for all: Mental disorders in one dimension. American Journal of Psychiatry, 175(9), 831–844. https://doi.org/10.1176/appi.ajp.2018.17121383
20.
Castellanos-RyanN.BrièreF. N.O’Leary-BarrettM.BanaschewskiT.BokdeA.BrombergU.BüchelC.FlorH.FrouinV.GallinatJ.GaravanH.MartinotJ. L.NeesF.PausT.PausovaZ.RietschelM.SmolkaM. N.RobbinsT. W.WhelanR.. . . IMAGEN Consortium. (2016). The structure of psychopathology in adolescence and its common personality and cognitive correlates. Journal of Abnormal Psychology, 125(8), 1039–1052. https://doi.org/10.1037/abn0000193
21.
ChangL.ConnellyB. S.GeezaA. A. (2012). Separating method factors and higher order traits of the Big Five: A meta-analytic multitrait–multimethod approach. Journal of Personality and Social Psychology, 102(2), 408–426. https://doi.org/10.1037/a0025559
22.
ClarkD. A.HicksB. M.AngstadtM.RutherfordS.TaxaliA.HydeL.WeigardA. S.HeitzegM. M.SripadaC. (2021). The general factor of psychopathology in the Adolescent Brain Cognitive Development (ABCD) study: A comparison of alternative modeling approaches. Clinical Psychological Science, 9(2), 169–182. https://doi.org/10.1177/2167702620959317
23.
ConnellyB. S.OnesD. S. (2010). An other perspective on personality: Meta-analytic integration of observers’ accuracy and predictive validity. Psychological Bulletin, 136(6), 1092–1122. https://doi.org/10.1037/a0021212
24.
De Los ReyesA. (2011). Introduction to the Special Section: More than measurement error: Discovering meaning behind informant discrepancies in clinical assessments of children and adolescents. Journal of Clinical Child & Adolescent Psychology, 40(1), 1–9. https://doi.org/10.1080/15374416.2011.533405
25.
De Los ReyesA.AugensteinT. M.WangM.ThomasS. A.DrabickD. A. G.BurgersD. E.RabinowitzJ. (2015). The validity of the multi-informant approach to assessing child and adolescent mental health. Psychological Bulletin, 141(4), 858–900. https://doi.org/10.1037/a0038498
26.
De Los ReyesA.HenryD. B.TolanP. H.WakschlagL. S. (2009). Linking informant discrepancies to observed variations in young children’s disruptive behavior. Journal of Abnormal Child Psychology, 37(5), 637–652. https://doi.org/10.1007/s10802-009-9307-3
27.
De Los ReyesA.KazdinA. E. (2005). Informant discrepancies in the assessment of childhood psychopathology: A critical review, theoretical framework, and recommendations for further study. Psychological Bulletin, 131(4), 483–509.
28.
De Los ReyesA.LernerM. D.KeeleyL. M.WeberR. J.DrabickD. A. G.RabinowitzJ.GoodmanK. L. (2019). Improving interpretability of subjective assessments about psychological phenomena: A review and cross-cultural meta-analysis. Review of General Psychology, 23(3), 293–319. https://doi.org/10.1177/1089268019837645
29.
De Los ReyesA.MakolB. A. (2022). Informant reports in clinical assessment. In SellbomM.BagbyR. M. (Eds.), Comprehensive clinical psychology. Volume 4: Assessment (pp. 105–122). Elsevier. https://doi.org/10.1016/B978-0-12-818697-8.00113-8
30.
De Los ReyesA.ThomasS. A.GoodmanK. L.KundeyS. M. A. (2013). Principles underlying the use of multiple informants’ reports. Annual Review of Clinical Psychology, 9(1), 123–149. https://doi.org/10.1146/annurev-clinpsy-050212-185617
31.
DeYoungC. G. (2006). Higher-order factors of the Big Five in a multi-informant sample. Journal of Personality and Social Psychology, 91(6), 1138–1151. https://doi.org/10.1037/0022-3514.91.6.1138
EidM.LischetzkeT.NussbeckF. W.TrierweilerL. I. (2003). Separating trait effects from trait-specific method effects in multitrait-multimethod models: A multiple-indicator CT-C(M-1) model. Psychological Methods, 8(1), 38–60. https://doi.org/10.1037/1082-989X.8.1.38
34.
EidM.NussbeckF. W.GeiserC.ColeD. A.GollwitzerM.LischetzkeT. (2008). Structural equation modeling of multitrait-multimethod data: Different models for different types of methods. Psychological Methods, 13(3), 230–253. https://doi.org/10.1037/a0013219
ForbesM. K.GreeneA. L.Levin-AspensonH. F.WattsA. L.HallquistM.LaheyB. B.MarkonK. E.PatrickC. J.TackettJ. L.WaldmanI. D.WrightA.CaspiA.IvanovaM.KotovR.SamuelD. B.EatonN. R.KruegerR. F. (2021). Three recommendations based on a comparison of the reliability and validity of the predominant models used in research on the empirical structure of psychopathology. Journal of Abnormal Psychology, 130(3), 297–317. https://doi.org/10.1037/abn0000533
37.
ForgasJ. P.LahamS. M. (2017). Halo effects. In PohlR. F. (Ed.), Cognitive illusions: Intriguing phenomena in thinking, judgment and memory (2nd ed., pp. 276–290). Routledge/Taylor & Francis Group.
38.
GarnerW. R.HakeH. W.EriksenC. W. (1956). Operationism and the concept of perception. Psychological Review, 63(3), 149–159. https://doi.org/10.1037/h0042992
39.
GeiserC.LockhartG. (2012). A comparison of four approaches to account for method effects in latent state-trait analyses. Psychological Methods, 17(2), 255–283. https://doi.org/10.1037/a0026977
40.
GlennL. E.KeeleyL. M.SzollosS.OkunoH.WangX.RauschE.DerosD. E.KarpJ. N.QasmiehN.MakolB. A.AugensteinT. M.LiptonM. F.RaczS. J.ScharfsteinL.BeidelD. C.De Los ReyesA. (2019). Trained observers’ ratings of adolescents’ social anxiety and social skills within controlled, cross-contextual social interactions with unfamiliar peer confederates. Journal of Psychopathology and Behavioral Assessment, 41(1), 1–15. https://doi.org/10.1007/s10862-018-9676-4
41.
GreeneA. L.EatonN. R. (2017). The temporal stability of the bifactor model of comorbidity: An examination of moderated continuity pathways. Comprehensive Psychiatry, 72, 74–82. https://doi.org/10.1016/j.comppsych.2016.09.010
42.
HamlatE. J.SnyderH. R.YoungJ. F.HankinB. L. (2019). Pubertal timing as a transdiagnostic risk for psychopathology in youth. Clinical Psychological Science, 7(3), 411–429. https://doi.org/10.1177/2167702618810518
43.
HopwoodC. J.WrightA. G. C.DonnellanB. M. (2011). Evaluating the evidence for the general factor of personality across multiple inventories. Journal of Research in Personality, 45(5), 468–478. https://doi.org/10.1016/j.jrp.2011.06.002
44.
JacksonJ. J.ConnollyJ. J.GarrisonS. M.LeveilleM. M.ConnollyS. L. (2015). Your friends know how long you will live: A 75-year study of peer-rated personality traits. Psychological Science, 26(3), 335–340. https://doi.org/10.1177/0956797614561800
45.
JennessJ. L.PeverillM.MillerA. B.HeleniakC.RobertsonM. M.SambrookK. A.SheridanM. A.McLaughlinK. A. (2021). Alterations in neural circuits underlying emotion regulation following child maltreatment: A mechanism underlying trauma-related psychopathology. Psychological Medicine, 51(11), 1880–1889. https://doi.org/10.1017/S0033291720000641
46.
JerniganT. L.BrownS. A.DowlingG. J. (2018). The adolescent brain cognitive development study. Journal of Research on Adolescence, 28(1), 154–156. https://doi.org/10.1111/jora.12374
47.
KlonskyE. D.OltmannsT. F.TurkheimerE. (2002). Informant-reports of personality disorder: Relation to self-reports and future research directions. Clinical Psychology: Science and Practice, 9(3), 300–311. https://doi.org/10.1093/clipsy.9.3.300
48.
KraemerH. C.MeaselleJ. R.AblowJ. C.EssexM. J.BoyceW. T.KupferD. J. (2003). A new approach to integrating data from multiple informants in psychiatric assessment and research: Mixing and matching contexts and perspectives. American Journal of Psychiatry, 160(9), 1566–1577. https://doi.org/10.1176/appi.ajp.160.9.1566
49.
KruegerR. F.TackettJ. L. (2003). Personality and psychopathology: Working toward the bigger picture. Journal of Personality Disorders, 17(2), 109–128. https://doi.org/10.1521/pedi.17.2.109.23986
50.
LaceulleO. M.VolleberghW. A. M.OrmelJ. (2015). The structure of psychopathology in adolescence: Replication of a general psychopathology factor in the TRAILS study. Clinical Psychological Science, 3(6), 850–860. https://doi.org/10.1177/2167702614560750
51.
LaheyB. B.ApplegateB.HakesJ. K.ZaldD. H.HaririA. R.RathouzP. J. (2012). Is there a general factor of prevalent psychopathology during adulthood?Journal of Abnormal Psychology, 121(4), 971–977. https://doi.org/10.1037/a0028355
52.
LaheyB. B.MooreT. M.KaczkurkinA. N.ZaldD. H. (2021). Hierarchical models of psychopathology: Empirical support, implications, and remaining issues. World Psychiatry, 20(1), 57–63. https://doi.org/10.1002/wps.20824
53.
LaheyB. B.ZaldD. H.PerkinsS. F.Villalta-GilV.WertsK. B.Van HulleC. A.RathouzP. J.ApplegateB.ClassQ. A.PooreH. E.WattsA. L.WaldmanI. D. (2018). Measuring the hierarchical general factor model of psychopathology in young adults. International Journal of Methods in Psychiatric Research, 27(1), Article e1593. https://doi.org/10.1002/mpr.1593
54.
LanceC. E.NobleC. L.ScullenS. E. (2002). A critique of the correlated trait-correlated method and correlated uniqueness models for multitrait-multimethod data. Psychological Methods, 7(2), 228–244. https://doi.org/10.1037/1082-989x.7.2.228
55.
LeisingD.BurgerJ.ZimmermannJ.BäckströmM.OltmannsJ. R.ConnellyB. S. (2020). Why do items correlate with one another? A conceptual analysis with relevance for general factors and network models. PsyArXiv. https://doi.org/10.31234/osf.io/7c895
56.
LenthR. V. (2021). emmeans: Estimated marginal means, aka least-squares means (Version 1.5.5-1) [Computer software]. Comprehensive R Archive Network. https://CRAN.R-project.org/package=emmeans.
57.
LippoldM. A.GreenbergM. T.CollinsL. M. (2013). Parental knowledge and youth risky behavior: A person oriented approach. Journal of Youth and Adolescence, 42(11), 1732–1744. https://doi.org/10.1007/s10964-012-9893-1
58.
LittlefieldA. K.LaneS. P.GetteJ. A.WattsA. L.SherK. J. (2021). The “Big Everything”: Integrating and investigating dimensional models of psychopathology, personality, personality pathology, and cognitive functioning. Personality Disorders: Theory, Research, and Treatment, 12(2), 103–114. https://doi.org/10.1037/per0000457
59.
MakolB. A.De Los ReyesA.GarridoE.HarlaarN.TaussigH. (2021). Assessing the mental health of maltreated youth with child welfare involvement using multi-informant reports. Child Psychiatry & Human Development, 52(1), 49–62. https://doi.org/10.1007/s10578-020-00985-8
60.
MakolB. A.YoungstromE. A.RaczS. J.QasmiehN.GlennL. E.De Los ReyesA. (2020). Integrating multiple informants’ reports: How conceptual and measurement models may address long-standing problems in clinical decision-making. Clinical Psychological Science, 8(6), 953–970. https://doi.org/10.1177/2167702620924439
61.
MannF. D.AthertonO. E.DeYoungC. G.KruegerR. F.RobinsR. W. (2020). Big five personality traits and common mental disorders within a hierarchical taxonomy of psychopathology: A longitudinal study of Mexican-origin youth. Journal of Abnormal Psychology, 129(8), 769–787. https://doi.org/10.1037/abn0000633
62.
MeehlP. E. (1945). The dynamics of “structured” personality tests. Journal of Clinical Psychology, 1, 296–303.
63.
MillsapR. E. (2012). Statistical approaches to measurement invariance. Routledge.
64.
MurrayA. L.EisnerM.RibeaudD. (2016). The development of the general factor of psychopathology ‘p Factor’ through childhood and adolescence. Journal of Abnormal Child Psychology, 44(8), 1573–1586. https://doi.org/10.1007/s10802-016-0132-1
65.
MusekJ. (2007). A general factor of personality: Evidence for the Big One in the five-factor model. Journal of Research in Personality, 41(6), 1213–1233. https://doi.org/10.1016/j.jrp.2007.02.003
NeumannA.PappaI.LaheyB. B.VerhulstF. C.Medina-GomezC.JaddoeV. W.Bakermans-KranenburgM. J.MoffittT. E.van IJzendoornM. H.TiemeierH. (2016). Single nucleotide polymorphism heritability of a general psychopathology factor in children. Journal of the American Academy of Child & Adolescent Psychiatry, 55(12), 1038–1045e4. https://doi.org/10.1016/j.jaac.2016.09.498
68.
NICHD Early Child Care Research Network. (Ed.). (2005). Child care and child development: Results from the NICHD study of early child care and youth development. Guilford Press.
69.
NoordhofA.KruegerR. F.OrmelJ.OldehinkelA. J.HartmanC. A. (2015). Integrating autism-related symptoms into the dimensional internalizing and externalizing model of psychopathology. The TRAILS study. Journal of Abnormal Child Psychology, 43(3), 577–587. https://doi.org/10.1007/s10802-014-9923-4
70.
OlinoT. M.BufferdS. J.DoughertyL. R.DysonM. W.CarlsonG. A.KleinD. N. (2018). The development of latent dimensions of psychopathology across early childhood: Stability of dimensions and moderators of change. Journal of Abnormal Child Psychology, 46(7), 1373–1383. https://doi.org/10.1007/s10802-018-0398-6
71.
OltmannsJ. R.SmithG. T.OltmannsT. F.WidigerT. A. (2018). General factors of psychopathology, personality, and personality disorder: Across domain comparisons. Clinical Psychological Science, 6(4), 581–589. https://doi.org/10.1177/2167702617750150
72.
R Core Team. (2020). R: A language and environment for statistical computing (Version 4.0.2) [Computer software]. R Foundation for Statistical Computing. http://www.R-project.org
73.
RevelleW.WiltJ. (2013). The general factor of personality: A general critique. Journal of Research in Personality, 47(5), 493–504. https://doi.org/10.1016/j.jrp.2013.04.012
74.
RiemM. M. E.van HoofM. J.GarrettA. S.RomboutsS. A. R. B.van der WeeN. J. A.van IJzendoornM. H.VermeirenR. R. J. M. (2019). General psychopathology factor and unresolved-disorganized attachment uniquely correlated to white matter integrity using diffusion tensor imaging. Behavioural Brain Research, 359, 1–8. https://doi.org/10.1016/j.bbr.2018.10.014
75.
RushtonP.IrwingP. (2011). The general factor of personality: Normal and abnormal. In Chamorro-PremuzicT.von StummS.FurnhamA. (Eds.), The Wiley-Blackwell handbook of individual differences (pp. 132–161). Wiley Blackwell.
76.
SallisH.SzekelyE.NeumannA.Jolicoeur-MartineauA.IJzendoornM.van HillegersM.GreenwoodC. M. T.MeaneyM. J.SteinerM.TiemeierH.WazanaA.PearsonR. M.EvansJ. (2019). General psychopathology, internalising and externalising in children and functional outcomes in late adolescence. Journal of Child Psychology and Psychiatry, 60(11), 1183–1190. https://doi.org/10.1111/jcpp.13067
SnyderH. R.YoungJ. F.HankinB. L. (2017). Strong homotypic continuity in common psychopathology-, internalizing-, and externalizing-specific factors over time in adolescents. Clinical Psychological Science, 5(1), 98–110. https://doi.org/10.1177/2167702616651076
79.
SuzukiT.SamuelD. B.PahlenS.KruegerR. F. (2015). DSM-5 alternative personality disorder model traits as maladaptive extreme variants of the five-factor model: An item-response theory analysis. Journal of Abnormal Psychology, 124(2), 343–354. https://doi.org/10.1037/abn0000035
80.
TackettJ. L.HerzhoffK.ReardonK. W.De ClercqB.SharpC. (2014). The externalizing spectrum in youth: Incorporating personality pathology. Journal of Adolescence, 37(5), 659–668. https://doi.org/10.1016/j.adolescence.2013.10.009
81.
van der LindenD.PekaarK. A.BakkerA. B.SchermerJ. A.VernonP. A.DunkelC. S.PetridesK. V. (2017). Overlap between the general factor of personality and emotional intelligence: A meta-analysis. Psychological Bulletin, 143(1), 36–52. https://doi.org/10.1037/bul0000078
82.
WattsA. L.LaneS. P.BonifayW.SteinleyD.MeyerF. A. C. (2020). Building theories on top of, and not independent of, statistical models: The case of the p-factor. Psychological Inquiry, 31(4), 310–320. https://doi.org/10.1080/1047840X.2020.1853476
83.
WattsA. L.PooreH. E.WaldmanI. D. (2019). Riskier tests of the validity of the bifactor model of psychopathology. Clinical Psychological Science, 7(6), 1285–1303. https://doi.org/10.1177/2167702619855035
84.
WeissmanD. G.BitranD.MillerA. B.SchaeferJ. D.SheridanM. A.McLaughlinK. A. (2019). Difficulties with emotion regulation as a transdiagnostic mechanism linking child maltreatment with the emergence of psychopathology. Development and Psychopathology, 31(3), 899–915. https://doi.org/10.1017/S0954579419000348
85.
WidigerT. A.SellbomM.ChmielewskiM.ClarkL. A.DeYoungC. G.KotovR.KruegerR. F.LynamD. R.MillerJ. D.Mullins-SweattS.SamuelD. B.SouthS. C.TackettJ. L.ThomasK. M.WatsonD.WrightA. G. C. (2019). Personality in a hierarchical model of psychopathology. Clinical Psychological Science, 7(1), 77–92. https://doi.org/10.1177/2167702618797105
86.
WoodS.ScheiplF. (2020). Generalized additive mixed models using “mgcv” and “lme4” (Version 0.2-6) [Computer software]. Comprehensive R Archive Network. https://cran.r-project.org/package=gamm4
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
