We advanced several “riskier tests” of the validity of bifactor models of psychopathology, which included that the general and specific psychopathology factors should be reliable and well represented by their respective indicators and that including a general factor should improve on the correlated factor model’s external validity. We compared bifactor and correlated factors models of psychopathology using data from a community sample of youth (N = 2,498) whose parents provided ratings on psychopathology and theoretically relevant external criteria (i.e., personality, aggression, antisociality). Bifactor models tended to yield either general or specific factors that were unstable and difficult to interpret. The general factor appeared to reflect a differentially weighted amalgam of psychopathology rather than a liability for psychopathology broadly construed. With rare exceptions, bifactor models did not explain additional variance in first-order psychopathology symptom dimensions or external criteria compared with correlated factors models. Together, our findings call into question the validity of bifactor models of psychopathology and the p factor more broadly.
AchenbachT. M. (1966). The classification of children’s psychiatric symptoms: A factor-analytic study. Psychological Monographs: General and Applied, 80, 1–37.
2.
AchenbachT. M.EdelbrockC. S. (1984). Psychopathology of childhood. Annual Review of Psychology, 35, 227–256.
3.
AchenbachT. M.KrukowskiR. A.DumenciL.IvanovaM. Y. (2005). Assessment of adult psychopathology: Meta-analyses and implications of cross-informant correlations. Psychological Bulletin, 131, 361–382.
4.
American Psychiatric Association. (1994). Diagnostic and statistical manual of mental disorders (4th ed.). Washington, DC: Author.
5.
AriasV. B.PonceF. P.NúñezD. E. (2018). Bifactor models of attention-deficit/hyperactivity disorder (ADHD): An evaluation of three necessary but underused psychometric indexes. Assessment, 25, 885–897.
6.
BagbyR. M.TaylorG. J.QuiltyL. C.ParkerJ. D. (2007). Reexamining the factor structure of the 20-item Toronto alexithymia scale: Commentary on Gignac, Palmer, and Stough. Journal of Personality Assessment, 89, 258–264.
7.
BeauchaineT. P.ThayerJ. F. (2015). Heart rate variability as a transdiagnostic biomarker of psychopathology. International Journal of Psychophysiology, 98, 338–350.
8.
BonifayW. (2015). An integrative framework of model evaluation (Unpublished doctoral dissertation). University of California, Los Angeles.
9.
BonifayW.CaiL. (2017). On the complexity of item response theory models. Multivariate Behavioral Research, 52, 465–484.
10.
BonifayW.LaneS. P.ReiseS. P. (2017). Three concerns with applying a bifactor model as a structure of psychopathology. Clinical Psychological Science, 5, 184–186.
11.
BrunnerM.NagyG.WilhelmO. (2012). A tutorial on hierarchically structured constructs. Journal of Personality, 80, 796–846.
12.
BussA. H.PlominR. (1984). Theory and measurement of EAS. In BussA. H.PlominR. (Eds.), Temperament: Early developing personality traits (pp. 84–104). Hillsdale, NJ: Erlbaum.
13.
CarragherN.TeessonM.SunderlandM.NewtonN. C.KruegerR. F.ConrodP. J.. . . SladeT. (2016). The structure of adolescent psychopathology: A symptom-level analysis. Psychological Medicine, 46, 981–994.
14.
CarverC. S.JohnsonS. L.TimpanoK. R. (2017). Toward a functional view of the p factor in psychopathology. Clinical Psychological Science, 5, 880–889.
15.
CaspiA.HoutsR. M.BelskyD. W.Goldman-MellorS. J.HarringtonH.IsraelS.. . . MoffittT. E. (2014). The p factor: One general psychopathology factor in the structure of psychiatric disorders?Clinical Psychological Science, 2, 119–137.
16.
CaspiA.MoffittT. E. (2018). All for one and one for all: Mental disorders in one dimension. American Journal of Psychiatry, 175, 831–844. doi:10.1176/appi.ajp.2018.17121383
17.
Castellanos-RyanN.BrièreF. N.O’Leary-BarrettM.BanaschewskiT.BokdeA.BrombergU.. . . GaravanH. (2016). The structure of psychopathology in adolescence and its common personality and cognitive correlates. Journal of Abnormal Psychology, 125, 1039–1052.
18.
ClarkL. A.WatsonD. (2006). Distress and fear disorders: An alternative empirically based taxonomy of the “mood” and “anxiety” disorders. The British Journal of Psychiatry, 189, 481–483.
19.
ConwayC. C.ForbesM. K.ForbushK. T.FriedE. I.HallquistM. N.KotovR.. . . SunderlandM. (2019). A hierarchical taxonomy of psychopathology can transform mental health research. Perspectives on Psychological Science, 14, 419–436. doi:10.1177/1745691618810696
20.
CuthbertB. N. (2005). Dimensional models of psychopathology: Research agenda and clinical utility. Journal of Abnormal Psychology, 114, 565–569.
21.
DodgeK. A.CoieJ. D. (1987). Social-information-processing factors in reactive and proactive aggression in children’s peer groups. Journal of Personality and Social Psychology, 53, 1146–1158.
22.
EndersC. K.BandalosD. L. (2001). The relative performance of full information maximum likelihood estimation for missing data in structural equation models. Structural Equation Modeling, 8, 430–457.
23.
ForbesM. K.KotovR.RuggeroC. J.WatsonD.ZimmermanM.KruegerR. F. (2017). Delineating the joint hierarchical structure of clinical and personality disorders in an outpatient psychiatric sample. Comprehensive Psychiatry, 79, 19–30.
24.
FrickP. J.HareR. D. (2001). The Antisocial Process Screening Device (ASPD). Toronto, Ontario, Canada: Multi Health Systems.
25.
GignacG. E. (2016). The higher-order model imposes a proportionality constraint: That is why the bifactor model tends to fit better. Intelligence, 55, 57–68.
26.
GrahamJ. M. (2006). Congeneric and (essentially) tau-equivalent estimates of score reliability: What they are and how to use them. Educational and Psychological Measurement, 66, 930–944.
27.
GreeneA. L.EatonN. R.LiK.ForbesM. K.MarkonK.WaldmanI.. . . KotovR. (in press). Are fit indices used to test psychopathology structure biased? A simulation study. Journal of Abnormal Psychology.
28.
GrevenK. M. (2005). The problem of “lumping versus splitting” [Editorial]. Gynecologic Oncology, 99, 527–529.
29.
HancockG. R.MuellerR. O. (2001). Rethinking construct reliability within latent variable systems. In CudeckR.du ToitS.SörbomD. (Eds.), Structural equation modeling: Present and future—A Festschrift in honor of Karl Jöreskog (pp. 195–216). Lincolnwood, IL: Scientific Software International.
30.
HolzingerK. J.SwinefordF. (1937). The bi-factor method. Psychometrika, 2, 41–54.
31.
KimH.EatonN. R. (2015). The hierarchical structure of common mental disorders: Connecting multiple levels of comorbidity, bifactor models, and predictive validity. Journal of Abnormal Psychology, 124, 1064–1078.
32.
KotovR.ChangS. W.FochtmannL. J.MojtabaiR.CarlsonG. A.SedlerM. J.BrometE. J. (2010). Schizophrenia in the internalizing-externalizing framework: A third dimension?Schizophrenia Bulletin, 37, 1168–1178.
33.
KotovR.KruegerR. F.WatsonD.AchenbachT. M.AlthoffR. R.BagbyR. M.. . . EatonN. R. (2017). The Hierarchical Taxonomy of Psychopathology (HiTOP): A dimensional alternative to traditional nosologies. Journal of Abnormal Psychology, 126, 454–477.
34.
KotovR.RuggeroC. J.KruegerR. F.WatsonD.YuanQ.ZimmermanM. (2011). New dimensions in the quantitative classification of mental illness. Archives of General Psychiatry, 68, 1003–1011.
35.
KruegerR. F. (1999). The structure of common mental disorders. Archives of General Psychiatry, 56, 921–926.
36.
KruegerR. F.KotovR.WatsonD.ForbesM. K.EatonN. R.RuggeroC. J.. . . BagbyR. M. (2018). Progress in achieving quantitative classification of psychopathology. World Psychiatry, 17, 282–293.
37.
KruegerR. F.MarkonK. E. (2006). Reinterpreting comorbidity: A model-based approach to understanding and classifying psychopathology. Annual Review in Clinical Psychology, 2, 111–133.
38.
LaceulleO. M.VolleberghW. A.OrmelJ. (2015). The structure of psychopathology in adolescence: Replication of a general psychopathology factor in the TRAILS study. Clinical Psychological Science, 3, 850–860.
39.
LaheyB. B.ApplegateB.HakesJ. K.ZaldD. H.HaririA. R.RathouzP. J. (2012). Is there a general factor of prevalent psychopathology during adulthood?Journal of Abnormal Psychology, 121, 971–977.
40.
LaheyB. B.RathouzP. J.KeenanK.SteppS. D.LoeberR.HipwellA. E. (2015). Criterion validity of the general factor of psychopathology in a prospective study of girls. Journal of Child Psychology and Psychiatry, 56, 415–422.
41.
Levin-AspensonH. F.WatsonD.ClarkL. A.ZimmermanM. (2018, September). What is the general factor of psychopathology? Consistency of the p factor across samples. Poster presented at the annual meeting of the Society for Research in Psychopathology, Indianapolis, IN.
42.
LoehlinJ. C. (2004). Latent variable models: An introduction to factor, path, and structural equation analysis. New York, NY: Psychology Press.
43.
MansolfM.ReiseS. P. (2017). When and why the second-order and bifactor models are distinguishable. Intelligence, 61, 120–129.
44.
MarkonK. E. (2019). Bifactor and hierarchical models: Specification, inference, and interpretation. Annual Review of Clinical Psychology, 15, 51–69. doi:10.1146/annurev-clinpsy-050718-095522
45.
MarkonK. E.KruegerR. F. (2006). Information-theoretic latent distribution modeling: Distinguishing discrete and continuous latent variable models. Psychological Methods, 11, 228–243.
46.
MartelM. M.MarkonK.SmithG. T. (2017). Research review: Multi-informant integration in child and adolescent psychopathology diagnosis. Journal of Child Psychology and Psychiatry, 58, 116–128.
47.
MartelM. M.PanP. M.HoffmannM. S.GadelhaA.do RosárioM. C.MariJ. J.. . . RohdeL. A. (2017). A general psychopathology factor (P factor) in children: Structural model analysis and external validation through familial risk and child global executive function. Journal of Abnormal Psychology, 126, 137–148.
48.
Maydeu-OlivaresA.CoffmanD. L. (2006). Random intercept item factor analysis. Psychological Methods, 11, 344–362.
49.
McDonaldR. P. (1999). Test theory: A unified approach. Mahwah, NJ: Erlbaum.
50.
MeehlP. E. (1978). Theoretical risks and tabular asterisks: Sir Karl, Sir Ronald, and the slow progress of soft psychology. Journal of Consulting and Clinical Psychology, 46, 806–834.
51.
MeierM. A.MeierM. H. (2018). Clinical implications of a general psychopathology factor: A cognitive-behavioral transdiagnostic group treatment for community mental health. Journal of Psychotherapy Integration, 28, 253–268. doi:10.1037/int0000095
52.
MorganG. B.HodgeK. J.WellsK. E.WatkinsM. W. (2015). Are fit indices biased in favor of bi-factor models in cognitive ability research?: A comparison of fit in correlated factors, higher-order, and bi-factor models via Monte Carlo simulations. Journal of Intelligence, 3, 2–20.
53.
MurrayA. L.EisnerM.RibeaudD. (2016). The development of the general factor of psychopathology ‘p factor’ through childhood and adolescence. Journal of Abnormal Child Psychology, 44, 1573–1586.
54.
MurrayA. L.JohnsonW. (2013). The limitations of model fit in comparing the bi-factor versus higher-order models of human cognitive ability structure. Intelligence, 41, 407–422.
55.
MuthénB.MuthénL. (2014). Mplus Version 7.2. Los Angeles, CA: Author.
56.
NeumannA.PappaI.LaheyB. B.VerhulstF. C.Medina-GomezC.JaddoeV. W.. . . TiemeierH. (2016). Single nucleotide polymorphism heritability of a general psychopathology factor in children. Journal of the American Academy of Child & Adolescent Psychiatry, 55, 1038–1045.
57.
NiggJ. T. (2006). Temperament and developmental psychopathology. Journal of Child Psychology and Psychiatry, 47, 395–422.
58.
NoordhofA.KruegerR. F.OrmelJ.OldehinkelA. J.HartmanC. A. (2015). Integrating autism-related symptoms into the dimensional internalizing and externalizing model of psychopathology. The TRAILS Study. Journal of Abnormal Child Psychology, 43, 577–587.
59.
OlatunjiB. O.EbesutaniC.AbramowitzJ. S. (2017). Examination of a bifactor model of obsessive-compulsive symptom dimensions. Assessment, 24, 45–59.
60.
OlatunjiB. O.EbesutaniC.ReiseS. P. (2015). A bifactor model of disgust proneness: Examination of the Disgust Emotion Scale. Assessment, 22, 248–262.
61.
OlinoT. M.DoughertyL. R.BufferdS. J.CarlsonG. A.KleinD. N. (2014). Testing models of psychopathology in preschool-aged children using a structured interview-based assessment. Journal of Abnormal Child Psychology, 42, 1201–1211.
62.
OltmannsJ. R.SmithG. T.OltmannsT. F.WidigerT. A. (2018). General factors of psychopathology, personality, and personality disorder: Across domain comparisons. Clinical Psychological Science, 6, 581–589. doi:10.1177/2167702617750150
63.
PatalayP.FonagyP.DeightonJ.BelskyJ.VostanisP.WolpertM. (2015). A general psychopathology factor in early adolescence. The British Journal of Psychiatry, 207, 15–22.
64.
PetterssonE.LarssonH.LichtensteinP. (2016). Common psychiatric disorders share the same genetic origin: A multivariate sibling study of the Swedish population. Molecular Psychiatry, 21, 717–721.
65.
PopperK. (1959). The logic of scientific discovery. London, England: Hutchinson.
66.
ReiseS. P. (2012). The rediscovery of bifactor measurement models. Multivariate Behavioral Research, 47, 667–696.
67.
ReiseS. P.KimD. S.MansolfM.WidamanK. F. (2016). Is the bifactor model a better model or is it just better at modeling implausible responses? Application of iteratively reweighted least squares to the Rosenberg Self-Esteem Scale. Multivariate Behavioral Research, 51, 818–838.
68.
RodriguezA.ReiseS. P.HavilandM. G. (2016). Evaluating bifactor models: Calculating and interpreting statistical indices. Psychological Methods, 21, 137–150.
69.
SelzamS.ColemanJ. R.CaspiA.MoffittT. E.PlominR. (2018). A polygenic p factor for major psychiatric disorders. Translational Psychiatry, 8, Article 205. doi:10.1038/s41398-018-0217-4
70.
SmithG. T.McCarthyD. M.ZapolskiT. C. (2009). On the value of homogeneous constructs for construct validation, theory testing, and the description of psychopathology. Psychological Assessment, 21, 272–284.
71.
SnyderH. R.MiyakeA.HankinB. L. (2015). Advancing understanding of executive function impairments and psychopathology: Bridging the gap between clinical and cognitive approaches. Frontiers in Psychology, 6, Article 328. doi:10.3389/fpsyg.2015.00328
72.
SnyderH. R.YoungJ. F.HankinB. L. (2017). Strong homotypic continuity in common psychopathology-, internalizing-, and externalizing-specific factors over time in adolescents. Clinical Psychological Science, 5, 98–110.
73.
TackettJ. L.KushnerS. C.De FruytF.MervieldeI. (2013). Delineating personality traits in childhood and adolescence: Associations across measures, temperament, and behavioral problems. Assessment, 20, 738–751.
74.
TackettJ. L.LaheyB. B.Van HulleC.WaldmanI.KruegerR. F.RathouzP. J. (2013). Common genetic influences on negative emotionality and a general psychopathology factor in childhood and adolescence. Journal of Abnormal Psychology, 122, 1142–1153.
75.
VanheuleS.DesmetM.GroenvynckH.RosseelY.FontaineJ. (2008). The factor structure of the Beck Depression Inventory-II: An evaluation. Assessment, 15, 177–187.
76.
WaldmanI. D.PooreH. E.van HulleC.RathouzP. J.LaheyB. B. (2016). External validity of a hierarchical dimensional model of child and adolescent psychopathology: Tests using confirmatory factor analyses and multivariate behavior genetic analyses. Journal of Abnormal Psychology, 125, 1053–1066.
77.
WaldmanI. D.RheeS. H.FeigonS. A.BarJ. L. (1998). The Georgia Twin Registry: Sample description and research issues. Behavior Genetics, 28, 484.
78.
WaldmanI. D.RoweR.BoylanK.BurkeJ. D. (2018). External validation of a bifactor model of oppositional defiant disorder. Molecular Psychiatry. Advance online publication. doi:10.1038/s41380-018-0294-z
79.
WaldmanI. D.TackettJ. L.Van HulleC. A.ApplegateB.PardiniD.FrickP. J.LaheyB. B. (2011). Child and adolescent conduct disorder substantially shares genetic influences with three socioemotional dispositions. Journal of Abnormal Psychology, 120, 57–70.
80.
WidigerT. A. (2005). A dimensional model of psychopathology. Psychopathology, 38, 211–214.
81.
WidigerT. A.OltmannsJ. R. (2017a). The general factor of psychopathology and personality. Clinical Psychological Science, 5, 182–183.
82.
WidigerT. A.OltmannsJ. R. (2017b). Neuroticism is a fundamental domain of personality with enormous public health implications. World Psychiatry, 16, 144–145.
83.
WilloughbyM. T.FabianoG. A.SchatzN. K.VujnovicR. K.MorrisK. L. (2017). Bifactor models of attention deficit/hyperactivity symptomatology in adolescents: Criterion validity and implications for clinical practice. Assessment. Advance online publication. doi:1073191117698755
84.
WrightA. G.KruegerR. F.HobbsM. J.MarkonK. E.EatonN. R.SladeT. (2013). The structure of psychopathology: Toward an expanded quantitative empirical model. Journal of Abnormal Psychology, 122, 281–294.
85.
YuanK. H.BentlerP. M. (2000). Three likelihood-based methods for mean and covariance structure analysis with nonnormal missing data. Sociological Methodology, 30, 165–200.
86.
ZacharP. (2015). Popper, Meehl, and progress: The evolving concept of risky test in the science of psychopathology. Psychological Inquiry, 26, 279–285.
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
