Large-scale studies spanning diverse project sites, populations, languages, and measurements are increasingly important to relate psychological to biological variables. National and international consortia already are collecting and executing mega-analyses on aggregated data from individuals, with different measures on each person. In this research, we show that Asparouhov and Muthén’s alignment method can be adapted to align data from disparate item sets and response formats. We argue that with these adaptations, the alignment method is well suited for combining data across multiple sites even when they use different measurement instruments. The approach is illustrated using data from the Whole Genome Sequencing in Psychiatric Disorders consortium and a real-data-based simulation is used to verify accurate parameter recovery. Factor alignment appears to increase precision of measurement and validity of scores with respect to external criteria. The resulting parameter estimates may further inform development of more effective and efficient methods to assess the same constructs in prospectively designed studies.
AdamsR. J.WilsonM.WuM. (1997). Multilevel item response models: An approach to errors in variables regression. Journal of Educational and Behavioral Statistics, 22, 47-76.
2.
AndreasenN. C.FlaumM.ArndtS. (1992). The Comprehensive Assessment of Symptoms and History (CASH): An instrument for assessing diagnosis and psychopathology. Archives of General Psychiatry, 49, 615-623.
3.
AngS.Van DyneL.KohC.NgK. Y.TemplerK. J.TayC.ChandrasekarN. A. (2007). Cultural intelligence: Its measurement and effects on cultural judgment and decision making, cultural adaptation and task performance. Management and Organization Review, 3, 335-371.
4.
AntonakisJ.AvolioB. J.SivasubramaniamN. (2003). Context and leadership: An examination of the nine-factor full-range leadership theory using the Multifactor Leadership Questionnaire. Leadership Quarterly, 14, 261-295.
BeatomA. E.GonzalezE. (1995). NAEP primer. Chestnut Hill, MA: Boston College.
7.
BrodeyB. B.FirstM.LinthicumJ.HamanK.SasielaJ. W.AyerD. (2016). Validation of the NetSCID: An automated web-based adaptive version of the SCID. Comprehensive Psychiatry, 66, 67-70.
EndersC. K. (2010). Applied missing data analysis. New York, NY: Guilford Press.
15.
FirstM. B. (2014). Structured clinical interview for the DSM (SCID). In The encyclopedia of clinical psychology (pp. 1-6). New York, NY: Wiley.
16.
FlakeJ. K.McCoachD. B. (2018). An investigation of the alignment method with polytomous indicators under conditions of partial measurement invariance. Structural Equation Modeling, 25, 56-70.
17.
FragosoT. M.GioloS. R.PereiraA. C.De AndradeM.SolerJ. M. (2014). Using item response theory to model multiple phenotypes and their joint heritability in family data. Genetic Epidemiology, 38, 152-161.
GuC.GutmanR. (2017). Combining item response theory with multiple imputation to equate health assessment questionnaires. Biometrics, 73, 990-998.
20.
HoferS. M.PiccininA. M. (2009). Integrative data analysis through coordination of measurement and analysis protocol across independent longitudinal studies. Psychological Methods, 14, 150.
21.
HussongA. M.CaiL.CurranP. J.FloraD. B.ChassinL. A.ZuckerR. A. (2008). Disaggregating the distal, proximal, and time-varying effects of parent alcoholism on children’s internalizing symptoms. Journal of Abnormal Child Psychology, 36, 335-346.
22.
HussongA. M.HuangW.CurranP. J.ChassinL.ZuckerR. A. (2010). Parent alcoholism impacts the severity and timing of children’s externalizing symptoms. Journal of Abnormal Child Psychology, 38, 367-380.
23.
KamataA.BauerD. J. (2008). A note on the relation between factor analytic and item response theory models. Structural Equation Modeling, 15, 136-153.
24.
KaplanD.McCartyA. T. (2013). Data fusion with international large-scale assessments: A case study using the OECD PISA and TALIS surveys. Large-Scale Assessments in Education, 1(1), Article 6.
25.
KaufmanC. J. (1988). The application of logical imputation to household measurement. Journal of the Market Research Society, 30, 453-466.
26.
KolenM. J.BrennanR. L. (2014). Test equating, scaling, and linking: Methods and practices. Berlin, Germany: Springer Science & Business Media.
27.
LeeM.AggenS. H.OtowaT.CastelaoE.PreisigM.GrabeH. J.. . . HettemaJ. M. (2016). Assessment and characterization of phenotypic heterogeneity of anxiety disorders across five large cohorts. International Journal of Methods in Psychiatric Research, 25, 255-266.
28.
LeiteW. L.HuangI. C.MarcoulidesG. A. (2008). Item selection for the development of short forms of scales using an ant colony optimization algorithm. Multivariate Behavioral Research, 43, 411-431.
29.
MarcoulidesK. M.GrimmK. J. (2017). Data integration approaches to longitudinal growth modeling. Educational and Psychological Measurement, 77, 971-989.
30.
MarshH. W. (1989). Confirmatory factor analysis of multitrait-multimethod data: Many problems and a few solutions. Applied Psychological Measurement, 13, 335-361.
31.
MarshH. W.GuoJ.ParkerP. D.NagengastB.AsparouhovT.MuthénB.DickeT. (2017). What to do when scalar invariance fails: The extended alignment method for multi-group factor analysis comparison of latent means across many groups. Psychological Methods, 23, 524-545. Retrieved from http://psycnet.apa.org/psycinfo/2017-01642-001/
32.
McArdleJ. J.GrimmK. J.HamagamiF.BowlesR. P.MeredithW. (2009). Modeling life-span growth curves of cognition using longitudinal data with multiple samples and changing scales of measurement. Psychological Methods, 14, 126-149.
33.
McArdleJ. J.PrescottC. A.HamagamiF.HornJ. L. (1998). A contemporary method for developmental-genetic analyses of age changes in intellectual abilities. Developmental Neuropsychology, 14, 69-114.
34.
MislevyR. J. (1991). Randomization-based inference about latent variables from complex samples. Psychometrika, 56, 177-196.
35.
MislevyR. J.BeatonA. E.KaplanB.SheehanK. M. (1992). Estimating population characteristics from sparse matrix samples of item responses. Journal of Educational Measurement, 29, 133-161.
36.
MuthénB.AsparouhovT. (2014). IRT studies of many groups: The alignment method. Frontiers in Psychology, 5, Article 978.
37.
MuthénB.AsparouhovT. (2018). Recent methods for the study of measurement invariance with many groups: Alignment and random effects. Sociological Methods & Research, 47, 637-664.
38.
NurnbergerJ. I.BleharM. C.KaufmannC. A.York-CoolerC.SimpsonS. G.Harkavy-FriedmanJ.. . . ReichT. (1994). Diagnostic interview for genetic studies: Rationale, unique features, and training. Archives of General Psychiatry, 51, 849-859.
39.
PastorD. A. (2003). The use of multilevel item response theory modeling in applied research: An illustration. Applied Measurement in Education, 16, 223-243.
40.
PerlmanG.KotovR.FuJ.BrometE. J.FochtmannL. J.MedeirosH.. . . PatoC. N. (2016). Symptoms of psychosis in schizophrenia, schizoaffective disorder, and bipolar disorder: A comparison of African Americans and Caucasians in the Genomic Psychiatry Cohort. American Journal of Medical Genetics Part B: Neuropsychiatric Genetics, 171, 546-555.
41.
R Core Team. (2019). R: A language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing. Retrieved from http://www.R-project.org
42.
ReiseS. P.RevickiD. A. (Eds.). (2014). Handbook of item response theory modeling: Applications to typical performance assessment. New York, NY: Routledge.
43.
RubinD. B. (1976). Inference and missing data. Biometrika, 63, 581-592. doi:10.1093/biomet/63.3.581
44.
RuderferD. M.FanousA. H.RipkeS.McQuillinA.AmdurR. L.GejmanP. V.. . . KendlerK. S. (2014). Polygenic dissection of diagnosis and clinical dimensions of bipolar disorder and schizophrenia. Molecular Psychiatry, 19, 1017-1024.
45.
SamejimaF. (1968). Estimation of latent ability using a response pattern of graded scores. ETS Research Report Series, 1968(1), i-169.
46.
SamejimaF. (1997). Graded response model. In van der LindenW. M.HambletonR. K. (Eds.), Handbook of modern item response theory (pp. 85-100). New York, NY: Springer.
47.
SandersS. J.NealeB. M.HuangH.WerlingD. M.AnJ. Y.DongS.. . . DalyM. J. (2017). Whole genome sequencing in psychiatric disorders: The WGSPD consortium. Nature Neuroscience, 20, 1661-1668.
48.
SchwartzS. H.RubelT. (2005). Sex differences in value priorities: Cross-cultural and multimethod studies. Journal of Personality and Social Psychology, 89, 1010-1028.
49.
SenthilG.DutkaT.BingamanL.LeherT. (2017). Genomic resources for the study of neuropsychiatric disorders. Molecular Psychiatry, 22, 1659-1663. doi:10.1038/mp.2017.29
50.
SheehanD. V.LecrubierY.SheehanK. H.AmorimP.JanavsJ.WeillerE.. . . DunbarG. C. (1998). The Mini-International Neuropsychiatric Interview (MINI): The development and validation of a structured diagnostic psychiatric interview for DSM-IV and ICD-10. Journal of Clinical Psychiatry, 59, 22-33.
51.
TakaneY.De LeeuwJ. (1987). On the relationship between item response theory and factor analysis of discretized variables. Psychometrika, 52, 393-408.
52.
Van der LindenW. J.VeldkampB. P.CarlsonJ. E. (2004). Optimizing balanced incomplete block designs for educational assessments. Applied Psychological Measurement, 28, 317-331.
53.
VandenbergR. J.LanceC. E. (2000). A review and synthesis of the measurement invariance literature: Suggestions, practices, and recommendations for organizational research. Organizational Research Methods, 3, 4-70.
54.
VispoelW. P.KimH. Y. (2014). Psychometric properties for the Balanced Inventory of Desirable Responding: Dichotomous versus polytomous conventional and IRT scoring. Psychological Assessment, 26, 878-891.
55.
WarmT. A. (1989) Weighted likelihood estimation of ability in item response theory. Psychometrika, 54, 427-450.
56.
WidamanK. F.ReiseS. P. (1997). Exploring the measurement invariance of psychological instruments: Applications in the substance use domain. In BryantK. J.WindleM.WestS. G. (Eds.), The science of prevention: Methodological advances from alcohol and substance abuse research (pp. 281-324). Washington, DC: American Psychological Association.
57.
WuM. (2005). The role of plausible values in large-scale surveys. Studies in Educational Evaluation, 31, 114-128.
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
