Using Integrative Data Analysis to Investigate School Climate Across Multiple Informants

Abstract

Survey research frequently involves the collection of data from multiple informants. Results, however, are usually analyzed by informant group, potentially ignoring important relationships across groups. When the same construct(s) are measured, integrative data analysis (IDA) allows pooling of data from multiple sources into one data set to examine information from multiple perspectives within the same analysis. Here, the IDA procedure is demonstrated via the examination of pooled data from student and teacher school climate surveys. This study contributes to the sparse literature regarding IDA applications in the social sciences, specifically in education. It also lays the groundwork for future educational researchers interested in the practical applications of the IDA framework to empirical data sets with complex model structures.

Keywords

integrative data analysis survey research school climate

Get full access to this article

View all access options for this article.

References

Acar

(2011). Sample size in differential item functioning: An application of Hierarchical linear modeling. Educational Sciences: Theory and Practice, 11, 284-288. Retrieved from https://files.eric.ed.gov/fulltext/EJ919902.pdf

Asparouhov

Muthén

(2010, August 14). Weighted least squares estimation with missing data (Technical report). Retrieved from http://www.statmodel.com/download/GstrucMissingRevision.pdf

Bainter

S. A.

Curran

P. J.

(2015). Advantages of integrative data analysis for developmental research. Journal of Cognition and Development, 16, 1-10. doi:10.1080/15248372.2013.871721

Bauer

D. J.

(2017). A more general model for testing measurement invariance and differential item functioning. Psychological Methods, 22, 507-526.

Bauer

D. J.

Hussong

A. M.

(2009). Psychometric approaches for developing commensurate measures across independent studies: Traditional and new models. Psychological Methods, 14, 101-125. doi:10.1037/a0015583

Carlson

Miller

(1987). Explanation of the relation between negative mood and helping. Psychological Bulletin, 102, 91-108.

Cialdini

R. B.

Fultz

(1990). Interpreting the negative mood-helping literature via “mega”-analysis: A contrary view. Psychological Bulletin, 107, 210-214.

Crane

P. K.

van Belle

Larson

E. B.

(2004) Test bias in a cognitive test: Differential item functioning in the CASI. Statistics in Medicine, 23, 241-256.

Curran

P. J.

Cole

Giordano

Georgeson

A. R.

Hussong

A. M.

Bauer

D. J.

(2018). Advancing the study of adolescent substance use through the use of integrative data analysis. Evaluation and the Health Professions, 41, 216-245.

10.

Curran

P. J.

Hussong

A. M.

(2009). Integrative data analysis: The simultaneous analysis of multiple data sets. Psychological Methods, 14, 81-100.

11.

Curran

P. J.

Hussong

A. M.

Cai

Huang

Chassin

Sher

K. J.

Zucker

R. A.

(2008). Pooling data from multiple longitudinal studies: The role of item response theory in integrative data analysis. Developmental Psychology, 44, 365-380.

12.

Curran

P. J.

McGinley

J. S.

Bauer

D. J.

Hussong

A. M.

Burns

A. B.

Chassin

. . . Zucker

R. A.

(2014). A moderated nonlinear factor model for the development of commensurate measures in integrative data analysis. Multivariate Behavioral Research, 49, 214-231. doi:10.1080/00273171.2014.889594

13.

DiStefano

Mindrila

Monrad

D. M.

(2013). Investigating factorial invariance of teacher climate factors across school organizational levels. In Khine

M. S.

(Ed.), Application of structural equation modeling in educational research and practice (pp. 257-275). Rotterdam, Netherlands: Sense.

14.

DiStefano

Monrad

D. M.

May

R. J.

McGuiness

Dickenson

(2007, April). Using school climate surveys to categorize schools and examine relationships with school achievement. Paper presented at the meeting of the American Educational Research Association, Chicago, IL.

15.

Ene

Leighton

E. A.

Guo

McGrath

K. V.

DiStefano

Monrad

D. M.

(2016, April). Relationships between school climate and indicators of school effectiveness: A multilevel investigation. Paper presented at the meeting of the American Educational Research Association, Washington, DC.

16.

Finney

S. J.

DiStefano

(2013). Non-normal and categorical data in structural equation models. In Hancock

G. R.

Mueller

R. O.

(Eds.), Structural equation modeling: A second course (2nd ed., pp. 439-492). Charlotte, NC: Information Age.

17.

Gareau

May

R. J.

Mindrila

Ishikawa

DiStefano

Monrad

D. M.

Price

(2010, April). The relationship between school climate and school performance. Paper presented at the meeting of the American Educational Research Association, Denver, CO, USA.

18.

Glass.

G. V.

(1976). Primary, secondary, and meta-analysis of research. Educational Researcher, 5(10), 3-8.

19.

Greenberg

(2004). Climates for learning. Paper presented at the annual meeting of the American Educational Research Association, San Diego, CA.

20.

Hofer

S. M.

Piccinin

A. M.

(2009). Integrative data analysis through coordination of measurement and analysis protocol across independent longitudinal studies. Psychological Methods, 14, 150-164.

21.

Hox

Bechger

(1995). Comparing and combining different approaches to the multitrait-multimethod matrix. In Hox

J. J.

Mellenbergh

G. J.

Swanborn

P. G.

(Eds.) (Eds.), Facet theory: Analysis and design (pp. 135-144). Zeist, Netherlands: SETOS.

22.

Bentler

P. M.

(1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6, 1-55.

23.

Hussong

A. M.

Flora

D. B.

Curran

P. J.

Chassin

L. A.

Zucker

R. A.

(2008). Defining risk heterogeneity for internalizing symptoms among children of alcoholic parents. Development and Psychopathology, 20, 165-193.

24.

Kline

R. B.

(2005). Principles and practice of structural equation modeling. New York, NY: Guilford Press.

25.

Lee

V. E.

Burkham

D. T.

(1996, February). Gender differences in middle-grade science achievement: Subject, domain, ability level, and course emphasis. Science Edition, 80, 613-650.

26.

MacMillan

R. B.

(1999). Influences of workplace conditions on teachers’ job satisfaction. Journal of Educational Research, 93, 39-47.

27.

MacCallum

R. C.

Browne

M. W.

Sugawara

H. M.

(1996). Power analysis and determination of sample size for covariance structure modeling. Psychological Methods, 1, 130-149.

28.

Marcoulides

K. M.

Grimm

K. J.

(2017). Data integration approaches to longitudinal growth modeling. Educational and Psychological Measurement, 77, 971-989.

29.

Marcoulides

K. M.

Yuan

K.-H.

(2017). New ways to evaluate goodness of fit: A note on using equivalence testing to assess structural equation models. Structural Equation Modeling: A Multidisciplinary Journal, 24, 148-153.

30.

McArdle

J. J.

Grimm

K. J.

Hamagami

Bowles

R. P.

Meredith

(2009). Modeling life-span growth curves of cognition using longitudinal wave data with multiple samples and changing scales of measurement. Psychological Methods, 14, 126-149.

31.

McArdle

J. J.

Horn

J. L.

(2002, September–October). The benefits and limitations of mega-analysis with illustrations for the WAIS. Paper presented at the International Meeting of CODATA, Montreal, Quebec, Canada.

32.

Monrad

D. M.

May

R. J.

DiStefano

Smith

Gay

Mindrila

. . . Rawls

(2008, April). Parent, student, and teacher perceptions of school climate: Investigations across organizational levels. Paper presented at the meeting of the American Educational Research Association, New York, NY.

33.

Muthén

L. K.

Muthén

B. O.

(1998-2012). Mplus user’s guide (7th ed.). Los Angeles, CA: Muthen & Muthen.

34.

Muthén

L. K.

Muthén

B. O.

(2017). Mplus [Computer software]. Retrieved from https://www.statmodel.com/index.shtml

35.

Muthén

L. K.

Muthén

B. O.

(2019). Mplus FAQs: Factor scores. Retrieved from https://www.statmodel.com/faq.shtml

36.

Penuel

Meyer

Valladares

M. R.

(2016, November 15). Making the most of the Every Student Succeeds Act (ESSA): Helping states focus on school equity, quality and climate (Policy Brief). Retrieved from http://nepc.colorado.edu/publication/ESSA

37.

Rässler

(2003). A non-iterative Bayesian approach to statistical matching. Statistica Neerlandica, 57, 58-74.

38.

Stewart

E. B.

(2007, June). School structural characteristics, student effort, peer associations, and parental involvement: The influence of school- and individual-level factors on academic achievement. Education and Urban Society, 40, 179-204.

39.

Thapa

Cohen

Guffey

Higgins-D’Alessandro

(2013). A review of school climate. Review of Educational Research, 83, 357-385.

40.

Wang

M. T.

Degol

J. L.

(2016). School climate: A review of the construct, measurement, and impact on student outcomes. Educational Psychology Review, 28, 315-352. doi:10.1007/s10648-015-9319-1

41.

West

S. G.

Finch

J. F.

Curran

P. J.

(1995). Structural equation models with non-normal variables: Problems and remedies. In Hoyle

(Ed.), Structural equation modeling: Concepts, issues and applications (pp. 56-75). Newbury Park, CA: Sage.

42.

Widaman

K. F.

Ferrer

Conger

R. D.

(2010). Factorial invariance within longitudinal structural equation models: Measuring the same construct across time. Child Development Perspectives, 4, 10-18. doi:10.111/j.1750-8606.2009.00110.x

43.

Yuan

K.-H.

Chan

(2016). Measurement invariance via multigroup SEM: Issues and solutions with chi-square difference tests. Psychological Methods, 21, 405-426.

44.

Yuan

K.-H.

Chan

Marcoulides

G. A.

Bentler

P. M.

(2016). Assessing structural equation models by equivalence testing with adjusted fit indices. Structural Equation Modeling: A Multidisciplinary Journal, 23, 319-330.