Detecting and Establishing Measurement Invariance In Hospitality Research: Foundations,Challenges and Implications for Validity and Theory Development

Abstract

Measurement invariance (MI) has received increasing attention in the psychology and organizational management literatures with warnings about the peril of assuming psychometric equivalence (i.e., invariance) of a measure of a latent construct across different conditions (such as time periods) or groups (such as in cross-cultural comparisons) rather than testing for it. Heeding this warning, we offer a beginner’s introduction to MI and assess the MI paradigm in hospitality research by examining studies in the International Journal of Hospitality Management and the Journal of Hospitality and Tourism Education that are susceptible to MI issues. We go further, including a didactic example, humbly examining our own published work on intrapersonal longitudinal change in cultural intelligence and demonstrating how assuming rather than testing for MI led to flawed inferences. Suggestions for researchers wanting to test for MI are provided, as are suggestions for how to proceed when measures are non-invariant.

Keywords

measurement invariance measurement equivalence intrapersonal change cross-group comparisons measurement noninvariance

Get full access to this article

View all access options for this article.

References

Adolf

Schuurman

N. K.

Borkenau

Borsboom

Dolan

C. V.

(2014). Measurement invariance within and between individuals: A distinct problem in testing the equivalence of intra- and inter-individual model structures. Frontiers in Psychology, 5, 883, 1–14. https://doi.org/10.3389/fpsyg.2014.00883

Allport

G. W.

(1954). The nature of prejudice. Addison-Wesley.

Anderson

J. C.

Gerbing

D. W.

(1988). Structural equation modeling in practice: A review and recommended two-step approach. Psychological Bulletin, 103(3), 411–423. https://doi.org/10.1037/0033-2909.103.3.411

Ang

Van Dyne

Koh

K. Y.

Templer

K. J.

Tay

Chandrasekar

N. A.

(2007). Cultural intelligence: Its measurement and effects on cultural judgment and decision making, cultural adaptation and task performance. Management and Organization Review, 3, 335–371. https://doi.org/10.1111/j.1740-8784.2007.00082.x

Bashkov

B. M.

Finney

(2013). Applying longitudinal mean and covariance structures (LMACS) analysis to assess construct stability over two time points: An example using psychological entitlement. Measurement and Evaluation in Counseling and Development, 46(4), 289–314. https://doi.org/10.1177/0748175613497038

Bentler

P. M.

(1990). Comparative fit indexes in structural models. (1990). Psychological Bulletin, 107(2), 238–246. https://doi.org/10.1037/0033-2909.107.2.238

Bentler

P. M.

Bonett

D. G.

(1980). Significance tests and goodness of fit in the analysis of covariance structures. Psychological Bulletin, 88, 588–606. https://doi.org/10.1037/0033-2909.88.3.588

Bialosiewicz

Murphy

Berry

(2013). Do our measures measure up? The critical role of measurement invariance. [Demonstration session]. The American Evaluation Association. https://comm.eval.org/prk12/viewdocument/aea13-demonstration

Bollen

K. A.

(1989). Structural equations with latent variables. Wiley.

10.

Bollen

K. A.

(2011). Evaluating effect, composite, and causal indicators in structural equation models. MIS Quarterly, 35(2), 359–372. https://doi.org/10.2307/23044047

11.

Bollen

Lennox

(1991). Conventional wisdom on measurement: A structural equation perspective. Psychological Bulletin, 110(2), 305–314. https://doi.org/10.1037/0033-2909.110.2.305

12.

Browne

M. W.

Cudeck

(1992). Alternative ways of assessing model fit. Sociological Methods & Research, 21(2), 230–258. https://doi.org/10.1177/0049124192021002005

13.

Cheung

G. W.

Rensvold

R. B.

(2002). Evaluating goodness-of-fit indexes for testing measurement invariance. Structural Equation Modeling, 9(2), 233–255. https://doi.org/10.1207/S15328007SEM0902_5

14.

Cole

D. A.

Maxwell

S. E.

(2003). Testing mediational models with longitudinal data: questions and tips in the use of structural equation modeling. Journal of Abnormal Psychology, 112(4), 558–577. https://doi.org/10.1037/0021-843X.112.4.558

15.

Dave

H. P.

Keefer

K. V.

Snetsinger

S. W.

Holden

R. R.

Parker

J. D. A.

(2021). Stability and change in trait emotional intelligence in emerging adulthood: A four-year population-based study. Journal of Personality Assessment, 103(1), 57–66. https://doi.org/10.1080/00223891.2019.1693386

16.

Deshon

R. P.

(2004). Measures are not invariant across groups with error variance homogeneity. Psychological Science, 6(1), 137–149. https://doi.org/10.1027/1614-0001.46.1.137

17.

Dong

Dumas

(2020). Are personality measures valid for different populations? A systematic review of measurement invariance across cultures, gender, and age. Personality and Individual Differences, 160, Article 109956. https://doi.org/10.1016/j.paid.2020.109956

18.

Earley

P. C.

Peterson

R. S.

(2004). The elusive cultural chameleon: Cultural intelligence as a new approach to intercultural training for the global manager. Academy of Management Learning & Education, 3(1), 100–115. https://doi.org/10.5465/amle.2004.12436826

19.

Emerson

S. D.

Guhn

Gadermann

A. M.

(2017). Measurement invariance of the Satisfaction with Life Scale: Reviewing three decades of research. Quality of Life Research, 26, 2251–2264. https://doi.org/10.1007/s11136-017-1552-2

20.

Fischer

(2011). Cross-cultural training effects on cultural essentialism beliefs and cultural intelligence. International Journal of Intercultural Relations, 35(6), 767–775. https://doi.org/10.1016/j.ijintrel.2011.08.005

21.

Gersh

(2016). Culinary industry practitioners’ and educators’ perceptions of core competencies of a 4-year bachelor’s degree in the culinary arts. Journal of Hospitality & Tourism Education, 28(1), 32–43. https://doi.org/10.1080/10963758.2015.1127166

22.

Golembiewski

R. T.

Billingsley

Yeager

(1976). Measuring change and persistence in human affairs: Types of change generated by OD designs. Journal of Applied Behavioral Science, 12(2), 133–157. https://doi.org/10.1177/002188637601200201

23.

Green

A. J.

Repetti

(2015). Clickers: A strategy for active learning in a hospitality classroom. Journal of Hospitality & Tourism Education, 27(4), 171–179. https://doi.org/10.1080/10963758.2015.1089513

24.

Hamagami

McArdle

J. J.

(2012). Dynamic extensions of latent difference score models. In M. R.

Mehl

T. S.

Conner

(Eds.), Data analytic techniques for dynamical systems (pp. 47–85). Psychology Press.

25.

Horn

J. L.

McArdle

J. J.

(1992). A practical and theoretical guide to measurement invariance in aging research. Experimental Aging Research, 18(3), 117–144. https://doi.org/10.1080/03610739208253916

26.

Kern

J. L.

McBride

B. A.

Laxman

D. J.

Dyer

W. J.

Santos

R. M.

Jeans

L. M.

(2016). The role of multiple-group measurement invariance in family psychology research. Journal of Family Psychology, 30(3), 364–374. https://doi.org/10.1037/fam0000184

27.

Kim

Cao

Liu

Wang

Dedrick

(2023) Testing measurement invariance over time with intensive longitudinal data and identifying a source of noninvariance. Structural Equation Modeling: A Multidisciplinary Journal, 30(3), 393–411. https://doi.org/10.1080/10705511.2022.2130331

28.

Leitgöb

Seddig

Schmidt

Sosu

Davidov

Cernat

Sakshaug

J. W.

(2021). Longitudinal measurement (non-)invariance in latent constructs: conceptual insights, model specifications, and testing strategies. In Cernat

Sakshaug

J. W.

(Eds.), Measurement error in longitudinal data (pp. 211–257). Oxford University Press.

29.

Liu

West

S. G.

(2018). Longitudinal measurement noninvariance with ordered categorical indicators: How are the parameters in second-order latent linear growth models affected? Structural Equation Modeling, 25(5), 762–777. https://doi.org/10.1080/10705511.2017.1419353

30.

Livingston

Pichler

Kossek

E. E.

Thompson

R. J.

Bodner

(2022). An alpha, beta, and gamma approach to evaluating occupational health organizational interventions: Learning from the measurement of work-family conflict change. Occupational Health Science, 6(4), 513–543. https://doi.org/10.1007/s41542-022-00122-y

31.

Leitgöb

Seddig

Schmidt

Sosu

Davidov

Cernat

Sakshaug

J. W.

(2021). Longitudinal measurement (non-)invariance in latent constructs: Conceptual insights, model specifications, and testing strategies. In A.

Cernat

J. W.

Sakshaug

(Eds.), Measurement error in longitudinal data (pp. 211–257). Oxford University Press.

32.

Leitgöb

Seddig

Asparouhov

Behr

Davidov

De Roover

Jak

Meitinger

Menold

Muthén

Rudnev

Schmidt

van de Schoot

(2023). Measurement invariance in the social sciences: Historical development, methodological challenges, state of the art, and future perspectives. Social Science Research, 110, Article 102805. https://doi.org/10.1016/j.ssresearch.2022.102805

33.

Little

T. D.

(1997). Means and covariance structures (MACS) analyses of cross-cultural data: Practical and theoretical issues. Multivariate Behavioral Research, 32, 53–76. https://doi.org/10.1207/s15327906mbr3201_3

34.

Lubke

G. H.

Dolan

C. V.

Kelderman

Mellenbergh

G. J.

(2003). Weak measurement invariance with respect to unmeasured variables: An implication of strict factorial invariance. British Journal of Mathematical and Statistical Psychology, 56(2), 231–248. https://doi.org/10.1348/000711003770480020

35.

Matusik

J. G.

Hollenbeck

J. R.

Mitchell

R. L.

(2021). Latent change score models for the study of development and dynamics in organizational research. Organizational Research Methods, 24(4), 772–801. https://doi.org/10.1177/1094428120963788

36.

McArdle

J. J.

Epstein

(1987). Latent growth curves within developmental structural equation models. Child Development, 58(1), 110–133. https://doi.org/10.2307/1130295.

37.

McDonald

R. P.

Marsh

H. W.

(1990). Choosing a multivariate model: Noncentrality and goodness of fit. Psychological Bulletin, 107, 247–255. https://doi.org/10.1037/0033-2909.107.2.247

38.

Mellenbergh

G. J.

(1989). Item bias and item response theory. International Journal of Educational Research, 13, 127–143. https://doi.org/10.1016/0883-0355(89)90002-5

39.

Meredith

(1993). Measurement invariance, factor analysis and factorial invariance. Psychometrika, 58(4), 525–543. https://doi.org/10.1007/BF02294825

40.

Miller

M. J.

Sheu

(2008). Conceptual and measurement issues in multicultural psychology research. In S. D.

Brown

R. W.

Lent

(Eds.), Handbook of counseling psychology (4th ed., pp. 103–120). Wiley.

41.

Min

Swanger

Gursoy

(2016). A longitudinal investigation of the importance of course subjects in the hospitality curriculum: An industry perspective. Journal of Hospitality & Tourism Education, 28(1), 10–20. https://doi.org/10.1080/10963758.2015.1127168

42.

Mustafa

Nazli

(2018). The effect of emotional labor on work alienation: A study at hotel business. Journal of Business Research – Turk, 10(3), 606–621. https://doi.org/10.20491/isarder.2018.490

43.

Muthén

L. K.

Muthén

B. O.

(1998–2017). Mplus user’s guide (8th ed). Muthén & Muthén.

44.

Ott

D. L.

Michailova

(2018). Cultural intelligence: A review and new research avenues. International Journal of Management Reviews, 20(1), 99–119. https://doi.org/10.1111/ijmr.12118

45.

Putnick

D. L.

Bornstein

M. H.

(2016). Measurement invariance conventions and reporting: The state of the art and future directions for psychological research. Developmental Review, 41, 71–90. https://doi.org/10.1016/j.dr.2016.06.004

46.

Rivera

(2016). A pre-/postcourse analysis of distance education students’ attitudes toward and perceptions of diversity and multicultural issues. Journal of Hospitality & Tourism Education, 28(2), 60–70. https://doi.org/10.1080/10963758.2016.1163496

47.

Riordan

C. M.

Richardson

H. A.

Schaffer

B. S.

Vandenburg

(2001). Alpha, beta and gamma change: A review of past research with recommendations for new directions. In C. A.

Schreisheim

I. I.

Neider

(Eds.), Equivalence in measurement (pp. 51–97). Information Age Publishing.

48.

Roesch

S. C.

Norman

G. J.

Merz

E. L.

Sallis

J. F.

Patrick

(2013). Longitudinal measurement invariance of psychosocial measures in physical activity research: An application to adolescent data. Journal of Applied Social Psychology, 43(4), 721–729. https://doi.org/10.1111/jasp.12000

49.

Ryu

Lee

H. R.

Kim

W. G.

(2012). The influence of the quality of the physical environment, food, and service on restaurant image, customer perceived value, customer satisfaction, and behavioral intentions. International Journal of Contemporary Hospitality Management, 24(2), 200–223. https://doi.org/10.1108/09596111211206141

50.

Scarpello

Hayton

J. C.

(2001). Identifying the sources of nonequivalence in measures of job satisfaction. In C. A.

Schreisheim

I. I.

Neider

(Eds.), Equivalence in measurement (pp. 131–160). Information Age Publishing.

51.

Sprangers

M. A. G.

Schwartz

C. E.

(1999). Integrating response shift into health-related quality of life research: A theoretical model. Social Science Medicine, 48(11), 1507–1515. https://doi.org/10.1016/S0277-9536(99)00045-3

52.

Thomas

(2006). Domain and development of cultural intelligence: The importance of mindfulness. Group & Organization Management, 31(1), 78–99. https://doi.org/10.1177/1059601105275266

53.

Tucker

T. R.

Lewis

(1973). A reliability coefficient for maximum likelihood factor analysis. Psychometrika, 38(1), 1–10. https://doi.org/10.1007/BF02291170

54.

Vandenberg

R. J.

Lance

C. E.

(2000). A review and synthesis of the measurement invariance literature: Suggestions, practices, and recommendations for organizational research. Organizational Research Methods, 3(1), 4–70. https://doi.org/10.1177/109442810031002

55.

Van de Schoot

Schmidt

Beuckelaer

A. D.

Lek

Zondervan-Zwijnenburg

(2015). Editorial: Measurement invariance. Frontiers in Psychology, 6, Article 1064. https://doi.org/10.3389/fpsyg.2015.01064.

56.

Vanier

Oort

F. J.

McClimans

Gulek

B. G.

Böhnke

J. R.

Sprangers

Sébille

Mayo

, & Response Shift-in Sync Working Group. (2021). Response shift in patient-reported outcomes: definition, theory, and a revised model. Quality of Life Research, 30(12), 3309–3322. https://doi.org/10.1007/s11136-021-02846-w

57.

Widaman

K. F.

Ferrer

Conger

R. D.

(2010). Factorial invariance within longitudinal structural equation models: Measuring the same construct across time. Child Development Perspectives, 4(1), 10–18. https://doi.org/10.1111/j.1750-8606.2009.00110.x

58.

Widaman

K. F.

Thompson

J. S.

(2003). On specifying the null model for incremental fit indices in structural equation modeling. Psychological Methods, 8(1), 16–37. https://doi.org/10.1037/1082-989X.8.1.16

59.

Wilson-Wunch

Beausaert

Tempelaar

Gijselaers

(2016). Expertise development of hospitality students: Do personality, emotional intelligence, and learning style matter? Journal of Hospitality & Tourism Education, 28(3), 155–167. https://doi.org/10.1080/10963758.2016.1190595

60.

Wolfe

K. L.

(2017). A longitudinal study of hospitality students’ emotional intelligence: Comparing high and low scores. Journal of Hospitality & Tourism Education, 29(2), 51–60. https://doi.org/10.1080/10963758.2017.1297715

61.

Young

C. A.

Haffejee

Corsun

D. L.

(2018). Developing cultural intelligence and empathy through diversified mentoring relationship. Journal of Management Education, 3(42), 219-346. https://doi.org/10.1177/1052562917710

62.

Zizka

Varga

(2021). Teaching sustainability in higher education institutions: Assessing hospitality students’ sustainability literacy. Journal of Hospitality & Tourism Education, 33(4), 242–257. https://doi.org/10.1080/10963758.2020.1726771