Linguistic Equivalence,Construct Validity,But Lack Measurement Invariance: An Illustration of Challenges in Cross-Cultural Research on Adolescent Adjustment

Abstract

In this study, we illustrated issues related to measure invariance in cross-cultural research involving instrument translation between Chinese and English. We translated and back-translated the third edition of the Behavioral Assessment for Children-Self Report of Personality (BASC-3-SRP) and administered it to 1,574 youth in China and 512 youth in the United States. We found that despite a rigorous approach to achieving linguistic equivalence, statistically demonstrating acceptable internal consistency and construct validity, measurement invariance tests revealed that six of the 16 BASC-3-SRP subscales lacked measurement invariance. Constructs for the first three of the six subscales that lacked measurement invariance (i.e., Negative Attitude toward School, Negative Attitude toward Teachers, and Self-Esteem) are known to be conceptualized differently in collectivistic societies, while constructs for the second three subscales (i.e., Atypicality, Sense of Inadequacy, and Hyperactivity) lacked measurement invariance without known cultural reasons. These results highlight instrument development issues and measurement variance issues that cross-cultural researchers must grapple with.

Keywords

BASC-3 linguistic equivalence measurement invariance cross-cultural studies behavioral health

Get full access to this article

View all access options for this article.

References

Aaronson

Ferres

(1987). The impact of language differences on language processing: An example from Chinese-English bilingualism. In Homel

Palij

Aaronson

(Eds.), Childhood bilingualism: Aspects of linguistic, cognitive, and social development (pp. 75–119). Lawrence Erlbaum.

Abdullah

Brown

T. L.

(2011). Mental illness stigma and ethnocultural beliefs, values, and norms: An integrative review. Clinical Psychology Review, 31, 934–948. https://doi.org/10.1016/j.cpr.2011.05.003

Ahn

C. M.

Ebesutani

(2015). Emotional and behavioral profile assessment using the BASC-2 with Korean middle school students. Psychology in the Schools, 52(8), 756–772. https://doi.org/10.1002/pits.21857

Ahn

C. M.

Ebesutani

Kamphaus

(2014). A psychometric analysis and standardization of the Behavior Assessment System for Children-2, Self-Report of Personality, Child Version among a Korean sample. School Psychology Quarterly, 29(2), 198–212. https://doi.org/10.1037/spq0000034

Asparouhov

Muthén

(2014). Multiple-group factor analysis alignment. Structural Equation Modeling: A Multidisciplinary Journal, 21(4), 495–508. https://doi.org/10.1080/10705511.2014.919210

Bentler

P. M.

Bonett

D. G.

(1980). Significance tests and goodness of fit in the analysis of covariance structures. Psychological Bulletin, 88, 588–606.

Bialosiewicz

Murphy

Berry

(2013). An introduction to measurement invariance testing: Resource packet for participants. http://comm.eval.org/HigherLogic/System/DownloadDocumentFile.ashx?DocumentFileKey=63758fed-a490-43f2-8862-2de0217a08b8

Bond

M. H.

(1986). The psychology of the Chinese people. Oxford University Press.

Brislin

R. W.

(1970). Back-translation for cross-cultural research. Journal of Cross-Cultural Psychology, 1(3), 185–216. https://doi.org/10.1177/135910457000100301

10.

Brown

T. A.

(2006). Confirmatory factor analysis for applied research. Guilford Press.

11.

Bush

K. R.

Peterson

G. W.

Cobas

J. A.

Supple

A. J.

(2002). Adolescents’ perceptions of parental behaviors as predictors of adolescent self-esteem in mainland China. Sociological Inquiry, 72(4), 503–526. https://doi.org/10.1111/1475-682X.00031

12.

Byrne

B. M.

(2006). Structural equation modeling with EQS: Basic concepts, applications, and programming (2nd ed.). Lawrence Erlbaum.

13.

Byrne

B. M.

(2016). Adaptation of assessment scales in cross-national research: Issues, guidelines, and caveats. International Perspectives in Psychology: Research, Practice, Consultation, 5(1), 51–65. https://doi.org/10.1037/ipp0000042

14.

Byrne

B. M.

Campbell

T. L.

(1999). Cross-cultural comparisons and the presumption of equivalent measurement and theoretical structure: A look beneath the surface. Journal of Cross-Cultural Psychology, 30(5), 555–574. https://doi.org/10.1177/0022022199030005001

15.

Chen

F. F.

(2007). Sensitivity of goodness of fit indexes to lack of measurement invariance. Structural Equation Modeling, 14(3), 464–504. https://doi.org/10.1080/10705510701301834

16.

Cheng

Cheung

S.-F.

Chio

H.-M.

Chan

M.-P. S.

(2013). Cultural meaning of perceived control: A meta-analysis of locus of control and psychological symptoms across 18 cultural regions. Psychological Bulletin, 139(1), 152–188. https://doi.org/10.1037/a0028596

17.

Cheng

S.-T.

Hamid

P. N.

(1995). An error in the use of translated scales: The Rosenberg Self-Esteem Scale for Chinese. Perceptual and Motor Skills, 81(2), 431–434. https://doi.org/10.2466/pms.1995.81.2.431

18.

Cheung

G. W.

Rensvold

R. B.

(2000). Assessing extreme and acquiescence response sets in cross-cultural research using structural equations modeling. Journal of Cross-Cultural Psychology, 31, 187–212. https://doi.org/10.1177/0022022100031002003

19.

Cheung

G. W.

Rensvold

R. B.

(2002). Evaluating goodness-of-fit indexes for testing measurement invariance. Structural Equation Modeling, 9(2), 233–255. https://doi.org/10.1207/S15328007SEM0902_5

20.

Chiao

J. Y.

Blizinsky

K. D.

(2010). Culture-gene coevolution of individualism-collectivism and the serotonin transporter gene. Proceedings of the Royal Society B: Biological Sciences, 277(1681), 529–537. https://doi.org/10.2307/40506151

21.

Chiswick

B. R.

Miller

P. W.

(2008). Linguistic distance: A quantitative measure of the distance between English and other languages. Journal of Multilingual & Multicultural Development, 26(1), 1–11. https://doi.org/10.1080/14790710508668395

22.

Farh

J.-L.

Cheng

B.-S.

(1997). Modesty bias in self-rating in Taiwan: Impact of item wording, modesty value, and self-esteem. Chinese Journal of Psychology, 39, 103–118.

23.

Flake

J. K.

McCoach

D. B.

(2018). An investigation of the alignment method with polytomous indicators under conditions of partial measurement invariance. Structural Equation Modeling: A Multidisciplinary Journal, 25(1), 56–70. https://doi.org/10.1080/10705511.2017.1374187

24.

Frick

P. J.

Barry

C. T.

Kamphaus

R. W.

(2010). Clinical assessment of child and adolescent personality and behavior (3rd ed.). Springer. https://doi.org/10.1007/978-1-4419-0641-0_6

25.

Hamid

P. N.

(1994). Self-monitoring, locus of control, and social encounters of Chinese and New Zealand students. Journal of Cross-Cultural Psychology, 25(3), 353–368. https://doi.org/10.1177/0022022194253004

26.

Harkness

J. A.

(2003). Questionnaire translation. In Harkness

J. A.

van de Vijver

Mohler

P. P.

(Eds.), Cross-cultural survey methods (pp. 35–56). John Wiley.

27.

Heine

Lehman

(1997). The cultural construction of self-enhancement: An examination of group-serving biases. Journal of Personality and Social Psychology, 72, 1268–1283. https://doi.org/10.1037/0022-3514.72.6.1268

28.

Hofstede

(1986). Cultural differences in teaching and learning. International Journal of Intercultural Relations, 10(3), 301–320. https://doi.org/10.1016/0147-1767(86)90015-5

29.

Hui

C. H.

Triandis

H. C.

(1985). Measurement in cross-cultural psychology: A review and comparison of strategies. Journal of Cross-Cultural Psychology, 16, 131–152.

30.

Hulin

C. L.

(1987). A psychometric theory of evaluations of item and scale translations: Fidelity across languages. Journal of Cross-Cultural Psychology, 18(2), 115–142. https://doi.org/10.1177/0022002187018002001

31.

International Test Commission. (2017). The ITC guidelines for translating and adapting tests (2nd ed.). www.InTestCom.org

32.

Kenny

D. A.

McCoach

D. B.

(2003). Effect of the number of variables on measures of fit in structural equation modeling. Structural Equation Modeling: A Multidisciplinary Journal, 10(3), 333–351. https://doi.org/10.1207/S15328007SEM1003_1

33.

King

L. M.

(1978). Social and cultural influences on psychopathology. Annual Review of Psychology, 29, 405–433.

34.

Kirmayer

L. J.

Bhugra

(2009). Culture and mental illness: Social context and explanatory models. In Salloum

I. M.

Mezzich

J. E.

(Eds.), Psychiatric diagnosis: Patterns and prospects (pp. 29–37). John Wiley.

35.

Korabik

van Rhijn

(2018). Best practices in scale translation and establishing measurement equivalence. In Shockley

K. M.

Shen

Johnson

R. C.

(Eds.), The Cambridge handbook of the global-family interface (pp. 212–230). Cambridge University Press.

36.

Kwan

V. S. Y.

Kuang

L. L.

Hui

N. H. H.

(2009). Identifying the sources of self-esteem: The mixed medley of benevolence, merit and bias. Journal of Self and Identity, 8(2–3), 176–195. https://doi.org/10.1080/15298860802504874

37.

Mann

E. M.

Ikeda

Mueller

C. W.

Takahashi

Tao

K. T.

Humris

B. L.

Chin

(1992). Cross-cultural differences in rating hyperactive-disruptive behaviors in children. American Journal of Psychiatry, 149, 1539–1542. https://doi.org/10.1176/ajp.149.11.1539

38.

Millsap

R. E.

(2011). Statistical approaches to measurement invariance. Routledge.

39.

Muthén

L. K.

Muthén

B. O.

(1998-2017). Mplus user’s guide (8th ed.).

40.

Nida

E. A.

(1964). Toward a science of translation. E. J. Brill.

41.

Peña

E. D.

(2007). Lost in translation: Methodological considerations in cross-cultural research. Child Development, 78(4), 1255–1264. https://doi.org/10.1111/j.1467-8624.2007.01064.x

42.

Reynolds

C. R.

Kamphaus

R. W.

(2015). Behavior assessment system for children, third edition (BASC-3). Pearson.

43.

Spencer-Rodgers

Peng

Wang

Hou

(2004). Dialectical self-esteem and East-West differences in psychological well-being. Personality and Social Psychology, 30(11), 1416–1432. https://doi.org/10.1177/0146167204264243

44.

C.-T.

Parham

L. D.

(2002). Case report: Generating a valid questionnaire translation for cross-cultural use. American Journal of Occupational Therapy, 56, 581–585. https://doi.org/10.5014/ajot.56.5.581

45.

Timimi

Taylor

(2003). ADHD is best understood as a cultural construct. British Journal of Psychiatry, 184(1), 8–9. https://doi.org/10.1192/bjp.184.1.8

46.

Uchida

Norasakkunkit

Kitayama

(2004). Cultural constructions of happiness: Theory and empirical evidence. Journal of Happiness Studies, 5(3), 223–239. https://doi.org/10.1007/s10902-004-8785-9

47.

van de Schoot

Kluytmans

Tummers

Lugtig

Hox

Muthén

(2013). Facing off with Scylla and Charybdis: A comparison of scalar, partial, and the novel possibility of approximate measurement invariance. Frontiers in Psychology, 4, Article 770. https://doi.org/10.3389/fpsyg.2013.00770

48.

van de Vijver

F. J. R.

Hambleton

R. K.

(1996). Translating tests. European Psychologist, 1(2), 89–99. https://doi.org/10.1027/1016-9040.1.2.89

49.

van de Vijver

F. J. R.

Leung

(2000). Methodological issues in psychological research on culture. Journal of Cross-Cultural Psychology, 31(1), 33–51. https://doi.org/10.1177/0022022100031001004

50.

van de Vijver

F. J. R.

Poortinga

Y. H.

(1997). Towards an integrated analysis of bias in cross-cultural assessment. European Journal of Psychological Assessment, 13, 29–37. https://doi.org/10.1027/1015-5759.13.1.29

51.

van Widenfelt

B. M.

Treffers

P. D. A.

de Beurs

Siebelink

B. M.

Koudijs

(2005). Translation and cross-cultural adaptation of assessment instruments used in psychological research with children and families. Clinical Child and Family Psychology Review, 8(2), 135–147. https://doi.org/10.1007/s10567-005-4752-1

52.

Weis

Smenner

(2007). Construct validity of the behavior assessment system for children (BASC) Self-Report of Personality: Evidence from adolescents referred to residential treatment. Journal of Psychoeducational Assessment, 25(2), 111–126. https://doi.org/10.1177/0734282906293762