Statistically significant differences in culture means may or may not reflect practically important differences between people of different cultures. To determine whether differences between culture means represent meaningful differences between individuals, further data analyses involving measures of cultural effect sizes are necessary. In this article the authors recommend four such measures and demonstrate their efficacy on two data sets from previously published studies. They argue for their use in future cross-cultural research as a complement to traditional tests of mean differences.
Get full access to this article
View all access options for this article.
References
1.
Abelson, R. P. (1997). On the surprising longevity of flogged horses: Why there is a case for the significance test. Psychological Science, 8, 12-15.
2.
Bond, M. H. , & Tedeschi, J. T. (in press). Polishing the jade: A modest proposal for improving the study of social psychology across cultures. In D. Matsumoto (Ed.), Handbook of culture and psychology. New York: Oxford University Press.
3.
Bresnahan, M. J. , Ohashi, R., Liu, W. Y., Nebashi, R., & Liao, C. (1999). A comparison of response styles in Singapore and Taiwan. Journal of Cross-Cultural Psychology, 30, 342-358.
4.
Cohen, J. (1962). The statistical power of abnormal-social psychological research. Journal of Abnormal and Social Psychology, 65, 145-153.
5.
Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). New York: Academic Press.
6.
Cortina, J. M. , & Nouri, H. (2000). Effect sizes for ANOVA designs. Thousand Oaks, CA: Sage.
7.
Dunlap, W. P. (1999). A program to compute McGraw and Wong’s common language effect size indicator. Behavior Research Methods, Instruments, and Computers, 31, 706-709.
8.
Feingold, A. (1992). Sex differences in variability in intellectual abilities: A new look at an old controversy. Review of Educational Research, 62, 61-84.
9.
Feingold, A. (1995). The additive effects of differences in central tendency and variability are important in comparisons between groups. American Psychologist, 50, 5-13.
10.
Feingold, A. , & Mazzella, R. (1998). Gender differences in body image are increasing. Psychological Science, 9, 190-195.
11.
Goodwin, R. , Nizharadze, G., Luu, L. A. N., Kosa, E., & Emelyanova, T. (1999). Glasnost and the art of conversation: A multilevel analysis of intimate disclosure across three former communist cultures. Journal of Cross-Cultural Psychology, 30, 72-90.
12.
Grissom, R. J. (1994). Probability of the superior outcome of one treatment over another. Journal of Applied Psychology, 79, 314-316.
13.
Harris, R. J. (1997). Significance tests have their place. Psychological Science, 8, 8-11.
14.
Hedges, L. V. , & Olkin, L. (1985). Statistical methods for meta-analysis. San Diego, CA: Academic Press.
15.
Henderson, B. B. , Marx, M. H., & Kim, Y. C. (1999). Academic interests and perceived competence in American, Japanese, and Korean children. Journal of Cross-Cultural Psychology, 30, 32-50.
16.
Hunter, J. E. (1997). Needed: A ban on the significance test. Psychological Science, 8, 3-7.
17.
Hunter, J. E. , & Schmidt, F. L. (1990). Methods and meta-analysis. Newbury Park, CA: Sage.
18.
Keltikangas-Jarvinen, L. , Terav, T., & Pakaslahti, L. (1999). Moral reasoning among Estonian and Finnish adolescents: A comparison of collectivist and individual settings. Journal of Cross-Cultural Psychology, 30, 267-290.
19.
Kleinknecht, R. A. , Dinnel, D. L., Kleinknecht, E. E., Hiruma, N., & Harada, N. (1997). Cultural factors in social anxiety: A comparison of social phobia symptoms and taijin kyofusho. Journal of Anxiety Disorders, 11, 157-177.
20.
Loftus, G. R. (1996). Psychology will be a much better science when we change the way we analyze data. Current Directions in Psychological Science, 5, 161-170.
21.
Matsumoto, D. (1999). Culture and self: An empirical assessment of Markus and Kitayama’s theory of independent and interdependent self-construals. Asian Journal of Social Psychology, 2, 289-310.
22.
Matsumoto, D. , & Ekman, P. (1989). American-Japanese cultural differences in intensity ratings of facial expressions of emotion. Motivation and Emotion, 13, 143-157.
23.
McGraw, K. O. , & Wong, S. P. (1992). A common language effect size statistic. Psychological Bulletin, 111, 361-365.
24.
Ohbuchi, K. , Fukushima, O., & Tedeschi, J. T. (1999). Cultural values in conflict management: Goal orientation, goal attainment, and tactical decision. Journal of Cross-Cultural Psychology, 30, 51-71.
25.
Poortinga, Y. H. , Van de Vijver, F. J. R., Joe, R. C., & van de Koppel, J. M. H. (1987). Peeling the onion called culture: A synopsis. In C. Kagitcibasi (Ed.), Growth and progress in cross-cultural psychology (pp. 22-34). Berwyn, PA: Swets North America.
26.
Rosenthal, R. (1991). Meta-analytic procedures for social research. Newbury Park, CA: Sage.
27.
Rosenthal, R. , Rosnow, R. L., & Rubin, D. B. (2000). Contrasts and effect sizes in behavioral research: A correlational approach. Cambridge, UK: Cambridge University Press.
28.
Rosenthal, R. , & Rubin, D. B. (1982). A simple, general purpose display of magnitude of experimental effect. Journal of Educational Psychology, 74, 166-169.
29.
Rosnow, R. L. , & Rosenthal, R. (1996). Computing contrasts, effect sizes, and counternulls on other people’s published data: General procedures for research consumers. Psychological Methods, 1, 331-340.
30.
Shrout, P. E. (1997). Should significance tests be banned? Introduction to a special section exploring the pros and cons. Psychological Science, 8, 1-2.
31.
Singelis, T. M. , Bond, M. H., Sharkey, W. F., & Lai, C. S. Y. (1999). Unpackaging culture’s influence on self-esteem and embarassability: The role of self-construals. Journal of Cross-Cultural Psychology, 30, 315-341.
32.
Takano, Y. & Osaka, M. (in press). An unsupported common view: Comparing Japan and the U.S. on individual/collectivism. Asian Journal of Social Psychology.
33.
Van de Vijver, F. J. R. , & Leung, K. (1997). Methods and data analysis for cross-cultural research. Thousand Oaks, CA: Sage.
34.
Wilcox, R. R. (1997). Introduction to robust estimation and hypothesis testing. San Diego, CA: Academic Press.
35.
Wilcox, R. R. (1998). How many discoveries have been lost by ignoring modern statistical methods?American Psychologist, 53, 300-314.
36.
Wilkinson, L. , The Task Force on Statistical Inference, American Psychological Association Board of Scientific Affairs. (1999). Statistical methods in psychology journals: Guidelines and explanations. American Psychologist, 54, 594-604.
37.
Yrizarry, N. , Matsumoto, D., & Wilson Cohn, C. (1998). American-Japanese differences in multiscalar intensity ratings of universal facial expressions of emotion. Motivation and Emotion, 22, 315-328.
