Enhancing the Cross-Cultural Comparability of Self-Reports Using the Overclaiming Technique: An Analysis of Accuracy and Exaggeration in 64 Cultures

Abstract

The overclaiming technique (OCT) is a novel way of measuring how socially desirable responding influences survey responses. It has the potential to enhance the cross-cultural comparability of survey data. It allows the identification of respondents’ knowledge accuracy and exaggeration by comparing their assessments of familiarity with existing and nonexisting concepts in a particular field of knowledge. We aim to compare the response patterns of countries and world regions based on their OCT accuracy and exaggeration-index values and validate these OCT scores using external variables. We also introduce a general model for the categorization of respondents based on their OCT indices values. We use the Programme for International Student Assessment (PISA) 2012 data from 64 countries (N = 275,904). We found considerable differences in response patterns across world regions: high accuracy values in East Asia, low accuracy values in Southern and Central America, high exaggeration values in Southern Europe, and low exaggeration values in Western Europe. Furthermore, we show that familiarity with math concepts changes substantially after adjustment using the OCT. The correlation between unadjusted math familiarity and math test scores is weak and nonsignificant (.13) whereas after adjustment the correlation becomes strong and significant (.66). Concerning other indicators such as gross domestic product (GDP), public expenditure in education, and Corruption Perceptions Index (CPI), before adjustment the correlation is negative and even significant for CPI (–.11, –.22, and –.45) whereas after adjustment the correlation becomes, though nonsignificant, positive (.05, .19, and .07). We also discuss the OCT in the context of other methods indicative of culturally preferred scale usage.