Not Random and Not Ignorable. An Examination of Nonresponse to Income Question in the European Social Survey,2008–2018

Abstract

This study analyzes the consequences of item nonresponse to the question about a household’s total net income in the European Social Survey (2008–2018). We recognize two mechanisms in avoiding answering the income question: task complexity and question sensitivity, and apply multilevel logistic regressions to predict the probability of refusals or “Do not know” across respondents of different income levels. We find that the refusal to answer the income question is the highest for respondents with lower incomes, while the probability of selecting “Do not know” answers or refusal to answer is the same among respondents with higher incomes. The bias resulting from the correlation between response propensities and household income affects the accuracy of estimates for several attitudinal measures when income is included as an explanatory variable. We recommend reducing the risk of bias by limiting the complexity and sensitivity of the income question and accounting for nonresponse bias.

Introduction

Information about citizens’ income is commonly used in social science research to understand the diversity of opinions and behaviors related to an individual’s socioeconomic status (Golsteyn and Hirsch 2019; Turrell 2000), yet it is challenging to measure in surveys. Past studies outline two main mechanisms contributing to income nonresponse: question sensitivity and task complexity. On the one hand, members of some income groups find the question too sensitive due to the perceived implications of sharing uncomfortable information in a survey interview (Frick and Grabka 2014). On the other hand, the income question often requires performing calculations during an interview, and due to the complex character of the task, members of some income groups might be more likely to reply, “Do not know” (Hansen and Kneale 2013).

Data missingness is unavoidable in surveys (Groves 2006; Kulas et al. 2017); thus, it is not surprising that many surveys also report missing sensitive and complex information on income. Nonetheless, income nonresponse might severely threaten survey quality only if the data missingness is not random (Little and Rubin 2019). The latter occurs when the missingness is directly related to the variables the research intends to measure (Groves 2006). Since the correlation between response propensity and income itself builds the foundation for nonresponse bias (Bethlehem 2002), statistical procedures cannot quickly correct nonrandom missingness (Pleis and Dahlhamer 2003), and the missingness mechanism must be inferred from the auxiliary variables (Giusti and Little 2011).

Our study uses the European Social Survey (ESS) data (rounds 4–9) and demonstrates that the respondents’ propensity to answer a question about total net household income from all sources is related to income level, which signals that income data missingness is not random. Further, we evaluate other variables in this data that are affected by the item nonresponse bias in the income question. We conclude with recommendations for reducing the risk of bias at the survey design and data analysis stages.

Two Mechanisms of Income Nonresponse: Question Sensitivity and Task Complexity

Converse (1976) identified two distinctive forms of item nonresponse: the refusal to answer or a situation in which respondents declare they do not have enough knowledge to provide an answer. “Do not know” responses refer to not knowing or too much cognitive effort when answering the question, whereas refusals are a product of finding the question too sensitive (Alwin and Krosnick 1991; Skelton 1963). Consequently, respondents who cannot calculate their income due to a lack of information may differ from those who refuse to answer due to the sensitive nature of the income question (Locander and Burton 1976; Schräpler 2006).

Findings from previous studies indicate that income nonresponse due to question sensitivity is related to two issues. First, social desirability bias affects item nonresponse for many sensitive survey questions, including income (Kulas et al. 2017). Income is a piece of very personal information; thus, respondents often worry about an interviewer’s reaction (Tourangeau et al. 2010), especially when they fear that their responses are socially undesirable and discomfort or embarrassment might drive item nonresponse in the form of refusal (Tourangeau and Yan 2007). ‘Socially desirable responding’ is more common among people with higher conformity and security values, that is, the conservation measures on the Schwartz scale (Schwartz et al. 1997), as such individuals are less keen to violate social expectations and norms. Second, question sensitivity also correlates with the respondents’ general trust in other people, that is, low-trust respondents are likelier to conceal their incomes than high-trusting ones (D’Hernoncourt and Méon 2012; Kim et al. 2015).

The second item nonresponse mechanism recognized in previous studies is related to the complexity of the task given to the respondents. Here, the respondents are asked to perform a calculation that is too difficult or impossible to perform during an interview, increasing the likelihood of “Do not know” responses. Hansen and Kneale (2013) differentiated between two cognitive processes associated with this nonresponse option—recall and reconciliation. When asked a question, respondents assimilate new information, interpret the question, and retrieve the information required to answer it. If they have problems in the recall stage, they cannot remember information during an interview and are likely to say, “Do not know.”

Another possibility for the “Do not know” response is that respondents remember some or even all information, but the way the information needs to be reported is too difficult. For example, a question might ask about the income of all household members, including employment and other sources, after extracting taxes (Lynn et al. 2006). This is a case of the ESS, in which respondents are requested to add income sources of all household members and fit a final net figure into the provided income brackets. Interviewers show a card with three sets of income brackets, so there is a high volume of numerical information in front of them, increasing task complexity (see Data section). Additionally, the household income question is more challenging for respondents with multiple income sources and those who rely on nonregular jobs and incomes (Frick and Grabka 2014; Pleis and Dahlhamer 2003; Riphahn and Serfling 2005). Households with various income sources have a more complex income situation across the income spectrum; those with lower incomes often have temporary jobs and less regular earnings, while those in higher income brackets might rely on multiple income sources. The complexity of the income situation increases with household size (Kim et al. 2015), which, beyond the numerous income sources, can reflect the complexity of living arrangements (e.g., divorced or blended families) and multigenerational households (Hansen and Kneale 2013).

Income Nonresponse and Response Bias

Many item nonresponses are patterned, meaning that the survey estimator might be biased as a result of who decided not to answer it. For example, when measuring prejudice against immigrants, more tolerant people might be more inclined to answer the question (Herda 2013; Piekut 2021), whereas when asked about political behavior, those who have not voted may avoid admitting their lack of participation in elections (Lahtinen et al. 2019). A similar mechanism patterning response propensities might happen in the case of the income question, in which people of a specific income group may be more likely to refrain from disclosing it.

Previous evidence from the German Socio-Economic Panel revealed that income nonresponse concentrates in the “tails of income distribution”; thus, high and low earners are more likely not to answer the income question (Frick and Grabka 2014). Less conclusive evidence comes from a study based on the British Household Panel Survey (BHPS), which measured the respondents’ gross income of employed individuals (Schräpler 2006). The author found that—compared to middle-status individuals—respondents with higher status were less likely to select the “Do not know” option in wave 1 of the BHPS, while those of lower status were less likely to do so in wave 2. Despite the conflicting findings, both studies agree with the possibility of significant response bias due to income nonresponse.

Our study examines whether income data are missing at random or related to income levels. Income nonresponse in survey data distorts population estimates of social mobility, health inequalities, and wealth distribution (Golsteyn and Hirsch 2019; Hansen and Kneale 2013; Turrell 2000). Thus, recognizing the patterns of income nonresponse and the potential for bias is crucial for methodological reasons, but it is also essential when performing substantial analyses of survey data. Income is the most straightforward measure of socioeconomic status and is highly correlated with other outcomes (Daniele and Geys 2015; Korinek et al. 2006; Lelkes 2006). Hence, misreporting income might lead to distorted conclusions regarding other items when income is included as an explanatory variable (Hariri and Lassen 2017; Lahtinen et al. 2019). To determine whether the bias introduced by item nonresponse, if detected, has implications for further data analysis, we evaluate whether estimates of any other variables in the ESS data are affected.

Data and Methods

Data

This study analyzes data from the ESS, a cross-national survey conducted biennially since 2002 in many European countries (Fitzgerald and Jowell 2010). We restricted our investigation to 18 countries participating in all rounds 4–9, 2008–2018, resulting in a sample of 18 countries, six rounds, and 199,507 respondents (the list of countries with their respective sample sizes presents Figure A1 in Supplementary Online Appendix; henceforth, SOA). The decision to exclude data from the first three rounds was based on the ESS introducing a different method for measuring household total net income in the fourth round in 2008. From round 4, respondents answering a question on income had to assign their household total net income from all sources (after tax and compulsory deductions) to one of 10 categories (expressed in national currency) based on the deciles of the household income distribution in a given country. Interviewers displayed a showcard with approximate weekly, monthly, or annual income. The response options on the showcard corresponded to the income decile groups and were preceded by letters that were not ordered alphabetically. It is worth noting that the two nonresponse options (i.e., refusal to answer and “Do not know”) were not explicitly offered to respondents; however, the interviewers could note one of the options regardless (SOA Section A presents the exact wording of the income question in the ESS rounds 4–9).

Specification of Multilevel Logistic Regression for Predicting the Propensity of Item Nonresponse

We estimated cross-classified multilevel logistic regressions with respondents nested within interviewers and countries. We used R software (R Core Team 2021) and implemented the lme4 package (Bates et al. 2015). Given the technical limitations of the lmer4 package, which does not estimate the regression model in its multinomial form, we followed Silber et al. (2021), arguing that the models specified as multinomial are mathematically equivalent to a set of binary logistic regressions. Thus, we predicted the likelihood of providing the “Do not know” option versus a response and the likelihood of a refusal versus a response as separate exercises of two binary logistic regressions. The model applies a combination of the poststratification and population size weights provided in the ESS data.

We defined the outcome variable (hereafter ${resp}_{ijk}$ ) such that $E\left({resp}_{ijk}=0\right)={\pi }_{ijk,0}$ is the probability that respondent i in country j surveyed by interviewer k will answer a question about household total net income, $E\left({resp}_{ijk}=1\right)={\pi }_{ijk,1}$ is the probability of a refusal, and $E\left({resp}_{ijk}=2\right)={\pi }_{ijk,2}$ is the probability of a “Do not know” response. Thus, our regression models work on a set of responding and nonresponding units to the income question; those who answered the question were assigned the same value, regardless of the income decile they declared. We used a logit link function, where the logit coefficient ${\eta }_{ijk,c}=\mathit{log}\left(\frac{{\pi }_{ijk,c}}{{\pi }_{ijk,0}}\right)$ is the log of the odds of the event ${resp}_{ijk}=c$ as opposed to ${resp}_{ijk}=0$ , where c = 1, 2. For $c\in \left\{\mathrm{1,2}\right\}$ , the model’s specification is as follows

${\eta }_{ijk,c}=\mathit{log}\left(\frac{{\pi }_{ijk,c}}{{\pi }_{ijk,0}}\right)={\beta }_{0,c}+{\gamma }_{j,c}+{\gamma }_{k,c}+{\beta }_{1,c}\times \left({{\gamma }_{1j,c}+HH\_size}_{ijk,c}\right)+{\beta }_{2,c}{\times income\_source}_{ijk,c}+{\beta }_{3,c}\times \left({\gamma }_{3j,c}+{conservation}_{ijk,c}\right)+{\beta }_{4,c}\times \left({\gamma }_{4j,c}+{social\_trust}_{ijk,c}\right)+{\beta }_{5,c}{\times gender}_{ijk,c}+{\beta }_{6,c}\times {age}_{ijk,c}+{\beta }_{7,c}\times {education}_{ijk,c}+{\beta }_{8,c}{\times ESS\hspace{0.17em}round}_{jk,c}+{\beta }_{9,c}\times {Response\hspace{0.17em}rate}_{jk,c}+{\beta }_{10,c}\times {Survey\hspace{0.17em}mode}_{jk,c}$

where ${\beta }_{0,c}$ is the grand intercept, ${\gamma }_{j,c}$ denotes between-country random intercepts, ${\gamma }_{k,c}$ refers to between-interviewers random intercepts, ${\gamma }_{1j,c}$ , ${\gamma }_{3j,c}$ , and ${\gamma }_{4j,c}$ represent random components of between-country variation in the slopes of three continuous covariates (i.e., household size, conservation, and social trust index), and $\beta$ is a vector of regression coefficients on all explanatory and control variables. We assumed that the random effects are mutually independent of each other and that they are normally distributed with a zero mean, such that ${\gamma }_{j,c}\sim N\left(0;{\sigma }_{j,c}^2\right)$ , ${\gamma }_{k,c}\sim N\left(0;{\sigma }_{kc}^2\right)$ , ${\gamma }_{1j,c}\sim N\left(0;{\sigma }_{1j,c}^2\right)$ , ${\gamma }_{3j,c}\sim N\left(0;{\sigma }_{3j,c}^2\right)$ , and ${\gamma }_{4j,c}\sim N\left(0;{\sigma }_{4j,c}^2\right)$ .

The models incorporate variables that previous studies found related to the task complexity mechanism—i.e. (1) household size and (2) the main source of household income—as well as those related to the question sensitivity argument, i.e. (3) conservation (proxy of social desirability) and (4) the social trust index (see section: Two Mechanisms of Income Nonresponse). We used the conservation (vs. openness to change) dimension of basic human values instead of the measure of social desirability responding, as the latter concept is not measured in the ESS. Schwartz et al. (1997) pointed out that conservation items strongly correlate with the scale of social desirability responding. Following previous studies on the item nonresponse analysis, we also added three control variables at the respondent level: (1) gender, (2) age, and (3) respondents’ level of education (e.g., Alexander 2017; Montagni et al. 2019; Piekut 2021; Yan et al. 2010). SOA section A includes the exact wording of all ESS questions used to define all variables included in the regression analysis. The operationalizations are presented in SOA section B, while section C presents descriptive statistics and an additional analysis of the explanatory variables.

Recognizing between-round variation in income measurement, we included the ESS round as a covariate at the survey level and two additional survey characteristics derived from ESS Documentation Reports—overall survey response rate and mode of data collection—as previous studies found item nonresponse conditional on both features (e.g., Klíma et al. 2023; Yan and Curtin 2010). We also found a significant amount of variation attributed to the interviewer and country level; the intraclass correlation coefficient for interviewers in the null model was equal to 0.297 for “Do not know” and 0.309 for refusal, while respective values for countries were equal to 0.050 and 0.238.

Procedures for Evaluating the Missingness Mechanism

Nonresponse bias is challenging to measure without validating survey responses with external administrative records (Hariri and Lassen 2017; Lahtinen et al. 2019). Thus, we indirectly assessed the risk of item nonresponse bias occurrence by examining variation in the estimated—by regression models—propensities to respond “Do not know” or to refuse to answer the income question in the subset of those survey participants who provided the information about their household income. As we know the income category to which the responding unit belongs—assuming the response is true—we used the predicted probabilities to evaluate the distribution of nonresponse propensities across the income deciles.

Following Schouten et al. (2009), we assumed that if estimated propensities for respondents are equal on average in the deciles of declared income and if the response of a particular respondent is independent of all other answers to the income question, then the missing data mechanism could be treated as completely at random. In such a case, item nonresponse reduces the sample size and increases the estimates’ variance; however, if response propensities are not connected with income level, it signals a low risk of bias in a group of responding units (see Groves 2006). By contrast, if the income itself is the cause of the response propensity, then the survey question and response propensity are correlated (the mean propensities to respond are not equal across income deciles), and the missingness leads to nonignorable bias (consult Little and Rubin 2019; Yan et al. 2010).

We recognize, however, that our analytical strategy introduces a notable limitation to the study, as the examination of the distribution of nonresponse propensities is restricted to respondents who answered the income question (we did not investigate income distribution among units nonresponding to the question). Nonetheless, suppose we find that the distribution of response propensities is not equal across declared deciles. In that case, this signals serious measurement quality issues caused by item nonresponse, even if we do not know the income distribution of the nonresponding units.

Assessing the Impact of Income Nonresponse on Other Survey Measures

When the missingness mechanism is not random, income-missing cases may distort the income distribution and diminish the quality of other survey measures. The latter might occur when researchers include income as an explanatory variable, and missing income cases are excluded from the analysis, even if respondents provided valid answers to a survey question of their main interest. Thus, besides analyzing the income nonresponse mechanism, we also assessed the size of the bias introduced to the mean values of the ESS variables when cases with missing income were dropped from the analysis. Following Bethlehem (2002), we used the estimated propensities of nonresponse occurrence and approximated the size of bias of the mean value of a selected ESS variable $Y$ , caused by income nonresponse options ( $c=1,2$ ), as follows

$B_c\left(\overline{Y}\right)\approx {\left(1-{\overline{\pi }}_{ijk,c}\right)}^{-1}Cov\left(\left(1-{\pi }_{ijk,c}\right),Y\right)$

where ${\overline{\pi }}_{ij,c}$ is a mean predicted propensity of income nonresponse occurrence, and $Cov\left(\left(1-{\pi }_{ijl,c}\right),Y\right)$ is a covariance between ${\overline{\pi }}_{ijk,c}$ and observed values of variable $Y$ in a group of respondents answering the income question and providing answers to a question related to variable $Y$ . The non-zero covariance signal bias was introduced (in the mean values of variable $Y$ ) by the data missingness for the income question. Values of $B_c\left(\overline{Y}\right)$ above 0 indicate that the decision to include income as a covariate of $Y$ will result in overestimating the average value of $Y$ ; similarly, values below 0 indicate that the decision underestimates the average value of $Y$ .

To assess the statistical significance of $B_c\left(\overline{Y}\right)$ introduced by refusals and “Do not know” responses, we calculated t-values as a standardized difference between (a) the Horwitz and Thomson (1952) estimator of the mean value of $Y$ (hereafter, ${\overline{y}}_{HT}$ ) weighted by design weights, and (b) the modified Horwitz–Thomson estimator (hereafter, ${\overline{y}}_{HT}^{*}$ ; see Bethlehem 2002:276) weighted by the product of design weights and the inverse estimated (by regression models) propensity of answering the income question. The standardized difference is as follows

$t_c\left(\overline{Y}\right)=\frac{{{\overline{y}}_{HT}-\overline{y}}_{HT}^{*}}{SE\left({\overline{y}}_{HT}\right)}\sim N\left(\mathrm{0,1}\right)$

where $SE\left({\overline{y}}_{HT}\right)$ is the standard error of ${\overline{y}}_{HT}$ , and all remaining components of the equation stand as previously defined. We also assessed the effect size of the differences by calculating Cohen’s d value (Cohen 1988).

Results

Before we move to the main results, we briefly summarize income nonresponse patterns for the control variables (see SOA Tables A6 and A7). “Don’t know” responses were more common among larger households, those having additional income sources (like investments) or self-employed, and those with lower levels of formal education. Meanwhile, those more prone to refuse to answer the income question were those in larger households (but the effect of household size for refusals is smaller compared to the effect for “Don’t know” responses); those who had additional income sources (e.g., investments; again, the effect is smaller compared to those for “Don’t know” responses); those who were self-employed, as well as pensioners; and those with the lowest and the highest education, higher conservation values, and lower social trust. Thus, part of our results stands in line with the question sensitivity argument (conservation and social trust correlate with refusals and not with “Don’t know” responses), and another supports the task complexity argument (household size and source of income are related to “Don’t know” responses, even if both covariates also significantly correlate with refusals).

Nonrandom Missingness of Household Income Data

Turning now to the main results of our investigation, Figure 1 presents the mean value of the predicted probability of refusals and “Do not know” occurrence with 95% confidence intervals for the mean for each income decile. We used predicted propensities of income nonresponse occurrence for those who answered the income question; thus, we knew which income decile they indicated.

Figure 1.

Mean values of predicted probability for income nonresponse by deciles of the household’s total net income. Note: Number of countries 18; number of observations for predicting: Don’t Know = 173,268; Refusals = 178,791.

As the propensity to report income is related to the amount of the household’s income, there is a risk that data missingness due to refusals to answer and “Do not know” responses is not random. Interestingly, the relationship is nonlinear and in the opposite direction for both types of nonresponses. The correlation between income deciles and the refusal option is negative (i.e., refusal propensity decreases along the income deciles). Further, it is generally positive for not knowing (i.e., the propensity of stating “Do not know” slightly increases with income). For the refusals, there is an apparent discrepancy in the propensity of item nonresponse between respondents living in households with a total net income above or below the median income value. In turn, the difference in response propensities across deciles is lesser for the “Do not know” responses. Thus, the risk of bias introduced by providing a “Do not know” answer is smaller than the risk related to refusals.

The analysis demonstrates that the lower-income population will most likely refuse to answer the income question. The probabilities of obtaining a “Do not know” answer and a refusal are the same for the 10th income decile. These results suggest that respondents from lower-income households find the question more sensitive than those with higher incomes. However, both patterns operate for people on the higher tail of the income distribution: the wealthiest respondents find the question similarly sensitive and complex. Although these results align with findings from previous studies demonstrating that low-earning respondents are less likely to provide information about their income (Frick and Grabka 2014; Hariri and Lassen 2017), our study provides evidence for the existence of nonrandom mechanisms of missing data in the income measurement in the ESS.

Impact of Income Missingness on the Accuracy of Data Analysis

As income remains a crucial determinant of an individual’s attitudes and a standard correlate of other variables in social science survey research, there is a possibility that income nonresponse (which is not random) will affect the accuracy of any empirical analysis when income is incorporated as an explanatory variable. Notably, the risk of bias exists when the dependent variable’s values differ significantly between respondents who did not answer and those who answered the income question, as the latter is excluded from the analysis due to missing data.

Table 1 contains 10 ESS core module items (all measured on an 11-point scale ranging from 0 to 10) with the highest $B_c\left(\overline{Y}\right)$ values, introduced by refusals and “Do not know” answers. Lists of all items included in the evaluation of $B_c\left(\overline{Y}\right)$ stand in replication codes for Tables A8 and A9 in SOA.

Table 1.

The ESS Core Module Items with the Highest Bias Introduced by Refusals and “Do Not Know” Answers to the Question Measuring Household’s Total Net Income.

Bias introduced by refusals	Bias	t-value	Cohen’s d
1) stfhlth: State of health services in country nowadays	0.049	11.08***	0.03
2) trstlgl: Trust in the legal system	0.045	10.30***	0.03
3) ppltrst: Most people can be trusted or you can’t be too careful	0.044	11.22***	0.03
4) trstprl: Trust in country’s parliament	0.043	9.16***	0.03
5) imueclt: Country’s cultural life undermined or enriched by immigrants	0.042	9.46***	0.03
6) pplfair: Most people try to take advantage of you, or try to be fair	0.041	11.14***	0.03
7) stfeco: How satisfied with present state of economy in country	0.040	8.92***	0.03
8) trstplt: Trust in politicians	0.036	8.89***	0.03
9) stfdem: How satisfied with democracy in country	0.032	6.31***	0.02
10) trstplc: Trust in the police	0.031	6.45***	0.02

Bias introduced by “Do not know” answers	Bias	t-value	Cohen’s d
1) trstep: Trust in the European parliament	−0.016	−4.92***	−0.01
2) polintr: How interested in politics	−0.012	−7.32***	−0.02
3) health: Subjective general health	0.010	7.31***	0.02
4) pplfair: Most people try to take advantage of you, or try to be fair	0.008	2.87**	0.01
5) stfeco: How satisfied with present state of economy in country	0.008	3.24**	0.01
6) ppltrst: Most people can be trusted or you can’t be too careful	0.007	2.38*	0.01
7) trstun: Trust in the United Nations	−0.007	−1.59n.s	0.00
8) trstlgl: Trust in the legal system	0.006	1.79n.s	0.01
9) trstplc: Trust in the police	0.006	1.94n.s	0.01
10) stflife: How satisfied with life as a whole	−0.005	−1.58n.s	0.00

Notes: ***p-value <.001; **p-value <.01; *p-value <.05; ^n.s. nonsignificant. Number of countries = 18; Number of observations for predicting: Don’t Know = 173,268; Refusals = 178,791.

For all 10 ESS core variables with the highest bias introduced by refusals (upper panel), the bias is significant. By contrast, for variables with the highest bias introduced by replying “Do not know” (lower panel), the bias is significant for 6 out of 10 items with the highest bias. Moreover, for refusals, the biases and t-values are at least twice (or even threefold) as high as for “Do not know” answers, which indicates the consequences of refusing to answer the income question are much more severe than those introduced by providing a “Do not know” response. The finding aligns with the data presented in Figure 1, which shows that between household income deciles, the variability in propensities of refusing to answer the income question is greater than in the propensities of answering “Do not know.” Note, however, that the values of Cohen’s d effect sizes are minor for respective variables, despite the significant value of bias.

Discussion

The analysis based on the ESS revealed that item nonresponse to the household net total income question is not random. The propensities to not answer the question vary across income deciles and differ between refusals to answer and “Do not know” responses. While the propensity for refusal decreases across the income deciles, it slightly increases for not knowing. We observe a higher risk of nonresponse bias resulting from the refusal to answer the income question among respondents with lower incomes, indicating that for this group, the question is particularly sensitive (Tourangeau et al. 2010). Interestingly, there is the same probability of obtaining “Do not know” answers or refusals among respondents with the highest income. Hence, for wealthier respondents, the cognitive response process during a survey interview and, therefore, the difficulties in recalling and reconciling necessary information (Hansen and Kneale 2013) are an equal cause of income nonresponse.

Our study demonstrates that to minimize the risk of item nonresponse bias when measuring income, researchers should focus on respondents’ concerns about providing sensitive information and reducing task complexity. First, respondents could be reminded about the confidentiality of the data collection process before being asked the income question. Although participants use letters linked to different income deciles, the order of letters is fixed, so they might suspect that interviewers can connect them with low incomes. In the case of face-to-face, computer-assisted interviews, it would be better to use random letters or allow self-completion for the income item. If respondents still do not reply, alternative measurement options, such as unfolding brackets, could be used (e.g., Wang 2010). Furthermore, as calculating “net total household income” requires time, splitting the question into a few less cognitively demanding items (i.e., corresponding to different income sources) could also improve the accuracy of income measurement. Consequently, the burden of calculating the total net figure is moved from the respondents to the researchers.

The bias resulting from the correlation between response propensities and household income has implications for the accuracy of estimates for several attitudinal measures. One possible way of handling data to account for income nonresponse bias is the implementation of modified Horvitz–Thomson estimators (Bethlehem 2002), which, beyond selection probabilities, also consider respondents’ propensities to answer the question. Even if the propensities are unknown and must be estimated from auxiliary variables (as we did), they may be incorporated into the data analysis as a component of weights. This approach was suggested in methodological studies addressing the negative consequences of unit nonresponse bias (Iannacchione 2003; Kim and Kim 2007); however, its extension to an item nonresponse issue seems reasonable. In the case of missing data imputation, we recommend adding the four variables we identified as key drivers of task complexity and question sensitivity mechanisms: household size, household structure, social trust, and conservation values.

Noticeable limitations of our study stem from the fact that we exclusively rely on data provided in the ESS. Hence, we could not account for possible income misreporting in estimating nonresponse bias. For instance, due to socially desirable reporting, some participants could decide to select a different than actual income bracket. Furthermore, if survey respondents did not know their household income, they might have been guessing the answer. Future studies could replicate our analysis with surveys where linkage with external data on household income is possible. Additionally, the analysis could be repeated based on other cross-national comparative surveys, where the income question is differently worded and measured in a way which allows for controlling question design issues. Finally, a dataset with a more comprehensive set of variables corresponding to the two tested mechanisms of avoiding answering income question, such as any income change in the last year, household composition change, or a more direct measure of social desirability responding, would lead to more robust results.