Sage Journals: Discover world-class research

Abstract

A major challenge in web-based cross-cultural data collection is varying response rates, which can result in low data quality and non-response bias. Country-specific factors such as the political and demographic, economic, and technological factors as well as the socio-cultural environment may have an effect on the response rates to web surveys. This study evaluates web survey response rates using meta-analytical methods based on 110 experimental studies from seven countries. Three dependent variables, so-called effect sizes, are used: the web response rate, the response rate to the comparison survey mode, and the difference between the two response rates. The meta-analysis indicates that four country-specific factors (political and demographic, economic, technological, and socio-cultural) impact the magnitude of web survey response rates. Specifically, web surveys achieve high response rates in countries with high population growth, high internet coverage, and a high survey participation propensity. On the other hand, web surveys are at a disadvantage in countries with a high population age and high cell phone coverage. This study concludes that web surveys can be a reliable alternative to other survey modes due to their consistent response rates and are expected to be used more frequently in national and international settings.

Keywords

response rates cross-national cross-cultural web surveys meta-analysis

Survey Response in a Cross-Cultural Setting

In an increasingly globalized world, cross-national research and thus cross-national datasets have grown in popularity in academic research. At the same time, concerns about methodological issues related to cross-national surveys have been growing. One such methodological concern is the differences in response rates across countries. Using data from large, representative samples of consumers from 11 countries of the European Union, Baumgartner and Steenkamp (2001) found that survey participants' response styles had systematic effects, in particular with respect to the proportion of reverse-scored items and midpoint scores. They also found that these style differences could result in varying response rates across countries.

Given the cost-intensive and inflexible nature of face-to-face surveys, there has been an international shift toward using web-based cross-cultural surveys for data collection (e.g., GfK consumer panel¹, CRONOS², and OPPA³). Using web-based data collection methods poses methodological challenges such as non-response bias, with systematic non-response leading to errors (Bethlehem, 2010; Groves & Peytcheva, 2008; Kreuter, 2013). Although the non-response rate (percentage of non-response) is not equal to non-response bias⁴, these two concepts are strongly related (Groves & Peytcheva, 2008).

While previous cross-cultural research on non-response has often focused on comparing response rates, (De Leeuw et al., 2018; Hox and De Leeuw, 1994; Johnson et al., 2020) it has not examined related non-response bias for feasibility reasons (i.e., response bias analyses require the characteristics of a sample to be known before it is recruited, as demonstrated by Jans et al. (2018). However, as low response rates are likely to result in more systematic non-response bias than high response rates, comparing bias across studies with varying response rates from different countries can be problematic⁵.

The literature on factors that influence non-response at the micro-level is diverse and well explored. Focusing on web response rates, Keusch (2015) provides a theoretical overview of the correlates of web response rates and shows that, at the micro-level, survey design attributes—such as reminders, prenotifications, and timing of invitations—improve web response rates. Keusch (2015) found that sample characteristics (e.g., female, non-Black, topic interest) can also affect web response rates. Similarly, Daikeler et al. (2019) found higher web response rates to web surveys using prenotifications, one and two contact attempts, longitudinal samples, and specific target groups. Previous research comparing modes of data collection (e.g., Daikeler et al., 2019; Lozar Manfreda et al., 2008) found that the response rates to web surveys were on average 12 percentage points lower than for the comparison survey mode, that is, the other survey mode to which web surveys were compared (face-to-face, mail, or telephone). However, these studies did not fully explain the response rate heterogeneity within and across countries. This could be because they only included survey design factors as explanatory variables and marginally considered cross-national differences on the country level, even though international databases were used.

Differences in cross-national and cross-cultural non-survey design factors may also influence response rates. Groves & Couper (2012) implicitly addressed this possibility in their non-response framework for household surveys, which points out that social environment can be a source of non-response. Blom (2012) explicitly addressed the sampling unit representing macro-level factors, such as a country’s gross domestic product (GDP) or the culture in which a survey is conducted, in her conceptual model of country-level contact rates. However, her analysis focused on survey characteristics, and she did not test for macro-level factors. While many research studies have examined survey design factors and sample characteristics, research on the differences in cross-country non-response is still in its infancy (Johnson et al., 2010). Previous research in this area has focused on cultural dimensions and their link to non-response in interviewer-administered surveys (Jans et al., 2018; Johnson et al., 2010) and on cross-cultural dimensions of internet consumption (Hermeking, 2005). To date, macroeconomic and country-level factors have mostly been addressed only in theoretical models. To the best of our knowledge, these factors have never been examined in the context of web non-response.

Our study addresses this research gap related to the differences in response rates between countries⁶ with two goals in mind. The first goal is to identify countries in which web surveys provide a valuable alternative as a survey mode. The second goal is to understand the role played by country-specific indicators—such as politics and demographics, economics, technology, and socio-culture—in the response rates to web surveys in comparison to other survey modes. Collectively, this will help us to determine why web surveys work better in some countries than others. Our study uses the strongest methodology available for comparative research, which is a meta-analysis based solely on experimental studies (APA, 2006; Vandenbroucke, 1998). Thus, our study is a cross-national extension of previous research by Daikeler et al., 2019; Lozar Manfreda et al., 2008. Specifically, we focus on experimental web mode comparisons and analyze the response rates to web surveys, the response rates to the comparison mode, and the difference in the two response rates.

Country-Specific Predictors of Web Survey Response Rates

To derive cross-cultural factors that may be relevant for the response rate to web surveys, we drew on the macroeconomic PEST⁷ (political, economic, socio-cultural, technological) approach (Cadle et al., 2014, p. 3). The PEST approach covers a wide range of topics and thus serves as a basis for our analytic model. After some adaptations, we derived four country-specific macro factors that might influence country-specific web response rates (see Figure 1).

Figure 1.

Macro-level factors for web response.

Political and Demographic Factors

The country-specific political and demographic environment affects the acceptance and success of web surveys. The relative performance of web surveys in comparison to other survey modes depends largely on the perceived burden of web surveys (Crawford et al., 2001). The lower the perceived participation burden for a web survey, the more likely respondents are to participate. We expected education and age to be two key determining factors regarding the perceived burdensomeness of participating in a web survey (De Leeuw, 2018; Yetter & Capaccioli, 2010). With respect to educational attainment, we expected that the higher the level of education in a country, the more the use of web surveys would be accepted by the public. More specifically, higher education is often linked to a higher cognitive ability (Falch & Sandgren Massih, 2011), which is thought to reduce the effort of operating an online device and thus facilitate the completion of a survey online (Gordon et al., 2019). In contrast, we did not expect education to have a similar effect on the response rate of other survey modes. Therefore, we assumed that the higher the level of education in a country, the lower the difference in response rates between survey modes.

Furthermore, we expected web surveys to be less accepted in countries with an aging population (Czaja et al., 2006; Gordon et al., 2019) because older people are often less open-minded about new developments and find it harder to learn new skills (Charness & Boot, 2009; Czaja et al., 2006). Hence, older populations often do not have the basic computer skills necessary to complete web-based surveys. Thus, we expected higher response rates of comparison survey modes in countries with a demographically older population, but lower response rates to the web survey mode. This would result in a large difference in response rates, with higher rates for the comparison survey modes.

In addition, we expected web survey response rates to be higher than response rates for other survey modes in countries with high population growth and a young population. Young people are more open to new ideas and use the internet as part of their everyday life. Thus, they generally have greater digital literacy (Czaja et al., 2006; Fang et al., 2009) than older respondents. For example, during the Arab Spring, the internet played a central role as a communication medium for young protestors (Howard et al., 2011). Moreover, the response patterns of young adults vary across survey modes. They have higher response rates for web-based surveys than for other modes of data collection. This means that it may be more difficult to have a representative sample if only traditional survey modes, such as paper-and-pencil, are used. While the low proportion of older people in a country can be strongly correlated to population growth, this assumption is not always the case (Fehr et al., 2008). In this respect, we expected that the higher the population growth in a country is, the higher the web response rate would be within that country. We also assumed that this link between population growth and the response rate to web-based surveys would lower the response rates to the comparison survey modes, which should minimize the differences between the survey modes. All of our hypotheses are summarized in Table 1.

Table 1.

Hypotheses Regarding the Political and Demographic, Economic, Technological, and Socio-Cultural Factors Influencing Web Response.

Consequence for		Web response rate	Comparison mode⁸ response rate	Response rate difference
Political and demographic factors
H1 Education	The higher the overall level of education in a country, the more frequently web surveys are used in that country. This should have no effect on the comparison survey modes and decrease the difference in response rates.	Higher	No effect	Smaller
H2 Age	Web surveys are less frequently used in countries with an aging population. Older populations prefer comparison modes, which should increase the difference in response rates.	Lower	Higher	Larger
H3 Population growth	Countries with a high population growth have high web survey response rates compared to other survey modes, as young people prefer web-based surveys. This should result in a smaller difference in response rates.	Higher	Lower	Smaller
Economic factors
H4 GDP	Countries with a high GDP have high web survey response rates as people in these countries are more familiar with using the internet and technology in general. In contrast, there should not be an effect on the comparison modes, which should result in a smaller difference in response rates.	Higher	No effect	Smaller
Technological development factors
H5 Internet coverage	Countries with higher internet coverage have high web survey response rates. This should have no effect on the comparison modes and should reduce the difference in response rates.	Higher	No effect	Smaller
H6 Internet users	The more the internet is used, the higher the web survey response rates. This should have no effect on the comparison modes and should decrease the difference in response rates.	Higher	No effect	Smaller
H7 Cellphone coverage	In countries with broad cell phone network coverage, we expected high web survey response rates. This should have no effect on the comparison modes and should decrease the difference in response rates.	Higher	No effect	Smaller
Socio-cultural factors
H8 Response climate	The higher the response rates to previous surveys in a country, the higher the web survey response rates. This should have no effect on the comparison modes and should reduce the difference in response rates.	Higher	No effect	Smaller

GDP: gross domestic product.

Economic Factors

The wealth of a country plays an essential role in the participation in web surveys compared to other survey modes. In countries with a high level of wealth, all population strata supposedly have internet access (Van Dijk, 2006). As a result, internet usage is socially desirable and embedded in everyday life, and large parts of the population have the necessary skills to use it (Galperin & Fernanda Viecens, 2017). Furthermore, in countries with a high level of prosperity, residents often have a job that requires them to be able to use computers or other internet devices. The general population within these countries are familiar with and use technology every day, and therefore the perceived burden of internet use is minimal (Garrote Sanchez et al., 2021). Thus, for countries with a relatively high GDP, we expected participation in web surveys to be more accepted by the population and there to be a smaller difference in the response rates compared to other survey modes.

Technological Development Factors

Another important factor that may influence the web survey response rate in a specific country is the degree of technological advancement (e.g., Bosnjak et al., 2005; Couper, 2000; Couper et al., 2007; Rookey et al., 2008; Silber et al., 2018). The higher the internet coverage in a country, the easier it is for the population to use it regularly. As a result, people develop web skills, and the burden of participating in a web survey is reduced (Van Deursen & Van Dijk, 2011; Galperin & Fernanda Viecens, 2017). The higher the willingness to participate in a web survey, the lower the difference in response rates between a web survey and other survey modes. The same applies to the proportion of internet users in a population. The more popular the internet is among various social strata, the more likely people are to use it in their everyday life (Teo et al., 1999). Also, the more the internet is used, the lower the burden of using it to answer a survey. This may lead to higher web survey response rates and a lower difference in response rates because the response rates for comparison survey modes should not be affected by the number of internet users.

Finally, increased cell phone network coverage in a country is expected to have a positive effect on web survey response. The provision of mobile internet access throughout a country means that geographical or time limits no longer affect a survey participant’s ability to respond to a web survey (Wright, 2005). On the other hand, for other survey modes, such as face-to-face and mail surveys, these constraints still impact response rates. Therefore, in countries with broad cell phone network coverage, we expected high web survey response rates and a low difference in response rates compared to other survey modes.

Socio-Cultural Factors

Comparative studies have investigated the decreasing response rates to different survey modes in many countries (Atrostic et al., 2001; Brick & Williams, 2013; Curtin et al., 2005; Kreuter, 2013; Rogers et al., 2004; Williams & Brick, 2018) to determine the factors that impact response rates, such as the country-specific survey climate and response propensity (e.g., Barbier et al., 2015; Beullens et al., 2018). One indicator of the acceptance of surveys in a country might be the willingness of its citizens to participate in surveys of any mode (Brüggen et al., 2011; Keusch et al., 2014; Petrova et al., 2007). Unwillingness to participate in a national survey might be fueled by data protection concerns (Gummer & Daikeler, 2018). These concerns may be stimulated by the media, word-of-mouth, or norms. They could also be related to the legislative history of a country (e.g., state security [Stasi] in the GDR). Following this argument, we assumed that the higher the willingness of citizens to participate in previous surveys of any mode in a country (influenced, for instance, by low data protection concerns and a positive attitude toward surveys), the higher their participation in web surveys. This may be because people with positive survey attitudes and low data protection concerns may consider web surveys to be a more convenient and less burdensome way of participating in surveys. Consequently, a positive survey climate may positively impact response to web surveys (Loosveldt & Joye, 2016; Porter et al., 2004). Thus, in countries with high response rates in previous surveys, web surveys should have similarly high response rates, and lower differences in response rates between survey modes can be expected. An overview of all hypotheses is provided in Table 1.

Survey Design Predictors of Web Survey Response Rates

The success of a (web) survey, which is contingent on a favorable response rate, depends strongly on the survey setting and characteristics (Groves & Peytcheva, 2008; Keusch, 2015). The selection of the survey mode not only influences the costs of a survey but the mode’s attributes can also impact the response rate. For example, a paper questionnaire on a desk can act as an automatic reminder for participants, or a personal contact with a telephone interviewer can convince a reluctant respondent of the importance of a survey. In addition, a survey’s methodological toolbox offers a variety of other means for increasing response rates. Incentives can compensate respondents for their time spent; respondents who are pre-recruited from longitudinal surveys are more likely to participate in follow-up studies, and repeated contact attempts can underline the value of a survey. To date, three meta-analyses by Daikeler et al., (2019); Lozar Manfreda et al., 2008 and Shih & Xitao Fan, 2008 have examined these tools and their effectiveness in boosting response rates. They found that prenotifications, one and two contact attempts, the use of longitudinal samples, and specific target groups, such as students, increased the response rate to web surveys in comparison to other survey modes. Notably, the meta-analytic studies showed that incentives are equally effective for all survey modes.

The Present Study

The present study addresses the question of cross-country differences in web survey participation behavior, building on previous studies, such as the study of Shih & Xitao Fan (2008) who found a large part of the heterogeneity in effect sizes remains unexplained. Daikeler et al., 2019; Lozar Manfreda et al. (2008) followed a similar meta-analytic approach, primarily focusing on a survey’s methodological characteristics (such as the use of incentives). Using meta-analytic methods, our research strove to identify the countries in which web surveys had favorably high response rates compared to other survey modes and the country-level indicators that determined the response rates. Thus, we examined whether political and demographic, economic, technological, and socio-cultural country-specific factors influenced the success of web surveys, focusing on three effect sizes: the response rate to web surveys, the response rate to the comparison survey mode, and the difference in response rates between the two. Our findings may provide helpful information for researchers who aim at evaluating whether a web survey is likely to be a successful mode of data collection in a specific country or cross-nationally.

Method

The present study uses the study of Lozar Manfreda et al. (2008) as a starting point and supplements it with more recent studies (see Daikeler et al., 2019).

Measurement of Political and Demographic, Economic, Technological, and Socio-Cultural Factors

To investigate whether political and demographic, economic, technological, and socio-cultural country-specific factors impact survey response rates, we added country-specific information to our dataset. These supplementary data were based on the operationalization of political and demographic, economic, technological, and socio-cultural factors. The country-level indicators were selected with the aid of two criteria. First, to maintain high-quality standards, only indicators referring to the respective country in the exact year were used. Therefore, the linked political and demographic, economic, technological, and socio-cultural data had to cover a period of more than 20 years on a yearly basis. Second, the country indicators had to include all countries presented in this study. After extensive literature and database research, we selected country-level data from the World Bank, the Organisation for Economic Co-operation and Development (OECD), and the International Social Survey Programme (ISSP) databases.

As a proxy for the political and demographic status of a country, we used its average education, population growth, and proportion of people aged over 65 (see Table 2). We operationalized the economic status of a country with its GDP. We measured the country-specific degree of technology based on a country’s internet and cell phone coverage as well as on its proportion of internet users. To map the socio-culture of a specific country, we examined a variety of factors. First, we determined whether the survey response rate differences were significantly influenced by the web survey response rate or the response rate of the comparison survey mode. In addition, we included the aggregated survey response rates at the country level from the last 5 years and added the response rate to the ISSP in the respective publication year to reflect the survey climate of a country. Table 2 provides an overview of the sources of this additional information.

Table 2.

Overview of the Country-Specific Predictors of Web Survey Response Rates.

Factor	Variable	Source	Description
Political and demographic factors	Education	World bank	Education index
	Annual population growth	World bank	Annual population growth in a country by year and country
	Population aged 65 and over	OECD	The elderly population is defined as the share of people aged 65 and over (vs. the working-age population—15–64 years) by year and country
Economic factors	GDP	OECD	Gross domestic product (GDP) at market prices is the expenditure on final goods and services minus imports by year and country
Technological advancements	Internet coverage	World bank	Individuals using the internet (% of the population) by year and country
	Cellphone coverage	World bank	Mobile cellular subscriptions (per 100 people) in the respective publication year and country
	Internet users in %	World values survey	Frequency of internet use (daily, weekly, monthly, less than monthly, never) by year and country
Socio-cultural factor	Web response rate	Calculated: Ratio invited versus participated	Primary study (in %)
	Other mode response rate	Calculated: Ratio invited versus participated	Primary study (in %)
	Country-level aggregated web response rate	Calculated: Mean response rate for respective country	Country-level 5-year aggregated value of current paper (in %)
	Country-level aggregated other mode response rate	Calculated: Mean response rate for respective country	Country-level 5-year aggregated value of current paper (in %)
	ISSP response rate of publication year	ISSP	Response rate by publication year and country of the last ISSP round (in %)

GDP: gross domestic product; OECD: organisation for economic co-operation and development.

The indicators for internet coverage, cellphone coverage, GDP, and persons aged over 65 were highly correlated (see the correlation map in Supplemental Figure A2).⁹ In the meta-analyses, inter-item correlation was not found to be problematic because we calculated separate models for each factor, which is a standard approach used in meta-analyses to prevent overfitting and facilitate interpretation (Harrer et al., 2019; Viechtbauer, 2007).

Overview of Meta-Analytic Procedure

Our meta-analysis comprised four steps. First, we conducted a comprehensive literature search using specific search terms. Second, we compared the studies identified by this literature search with our eligibility criteria. We excluded studies that did not meet our criteria. Third, we coded the data needed to calculate the response rates and the survey country. Fourth, we carried out the moderator analysis. Each of these four steps is explained in the following sections.

Eligibility Criteria and Search Strategy

The eligible studies had to meet the following criteria: (1) A split-sample experimental design had to be performed on subjects from the same population who were randomly assigned to different survey modes. (2) One of the survey modes had to be a web-based survey (i.e., a survey using a web questionnaire to collect respondents' answers online, which were entered using a PC or laptop; we excluded mobile-only studies). This web-based survey had to be compared with data from at least one other survey mode (i.e., mail, telephone, face-to-face, or fax surveys). (3) Data had to be available on the response rates to the web and other survey modes as well as on the survey country: that is, the country in which the survey was conducted. (4) Survey participants had to remain in the mode to which they were randomly assigned; that is, studies in which participants could change modes were not eligible. (5) The implementation of the compared survey modes had to be identical, meaning that the survey methodological tools needed to be identical; for example, both modes had to use incentives. We did not impose restrictions related to the population of participants, time period, or geography. Thus, we included studies in our meta-analysis regardless of the respondent population (e.g., student surveys) used in the study and also of the country where the study was conducted. As a first important step to ensure the quality of our meta-analysis, we conducted a comprehensive literature search (see Figure 2 for an overview of our systematic search strategy) using the following search terms: web survey, internet survey, online survey, web-based survey, internet-based survey, electronic survey, which we supplemented with response rate, return rate, participation rate, and non-response rate.

Figure 2.

Search strategy

Using the above-mentioned search terms, we searched a list of search engines (see Figure 2). To overcome the publication bias problem (Rosenthal, 1979), we employed several techniques, such as including conference presentations and grey literature in our search. In addition, with the aid of a snowballing technique, we searched the reference lists of the selected publications. In addition, to explicitly collect grey literature, we examined conference abstracts from several relevant conferences (see Figure 2) from the years 2005–2016.

After the literature search and removal of duplicates, 142 manuscripts were identified for screening, of which 75 manuscripts matched our inclusion criteria. In these 75 manuscripts, we identified 110 studies that compared web response rates experimentally to another survey mode (see PRISMA flow chart in Supplemental Figure A1 for more information). These effects were included in our analysis (see Supplemental Table A4 for an overview). The coding reliability between the two independent coders was 92% (Krippendorff, 2004). Discrepancies in coding were resolved in a discussion involving the two coders.

Statistical Methodology and Effect Sizes

Our effect sizes were the web response rate, the response rate to the comparison survey mode, and the difference between the two response rates. Accordingly, we calculated the number of invited and eligible subjects for each mode and compared the results to obtain the difference in response rates. However, raw frequencies were essential for calculating the confidence interval for each effect size. We used the primary study authors’ definition of the response rate and calculated the raw frequencies for cases where insufficient data were provided. We created a dummy variable based on whether the authors provided raw frequencies or response rates. A robustness analysis did not reveal a significant moderation effect of the dummy variable on the average difference in response rates. We calculated the three effect sizes as follows

d_{Web} = \frac{N_{web}}{I_{web}}

(1)

d_{Comparison Mode} = \frac{N_{other}}{I_{other}}

(2)

d_{RDD} = \frac{N_{web}}{I_{web}} - \frac{N_{other}}{I_{other}}

(3)

note: with N_web=Web respondents; I_Web =Number of invited/eligible subjects for web mode; N_other=Comparison mode respondents; I_other=Number of invited/eligible subjects for comparison mode.

The interpretation of the web and comparison mode response rates is intuitive; a positive response rate difference (d_RRD) means that the response rate for the web mode is higher than for the other modes, and a negative response rate difference means that the response rate is lower for the web mode. Our three effect sizes are closely linked and can be derived from each other. Nevertheless, we decided to report all three effect sizes, as further analyses showed that 32% of the heterogeneity in effect sizes (σ²) at the country level (effect sizes clustered in countries) could be explained by the response rate to the comparison mode (see Supplemental Table A1). Therefore, it was also important to consider the response rate to the comparison mode in order to assess whether web surveys could be the more suitable survey tool for a specific country.

In general, our statistical analysis comprised four steps (see Lipsey & David, 2001). First, we calculated the weighted average response rate difference and the confidence interval per country by weighting each effect size with the inverse value of its variance. This variance component consisted of the variance of sampling errors at the study level and an estimate of the variance between the studies (Borenstein et al., 2009). We performed a random-effect analysis because we wanted to draw conclusions for the population going beyond the included studies (Hedges & Vevea, 1998).

Due to the limited number of countries in our study, we were not able to perform a multilevel meta-analysis, which would have enabled us to disentangle the differences between the country level and the characteristics of the studies (study level) (Cheung, 2014). To ensure that study-level characteristics did not confound our substantive conclusions, we first conducted a sensitivity analysis (see Sensitivity Analyses in the Results Section), in which we correlated the relevant study characteristics with the country variable. Second, we performed a homogeneity analysis (as suggested by Viechtbauer, 2007, 2010) at the country level to determine whether the effect sizes came from the same population and to test whether a moderator analysis was appropriate. Third, we checked the robustness and quality of our results with a second sensitivity analysis focusing on sample distortions by mode and survey country (Cooper et al., 2019; Littell et al., 2008, p. 126; Viechtbauer, 2010, pp. 418–431) and also conducted an outlier analysis (Viechtbauer, 2010) and a publication bias analysis (Viechtbauer, 2010). In addition, we accounted for the multilevel structure of our data (effect sizes nested in studies nested in countries) by performing a multilevel analysis (see Supplemental Table A1). Fourth, in our main analysis, we examined which country-specific factors had a significant influence on differences in response rates. We estimated separate models for each factor, which is a standard approach in meta-analyses used to address overfitting and facilitate interpretation (Harrer et al., 2019; Viechtbauer, 2007). For the analyses, we used the R-package metafor (Version 1.9–9) (Viechtbauer, 2010). We chose “RD” (risk difference) as the effect size measure. The metafor package automatically transforms a risk difference into a log of the effect size, which makes these outcome measures symmetric around zero and enables a distribution of measures that is closer to a normal distribution.

Results

In this section, first, we report on the descriptive characteristics of the studies we examined. Following this, we investigate whether cross-cultural differences exist in the response rates of the seven countries in our study and, finally, identify which of the four country-specific factors might moderate these differences.

Study Characteristics and Sensitivity Analyses

The 110 effect sizes that we identified as eligible for our study were dominated by two characteristics. First, most (63%) of the web surveys were compared with mail surveys; second, most of the included studies (73%) were conducted in the United States (see Figure 3).

Figure 3.

Included experiments in percent by comparison mode and survey country (N = 110).

To prevent the strong presence of US studies and mail comparisons from distorting our analyses, we conducted two additional sensitivity analyses to ensure the robustness of our results. To this end, we replicated our results with a subpopulation of the US studies and the mail comparison studies. First, we drew a random sample from a selection of US studies. Second, we performed an analysis on studies including mail comparisons only. However, our results might have been biased, because a correlation may have existed between the comparison mode and the survey country. For this reason, we calculated the results not only for the mail mode but also for the telephone and face-to-face modes. All these further robustness checks replicated the subsequent findings (see Supplemental Table A2, Parts “b” and “c”). In addition, Daikeler et al., 2019; Lozar Manfreda et al., 2008 concluded that the contact mode of a survey, the sample population, and the number of contact attempts determined the difference between the response rates to web surveys and comparison survey modes. Therefore, to avoid pseudo-correlations, we investigated whether these variables correlated with the survey country. Our results showed that these three factors correlated only marginally with the survey country (contact mode r = .13 & p = .17; sample population r = .12 & p = .22; number of contact attempts r = .05 & p = .69).

Before exploring the cultural differences in web survey response rates, we addressed two questions regarding the validity of findings: publication bias and robustness. Publication bias refers to the problem that significant results have a higher probability of being published, which may distort the results of our meta-analysis. Our sensitivity analysis, which was implemented as a funnel plot, did not identify publication bias in our data, as the funnel plot was symmetrical. We also performed several robustness checks, such as excluding outliers, performing separate analyses for old and more recent studies, and applying a multilevel approach for effect sizes nested in papers. All differences in the mean response rates of these additional models were similar to those presented here; we did not detect any substantial differences. This suggests that the overall effect size was robust in terms of magnitude and direction. A detailed description of validity testing is provided in Section A2 of the online Appendix.

Cultural Differences in Web Surveys

For all 110 studies, the difference between the response rates to web surveys and to the comparison survey modes was 12 percentage points (confidence interval: 9 percentage points /16 percentage points). Thus, on average, web surveys recorded a 36% response rate, and the response rate to the comparison survey mode was 48% (see Figure 4). Consequently, the response rates to web surveys were robustly weaker than other survey modes. All three effect sizes were heterogeneous (significant Q-score of the difference 7.501 (df = 114, p ≤ .0001)). A heterogeneous effect size means that the value of the effect size, such as the difference in response rates, was not consistent across the studies, but varied significantly. Consequently, it was advisable to conduct a moderator analysis to explain this heterogeneity (Borenstein et al., 2009). We addressed this heterogeneity by testing the country itself as a possible moderator in a mixed-effects model, and our results showed significant differences in the response rates at the country level for all three effect sizes (see Table A2, Part “a”). This indicated that the participation rate in a web survey depends partly on the respective country in which a survey is conducted.

Figure 4.

Response rate overview across countries (N = 110).

Focusing on the differences between the average effect sizes of the 110 studies in Figure 4, it can be seen that the effect sizes of the web survey mode in the left forest plot are more similar than the effect sizes of the comparison modes in the middle forest plot and the response rate differences shown in the right forest plot. The right forest plot shows that the difference in response rates is significantly different to zero in all three examined countries. The smallest difference between response rates to web surveys and to other survey modes was recorded in the United States, with a difference of 9 percentage points. The United Kingdom and the Netherlands showed larger differences of 25 percentage points each. As the other countries in our sample had fewer than five effect sizes per country, we did not show their individual country values in this forest plot. The overall gap in response rates was 11 percentage points. However, when all studies conducted in the United States were excluded, this number rose to 18 percentage points¹⁰.

In the next section, we investigate in detail whether the cross-country differences in effect sizes can be explained by political and demographic, economic, technological, and socio-cultural factors.

Country-Specific Predictors of the Success of Web Surveys¹¹

Table 3 shows how country-specific political and demographic factors influence web survey response rates, especially with respect to country-specific proportions of young and old populations (see H1-H3 in Table 1). It shows that the higher the population growth and the lower the proportion of older people in a country, the smaller the difference between response rates to web surveys and their comparison survey mode. Specifically, the difference between response rates to web and comparison modes increases by 18 percentage points for each percentage point increase in population growth and decreases by 2 percentage points for each percentage point increase in persons over the age of 65 (see Rows 3 and 4 in Table 3). However, the difference in response rates seems to be especially influenced by the success or failure of the comparison survey mode. The younger people are, the lower the response rates to face-to-face, mail, telephone, and other survey modes. One percentage point more of population growth translates into a 26% lower response rate for comparison modes (see Row 4 of Column 5 in Table 3). The opposite effect can be observed for the proportion of persons aged over 65 years. The greater the proportion of older people within a country, the higher the response rate to comparison survey modes (see Row 3 of Column 5 in Table 3). In summary, the success of web surveys depends less on classic political and demographic factors such as education and more on the age structure of a population.

Table 3.

Political and Demographic, Economic, Technological, and Socio-Cultural Determinants for the Success of Web Surveys.

Measure		Response rate difference	Web response rate	Comparison mode response rate
Moderator
Political and demographic factors	Education	n.s.	n.s.	n.s.
	Annual population growth	.18 **	n.s.	−.26 ***
	Population aged 65 and over	−.02 *	n.s.	.03 ***
Economic factors	GDP	n.s.	n.s.	−.00 +
Technological advancements	Internet coverage	.01 +	.01 +	.01 ***
	Cellphone coverage	−.01 +	.01 +	.01 **
	Internet users	n.s.	n.s.	n.s.
Socio-cultural factors	Web response rate (reported in the study)	.01 ***		.01 ***
	Other mode response rate (reported in the study)	−.01 ***	.01 ***
	Web response rate (aggregated)	n.s.	.99 ***	1.29 ***
	Other mode response rate (aggregated)	−.42 *	.46 +	.88 **
	ISSP response rate	.01 *	n.s.	n.s.

GDP: gross domestic product.

Sig. level: ***p < .001 **p < .01 *p < .05 +p < .10; smaller coefficients than .01 or −.01 are rounded to display directions; the table displays three separately estimated mixed-effects models for each factor.

With respect to economic factors (H4), we expected a higher GDP to be linked to higher prosperity and innovation. Thus, we expected more of the population to have regular access to the internet. Therefore, for prosperous countries, we expected a high web response rate and a low difference between the response rates to different modes. However, the results of our analysis did not support this expected positive effect of GDP on the success of web surveys compared to other survey modes (see Row 6 in Table 3).

The technological advancements of a country (see H5–H7 in Table 1) should also be considered in researchers' decisions to employ a survey mode. Our results showed that the higher the internet and cell phone coverage in a country, the higher the response rates to the web-based mode relative to other survey modes (one percentage point more coverage results in one percentage point more web response rate, see Rows 7 and 8 of Column 4 in Table 3). With respect to the difference in response rates, Table 3 shows that for every percentage point more internet coverage in a country, the difference between the response rates to web-based surveys and to comparison survey modes shrinks by one percentage point (see Row 7 of Column 3 in Table 3). The opposite effect occurs regarding cell phone coverage. The higher the cell phone coverage in a country, the larger the difference between the response rates to web surveys and to comparison survey modes (see Row 8 of Column 3 in Table 3). In other words, the higher the cell phone coverage, the greater the disadvantage of conducting web surveys compared to other modes of data collection. The number of actual internet users had no significant effect on response rates. To summarize our results regarding the technological advancements of a country, internet and cell phone coverage moderated the response rates for web surveys. More specifically, our results showed that the better the cellphone coverage, the higher the web survey response rate, but, at the same time, the difference between the response rates to web surveys and comparison survey modes was also larger.

Finally, to address Hypothesis 8 (see Table 1), we examined whether the socio-culture influenced country-specific survey response. Our analysis showed that a positive propensity to participate in surveys led to higher response rates. For every percentage point more web response, the response rate for the comparison mode also grew by one percentage point (see Rows 11 and 12 in Table 3). However, considering the aggregated response rates from the last 5 years, our analysis also revealed that the comparison survey mode benefited almost twice as much (.46 percentage points increase in web response rate compared to a .88 percentage points increase in the response rate for the other mode) from a positive propensity to participate in surveys. On the whole, if the country-specific response rates for the comparison survey modes were generally high, web surveys also achieved higher response rates, and vice versa. The difference in response rates was mainly moderated by the response rates to the comparison survey mode, absolute, and aggregated. The higher the response rates to the comparison survey mode, the larger the difference in response rates. This phenomenon is also reflected in the response rate to the ISSP. The higher the response rate to the ISSP in a country, the larger the difference between the response rates to web surveys and their comparison survey mode (for each percentage point more response rate, the difference in response rates rises by one percentage point). This suggests that if the ISSP has a high response rate in a country, a large difference can be expected between web-based and comparison survey modes.

Furthermore, following Cadle et al’s. (2014) suggestion that socio-cultural factors are important for evaluating a country’s business environment, we conducted an additional moderator analysis using several cultural concepts: Hofstede’s individualism and uncertainty avoidance dimensions (Hofstede, 2016); Schwartz values concerning hedonism, conservation, and openness to change (Schwartz & Boehnke, 2004); the trust values of the World Values Survey (Inglehart et al., 2014). We did not find any significant effects of these moderators. As we did not have a strong theoretical expectation that linked web survey participation behavior to these cultural concepts (Daikeler et al., 2019), we decided against reporting these supplementary analyses in our Results section; instead, we report the complete results of these supplementary analyses in the Supplemental in Table A3.

Discussion

Summary of Results

Our research investigates whether the success of a web survey depends on country-specific factors and which country-specific indicators favor high web survey response rates. We developed our research questions based on an examination of previous meta-analytic studies (Daikeler et al., 2019; Lozar Manfreda et al., 2008) that used an international database, but did not investigate the heterogeneity of effect sizes at the country level. In our study, we meta-analyzed a dataset that consisted of 110 random split-sample experiments. Our results show that within-country factors are a source of heterogeneity for each of the three effect sizes: the web survey response rate, the response rate to the comparison survey mode, and the difference between the response rates to these survey modes. In an attempt to understand why cost-effective and time-saving web surveys may be more appropriate in some countries and less appropriate in others, our meta-analytical study investigated whether country-specific factors had an impact on web survey response rates. We examined four country-specific factors: political and demographic, economic, technological, and socio-cultural factors.

One of our main findings was that the heterogeneity of the differences between the response rates to web-based survey modes and other survey modes within countries was, to a large degree, due to the response rates of the comparison survey mode. Web surveys performed similarly across countries, unlike other survey modes. On average, the United States had 9% percentage points difference in survey response rates, whereas the U.K. and the Netherlands had 25% percentage points difference in survey response rates. However, the average response rates to web surveys in these countries were quite similar, 36%, 46%, and 33%, respectively, whereas the response rates to the comparison survey mode varied more, with 45%, 71%, and 58%, respectively. Given that the response rates to the comparison survey mode varied considerably across countries, the decision about whether to implement a web survey in a specific country should always take into account the expected response rates for alternative survey modes. As Kunert et al. (2002) and De Heer (1999) showed, preference for certain survey modes depends on the individual resources of a country.

With respect to the country-specific factors that moderate cross-country differences in survey response rates, our results showed that three out of four macroeconomic factors had an impact on the web survey response rate. A higher web survey response rate was linked with high population growth, high internet coverage, and a high response propensity. However, web surveys had a serious disadvantage compared to other survey modes when a country’s population was older and cell phone coverage was high. Previous research also reports age as a moderating factor of response rates to different modes in survey research (*Bech & Kristensen, 2009; Revilla et al., 2017).

With respect to political and demographic factors, we found that the higher the population growth and the lower the proportion of older people in a country, the better web surveys performed within a country. For countries with a high level of education, we expected web surveys to perform better, as most of the literature reports that higher educated individuals exhibit a higher propensity to respond to web surveys (e.g., Millar et al., 2009; Mulder & de Bruijne, 2019). However, we did not find this positive effect in our analysis. This could be due to the similarity of the educational levels in the countries in our study.

As a proxy for economic factors, we examined the GDP of the countries in our study and expected a positive association with web response rates. However, we found that this factor did not have an impact on the response rates to web-based survey or comparison survey modes. Again, a possible reason for this non-significant effect could be that the countries in our study had a similar economic status, as previous literature reports a link between income and mode preferences (e.g., Millar et al., 2009)

Regarding the technological factors, we found that web surveys were more appropriate for countries with high internet coverage rates. This finding was in line with our expectations and the literature (see, e.g., Smyth et al., 2014). We assumed that high internet coverage rates enable a population to use the internet regularly, and regular usage reduces the burden of completing a web survey. However, our results also showed that the better the cell phone coverage of a country, the higher the response rates for the comparison survey modes. Consequently, the difference in the response rates to web surveys and comparison survey modes was larger in countries with high cell phone coverage. From today’s perspective, this finding is surprising as we often equate cell phone coverage with mobile internet coverage. However, most of the studies that we analyzed were completed at a time when the mobile internet was not yet available. Therefore, this indicator may influence web survey response rate differently in the future, and our study should be replicated when more recent studies are available.

With respect to country-based socio-cultural factors, we found that web surveys work well in countries with high response rates to comparison survey modes and high ISSP response rates. This finding is in line with the results of a study conducted in a region in Belgium. This study showed that a low propensity to participate in surveys leads to lower web response rates and a higher number of contact attempts (Barbier et al., 2015). Our study, therefore, enabled us to generalize Barbier et al. (2015) results regarding survey participation propensity and response rates across countries. With respect to decisions about which survey mode to use in a country, we conclude that web surveys probably work well in a country where comparative survey modes also perform well. In addition, although switching the survey mode might decrease the survey response rate, it also might bring organizational and financial benefits. Overall, web surveys are a useful alternative to traditional survey modes especially when a country has a young, technology-oriented, and survey-friendly population.

Limitations and Further Research

First, 73% of the studies we examined were conducted in the United States and, despite our robustness analysis, evidence from other countries is needed to further strengthen our findings. Additional evidence is especially important because most of the experimental studies were conducted in western countries. Including more (diverse) countries would lead to more statistical power and a deeper understanding of the moderating factors. Furthermore, including more countries would enable researchers to use multilevel meta-analytic models, enabling an examination of country-level variance.

A second limitation is that our sample largely consists of comparisons between web-based and mail surveys. On the one hand, this is not surprising because many studies compare self-administered modes. On the other hand, mail surveys have only recently begun to play a more prominent role as a survey mode again.

Third, some empirical studies (e.g., Johnson et al., 2018; Rammstedt et al., 2017) and theoretical approaches (e.g., Cadle et al., 2014) include value-oriented concepts, such as the models suggested by Hofstede (2016) or Schwartz and Boehnke (2004). We tested several value concepts such as Hofstede’s individualism and uncertainty avoidance dimensions (Hofstede, 2001); the Schwartz values concerning hedonism, conservation, and openness to change (Schwartz & Boehnke, 2004); and the trust values of the World Values Survey (Inglehart et al., 2014). However, we did not find any significant effects of these concepts. Nevertheless, the concepts we tested are not optimal because, for example, Hofstede’s values are not available as a time series.

Fourth, there are several possible alternative political and demographic, economic, technological, and socio-cultural factors that influence the response rate of web surveys. The selection presented was made based on the feasibility of the database and the inclusion criteria outlined in the Methods section. We recommend conducting further research to explore alternative operationalizations, which would help test the generalizability of the findings of the present study.

Fifth, we decided against including survey-based indicators such as incentives, sample population, or contact attempts because Daikeler et al. (2019) showed that these indicators explain only a very small amount of the heterogeneity of the difference in survey response rates. While the studies we examined were not randomized across countries, comparison modes, or survey-based indicators, our sensitivity analyses, separated by mode, did not indicate a systematic mode bias in our results.

Sixth, the costs and the effort required to implement a certain survey mode are two determining factors that impact survey mode choice. In our meta-analysis, we could not take these two factors into account due to a lack of information in the primary studies we examined. In a cross-country context, the costs for web surveys should be quite homogeneous across countries, whereas the costs for interviewer-administered comparison survey modes depend strongly on country-specific conditions, such as the infrastructure, wages, availability of interviewers, and the competition between organizations conducting the interviews.

Seventh, as outlined in the Introduction, response rates are likely to be directly related to non-response bias, but this is not necessarily always the case and should be investigated in future studies. Such studies require that the characteristics of the population are known before the sample is drawn (and in the case of meta-analyses that they are published and well documented). Our findings reveal cross-national differences in the country-level indicators that influence survey response rates (see Table 3), thus raising the question as to whether differences in the social structure across countries are systematically related to non-response within a country. Such systematic non-response bias would, in turn, affect cross-national comparability in cross-national settings (Jans et al., 2018).

Last, the increasing popularity of mobile web surveys calls for them to be included in future survey mode comparisons and in meta-analyses. This avenue of research seems especially promising in a cross-cultural context because many people in Asian and African countries access the internet mainly with their smartphones (Statista, 2018).

Conclusion and Practical Implications

Because web surveys perform consistently with relatively high response rates, our findings suggest that they can be used as an alternative to other survey modes in all of the countries included in the study. Web surveys can be a reliable alternative, especially when other survey modes are not feasible due to high survey costs, long field times, or decreasing response rates. At the same time, in almost all countries of our study, the web survey response rate was lower than the rates of other survey modes. Consequently, using a web survey is likely to decrease the response rate, and survey designers will have to account for this during the study planning phase (e.g., by drawing larger samples, sending more reminders, or providing higher incentives) ¹². This is especially relevant in cross-national settings where more traditional survey modes perform less consistently, and fieldwork is often challenging and time-consuming. In such settings, web surveys are expected to be increasingly the survey mode of choice.

Supplemental Material

sj-pdf-1-mre-10.1177_14707853211050916 – Supplemental Material for A Meta-Analysis of How Country-Level Factors Affect Web Survey Response Rates

Supplemental Material, sj-pdf-1-mre-10.1177_14707853211050916 for A Meta-Analysis of How Country-Level Factors Affect Web Survey Response Rates by Jessica Daikeler, Henning Silber and Michael Bošnjak in International Journal of Market Research

Footnotes

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

Data Availability

After publication, primary data and R code are available on PsychArchives

ORCID iDs

Jessica Daikeler

Henning Silber

Supplementary material

Supplementryl material for this article is available online.

Notes

References

*Al Baghal

Lynn

(2015). Using motivational statements in web-instrument design to reduce item-missing rates in a mixed-mode context. Public Opinion Quarterly, 79(2), 568-579. http://dx.doi.org/10.1093/poq/nfv023.

*Allum

Auspurg

Blake

Booker

Crossley

D’Ardenne

Fairbrother

Iacovou

Jäckle

Kaminska

Lynn

Nicoletti

Oldfield

Pudney

Schnettler

Uhrig

S. C. N.

Winter

(2014). Understanding Society Innovation Panel Wave 6: Results from Methodological Experiments . Understanding Society Working Paper Series 2014-04.

*Al-Subaihi

A. A.

(2008). Comparison of web and telephone survey response rates in Saudi Arabia. The Electronic Journal of Business Research Methods, 6(2), 123-132.

APA (2006). Evidence-based practice in psychology. The American Psychologist, 61(4), 271‐285.https://doi.org/10.107/0003-066X.61.4.271.

Atrostic

B. K.

Bates

Burt

Silberstein

(2001). Non-response in U.S. government household surveys: Consistent measures, recent trends, and new insights Journal of Official Statistics172209226

*Auspurg

Burton

Cullinane

Delavande

Laura

Iacovou

Jäckle

Kaminska

Lynn

Mathews

Nicolaas

Nicoletti

Zafar

(2013). Understanding Society Innovation Panel Wave 5: Results from Methodological Experiments . Understanding Society Working Paper Series 2013-06.

Barbier

Loosveldt

Carton

(2015). The flemish survey climate: An analysis based on the survey of social-cultural changes in flanders. International workshop on household survey nonresponse, Leuven, Belgium, 02–04 September 2015.

Bason

(2000). Comparison of telephone, mail, web and IVR surveys of drug and alcohol use among University of Georgia Students. In: 55th Annual Conference of the American Association for Public Opinion Research, Portland, Oregon, July 22, 2013.

Bates

(2001). Internet versus mail as a data collection methodology from a high-coverage population, The American association for public opinion research 56th annual conference, Montreal, Quebec Canada, May 19, 2001.

10.

Baumgartner

Steenkamp

J.-B. E. M.

(2001). Response styles in marketing research: A cross-national investigation. Journal of Marketing Research, 38(2), 143‐156. https://doi.org/10.1509/jmkr.38.2.143.18840.

11.

Beach

Musa

(2012). Non-response and mode effects in a two-wave randomized mode experiment, The American association for public opinion research. 67th annual conference; Orlando, Florida, June 2, 2009.

12.

Beach

Musa

Beeson

Sparks

(2008). Mode effects and non-response bias in an undergraduate student satisfaction survey: Results from a randomized experiment comparing telephone and web administration, The American association for public opinion research 63rd annual conference & WAPOR 61st annual conference,

New Orleans, USA

June 2, 2009.

13.

*Bech

Kristensen

(2009). Differential response rates in postal and Web-based surveys in older respondents. Survey Research Methods, 3(1), 1-6. https://doi.org/10.18148/srm/2009.v3i1.592.

14.

Bethlehem

(2010). Selection bias in web surveys. International Statistical Review, 78(2), 161‐188. https://doi.org/10.1111/j.1751-5823.2010.00112.x.

15.

Beullens

Loosveldt

Vandenplas

(2017). Classes of nonrespondents in the ESS: which classes are prioritized and which classes should be prioritized in order to reduce nonresponse bias. In ESS ERIC work programme. Leuven, Belgium: Centre for Sociological Research.

16.

Beullens

Vandenplas

Loosveldt

Ineke

(2018). Response rates in the European social survey: Increasing, decreasing, or a matter of fieldwork efforts? In Survey methods: Insights from the field. .. https://surveyinsights.org/?p=9673.

17.

Blom

A. G.

(2012). Explaining cross‐country differences in survey contact rates: Application of decomposition methods. Journal of the Royal Statistical Society: Series A (Statistics in Society), 175(1), 217-242. https://doi.org/10.1111/j.1467-985X.2011.01006.x.

18.

*Blom

A. G.

Burton

Booker

Cernat

Fairbrother

Jäckle

Kaminska

Keusch

Krosnick

J. A.

Lynn

Oberski

Pudney

Sala

Schnettler

Silber

Stark

T. H.

Uhrig

S. C. N.

Yan

(2015). Understanding Society Innovation Panel Wave 7: Results from Methodological Experiments , Understanding Society Working Paper Series 2015-03.

19.

Borenstein

Hedges

L. V.

Higgins

J. P. T.

Hannah

(2009). Introduction to meta-analysis. Hoboken, NJ: John Wiley & Sons.

20.

*Börkan

(2010). The mode effect in mixed-mode surveys: Mail and web surveys. Social Science Computer Review, 28(3), 371-380. https://doi.org/10.1177/0894439309350698.

21.

Bosnjak

Forsman

Isaksson

(2005). Preface to JOS special issue on web surveys, Journal of Official Statistics, 21(2iii).

22.

Brick

J. M.

Williams

(2013). Explaining rising non-response rates in cross-sectional surveys. The ANNALS of the American Academy of Political and Social Science, 645(1), 36‐59. https://doi.org/10.1177/0002716212456834.

23.

Brüggen

Wetzels

De Ruyter

Schillewaert

(2011). Individual differences in motivation to participate in online panels: The effect on response rate and response quality perceptions. International Journal of Market Research, 53(3), 369‐390. https://doi.org/10.2501/IJMR-53-3-369-390.

24.

Cadle

Paul

Turner

(2014). Business analysis techniques. Swindon, UK: BCS Learning & Development Limited.

25.

Charness

Boot

W. R.

(2009). Aging and information technology use. Current Directions in Psychological Science, 18(5), 253-258. https://doi.org/10.1111/j.1467-8721.2009.01647.x.

26.

Chatt

Dennis

Motta-Stanko

Pulliam

(2005). Data collection mode effects controlling for sample origins in a panel survey: Telephone versus Internet, 60th Annual Conference of the American Association for Public Opinion Research, Miami Beach, Florida, 12–15 May 2015.

27.

Cheung

M. W.-L.

(2014). Modeling dependent effect sizes with three-level meta-analyses: A structural equation modeling approach. Psychological Methods, 19(2), 211‐229. https://doi.org/10.1037/a0032968.

28.

Clark

Rogers

Foster

Dvorchak

Saadeh

Weaver

Mor

(2011). A randomized trial of the impact of survey design characteristics on response rates among nursing home providers. Evaluation & the Health Professions, 34(4), 464-486. https://doi.org/10.1177/0163278710397791.

29.

*Cobanoglu

Moreo

P. J.

Warde

(2001). A comparison of mail, fax and web-based survey methods. International Journal of Market Research, 43(4), 1-15. https://doi.org/10.1177/147078530104300401.

30.

*Cole

S. T.

(2005). Comparing mail and web-based survey distribution methods: Results of surveys to leisure travel retailers. Journal of Travel Research, 43(4), 422-430. https://doi.org/10.1177/0047287505274655.

31.

*Cooper

Hedges

L. V.

Valentine

J. C.

(2019). The Handbook of Research Synthesis and Meta-Analysis. New York, NY: Russell Sage Foundation.

32.

Couper

M. P.

(2000). Review: Web surveys: A review of issues and approaches. Public Opinion Quarterly, 64(4), 464‐494. https://doi.org/10.1086/318641.

33.

Couper

M. P.

Kapteyn

Schonlau

Winter

(2007). Noncoverage and non-response in an Internet survey. Social Science Research, 36(1), 131‐148. https://doi.org/10.1016/j.ssresearch.2005.10.002.

34.

Crawford

S. D.

Couper

M. P.

Lamias

M. J.

(2001). Web surveys: perceptions of burden. Social Science Computer Review, 19(2), 146-162. https://doi.org/10.1177/089443930101900202.

35.

Crawford

McCabe

Couper

M. P.

Boyd

C. J.

(2002). From Mail to Web: Improving Response Rates and Data Collection Efficiencies. In: International Conference on Improving Surveys. Copenhagen, Denmark, 25 May 2004.

36.

*Croteau

A.-M.

Dyer

Miguel

(2010). Employee reactions to paper and electronic surveys: An experimental comparison. IEEE Transactions on Professional Communication, 53(3), 249-259. https://doi.org/10.1109/TPC.2010.2052852.

37.

Curtin

Presser

Singer

(2005). Changes in telephone survey nonresponse over the past quarter century. Public Opinion Quarterly, 69(1), 87‐98.https://doi.org/10.1093/poq/nfi002.

38.

Czaja

S. J.

Charness

Fisk

A. D.

Hertzog

Nair

S. N.

Rogers

W. A.

Sharit

(2006). Factors predicting the use of technology: Findings from the center for research and education on aging and technology enhancement (CREATE). Psychology and Aging, 21(2), 333‐352. https://doi.org/10.1037/0882-7974.21.2.333.

39.

Daikeler

Bošnjak

Lozar Manfreda

(2019). Web versus other survey modes: An updated and extended meta-analysis comparing response rates. Journal of Survey Statistics and Methodology, 8(3), 513‐539. https://doi.org/10.1093/jssam/smz008.

40.

Das

Kapteyn

Bosnjak

(2018). Open probability-based panel infrastructures. In Vannette

D. L.

Krosnick

J. A.

(Eds.), The Palgrave Handbook of Survey Research (pp. 199-209). Cham, CH: Palgrave Macmillan.

41.

De Heer

(1999). International response trends: Results of an international survey. Journal of Official Statistics, 15(2), 129‐142.

42.

De Leeuw

(2018). Mixed-mode: Past, present, and future. Survey Research Methods, 12(2), 75‐89. https://doi.org/10.18148/srm/2018.v12i2.7402.

43.

De Leeuw

Hox

Luiten

(2018). International non-response trends across countries and years: an analysis of 36 years of Labour Force Survey data. Survey Methods: Insights from the Field, 1-11. https://surveyinsights.org/?p=10452.

44.

De Leeuw

Nicolaas

Campanelli

Hox

(2012). Question or mode effects in mixed-mode surveys: A cross-cultural study in the Netherlands, Germany, and the UK, The American association for public opinion research 67th annual conference, Orlando, Florida, 17–20 May, 2012.

45.

*DiNitto

D. M.

Bridget Busch-Armendariz

Bender

Hyeyoung Woo

Tackett-Gibson

Dyer

(2008). Testing telephone and web surveys for studying men’s sexual assault perpetration behaviors. Journal of Interpersonal Violence, 23(10), 1483-1493. https://doi.org/10.1177/0886260508314341.

46.

*Edwards

M. L.

Dillman

D. A.

Smyth

J. D.

(2014). An experimental test of the effects of survey sponsorship on Internet and mail survey response. Public Opinion Quarterly, 78(3), 734-750. https://doi.org/10.1093/poq/nfu027.

47.

Eklund

Järnbert

Orusild

Persson

Scheffer

(2015). Mixed-mode experiment: Evaluation of effects on data quality and response rates. In: The 6th Conference of the European Survey Research Assoication, Reykjavik, Iceland, July 13–17, 2015.

48.

*Elder

Incalcatera

(2000). Pushing the envelope: Moving a major syndicated study to the web. Net Effects, 3.

49.

*Ellis

Rexrode

(2012). Addressed-based sampling – a better sample? exploring the benefits of using addressed-based sampling in a state-wide targeted sub-population. The American Association for Public Opinion Research (AAPOR) 67th Annual Conference, 2012.

50.

Falch

Sandgren Massih

(2011). The effect of education on cognitive ability. Economic Inquiry, 49(3), 838‐856. https://doi.org/10.1111/j.1465-7295.2010.00312.x.

51.

Fang

Shao

Lan

(2009). Effects of innovativeness and trust on web survey participation. Computers in Human Behavior, 25(1), 144‐152. https://doi.org/10.1016/j.chb.2008.08.002.

52.

Fehr

Jokisch

Kotlikoff

L. J.

(2008). Fertility, mortality and the developed world’s demographic transition. Journal of Policy Modeling. 30(3), 455‐473. https://doi.org/10.1016/j.jpolmod.2008.01.002.

53.

*Fisher

S. H.

Herrick

(2013). Old versus new: The comparative efficiency of mail and internet surveys of state legislators. State Politics & Policy Quarterly, 13(2), 147-163. https://doi.org/10.1177/1532440012456540.

54.

*Fraze

S. D.

Hardin

K. K.

Brashears

M. T.

Haygood

J. L.

Smith

J. H.

(2003). The effects of delivery mode upon survey response rate and perceived attitudes of Texas agri-science teachers. Journal of Agricultural Education, 44(2), 27-37. https://doi.org/10.5032/jae.2003.02027.

55.

*Fricker

Galesic

Tourangeau

Yan

(2005). An experimental comparison of web and telephone surveys. Public Opinion Quarterly, 69(3), 370-392. https://doi.org/10.1093/poq/nfi027.

56.

Fuchs

Busse

(2009). The coverage bias of mobile web surveys across European countries. International Journal of Internet Science, 4(1), 21‐33.

57.

Galperin

Fernanda Viecens

(2017). Connected for development? Theory and evidence about the impact of internet technologies on poverty alleviation. Development Policy Review, 35(3), 315‐336. https://doi.org/10.1111/dpr.12210.

58.

Garrote Sanchez

Gomez Parra

Ozden

Rijkers

Viollaz

Winkler

(2021). Who on earth can work from home? The World Bank Research Observer, 36(1), 67‐100. https://doi.org/10.1093/wbro/lkab002.

59.

Gordon

M. L.

Gatys

Guestrin

Bigham

J. P.

Trister

Patel

(2019). App usage predicts cognitive ability in older adults. Paper presented at: The Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. Glasgow, Scotland, UK, 14–17 May, 2019.

60.

Grandjean

Nelson

Taylor

(2009). Comparing an internet panel survey to mail and phone surveys on willingness to pay for environmental quality: A national mode test In: The American association for public opinion research 64th annual conference,

Hollywood, Florida

14–17 May, 2009.

61.

*Greene

Speizer

Wiitala

(2008). Telephone and web: Mixed‐mode challenge. Health Services Research, 43(1p1), 230-248. https://doi.org/10.1111/j.1475-6773.2007.00747.x.

62.

*Greenlaw

Brown-Welty

(2009). A comparison of web-based and paper-based survey methods: Testing assumptions of survey mode and response cost. Evaluation Review, 33(5), 464-480. https://doi.org/10.1177/0193841X09340214.

63.

*Grigorian

Sederstrom

Hoffer

(2004). Web of intrigue? Evaluating effects on response rates of between web SAQ, CATI, and mail SAQ options in a national panel survey. Paper presented at:The American association for public opinion research 59th annual conference, Phoenix, Arizona, 13–16 May, 2004.

64.

Groves

R. M.

Couper

M. P.

(2012). Non-Response in Household Interview Surveys. Hoboken, NJ: John Wiley & Sons.

65.

Groves

R. M.

Peytcheva

(2008). The impact of nonresponse rates on nonresponse bias: A meta-analysis. The Public Opinion Quarterly, 72(2), 167‐189. https://doi.org/10.1093/poq/nfn011.

66.

Gummer

Daikeler

(2018). A note on how prior survey experience with self-administered panel surveys affects attrition in different modes. Social Science Computer Review, 38(4), 490‐498. https://doi.org/10.1177/0894439318816986.

67.

*Hardigan

P. C.

Succar

C. T.

Fleisher

J. M.

(2012). An analysis of response rate and economic costs between mail and web-based surveys among practicing dentists: A randomized trial. Journal of Community Health, 37(2), 383-394. https://doi.org/10.1007/s10900-011-9455-6.

68.

Harrer

Cuijpers

Furukawa

Ebert

(2019). Doing meta-analysis in R: A hands-on guide. PROTECT Lab, Erlangen: CRC Press/Chapman & Hall. https://doi.org/10.1201/9781003107347.

69.

*Hayslett

M. M.

Wildemuth

B. M.

(2004). Pixels or pencils? The relative effectiveness of Web-based versus paper surveys. Library & Information Science Research, 26(1), 73-93. https://doi.org/10.1016/j.lisr.2003.11.005.

70.

Hedges

L. V.

Vevea

J. L.

(1998). Fixed- and random-effects models in meta-analysis. Psychological Methods, 3(4), 486‐504. https://doi.org/10.1037/1082-989X.3.4.486.

71.

Hermeking

(2005). Culture and internet consumption: Contributions from cross-cultural marketing and advertising research. Journal of Computer-Mediated Communication, 11(1), 192‐216. https://doi.org/10.1111/j.1083-6101.2006.tb00310.x.

72.

Hofstede

(2001). Culture’s consequences: Comparing values, behaviors, institutions and organizations across nations. Thousand Oaks, CA: Sage publications, Inc.

73.

Hofstede

(2016). Cultural dimension. Available at: https://geert-hofstede.com/ (accessed 05 May, 2020).

74.

Howard

P. N.

Duffy

Freelon

Hussain

M. M.

Mari

Maziad

(2011). Opening closed regimes: What was the role of social media during the arab spring? SSRN Electronic Journal. http://dx.doi.org/10.2139/ssrn.2595096.

75.

*Hsu

McFall

(2015). Mode effects in mixed-mode economic surveys: Insights from a randomized experiment. Finance and Economics Discussion Series, 2015(8), 1-35.

76.

Inglehart

Haerpfer

Moreno

(2014). World values survey: All rounds - country-pooled datafile 1981-2014. Madrid: J.D. Systems Institute.

77.

Israel

(2013). Using mixed-mode contacts to facilitate participation in public agency client surveys. The American association for public opinion research 68th annual conference, Boston, Massachusetts, 23 May, 2013.

78.

*Israel

G. D.

Lamm

A. J.

(2012). Item nonresponse in a client survey of the general public. Survey Practice, 5(2), 1-5. https://doi.org/10.29115/SP-2012-0010.

79.

ISSP (2020). International social survey programme. Available at: http://www.issp.org. (Accessed 5 May, 2020).

80.

*Jacob

(2011). An experiment to test the feasibility and quality of a web-based questionnaire of teachers. Evaluation Review, 35(1), 40-70. https://doi.org/10.1177/0193841X11399376.

81.

Jans

McLaughlin

Viana

Grant

Park

Ponce

N. A.

(2018). Geographic correlates of nonresponse in California. In Johnson

T. P.

Pennell

B.-E.

Stoop

I. A. L.

Dorer

(Eds.), Advances in comparative survey methods: Multinational, multiregional, and multicultural contexts (3MC) (pp. 835-857). Hoboken, NJ: John Wiley & Sons, Inc.

82.

Johnson

T. P.

Lee

Cho

Y. I.

(2010). Examining the association between cultural environments and survey nonresponse. Survey Practice, 3(3), 1‐12. https://doi.org/10.29115/SP-2010-0013.

83.

Johnson

T. P.

Pennell

B.-E.

Stoop

I. A. L.

Dorer

(2018). The promise and challenge of 3MC research. In Johnson

T. P.

Pennell

B.-E.

Stoop

I. A. L.

Dorer

(Eds.), Advances in comparative survey methods: Multinational, multiregional, and multicultural contexts (3MC) (pp. 3-12). Hoboken, NJ: John Wiley & Sons, Inc.

84.

*Jones

Pitt

(1999). Health surveys in the workplace: Comparison of postal, email and World Wide Web methods. Occupational Medicine, 49(8), 556-558. https://doi.org/10.1093/occmed/49.8.556.

85.

*Kaplowitz

M. D.

Hadlock

T. D.

Levine

(2004). A comparison of web and mail survey response rates. Public Opinion Quarterly, 68(1), 94-101. https://doi.org/10.1093/poq/nfh006.

86.

Kennedy

Ackermann

Turakhia

Emerson

M. O.

James

(2012). Mode effects measurement and correction: A case study, The American association for public opinion research 67th annual conference, Orlando, Florida, 26 June, 2012.

87.

*Kerwin

Brick

Levin

Cantor

O’Brien

Campbell

Shipp

Wang

(2004). Web, mail, and mixed-mode data collection in a survey of advanced technology program applicants. Toronto, Canada: Joint Statistical Meetings.

88.

Keusch

(2015). Why do people participate in Web surveys? Applying survey participation theory to Internet survey data collection. Management Review Quarterly, 65, 183‐216. https://doi.org/10.1007/s11301-014-0111-y.

89.

Keusch

Batinic

Mayerhofer

(2014). Motives for joining nonprobability online panels and their association with survey participation behavior. Online Panel Research: A Data Quality Perspective, (pp. 171‐191).

90.

*Kiernan

N. E.

Kiernan

Oyler

M. A.

Gilles

(2005). Is a web survey as effective as a mail survey? A field experiment among computer users. American Journal of Evaluation, 26(2), 245-252. https://doi.org/10.1177/1098214005275826.

91.

*Kirchner

Felderer

(2017). The effect of nonresponse and measurement error on wage regression across survey modes. In Biemer

P. P.

De Leeuw

Eckman

Edwards

Kreuter

Lyberg

L. E.

Tucker

N. C.

West

B. T.

(Eds.), Total survey error in practice (pp. 531-556). Hoboken, NJ: John Wiley & Sons, Inc.

92.

Klausch

Hox

Schouten

(2012). Assessing the mode: Dependency of sample selectivity across the survey response process. The American Association for Public Opinion Research 67th Annual Conference,

Orlando, Florida

17–20, May 2012.

93.

*Knapp

Kirk

S. A.

(2003). Using pencil and paper, Internet and touch-tone phones for self-administered surveys: does methodology matter? Computers in Human Behavior, 19(1), 117-134. https://doi.org/10.1016/S0747-5632(02)00008-0.

94.

Kreuter

(2013). Facing the nonresponse challenge. The ANNALS of the American Academy of Political and Social Science, 645(1), 23‐35. https://doi.org/10.1177/0002716212456815.

95.

Krippendorff

(2004). Reliability in content analysis: Some common misconceptions and recommendations. Human Communication Research, 30(3), 411‐433. https://doi.org/10.1093/hcr/30.3.411.

96.

Kunert

Kloas

Kuhfeld

(2002). Design characteristics of national travel surveys: International comparison for 10 countries. Transportation Research Record: Journal of the Transportation Research Board, 1804(1), 107‐116. https://doi.org/10.3141/1804-15.

97.

*Kwak

Radler

(2002). A comparison between mail and web surveys: Response pattern, respondent profile, and data quality. Journal of Official Statistics, 18(2), 257-273.

98.

Lesser

Newton

(2001). Mail, email and web surveys: A cost and response rate comparison in a study of undergraduate research activity, The American association for public opinion research 56th annual conference,

Montreal, Quebec, Canada

May 17–20, 2001.

99.

Lipsey

M. W.

David

B. W.

(2001). Analysis issues and strategies. In Lipsey

M. W.

Wilson

D. B.

(Eds.), Practical meta-analysis (pp. 105-128). Thousand Oaks, CA: SAGE publications, Inc.

100.

Littell

J. H.

Corcoran

Pillai

(2008). Systematic reviews and meta-analysis. Oxford, UK: Oxford University Press.

101.

Loosveldt

Joye

(2016). Defining and assessing survey climate. In Wolf

Joye

Smith

T. W.

Y.-C.

(Eds.), The SAGE handbook of survey methodology (pp. 67-76). Thousand Oaks, CA: SAGE publications, Inc.

102.

Lozar Manfreda

Bosnjak

Berzelak

Haas

Vehovar

(2008). Web surveys versus other survey modes: A meta-analysis comparing response rates. International Journal of Market Research, 50(1), 79-104. https://doi.org/10.1177/147078530805000107.

103.

*Lozar Manfreda

Vehovar

Batagelj

(2000). Web versus mail questionnaire for an institutional survey. Paper presented at ‘The Challenge of the Internet’, organized by The Association for Survey Computing, Latimer Conference Centre, UK, May 11–12, 2001

104.

*Lugtig

Lensvelt-Mulders

G. J. L. M.

Frerichs

Greven

(2011). Estimating non-response bias and mode effects in a mixed-mode survey. International Journal of Market Research, 53(5), 669-686. https://doi.org/10.2501/IJMR-53-5-669-686.

105.

*McMorris

B. J.

Petrie

R. S.

Catalano

R. F.

Fleming

C. B.

Haggerty

K. P.

Abbott

R. D.

(2009). Use of web and in-person survey modes to gather data from young adults on sex and drug use: An evaluation of cost, time, and survey error based on a randomized mixed-mode design. Evaluation Review, 33(2), 138-158. https://doi.org/10.1177/0193841X08326463.

106.

McNeish

(2001). Using the Internet data collection: Just because we can, should we? The American association for public opinion research 56th annual conference, Montreal, Quebec, Canada, 17–20 May, 2001.

107.

*Mercer

McGeeney

Mathiowetz

McGeeney

(2015). From telephone to the web: The challenge of mode of interview effects in public opinion polls. In: The American association for public opinion research 70th annual conference,

Hollywood, Florida

14–17 May, 2015.

108.

*Messer

(2012). Pushing households to the web: Experiments of a ’Web+ Mail’ methodology for conducting general public surveys. Dissertation, Washington University, USA.

109.

*Messer

B. L.

Edwards

M. L.

Dillman

D. A.

(2012). Determinants of item non-response to web and mail respondents in three address-based mixed-mode surveys of the general public. Survey Practice, 5(2), 1-8. https://doi.org/10.29115/SP-2012-0012.

110.

*Millar

M. M.

Dillman

D. A.

(2011). Improving response to web and mixed-mode surveys. Public Opinion Quarterly, 75(2), 249-269. https://doi.org/10.1093/poq/nfr003.

111.

Millar

M. M.

O’Neill

A. C.

Dillman

D. A.

(2009). Are mode preferences real? Technical report 09-003. Washington State University: Social and Economic Sciences Research Center, USA.

112.

Miller

T. I.

Kobayashi

M. M.

Caldwell

Thurston

Collett

(2002). Citizen surveys on the Web: General population surveys of community opinion. Social Science Computer Review, 20(2), 124‐136. https://doi.org/10.1177/089443930202000203.

113.

Mulder

de Bruijne

(2019). Willingness of online respondents to participate in alternative modes of data collection. Survey Practice, 12(1), 1‐11. https://doi.org/10.29115/SP-2019-0001.

114.

Murphy

Brackbill

(2012). Disentangling mode and nonresponse effects in the world trade center health registry. The American association for public opinion research 67th annual conference, Orlando, Florida, 17–20 May, 2012.

115.

OECD . (2013). Technical report of the survey of adult skills (PIAAC). Available at: http://www.oecd.org/skills/piaac/_Technical%20Report_17OCT13.pdf. OECD.

116.

OECD (2020a). Elderly population. Available at: https://data.oecd.org/pop/elderly-population.htm. (Accessed 5 May, 2020).

117.

OECD (2020b). Gross domestic product (GDP). Available at: https://data.oecd.org/gdp/gross-domestic-product-gdp.htm, (Accessed 5 May, 2020).

118.

Petrova

P. K.

Cialdini

R. B.

Sills

S. J.

(2007). Consistency-based compliance across cultures. Journal of Experimental Social Psychology, 43(1), 104‐111. https://doi.org/10.1016/j.jesp.2005.04.002.

119.

Porter

S. R.

Whitcomb

M. E.

Weitzer

W. H.

(2004). Multiple surveys of students and survey fatigue. New Directions for Institutional Research, 2004(121), 63‐73. https://doi.org/10.1002/ir.101.

120.

*Pötschke

(2004). Paper and pencil or online? Methodological experiences from an employee survey. In: German Online Research Conference (GOR), Stuttgart.

121.

Rammstedt

Danner

Bosnjak

(2017). Acquiescence response styles: A multilevel model explaining individual-level and country-level differences. Personality and Individual Differences, 107, 190-194. https://doi.org/10.1016/j.paid.2016.11.038.

122.

Redline

Zukerberg

Rizzo

(2015). Hope springs eternal: Will a probability sample of schools and principals respond by web and provide email addresses? The American association for public opinion research 70th annual conference, Hollywood, Florida, 14–15 May, 2015.

123.

Revilla

Ochoa

Loewe

(2017). Using passive data from a meter to complement survey data in order to study online behavior. Social Science Computer Review, 35(4), 521–536. https://doi.org/10.1177/0894439316638457.

124.

Rogers

Murtaugh

M. A.

Edwards

Slattery

M. L.

(2004). Contacting controls: are we working harder for similar response rates, and does it make a difference? American Journal of Epidemiology, 160(1), 85–90. https://doi.org/10.1093/aje/kwh176

125.

Rookey

B. D.

Hanway

Dillman

D. A.

(2008). Does a probability-based household panel benefit from assignment to postal response as an alternative to internet-only? Public Opinion Quarterly, 72(5), 962–984. https://doi.org/10.1093/poq/nfn061.

126.

Rosenthal

(1979). The file drawer problem and tolerance for null results. Psychological Bulletin, 86(3), 638‐641. https://doi.org/10.1037/0033-2909.86.3.638.

127.

*Sax

L. J.

Gilmartin

S. K.

Bryant

A. N.

(2003). Assessing response rates and nonresponse bias in web and paper surveys. Research in Higher Education, 44(4), 409-432. https://doi.org/10.1023/A:1024232915870.

128.

Schwartz

S. H.

Boehnke

(2004). Evaluating the structure of human values with confirmatory factor analysis. Journal of Research in Personality, 38(3), 230–255. https://doi.org/10.1016/S0092-6566(03)00069-2.

129.

Shih

T.-H.

Xitao Fan

(2008). Comparing response rates from web and mail surveys: A meta-analysis. Field Methods, 20(3), 249-271. https://doi.org/10.1177/1525822X08317085.

130.

Silber

Daikeler

Weidner

Bosnjak

(2018). Web survey. In Balakrishnan

Colton

Everitt

Piegorsch

Ruggeri

Teugels

J. L.

(Eds.), Wiley Statsref: Statistics Reference Online (pp. 1-6). Hoboken, NJ: John Wiley & Sons, Ltd.

131.

*Sinclair

O’Toole

Malawaraarachchi

Leder

(2012). Comparison of response rates and cost-effectiveness for a community-based survey: Postal, internet and telephone modes with generic or personalised recruitment approaches. BMC Medical Research Methodology, 12(1), 132. https://doi.org/10.1186/1471-2288-12-132.

132.

*Smyth

J. D.

Dillman

D. A.

Christian

L. M.

O’Neill

A. C.

(2010). Using the Internet to survey small towns and communities: Limitations and possibilities in the early 21st century. American Behavioral Scientist, 53(9), 1423-1448. https://doi.org/10.1177/0002764210361695.

133.

Smyth

J. D.

Olson

Millar

M. M.

(2014). Identifying predictors of survey mode preference. Social Science Research, 48, 135-144. https://doi.org/10.1016/j.ssresearch.2014.06.002.

134.

Statista (2018). Mobile internet usage worldwide - Statistics & Facts, Statista. Available at: https://www.statista.com/topics/779/mobile-internet/.

135.

*Szoc

Pflieger

Barlas

(2013). Great expectations: Changing mode of survey data collection in military populations. In: Federal Committee on Statistical Methodology annual meeting, Washington, DC.

136.

Teo

T. S. H.

Lim

V. K. G.

Lai

R. Y. C.

(1999). Intrinsic and extrinsic motivation in Internet usage. Omega, 27(1), 25–37. https://doi.org/10.1016/S0305-0483(98)00028-0.

137.

Turner

Viera

Marsh

(2010). Offering a web option in a mail survey of young adults: Impact on survey quality. The American Association for Public Opinion Research 65th Annual Conference Chicago, Illinois, May 13-16, 2010.

138.

Van Deursen

Van Dijk

(2011). Internet skills and the digital divide. New Media & Society, 13(6), 893–911. https://doi.org/10.1177/1461444810386774.

139.

Van Dijk

(2006). Digital divide research, achievements and shortcomings. Poetics, 34(4‐5), 221-235. https://doi.org/10.1016/j.poetic.2006.05.004

140.

Vandenbroucke

J.P

(1998). Observational research and evidence-based medicine: What should we teach young physicians?. Journal of Clinical Epidemiology, 51(6), 467–472. https://doi.org/10.1016/s0895-4356(98)00025-0.

141.

*Vehovar

Manfreda

K. L.

Batagelj

(2001). Sensitivity of electronic commerce measurement to the survey instrument. International Journal of Electronic Commerce, 6(1), 31-51. https://doi.org/10.1080/10864415.2001.11044230.

142.

Viechtbauer

(2007). Accounting for heterogeneity via random-effects models and moderator analyses in meta-analysis. Zeitschrift für Psychologie/Journal of Psychology, 215(2), 104–121. https://doi.org/10.1027/0044-3409.215.2.104

143.

Viechtbauer

(2010). Conducting meta-analyses in R with the metafor package. Journal of Statistical Software, 36(3), 1–48. https://doi.org/10.18637/jss.v036.i03.

144.

*Weible

Wallace

(1998). Cyber research: The impact of the Internet on data collection. Marketing Research, 10(3), 18-31.

145.

Williams

Brick

J. M.

(2018). Trends in U.S. face-to-face household survey nonresponse and level of effort. Journal of Survey Statistics and Methodology, 6(2), 186–211. https://doi.org/10.1093/jssam/smx019.

146.

*Wolfe

E. W.

Converse

P. D.

Oswald

F. L.

(2008). Item-level non-response rates in an attitudinal survey of teachers delivered via mail and Web. Journal of Computer-Mediated Communication, 14(1), 35-66. https://doi.org/10.1111/j.1083-6101.2008.01430.x.

147.

*Woo

Kim

Couper

M. P.

(2015). Comparing a cell phone survey and a web survey of university students. Social Science Computer Review, 33(3), 399-410. https://doi.org/10.1177/0894439314544876.

148.

World Bank (2020). World Bank Open Data, Education/Individuals using the Internet (% of population)/ Mobile cellular subscriptions (per 100 people)/ Population growth (annual %). Available at: https://data.worldbank.org/, (accessed 5 May, 2020).

149.

Wright

K. B.

(2005). Researching internet-based populations: Advantages and disadvantages of online survey research, online questionnaire authoring software packages, and web survey services. Journal of Computer-Mediated Communication, 10(3). https://doi.org/10.1111/j.1083-6101.2005.tb00259.x.

150.

*Wygant

Lindorf

(1999). Surveying collegiate Net surfers—Web methodology or mythology. Quirk’s Marketing Research Review. Lindorf (1999), “Surveying Collegiate Net Surfers—Web Methodology or Mythology,” Quirk’s Marketing Research Review. https://www.quirks.com/articles/surveying-collegiate-net-surfersweb-methodology-or-mythology.

151.

Yetter

Capaccioli

(2010). Differences in responses to Web and paper surveys among school professionals. Behavior Research Methods, 42(1), 266–272. https://doi.org/10.3758/BRM.42.1.266.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.35 MB

A Meta-Analysis of How Country-Level Factors Affect Web Survey Response Rates

Abstract

Keywords

Survey Response in a Cross-Cultural Setting

Country-Specific Predictors of Web Survey Response Rates

Political and Demographic Factors

Economic Factors

Technological Development Factors

Socio-Cultural Factors

Survey Design Predictors of Web Survey Response Rates

The Present Study

Method

Measurement of Political and Demographic, Economic, Technological, and Socio-Cultural Factors

Overview of Meta-Analytic Procedure

Eligibility Criteria and Search Strategy

Statistical Methodology and Effect Sizes

Results

Study Characteristics and Sensitivity Analyses

Cultural Differences in Web Surveys

Country-Specific Predictors of the Success of Web Surveys 11

Discussion

Summary of Results

Limitations and Further Research

Conclusion and Practical Implications

Supplemental Material

sj-pdf-1-mre-10.1177_14707853211050916 – Supplemental Material for A Meta-Analysis of How Country-Level Factors Affect Web Survey Response Rates

Footnotes

Funding

Data Availability

ORCID iDs

Supplementary material

Notes

References

Supplementary Material

Country-Specific Predictors of the Success of Web Surveys¹¹