The Internet is used more and more to conduct surveys. However, moving from traditional modes of data collection to the Internet may threaten the comparability of the data if the mode has an impact on the way respondents answer. In previous research, Revilla and Saris (2012) find similar average quality (defined as the product of reliability and validity) for several survey questions when asked in a face-to-face interview and when asked online. But does this mean that the mode of data collection does not have an impact on the quality? Or may it be that for some respondents the quality is higher for Web surveys whereas for others it is lower, such that on an average the quality for the complete sample is similar? Comparing the quality for different groups of respondents in a face-to-face and in a Web survey, no significant impact of the background characteristics, the mode and the interaction between them on the quality is found.
AlwinDFKrosnickJA (1991) The Reliability of Survey Attitude Measurement. Sociological Methods and Research20(1):139–81.
2.
AndrewsF (1984) Construct Validity and Error Components of Survey Measures: A Structural Modeling Approach. Public Opinion Quarterly, 46:409–42. Reprinted in Saris WE and van Meurs A (1990) Evaluation of Measurement Instruments by Metaanalysis of Multitrait multimethod studies. Amsterdam: North-Holland.
3.
CampbellDTFiskeDW (1959) Convergent and Discriminant Validation by the Multitrait-Multimethod Matrix. Psychological Bulletin6:81–105.
4.
CouperMPMillerPV (2009) Introduction to the Special Issue. Public Opinion Quarterly72(5): 831–35.
5.
DillmanDAChristianLM (2005) Survey Mode as a Source of Instability in Responses across Surveys. Field Methods17(1): 30–52.
6.
FrickerSGalesicMTourangeauRYanT (2005) An Experimental Comparison of Web and Telephone Surveys. Public Opinion Quarterly69: 370–92.
7.
HeerweghD (2009) Mode Differences between Face-to-Face and Web Surveys: An Experimental Investigation of Data Quality and Social Desirability Effects. International Journal of Public Opinion Research21(1): 111–19.
8.
HeerweghDLoosveldtG (2009) Face-to-Face Versus Web Surveying in a High-Internet-Coverage Population. Differences in Response Quality. Public Opinion Quarterly72(5): 836–46.
9.
JöreskogKGSörbomD (1991) Lisrel VII: A Guide to the Program and Applications. Chicago, IL: SPSS.
10.
KaplowitzMDHadlockTDLevineR (2004) A Comparison of Web and Mail Survey Response Rates. Public Opinion Quarterly68(1): 94–101.
11.
KreuterFPresserSTourangeauR (2009) Social Desirability Bias in CATI, IVR and Web Surveys: The Effects of Mode and Question Sensitivity. Public Opinion Quarterly72(5): 847–65.
12.
KrosnickJA. (1999) Survey Research. Annual Review of Psychology50: 537–67.
13.
RevillaM (2010) Quality in Unimode and Mixed-Mode Designs: A Multitrait-Multimethod Approach. Survey Research Methods4(3): 151–64.
14.
RevillaMSarisWE (2010) A Comparison of Surveys Using Different Modes of Data Collection: ESS versus Liss Panel. RECSM Working paper13.
15.
RevillaMSarisWE (2012) A Comparison of the Quality of Questions in a Face-to-face and a Web Survey. International Journal of Public Opinion Research. Advance Access published 17 April 2012, doi:10.1093/ijpor/eds007.
16.
RhodesSDBowieDAHergenratherKC (2003) Collecting Behavioural Data Using the World Wide Web: Considerations for Researchers. Journal of Epidemiology and Community Health57(1): 68–73.
17.
SarisWE (2008) Something Has to Be Done to Protect the Public against Bad Web Surveys. WAPOR Conference Cadenabbia VII: On Misapprehended Quality Criteria, Online Polls and Horoscopes, Lake Como, Italy, 10-12 July 2008.
18.
SarisWE.AndrewsFM (1991) Evaluation of Measurement Instruments Using a Structural Modeling Approach. In BiemerPPGrovesRMLybergLEMathiowetzNASudmanS (eds) Measurement Errors in Surveys. New York City John Wiley & Sons, Inc., 575–97.
19.
SarisWESatorraACoendersG (2004) A New Approach to Evaluating the Quality of Measurement Instruments: The Split-ballot MTMM Design. Sociological Methodology34(1): 311–47.
20.
SarisWE.GallhoferI (2007) Design, Evaluation, and Analysis of Questionnaires for Survey Research. New York City John Wiley & Sons, Inc.
21.
SarisWESatorraAVan der VeldWM. (2009) Testing Structural Equation Models or Detection of Misspecifications?Structural Equation Modeling: A Multidisciplinary Journal16(4): 561–82.
22.
ScherpenzeelA (1995) Meta Analysis of a European Comparative Study. In SarisWMünnichA (eds) The Multitrait-Multimethod Approach to Evaluate Measurement Instruments. Budapest: Eötvös University Press, Budapest225–42.
SmythJDChristianLHDillmanDA (2008) Does “Yes or No” on the Telephone Mean the Same as “Check-all-that-apply” on the Web?Public Opinion Quarterly72(1): 103–13.
25.
ToepoelVDasMVan SoestA (2005) Design of Web Questionnaires: A Test for Number of Items per Screen. CentER Discussion Paper No. 2005-114. Available from SSRN at: http://ssrn.com/abstract=852704.
26.
Van der VeldWMSarisWESatorraA (2009) Jrule 2.0: User manual. Unpublished manuscript, Internal Report, Radboud University Nijmegen, The Netherlands.
27.
Van MeursASarisWE (1990) Memory Effects in MTMM Studies. In SarisWEvan MeursA (eds.) Evaluation of Measurement Instruments by Meta-analysis of Multitrait-multimethod Studies. Amsterdam: North Holland, 134–46.
28.
VoogtRJJSarisWE (2005) Mixed Mode Designs: Finding the Balance between Nonresponse Bias and Mode Effects. Journal of Official Statistics21: 367–87.
