This paper discusses the strengths and limitations of experimentally-orientated research in sociology. In principal, results from experiments cannot be generalized. Nevertheless they allow controlling for certain conditions that are impossible to measure during a survey. This paper gives an example: It presents findings from a quasi-experimental research on interviewer falsifications. Interviewers conducted real standardized interviews and subsequently falsified corresponding interviews in the lab. This enables a comparison of real and falsified survey data. Our research design has the limitation that there is no proof that actual falsifiers in an actual survey fieldwork environment would have falsified interviews in the same way as did the participants of our study. However, only this quasi-experimental approach allows us to know for sure which interviews are falsified.
AjzenI (1991) The Theory of Planned Behavior. Organizational Behavior and Human Decision Processes50: 179–211.
3.
AjzenIKrebsD (1994) Attitude Theory and Measurement - Implications for Survey Research. In: BorgIMohlerP (eds) Trends and Perspectives in Empirical Social Research. Berlin: de Gruyter, 250–65.
4.
AngristJDPischkeJ-S (2015) Mastering Metrics - The Path from Cause to Effect. Princeton: Princeton University Press, 1–33.
5.
BiemerPPStokesSL (1989) The Optimal Design of Quality Control Samples to Detect Interviewer Cheating. Journal of Official Statistics5(1): 23–39.
6.
BredlSWinkerPKötschauK (2012) A Statistical Approach to Detect Interviewer Falsification of Survey Data. Survey Methodology38(1): 1–10.
7.
BredlSStorfingerNMenoldN (2013) A Literature Review of Methods to Detect Fabricated Survey Data. In: WinkerPMenoldNPorstR (eds) Interviewers’ Deviations in Surveys. Frankfurt: Peter Lang, 3–24.
FranzenAPointnerS (2013) The External Validity of Giving in the Dictator Game. Experimental Economics16(2): 155–69.
10.
GESIS - Leibniz-Institut für Sozialwissenschaften (2013) German General Social Survey - ALLBUS 2010. Köln: GESIS Datenarchiv. ZA4612 Version 1.0.1.
11.
GESIS - Leibniz-Institut für Sozialwissenschaften (2015) German General Social Survey - ALLBUS 2008. Köln: GESIS Datenarchiv. ZA4602 Version 1.0.0.
12.
GrovesRMMagilavyLJ (1986) Measuring and Explaining Interviewer Effects in Centralized Telephone Surveys. Public Opinion Quarterly50(2): 251–66.
13.
ImbensGWRubinDR (2015) Causal Inference for Statistics, Social, and Biomedical Sciences - An Introduction. New York (NY): Cambridge University Press.
14.
KaaseMMarshA (1979) Political Action - A Theoretical Perspective. In: BarnesSHKaaseMAllerbeckKRFarahBGHeunksFInglehartRJenningsMKKlingemannDMarshARosenmayrL (eds) Political Action - Mass Participation in Five Western Democracies. Beverly Hills (CA): Sage, 27–56.
15.
KemperCJMenoldN (2014) Nuisance or Remedy? The Utility of Stylistic Responding as an Indicator of Data Fabrication in Surveys. Methodology10(3): 92–99.
16.
KochA (1995) Gefälschte Interviews - Ergebnisse der Interviewerkontrolle beim ALLBUS 1994. ZUMA-Nachrichten36: 89–105.
17.
KrejsaEADavisMCHillJM (1999) Evaluation of the Quality Assurance Falsification Interview used in the Census 2000 Dress Rehearsal. In: Proceedings of the American Statistical Association (Survey Research Methods Section), 635–40.
18.
KrosnickJAAlwinD (1987) An Evaluation of a Cognitive Theory of Response-order Effects in Survey Measurement. Public Opinion Quarterly51(2): 201–19.
19.
LandrockU (2017a) Explaining Political Participation - A Comparison of Real and Falsified Survey Data. Statistical Journal of the IAOS32(3): 447–58.
20.
LandrockU (2017b) How Interviewer Effects Differ in Real and Falsified Survey Data - Using Multilevel Analysis to Identify Interviewer Falsifications. mda: forthcoming.
21.
LandrockUMenoldN (2016) Validation of Theoretical Assumptions with Real and Falsified Survey Data. Statistical Journal of the IAOS32(3): 305–12.
22.
LevittSDListJA (2007) What Do Laboratory Experiments Measuring Social Preferences Reveal about the Real World?Journal of Economic Perspectives21(2): 153–74.
23.
LiJBrickJMTranBSingerP (2009) Using Statistical Models for Sample Design of a Reinterview Program. In: Proceedings of the American Statistical Association (Survey Research Methods Section), 4681–95.
24.
MangioneTWFowlerFJLouisTA (1992) Question Characteristics and Interviewer Effects. Journal of Official Statistics8(3): 293–307.
25.
MenoldNWinkerPStorfingerNKemperCJ (2013) A Method for Ex-post-identification of Falsifications in Survey Data. In: WinkerPMenoldNPorstR (eds) Interviewers’ Deviations in Surveys. Frankfurt: Peter Lang, 25–48.
26.
MenoldNKemperCJ (2014) How do Real and Falsified Data Differ? Psychology of Survey Response as a Source of Falsification Indicators in Face-to-face Surveys. International Journal of Public Opinion Research26(1): 41–65.
27.
OppKDFinkelSE (2001) Politischer Protest, Rationalität und Lebensstile. Eine empirische Überprüfung alternativer Erklärungsmodelle. In: KochASchmidtPWasmerM (eds) Politische Partizipation in der Bundesrepublik Deutschland. Empirische Befunde und theoretische Erklärungen. Opladen: Leske und Budrich, 73–108.
28.
PagèsFTribelMBonis-CharancleA (2006) Indélicatesse et manque de rigueur dans les sondages - Enquête auprès des professionnels du sondage. BMS (89): 31–48.
29.
ReubandKH (1990) Interviews, die keine sind. „Erfolge” und „Mißerfolge” beim Fälschen von Interviews. Kölner Zeitschrift für Soziologie und Sozialpsychologie42(4): 706–33.
30.
SchnellR (1991) Der Einfluss gefälschter Interviews auf Survey-Ergebnisse. Zeitschrift für Soziologie20(1): 25–35.
31.
SchraeplerJPWagnerGG (2003) Identification, Characteristics and Impact of Faked Interviews in Surveys. An Analysis by Means of Genuine Fakes in the Raw Data of SOEP. IZA Discussion Paper No. 969. Forschungsinstitut zur Zukunft der Arbeit.
32.
ShadishWRCookTDCampbellDT (2002) Experimental and Quasi-Experimental Designs for Generalized Causal Inference. Belmont (CA): Wadsworth Cengage Learning.
33.
SodeurW (2007) Entscheidungsspielräume von Interviewern bei der Wahrscheinlichkeitsauswahl. Methoden Daten Analysen1(2): 107–30.
34.
TurnerCFGribbeJNAl-TayyipAAChromyJR (2002) Falsification in Epidemiologic Surveys - Detection and Remediation (Prepublication Draft). Technical Papers on Health and Behavior Measurement No. 53. Research Triangle Institute. Washington (DC).
35.
van der ZouwenJ (2006) Interviewer and Survey Researcher - Mutual Dependencies. BMS (89): 49–64.
36.
van MeterK (2005) Studying Survey Interviewers - A Call for Research and an Overview. BMS (88): 51–71.
37.
WinkerP (2016) Assuring the Quality of Survey Data - Incentives, Detection and Documentation of Deviant Behavior. Statistical Journal of the IAOS32(3): 295–303.
38.
WinkerPKruseKWMenoldNLandrockU (2015) Interviewer Effects in Real and Falsified Interviews - Results from a Large Scale Experiment. Statistical Journal of the IAOS31(3): 423–34.
39.
WolbringTKeuschniggM (2015) Feldexperimente in den Sozialwissenschaften - Grundlagen, Herausforderungen, Beispiele. In: KeuschniggMWolbringT (eds) Experimente in den Sozialwissenschaften (Soziale Welt Special Issue22). Baden-Baden: Nomos, 219–45.
