Survey interviewers can negatively affect survey data by introducing variance and bias into estimates. When investigating these interviewer effects, research typically focuses on interviewer sociodemographics with only a few studies examining the effects of characteristics that are not directly visible such as interviewer attitudes, opinions, and personality. For the study at hand, self-reports of 1,212 respondents and 116 interviewers, as well as their interpersonal perceptions of each other, were collected in a large-scale, face-to-face survey of households in Germany. Respondents and interviewers were presented with the same questions regarding their opinions and mutual perceptions toward social and political issues in Germany. Analyses show that interviewer effects can be largely explained by how an interviewer is seen by respondents. This indicates that some respondents adjust their answers toward anticipated interviewer opinions. Survey practitioners ought to acknowledge this in their survey design and training of interviewers.
ADM, Arbeitskreis Deutscher Markt- und Sozialforschungsinstitute e.V. 2018. Quantitative Interviews by ADM Member Agencies and by Method of Interview. Retrieved September 16, 2019 (https://www.adm-ev.de/die-branche/mafo-zahlen/).
3.
AmbadyNaliniHallahanMarkRosenthalRobert. 1995. “On Judging and Being Judged Accurately in Zero-acquaintance Situations.” Journal of Personality and Social Psychology69(3):518–29.
4.
American Association for Public Opinion Research. 2016. Standard Definitions: Final Dispositions of Case Codes and Outcome Rates for Surveys. 9th ed. Lenexa, KS: American Association for Public Opinion Research.
5.
AtheyK. R.ColemanJoan E.ReitmanAudrey P.TangJenny. 1960. “Two Experiments Showing the Effect of the Interviewer’s Racial Background on Responses to Questionnaires Concerning Racial Issues.” Journal of Applied Psychology44(4):244–46.
6.
BiemerPaul. B.LybergLars. E.. 2003. “Errors due to Interviewers and Interviewing (Chapter 5).” Pp. 149–187 in Introduction to Survey Quality, edited by LybergLars E.BiemerPaul P.. Hoboken, NJ: Wiley.
7.
BorkenauPeterLieblerAnette. 1993. “Convergence of Stranger Ratings of Personality and Intelligence with Self-ratings, Partner Ratings, and Measured Intelligence.” Journal of Personality and Social Psychology65(3):546–53.
8.
BoydHarper W.WestfallRalph. 1955. “Interviewers as a Source of Error in Surveys.” Journal of Marketing19(4):311–24.
9.
BrunswikEgon. 1956. Perception and the Representative Design of Psychological Experiments. 2nd ed. Berkeley: University of California Press.
10.
Brunton-SmithIanSturgisPeterLeckieGeorge. 2017. “Detecting and Understanding Interviewer Effects on Survey Data by using a Cross-classified Mixed Effects Location-scale Model.” Journal of the Royal Statistical Society Series A180(2):551–68.
CannellC. F.1977. A Summary of Studies on Interviewing Methodology. Data Evaluation and Methods Research Series 2 Number 69, DHEW Publication No. (HRA) 77-1343, U.S. Department of Health, Education, and Welfare, National Center for Health Statistics, Rockville, MD.
13.
CannellC. F.MillerPeter V.OksenbergLois. 1981. “Research on Interviewing Techniques.” Pp. 389–437 in Sociological Methodology, edited by LeinhardtSamuel. San Francisco, CA: Jossey-Bass.
14.
CataniaJosephBinsonDiane. 1996. “Effects of Interviewer Gender, Interviewer Choice, and Item Wording on Responses to Questions Concerning Sexual Behavior.” Public Opinion Quarterly60(3):345–75.
15.
CohenJacob. 1960. “A Coefficient of Agreement for Nominal Scales.” Educational and Psychological Measurement20:37–46.
16.
DurrantGabriele B.D’ArrigoJulia. 2014. “Doorstep Interactions and Interviewer Effects on the Process Leading to Cooperation or Refusal.” Sociological Methods & Research43(3):490–518.
17.
DurrantGabriele B.GrovesRobert M.StaetskyLauraSteeleFiona. 2010. Effects of Interviewer Attitudes and Behaviors on Refusal in Household Surveys. Public Opinion Quarterly74(1):1–36.
18.
FendrichMichaelJohnsonTimothyWislarJoseph S.ShaligramChitra. 1999. “The Impact of Interviewer Characteristics on Cocaine Use Underreporting by Male Juvenile Arrestees.” Journal of Drug Issues29(1):1014–19.
19.
GerlitzJean-YvesSchuppJürgen. (2005). Zur Erhebung der Big-Five-basierten Persoünlichkeitsmerkmale im SOEP. Dokumentation der Instrumentenentwicklung BFI-S auf Basis des SOEP-Pretests 2005. DIW Research Note 4, Berlin. Retrieved September 16, 2019 (https://www.diw.de/documents/%20publikationen/73/diw_01.c.43490.de/rn4.pdf).
20.
GoebelJanGrabkaMarkus M.LiebigStefanKrohMartinRichterDavidCarstenSchroüderSchuppJürgen. (2018). “The German Socio-economic Panel (SOEP).”Journal of Economics and Statistics239(2):345–60.
21.
GrovesRobert. M.2004. Survey Error and Survey Costs. Hoboken, NJ: John Wiley.
22.
GrovesRobert M.FultzNancy. H.. 1985. “Gender Effects among Telephone Interviewers in a Survey of Economic Attitudes.” Sociological Methods & Research14(1):31–52.
23.
GrovesRobert M.MagilavyLou J.. 1986. “Measuring and Explaining Interviewer Effects in Centralized Telephone Surveys.” Public Opinion Quarterly50(2): 251–66.
24.
HealyAndrewMalhotraNeil. 2014. “Partisan Bias among Interviewers.” Public Opinion Quarterly78(2):485–99.
25.
HilgertLuisaKrohMartinRichterDavid. 2016. “The Effect of Face-to-face Interviewing on Personality Measurement.” Journal of Research in Personality63:133–36.
26.
HimeleinKristen. 2016. “Interviewer Effects in Subjective Survey Questions: Evidence from Timor-Leste.” International Journal of Public Opinion Research. 28(4):511–33.
27.
HymanHerbert H.1954. Interviewing in Social Research. Chicago: University of Chicago Press.
28.
JaückleAnetteLynnPeterSinibaldiJenniferTippingSarah. (2013). “The Effect of Interviewer Experience, Attitudes, Personality and Skills on Respondent Co-operation with Face-to-face Surveys.”Survey Research Methods7(1):1–15.
29.
KaneEmily W.MacaulayLaura J.. 1993. “Interviewer Gender and Gender Attitudes.” Public Opinion Quarterly57:1–28.
30.
KatzDaniel. 1942. “Do Interviewers Bias Poll Results?” Public Opinion Quarterly6:248–68.
31.
KishLeslie. 1962. “Studies of Interviewer Variance for Attitudinal Variables.” Journal of the American Statistical Association57(297):92–115.
32.
KrohMartinKuühneSimonSiegersRainerBelchevaVeronika. (2018). “SOEP-core—Documentation of Sample Sizes and Panel Attrition (1984 until 2016).” SOEP Survey Papers No. 480. Tech. rep. 480, DIW Berlin, Germany.
33.
KrumpalIvar. 2013. “Determinants of Social Desirability Bias in Sensitive Surveys: A Literature Review.” Quality & Quantity47:2025–47.
34.
KühneS.2018. “From Strangers to Acquaintances? Interviewer Continuity and Socially Desirable Responses in Panel Surveys.” Survey Research Methods12(2):121–46.
35.
LevesqueMaurice J.KennyDavid A.. 1993. “Accuracy of Behavioral Predictions at Zero Acquaintance: A Social Relations Analysis.” Journal of Personality and Social Psychology65(6):1178–87.
36.
LippsOliverLutzGeorg. 2010. “How Answers on Political Attitudes Are Shaped by Interviewers: Evidence from a Panel Survey.” Swiss Journal of Sociology36(2):345–58.
37.
LoosveldtG. (2008). “Face-to-face Interviews.” Pp. 201–220 in International Handbook of Survey Methodology, edited by LeeuwEdith D. DeHoxJoop J.DillmanDon A.. New York: Taylor & Francis Group.
38.
MahalabonisPrasanta C.1946. “Recent Experiments in Statistical Sampling in the Indian Statistical Institute.” Journal of the Royal Statistical Society109:325–70.
39.
McCraeRobert R.CostaPaul T.Jr (1987). “Validation of the Five-factor Model of Personality across Instruments and Observers.” Journal of Personality and Social Psychology52(1):81–90.
40.
NestlerSteffenBackMitja D.. 2013. “Applications and Extensions of the Lens Model to Understand Interpersonal Judgments at Zero Acquaintance.” Current Directions in Psychological Science22:374–79.
41.
NestlerSteffenEgloffBorisKuüfnerAlbrecht C. P.BackMitja D.. 2012. “An Integrative Lens Model Approach to Bias and Accuracy in Human Inferences: Hindsight Effects and Knowledge Updating in Personality Judgments.” Journal of Personality and Social Psychology103(4):689–717.
42.
NyboAndersenAnne-MarieOlsenJorn. 2002. “Do Interviewers’ Health Beliefs and Habits Modify Responses to Sensitive Questions? A Study Using Data Collected from Pregnant Women by Means of Computer-assisted Telephone Interviews.” American Journal of Epidemiology155(1):95–100.
43.
OlsonKristenPeytchevAndrew. (2007). “Effect of Interviewer Experience on Interview Pace and Interviewer Attitudes.” Public Opinion Quarterly71(2):273–86.
44.
O’MuircheartaighColm A.CampanelliPamela. 1998. “The Relative Impact of Interviewer Effects and Sample Design Effects on Survey Precision.” Journal of the Royal Statistical Society Series A161(1):63–77.
45.
RichterDavidSchuppJürgen. 2015. “The SOEP Innovation Sample (SOEP IS).” Schmollers Jahrbuch135(3):89–399.
46.
RosenthalRobertRubinDonald B.. 1978. “Interpersonal Expectancy Effects: The First 345 Studies.” Behavioral and Brain Sciences1(3):377–86.
47.
SamochowiecJakubMichaelaWaünkeFiedlerKlaus. 2010. “Political Ideology at Face Value.” Social Psychology and Personality Science1(3):206–12.
48.
SchaefferNora C. (2018). “Interviewer Deviations from Scripts.” Pp. 465–472 in The Palgrave Handbook of Survey Research, edited by VannetteDavid L.KrosnickJon A.. London, England: Palgrave Macmillan.
49.
SchaefferNora C.DykemaJenniferMaynardDouglas W.. 2010. “Interviewers and Interviewing.” Pp. 437–470Handbook of Survey Research, edited by MarsdenPeter V.WrightJames D.. Binley, England: Emerald.
50.
SchnellRainerKreuterFrauke. 2005. “Separating Interviewer and Sampling-point Effects.” Journal of Official Statistics21(3):389–410.
51.
SchumanHowardConverseJean M.. 1971. “The Effects of Black and White Interviewers on Black Responses in 1968.” Public Opinion Quarterly35(1):44–68.
52.
SingerEleanorFrankelMartin R.GlassmanMarc B.. 1983. “The Effect of Interviewer Characteristics and Expectations on Response.” Public Opinion Quarterly47:68–83.
53.
SingerEleanorKohnke-AguirreLuane. (1979). “Interviewer Expectation Effects: A Replication and Extension.” Public Opinion Quarterly43(2):245–60.
54.
SmitJohannes H.DijkstraWilZouwenJohannes van der. 1997. “Suggestive Interviewer Behavior in Surveys: An Experimental Study.” Journal of Official Statistics13(1):19–28.
55.
SmithHarry L.HymanHerbert H.. 1950. “The Biasing Effect of Interviewer Expectations on Survey Results.” Public Opinion Quarterly1:491–506.
56.
SpearmanCharles. 1904. “The Proof and Measurement of Association between Two Things.” American Journal of Psychology15:72–101.
57.
SudmanSeymourBradburnNorman M.. 1974. Response Effects in Surveys. Chicago: Aldine.
58.
SudmanSeymourBradburnNorman M.BlairEdStockingCarol. 1977. “Modest Expectations. The Effects of Interviewers’ Prior Expectations on Responses.” Sociological Methods & Research6(2):171–82.
59.
TourangeauRogerRipsLance J.RasinskiKenneth. 2000. The Psychology of Survey Response. Cambridge, England: Cambridge University Press.
60.
TulvingEndelSchacterDaniel L.. (1990). “Priming and Human Memory Systems.” Science247(4940):301–6.
61.
Von SandenNicholas D. (2004). “Interviewer effects in household surveys: estimation and design.” PhD thesis, School of Mathematics and Applied Statistics, University of Wollongong. (http://ro.uow.edu.au/theses/312)
62.
WebsterCynthia. 1996. “Hispanic and Anglo Interviewer and Respondent Ethnicity and Gender: The Impact on Survey Response Quality.” Journal of Marketing Research33:62–72.
63.
WeeksMichael F.Paul MooreR.. 1981. “Ethnicity-of-interviewer Effects on Ethnic Respondents.” Public Opinion Quarterly45(2):245–49.
64.
WestBrady T.BlomAnnelies G.. 2017. “Explaining Interviewer Effects: A Research Synthesis.” Journal of Survey Statistics and Methodology5(2):175–211.
65.
WestBrady T.KreuterFraukeJaenichenUrsula. 2013. “‘Interviewer’ Effects in Face-to-face Surveys: A Function of Sampling, Measurement Error, or Nonresponse?” Journal of Official Statistics29(2):277–97
66.
WilliamsJ. AllenJr. 1964. “Interviewer-respondent Interaction: A Study of Bias in the Information Interview.” Sociometry27(3):338–52.
67.
WilliamsJ. AllenJr. 1968. “Interviewer Role Performance: A Further Note on Bias in the Information Interview.” Public Opinion Quarterly32(2):287–94.
68.
WinkerPeterMenoldNataljaPorstRolf, eds. 2013. Interviewers’ Deviations in Surveys: Impact, Reasons, Detection and Prevention. New York: Peter Lang.
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
