This article is an empirical contribution to the evaluation of the randomized response technique (RRT), a prominent procedure to elicit more valid responses to sensitive questions in surveys. Based on individual validation data, we focus on two questions: First, does the RRT lead to higher prevalence estimates of sensitive behavior than direct questioning (DQ)? Second, are there differences in the effects of determinants of misreporting according to question mode? The data come from 552 face-to-face interviews with subjects who had been convicted by a court for minor criminal offences in a metropolitan area in Germany. For the first question, the answer is negative. For the second, it is positive, that is, effects of individual and situational determinants of misreporting differ between the two question modes. The effect of need for social approval, for example, tends to be stronger in RRT than in DQ mode. Interviewer experience turns out to be positively related to answer validity in DQ and negatively in RRT mode. Our findings support a skeptical position toward RRT, shed new light on long-standing debates within survey methodology, and stimulate theoretical reasoning about response behavior in surveys.
AquilinoWilliam S.WrightDebra L.SuppleAndrew J.. 2000. “Response Effects Due to Bystander Presence in CASI and Paper-and-Pencil Surveys of Drug Use and Alcohol Use.” Substance Use & Misuse35:845–67.
2.
BeldtSandra F.DanielWayne W.GarchaBikramjit S.. 1982. “The Takahasi-Sakasegawa Randomized Response Technique: A Field Test.” Sociological Methods & Research11:101–11.
3.
BernsteinRobertChadhaAnitaMontjoyRobert. 2001. “Overreporting Voting. Why it Happens and Why it Matters.” Public Opinion Quarterly65:22–44.
4.
BlankSara G.2008. Using the Randomized Response Technique to Investigate Illegal Fishing and Contribute to Abalone Management in Northern California. Wellington, New Zealand: Victoria University of Wellington.
5.
BoruchRobert F.1971. “Assuring Confidentiality of Responses in Social Research: A Note on Strategies.” American Sociologist6:308–11.
6.
BradburnNorman M.SudmanSeymour. 1979. Improving Interview Method and Questionnaire Design. San Francisco, CA: Jossey-Bass.
BurrisJane E.JohnsonTimothy P.O’RourkeDiane P.. 2003. Validating Self-reports of Socially Desirable Behaviors. Paper presented at the annual meeting of the American Association for Public Opinion Research, Sheraton Music City, Nashville, TN. Retrieved 2013-08-05, fromwww.amstat.org. American Association for Public Opinion Research—Section on Survey Research Methods.
9.
CouttsElisabethJannBen. 2011. “Sensitive Questions in Online Surveys: Experimental Results for the Randomized Response Technique (RRT) and the Unmatched Count Technique (UCT).” Sociological Methods & Research40:169–93.
10.
CrowneDouglas P.MarloweDavid. 1960. “A New Scale of Social Desirability Independent of Psychopathology.” Journal of Consulting Psychology24:349–54.
11.
DiekmannAndreasJannBen. 2001. “Anreizformen und Ausschöpfungsquoten bei postalischen Befragungen. Eine Prüfung der Reziprozitätshypothese.” ZUMA-Nachrichten48:18–27.
12.
deJongMartijnGPietersRikFoxJean-Paul. 2010. “Reducing Social Desirability Bias Through Item Randomized Response: An Application to Measure Underreported Desires.” Journal of Marketing Research47:14–27.
13.
EdgellStephen E.HimmelfarbSamuelDuchanKaren L.. 1982. “Validity of Forced Responses in a Randomized Response Model.” Sociological Methods & Research11:89–100.
14.
EsserHartmut. 1986. “Können Befragte lügen? Zum Konzept des ‘wahren Wertes im Rahmen der handlungstheoretischen Erklärung von Situationseinflüssen bei der Befragung.” Kölner Zeitschrift für Soziologie und Sozialpsychologie38:314–36.
15.
EsserHartmut. 1991. “Die Erklärung systematischer Fehler in Interviews: Befragtenverhalten als “rational choice.” Pp. 59–78 in Person—Situation—Institution—Kultur, edited by WittenbergR.. Berlin, Germany: Duncker & Humblot.
16.
FolsomRalph E.1974. A Randomized Response Validation Study: Comparison of Direct and Randomized Reporting of DUI Arrests. Chapel Hill, NC: Research Triangle Institute.
17.
FolsomRalph E.GreenbergBernard G.HorvitzDaniel G.AbernathyJames R.. 1973. “The Two Alternate Questions Randomized Response Model for Human Surveys.” Journal of the American Statistical Association68:525–30.
18.
FoxJames AlanTracyPaul E.. 1986. “Randomized Response. A Method for Sensitive Surveys.” Sage University Paper Series on Quantitative Applications in the Social Sciences Vol. 07-058, Sage, Newbury Park, CA.
19.
GansterDaniel C.HennesseyHarry W.LuthansFred. 1983. “Social Desirability Response Effects: Three Alternative Models.” Academy of Management Journal26:321–31.
20.
GreenbergBernard G.KueblerRoy R.AbernathyJames R.HorvitzDaniel G.. 1971. “Application of the Randomized Response Technique in Obtaining Quantitative Data.” Journal of the American Statistical Association66:243–50.
HolbrookAllyson L.KrosnickJon A.. 2010. “Measuring Voter Turnout by Using the Randomized Response Technique. Evidence Calling into Question the Method’s Validity.” Public Opinion Quarterly74:328–43.
23.
HorvitzDaniel G.ShahB. V.SimmonsWalt R.. 1967. “The Unrelated Question Randomized Response Model.” Proceedings of the American Statistical Association (Social Statistics Section):65–72.
24.
HoxJoop J.LeeuwEdith D. de. 2002. “The Influence of Interviewers’ Attitude and Behavior on Household Survey Nonresponse: An International Comparison.” Pp. 103–20 in Survey Nonresponse, edited by GrovesR. M.DillmanD. A.EltingeJ. L.LittleR. J. A.. New York: John Wiley.
25.
HymanHerbert. 1944. “Do They Tell the Truth?” Public Opinion Quarterly8:557–59.
26.
JannBen. 2005. RRLOGIT: Stata Module to Estimate Logistic Regression for Randomized Response Data. Boston, MA: Boston College Department of Economics.
27.
JannBen. 2008. RRREG: Stata Module to Estimate Linear Probability Model for Randomized Response Data. Boston, MA: Boston College Department of Economics.
28.
JohnsonTimothy P.FendrichMichaelMackesy-AmitiMary-Ellen. 2012. “An Evalutation of the Validity of the Crowne-Marlowe Need for Approval Scale.” Quality & Quantity46:1883–96.
29.
JonesElise F.ForrestJacqueline D.. 1992. “Underreporting of Abortion in Surveys of U.S. Women: 1976 to 1988.” Demography29:113–26.
30.
KingMaryon F.BrunerGordon C.. 2000. “Social Desirability Bias: A Neglected Aspect of Validity Testing.” Psychology & Marketing17:79–103.
31.
KreuterFraukePresserStanleyTourangeauRoger. 2008. “Social Desirability Bias in CATI, IVR, and Web Surveys. The Effects of Mode and Question Sensitivity.” Public Opinion Quarterly72:847–65.
32.
KrumpalIvar. 2009. Sensitive Questions and Measurement Error: Using the Randomized Response Technique to Reduce Social Desirability Bias in CATI Surveys. Leipzig, Germany: Universität Leipzig.
33.
KukAnthony Y. C. 1990. “Asking Sensitive Questions Indirectly.” Biometrika77:436–38.
34.
KulkaRichard A.WeeksMichael F.FolsomRobert E.. 1981. “A Comparison of the Randomized Approach and Direct Questioning Approach to Asking Sensitive Survey Questions.” Working Paper. Research Triangle Park, NC: Research Triangle Institute.
35.
LambCharles W.StemDonald E.. 1978. “An Empirical Validation of the Randomized Response Technique.” Journal of Marketing Research15:616–21.
36.
LandsheerJohannes A.HeijdenPeter van derGilsGer van. 1999. “Trust and Understanding. Two Psychological Aspects of Randomized Response.” Quality & Quantity33:1–12.
37.
LaraDianaStricklerJenniferOlavarrietaClaudia DíazEllertsonCharlotte. 2004. “Measuring Induced Abortion in Mexico. A Comparison of Four Methodologies.” Sociological Methods & Research32:529–58.
38.
Lensvelt-MuldersGerty J. L. M.BoeijeHennie R.. 2007. “Evaluating Compliance with a Computer Assisted Randomized Response Technique: A Qualitative Study into the Origins of Lying and Cheating.” Computers in Human Behavior23:591–608.
39.
Lensvelt-MuldersGerty J. L. M.HoxJoop J.PeterG. M.Heijdenvan der. 2005. “How to Improve the Efficiency of Randomised Response Designs.” Quality & Quantity39:253–65.
40.
Lensvelt-MuldersGerty J. L. M.HoxJoop J.Heijdenvan derPeterG. M.MaasCora J. M.. 2005. “Meta-analysis of Randomized Response Research. Thirty-five Years of Validation.” Sociological Methods & Research33:319–48.
41.
LiuP. T.ChowL. P.. 1976a. “The Efficiency of the Multiple Trial Randomized Response Technique.” Biometrics32:607–18.
42.
LiuP. T.ChowL. P.. 1976b. “A New Discrete Quantitative Randomized Response Model.” Journal of the American Statistical Association71:30–31.
43.
LocanderWilliamSudmanSeymourBradburnNorman. 1976. “An Investigation of Interview Method, Threat and Response Distortion.” Journal of the American Statistical Association71:269–75.
44.
MaddalaG. S.1983. Limited-dependent and Qualitative Variables in Econometrics. Cambridge, England: Cambridge University Press.
45.
McCraeRobert R.CostaPaul T.. 1983. “Social Desirability Scales: More Substance Than Style.” Journal of Consulting and Clinical Psychology51:882–88.
46.
MoormanRobert H.PodsakoffPhilip M.. 1992. “A Meta-analytic Review and Empirical Test of the Potential Confounding Effects of Social Desirability Response Sets in Organizational Behaviour Research.” Journal of Occupational and Organizational Psychology65:131–49.
47.
MoorsJ. J. A.1971. “Optimization of the Unrelated Question Randomized Response Model.” Journal of the American Statistical Association66:627–29.
48.
NordlundSturlaHolmeIngarTamsfossSteinar. 1994. “Randomized Response Estimates for the Purchase of Smuggled Liquor in Norway.” Addiction89:401–05.
49.
OstapczukMartinMuschJochenMoshagenMorten. 2009. “A Randomized-response Investigation of the Education Effect in Attitudes Towards Foreigners.” European Journal of Social Psychology39:920–31.
50.
PaulhusDelroy L.ReidDouglas B.. 1991. “Enhancement and Denial in Socially Desirable Responding.” Journal of Personality and Social Psychology60:307–17.
51.
PhillipsDerek L.ClancyKevin J.. 1972. “Some Effects of ‘Social Desirability’ in Survey Studies.” American Journal of Sociology77:921–40.
52.
RasinskiKenneth A.WillisGordon B.BaldwinAlison K.YehWenchiLeeLisa. 1999. “Methods of Data Collection, Perceptions of Risks and Losses, and Motivation to Give Truthful Answers to Sensitive Survey Questions.” Applied Cognitive Psychology13:465–84.
53.
ReckersPhilip M. J.WheelerStephen W.Wong-On-WingBernard. 1997. “A Comparative Examination of Auditor Premature Sign-offs Using the Direct and the Randomized Response Methods.” Auditing16:69–78.
54.
ReinmuthJames E.GeurtsMichael D.. 1975. “The Collection of Sensitive Information Using a Two-stage, Randomized Response Model.” Journal of Marketing Research12:402–07.
55.
SchnellRainerKreuterFrauke. 2005. “Separating Interviewer and Sampling-point Effects.” Journal of Official Statistics21:389–410.
56.
SilverBrian D.AbramsonPaul R.AndersonBarbara A.. 1986. “The Presence of Others and Overreporting of Voting in American National Elections.” Public Opinion Quarterly50:228–39.
57.
Skarbek-KozietulskaAnnaPreisendörferPeterWolterFelix. 2012. “Leugnen oder gestehen? Bestimmungsfaktoren wahrer Antworten in Befragungen.” Zeitschrift für Soziologie41:5–23.
58.
StadtmüllerSven. 2009. “Rücklauf gut, alles gut? Zu erwünschten und unerwünschten Effekten monetärer Anreize bei postalischen Befragungen.” Methoden—Daten—Analysen3:167–85.
59.
StockéVolker. 2004. “Entstehungsbedingungen von Antwortverzerrungen durch soziale Erwünschtheit. Ein Vergleich der Rational-choice Theorie und des Modells der Frame-Selektion.” Zeitschrift für Soziologie33:303–20.
60.
StockéVolker. 2007a. “Deutsche Kurzskala zur Erfassung des Bedürfnisses nach sozialer Anerkennung.” in ZUMA-Informationssystem. Elektronisches Handbuch sozialwissenschaftlicher Erhebungsinstrumente. ZIS Version 11.00, edited by A. Glöckner-Rist. Bonn, Germany: GESIS.
61.
StockéVolker. 2007b. “The Interdependence of Determinants for the Strength and Direction of Social Desirability Bias in Racial Attitude Surveys.” Journal of Official Statistics23:493–514.
62.
TezcanSabahatOmranAbdel R.. 1981. “Prevalence and Reporting of Induced Abortion in Turkey: Two Survey Techniques.” Studies in Family Planning12:262–71.
63.
TourangeauRogerYanTing. 2007. “Sensitive Questions in Surveys.” Psychological Bulletin133:859–83.
64.
TracyPaul E.FoxJames Alan. 1981. “The Validity of Randomized Response for Sensitive Measurements.” American Sociological Review46:187–200.
65.
UmeshU. N.PetersonRobert A.. 1991. “A Critical Evaluation of the Randomized Response Method. Applications, Validation, and Research Agenda.” Sociological Methods & Research20:104–38.
66.
van derHeijdenPeterG. M.GilsGer van. 1996. “Some Logistic Regression Models für Randomized Response Data.” Pp. 341–48 in Statistical Modelling. Proceedings of the 11th International Workshop on Statistical Modelling, edited by ForcinaA.MarchettiG. M.HatzingerR.PerugiaG. Galmacci. Rome: University of Perugia.
67.
van derHeijdenPeterG. M.GilsGer vanBoutsJanHoxJoop J.. 2000. “A Comparison of Randomized Response, Computer-assisted Self-interview, and Face-to-Face Direct Questioning. Eliciting Sensitive Information in the Context of Welfare and Unemployment Benefit.” Sociological Methods & Research28:505–37.
68.
WarnerStanley L.1965. “Randomized Response: A Survey Technique for Eliminating Evasive Answer Bias.” Journal of the American Statistical Association60:63–69.
69.
WimbushJames C.DaltonDan R.. 1997. “Base Rate for Employee Theft: Convergence of Multiple Methods.” Journal of Applied Psychology82:756–63.
70.
WolterFelix. 2012. Heikle Fragen in Interviews. Eine Validierung der Randomized Response-Technik. Wiesbaden: Springer VS.
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
