Gaining valid answers to so-called sensitive questions is an age-old problem in survey research. Various techniques have been developed to guarantee anonymity and minimize the respondent’s feelings of jeopardy. Two such techniques are the randomized response technique (RRT) and the unmatched count technique (UCT). In this study the authors evaluate the effectiveness of different implementations of the RRT (using a forced-response design) in a computer-assisted setting and also compare the use of the RRT to that of the UCT. The techniques are evaluated according to various quality criteria, such as the prevalence estimates they provide, the ease of their use, and respondent trust in the techniques. The results indicate that the RRTs are problematic with respect to several domains, such as the limited trust they inspire and nonresponse, and that the RRT estimates are unreliable due to a strong false no bias, especially for the more sensitive questions. The UCT, however, performed well compared to the RRTs on all the evaluated measures. The authors conclude that the UCT is a promising alternative to RRT in self-administered surveys and that future research should be directed toward evaluating and improving the technique.
Ahart, Alison M. and Paul R. Sackett.2004. ‘‘A New Method of Examining Relationships Between Individual Difference Measures and Sensitive Behavior Criteria: Evaluating the Unmatched Count Technique.’’ Organizational Research Methods7:101-14.
2.
Anderson, Drew A., Angela M. Simmons, Suzanne M. Milnes, and Mitchell Earleywine .2007. ‘‘Effect of Response Format on Endorsement of Eating Disordered Attitudes and Behaviors ’’ International Journal of Eating Disorders40:90-3.
3.
Barnett, Julie.1998. ‘‘Sensitive Questions and Response Effects: An Evaluation.’’ Journal of Managerial Psychology13:63-76.
4.
Boruch, Robert F.1971. ‘‘Assuring Confidentiality of Responses in Social Research: A Note on Strategies.’’The American Sociologist6:308-11.
5.
Chaudhuri, A. and A. Saha.2005. ‘‘Optional Versus Compulsory Randomized Response Techniques in Complex Surveys.’’ Journal of Statistical Planning and Inference135:516-27.
6.
Chaudhuri, Arijit and Tasos C. Christofides.2007. ‘‘Item Count Technique in Estimating the Proportion of People With a Sensitive Feature.’’Journal of Statistical Planning and Inference137:589-93.
7.
Clark, Stephen J. and Robert A. Desharnais.1998. ‘‘Honest Answers to Embarrassing Questions: Detecting Cheaters in the Randomized Response Model’’ Psychological Methods3:160-68.
8.
Dalton, Dan R., James C. Wimbush, and Catherine M. Daily .1994. ‘‘Using the Unmatched Count Technique (UCT) to Estimate Base Rates for Sensitive Behavior.’’Personnel Psychology47:817-28.
9.
Droitcour, Judith, Rachael A. Caspar, Michael L. Hubbard, Teresa L. Parsely, Wendy Vischer, and Trena M. Ezzati .1991. ‘‘The Item Count Technique as a Method of Indirect Questioning: A Review of Its Development and a Case Study Application.’’ Pp. 185-210 in Measurement Errors in Surveys, edited by P. Biemer, R. M. Groves, L. E. Lyberg, N. A. Mathiowetz, and S. Sudman.New York: John Wiley.
10.
Edgell, Stephen E., Samuel Himmelfarb, and Karen L. Duchan .1982. ‘‘Validity of Forced Responses in a Randomized-response Model.’’Sociological Methods & Research11:89-100.
11.
Folsom, Ralph E., Bernard G. Greenberg, Daniel G. Horvitz, and James R. Abernathy .1973. ‘‘The Two Alternate Questions Randomized Response Model for Human Surveys.’’Journal of the American Statistical Association68:525-30.
12.
Fox, James Alan and Paul E. Tracy.1986. Randomized Response: A Method for Sensitive Surveys . London: Sage.
13.
Goodstadt, Michael S. and Valerie Gruson.1975. ‘‘The Randomized Response Technique: A Test of Drug Use.’’Journal of the American Statistical Association70:814-18.
14.
Gosling, Samuel D., Simine Vazire, Sanjay Srivastava, and Oliver P. John.2004. ‘‘Should We Trust Web-based Studies? A Comparative Analysis of Six Preconceptions About Internet Questionnaires.’’American Psychologist59:93-104.
15.
Greenberg, Bernard G., Abdel-Latif A. Abul-Ela, Walt R. Simmons, and Daniel G. Horvitz .1969. ‘‘The Unrelated Question Randomized Response Model: Theoretical Framework.’’Journal of the American Statistical Association64:520-39.
16.
Himmelfarb, Samuel and Carl Lickteig.1982. ‘‘Social Desirability and the Randomized-response Technique.’’ Journal of Personality and Social Psychology43: 710-17.
17.
Holbrook, Allyson L. and Jon A. Krosnick.2010a. ‘‘Measuring Voter Turnout by Using the Randomized Response Technique: Evidence Calling Into Question the Method’s Validity.’’Public Opinion Quarterly74:328-43.
18.
Holbrook, Allyson L. and Jon A. Krosnick.2010b. ‘‘Social Desirability Bias in Voter Turnout Reports: Tests Using the Item Count Technique.’’Public Opinion Quarterly74:37-67.
19.
Jann, Ben.2005. ‘‘rrlogit: Stata Module to Estimate Logistic Regression for Randomized Response Data.’’Statistical Software ComponentsS456203. Boston College Department of Economics. ( http://ideas.repec.org/c/boc/bocode/s456203.html ).
20.
Krumpal, Ivar.2008. ‘‘Evaluation of the Effectiveness of the Randomized Response Technique and the Item Count Method in the Telephone Survey Mode .’’ In Proceedings of the 7th International Conference on Social Science Methodology, RC33-Logic and Methodology in Sociology (ISA) [CD-ROM], edited by S. Balbi, G. Scepi, G. Russolillo , and A. Stawinoga. Napoli: Jovene Editore.
21.
Kuk, Anthony Y. C.1990. ‘‘Asking Sensitive Questions Indirectly.’’Biometrika77:436-38.
22.
LaBrie, Joseph W. and Mitchell Earleywine.2000. ‘‘Sexual Risk Behaviors and Alcohol: Higher Base Rates Revealed Using the Unmatched-count Technique.’’Journal of Sex Research37:321-26.
23.
Lara, Diana, Jennifer Strickler, Claudia D. Olavarrieta, and Charlotte Ellertson .2004. ‘‘Measuring Induced Abortion in Mexico: A Comparison of Four Methodologies.’’Sociological Methods & Research32:529-58.
24.
Lee, Raymond M.1993. Doing Research on Sensitive Issues. London: Sage.
25.
Lensvelt-Mulders, Gerty J. L. M. and Hennie R. Boeije.2007. ‘‘Evaluating Compliance With a Computer Assisted Randomized Response Technique: A Qualitative Study Into the Origins of Lying and Cheating.’’Computers in Human Behavior23:591-608.
26.
Lensvelt-Mulders, Gerty J. L. M., Joop J. Hox, Peter G. M. van der Heijden, and Cora J. M. Maas.2005. ‘‘Meta-analysis of Randomized Response Research.’’Sociological Methods & Research33:319-48.
27.
Lensvelt-Mulders, Gerty J. L. M., Peter G. M. van der Heijden, Olav Laudy, and Ger van Gils.2006. ‘‘A Validation of a Computer-assisted Randomized Response Survey to Estimate the Prevalence of Fraud in Social Security.’’Journal of the Royal Statistical Society Series A169:305-18.
28.
Maddala, G.S.1983. Limited Dependent and Qualitative Variables in Econometrics . Cambridge, UK: Cambridge University Press.
29.
Musch, Jochen, Arndt Bröder, and Karl Christoph Klauer.2001. ‘‘Improving Survey Research on the World Wide Web Using the Randomized Response Technique.’’ Pp. 179-92 in Dimensions of Internet Science, edited by U.-D. Reips and M. Bosnjak.Lengerich, Germany: Pabst Science Publishers.
30.
Nathan, Gad and Monroe G. Sirken.1988. ‘‘Cognitive Aspects of Randomized Response.’’ Pp. 173-78 in Proceedings of the Section on Survey Research Methods. Alexandria, VA: American Statistical Association.
31.
Postoaca, Andrei.2006. The Anonymous Elect: Market Research Through Online Access Panels. Berlin: Springer .
32.
Raghavarao, Damaraju and Walter T. Federer.1979. ‘‘Block Total Response as an Alternative to the Randomized Response Method in Surveys.’’Journal of the Royal Statistical Society Series B (Statistical Methodology)41:40-5.
33.
Rasinski, K.A., G.B. Willis, A.K. Baldwin, W.C. Yeh, and L. Lee.1999. ‘‘Methods of Data Collection, Perceptions of Risks and Losses, and Motivation to Give Truthful Answers to Sensitive Survey Questions.’’Applied Cognitive Psychology13:465-84.
34.
Recker Rayburn, Nadine, Mitchell Earleywine, and Gerald C. Davison.2003. ‘‘Base Rates of Hate Crime Victimization Among College Students.’’Journal of Interpersonal Violence18:1209-221.
35.
Reckers, Philip M. J., Stephen W. Wheeler, and Bernard Wong-On-Wing.1997. ‘‘A Comparative Examination of Auditor Premature Sign-offs Using the Direct and the Randomized Response Methods.’’Auditing: A Journal of Practice & Theory16:69-78.
36.
Reips, Ulf-Dietrich.2002. ‘‘Standards for Internet-based Experimenting .’’ Experimental Psychology49:243-56.
37.
Scheers, N.J. and C. Mitchell Dayton.1987. ‘‘Improved Estimation of Academic Cheating Behavior Using the Randomized-response Technique.’’Research in Higher Education26:61-9.
38.
Scheers, N.J. and C. Mitchell Dayton.1988. ‘‘Covariate Randomized Response Models.’’Journal of the American Statistical Association83:969-74.
39.
Singer, Eleanor, Dawn R. von Thurn, and Esther Miller.1995. ‘‘Confidentiality Assurances and Response: A Quantitative Review of the Experimental Literature.’’Public Opinion Quarterly59:66-77.
40.
Snijders, Chris and Jeroen Weesie.2008. ‘‘The Online Use of Randomized Response Measurement.’’Paper presented at General Online Research2008, Hamburg, Germany.
41.
Stem, Donald E. and R. Kirk Steinhorst.1984. ‘‘Telephone Interview and Mail Questionnaire Applications of the Randomized Response Model.’’Journal of the American Statistical Association79:555-64.
42.
Tourangeau, R., Lance J. Rips, and Kenneth Rasinski .2000. The Psychology of Survey Response. Cambridge, UK: Cambridge University Press.
43.
Tourangeau, Roger and Ting Yan.2007. ‘‘Sensitive Questions in Surveys. ’’ Psychological Bulletin133:859-83.
44.
Tracy, Paul E. and James Allan Fox.1981. ‘‘The Validity of Randomized Response for Sensitive Measurements.’’American Sociological Review46:187-200.
45.
Tsuchiya, Takahiro, Yoko Hirai, and Shiguro Ono .2007. ‘‘A Study of the Properties of the Item Count Technique.’’ Public Opinion Quarterly71:253-72.
46.
van den Hout, Ardo and Peter Kooiman.2006. ‘‘Estimating the Linear Regression Model With Categorical Covariates Subject to Randomized Response.’’Computational Statistics & Data Analysis50:3311-323.
47.
van der Heijden, Peter G. M. , Ger van Gils, Jan Bouts, and Joop J. Hox.2000. ‘‘A Comparison of Randomized Response, Computer-assisted Self-interview, and Face-to-face Direct Questioning-Eliciting Sensitive Information in the Context of Welfare and Unemployment Benefit.’’Sociological Methods & Research28:505-37.
48.
Warner, Stanley L.1965. ‘‘Randomized-response: A Survey Technique for Eliminating Evasive Answer Bias.’’Journal of the American Statistical Association60:63-9.
49.
Wimbush, James C. and Dan R. Dalton.1997. ‘‘Base Rate for Employee Theft: Convergence of Multiple Methods.’’Journal of Applied Psychology82:756-63.
50.
Zdep, S.M. and Isabelle N. Rhodes.1976. ‘‘Making the Randomized Response Technique Work.’’Public Opinion Quarterly40:531-37.
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
