Researchers consistently have demonstrated the distorting effects that inattentive (responding without regard for survey content) and socially desirable (presenting one’s self in an overly positive way) responding can have on quantitative data. These threats to validity usually go undetected in counseling research, which is concerning given that counseling research often considers implications for practice and that research-based practice is in demand. In this article, the authors review practical approaches for detecting and addressing inattentive and socially desirable response tendencies in counseling research, thereby optimizing the validity of conclusions inferred from data.
American Counseling Association. (2014). 2014 ACA code of ethics. Alexandria, VA: Author.
2.
BäckströmM.BjörklundF. (2013). Social desirability in personality inventories: Symptoms, diagnosis, and prescribed cure. Scandinavian Journal of Psychology, 54, 152–159. doi:10.1111/sjop.12015
3.
BerryD. T. R.WetterM. W.BaerR. A.LarsenL.ClarkC.MonroeK. (1992). MMPI-2 random responding indices: Validation using a self-report methodology. Psychological Assessment, 4, 340–345. doi:10.1037/1040-3590.4.3.340
4.
BelliR. F.MooreS. E.VanHoewykJ. (2006). An experimental comparison of question forms used to reduce vote overreporting. Electoral Studies, 25, 751–759. doi:10.1016/j.electstud.2006.01.001
5.
ButcherJ. N.DahlstromW. G.GrahamJ. R.TellegenA.KaemmerB. (1989). The Minnesota Multiphasic Personality Inventory-2 (MMPI-2): Manual for administration and scoring. Minneapolis: University of Minnesota Press.
6.
ChapmanL. J.ChapmanJ. P. (1983). Infrequency scale. Unpublished test (copies available from T.R. Kwapil, Department of Psychology, University of North Carolina at Greensboro, Greensboro, NC, 27402-6170).
7.
CrowneD. P.MarloweD. (1960). A new scale of social desirability independent of psychopathology. Journal of Consulting Psychology, 24, 349–354. doi:http://dx.doi.org/10.1037/h0047358
8.
CrowneD. P.MarloweD. (1964). The approval motive. New York, NY: John Wiley.
9.
DeVellisR. F. (2003). Scale development: Theory and applications (2nd ed.). Thousand Oaks, CA: Sage.
10.
ElmoreP. B. (2010). Reporting standards for research publications. Counseling Outcome Research and Evaluation, 1, 19–29. doi:10.1177/2150137810386108
11.
FlemingP. (2012). Social desirability, not what it seems: A review of the implications for self-reports. The International Journal of Educational and Psychological Assessment, 11, 3–22.
12.
GignacG. E. (2013). Modeling the balanced inventory of desirable responding: Evidence in favor of a revised model of socially desirable responding. Journal of Personality Assessment, 95, 645–656. doi:10.1080/00223891.2013.816717
13.
Groth-MarnatG. (2009). Handbook of psychological assessment (5th ed.). Hoboken, NJ: John Wiley.
14.
HancockD. R.FlowersC. P. (2001). Comparing social desirability responding on World Wide Web and paper-administered surveys. Educational Technology, Research, & Development, 49, 5–13. doi:10.1007/BF02504503
15.
HathawayS. R.McKinleyJ. C. (1951). Minnesota multiphasic personality inventory manual. New York, NY: Psychological Corporation.
16.
HeppnerP. P.KivlighanD. M.WampoldB. E. (2008). Research design in counseling (3rd ed.). Belmont, CA: Thomson Brooks/Cole.
17.
HirshJ. B.PetersonJ. B. (2008). Predicting creativity and academic success with a “fake-proof” measure of the Big Five. Journal of Research in Personality, 42, 1323–1333. doi:10.1016/j.jrp.2008.04.006
18.
HoltgravesT.EckJ.LaskyB. (1997). Face management, question wording, and social desirability. Journal of Applied Social Psychology, 27, 1650–1671. doi:10.1111/j.1559-1816.1997.tb01618.x
19.
HuangJ. L.BowlingN. A.LiuM.LiY. (2015). Detecting insufficient effort responding with an infrequency scale: Evaluating validity and participant reactions. Journal of Business and Psychology, 30, 299–311. doi:10.1007/s10869-014-9357-6
20.
JohnsonJ. A. (2005). Ascertaining the validity of individual protocols from web-based personality inventories. Journal of Research in Personality, 39, 103–129. doi:10.1016/j.jrp.2004.09.009
21.
KamC. (2013). Probing item social desirability by correlating items with Balanced Inventory of Desirable Responding (BIDR): A validity examination. Personality and Individual Differences, 54, 513–518. doi:10.1016/j.paid.2012.10.017
22.
KrumpalI. (2013). Determinants of social desirability bias in sensitive surveys: A literature review. Quality & Quantity, 47, 2025–2047. doi:10.1007/s11135-011-9640-9
23.
LanyonR. I.CarleA. C. (2007). Internal and external validity scores on the Balanced Inventory of Desirable Responding and the Paulhus Deception Scales. Educational and Psychological Measurement, 67, 859–876. doi:10.1177/0013164406299104
24.
LeiteW. L.BeretvasS. N. (2005). Validation of scores on the Marlowe-Crowne Social Desirability Scale and the Balanced Inventory of Desirable Responding. Educational and Psychological Measurement, 65, 140–154. doi:10.1177/0013164404267285
25.
LiuX. S. (2013). Comparing sample size requirements for significance tests and confidence intervals. Counseling Outcome Research and Evaluation, 4, 3–12. doi:10.1177/2150137812472194
26.
ManiaciM. R.RoggeR. D. (2014). Caring about carelessness: Participant inattention and its effects on research. Journal of Research in Personality, 48, 61–83. doi:10.1016/j.jrp.2013.09.008
27.
McKibbenW. B.SilviaP. J. (in press). Evaluating the distorting effects of inattentive responding and social desirability on self-report scales creativity and the arts. Journal of Creative Behavior. doi:10.1002/jocb.86
28.
McKinleyJ. C.HathawayS. R.MeehlP. E. (1948). The Minnesota multiphasic personality inventory: VI. The K scale. Journal of Consulting Psychology, 12, 20–31. doi:http://dx.doi.org/10.1037/h0061377
29.
MeadeA. W.CraigS. B. (2012). Identifying careless responses in survey data. Psychological Methods, 17, 437–455. doi:http://dx.doi.org/10.1037/a0028085
30.
MuschJ.OstapczukM.KlaiberY. (2012). Validating an inventory for the assessment of egoistic bias and moralistic bias as two separable constructs of social desirability. Journal of Personality Assessment, 94, 620–629. doi:10.1080/00223891.2012.672505
31.
NäherA.KrumpalI. (2013). Asking sensitive questions: The impact of forgiving wording and question context on social desirability bias. Quality & Quantity, 46, 1601–1616. doi:10.1007/s11135-011-9469-2
32.
OppenheimerD. M.MeyvisT.DavidenkoN. (2009). Instructional manipulation checks: Detecting satisficing to increase statistical power. Journal of Experimental Social Psychology, 45, 867–872. doi:10.1016/j.jesp.2009.03.009
33.
PaulhusD. L. (1984). Two-component models of socially desirable responding. Journal of Personality and Social Psychology, 46, 598–609. doi:10.1037/0022-3514.46.3.598
34.
PaulhusD. L. (1991). Measurement and control of response bias. In RobinsonJ. P.ShaverP. R.WrightsmanL. S. (Eds.), Measures of personality and social psychological attitudes (pp. 17–59). San Diego, CA: Academic Press.
35.
PaulhusD. L. (1998). Paulhus Deception Scales (PDS): The Balanced Inventory of Desirable Responding-7: User’s manual. North Tanawanda, NY: Multi-Health Systems.
36.
PaulhusD. L.ReidD. B. (1991). Enhancement and denial in socially desirable responding. Journal of Personality and Social Psychology, 60, 334–356. doi:http://dx.doi.org/10.1037/0022-3514.60.2.307
37.
PresserS. (1990). Can changes in context reduce vote overreporting in surveys?Public Opinion Quarterly, 54, 586–593. doi:10.1086/269229
38.
ReynoldsW. M. (1982). Development of reliable and valid short forms of the Marlowe-Crowne Social Desirability Scale. Journal of Clinical Psychology, 38, 119–125. doi:10.1002/1097-4679
39.
SinkC. A.MvududuN. H. (2010). Statistical power, sampling, and effect sizes: Three keys to research relevancy. Counseling Outcome Research and Evaluation, 1, 1–18. doi:10.1177/2150137810373613
40.
SteenkampJ. E. M.de JongM. G.BaumgartnerH. (2010). Socially desirable response tendencies in survey research. Journal of Marketing Research, 47, 199–214. doi:http://dx.doi.org/10.1509/jmkr.47.2.199
41.
VazireS. (2006). Informant reports: A cheap, fast, and easy method for personality assessment. Journal of Research in Personality, 40, 472–481. doi:10.1016/j.jrp.2005.03.003
