There is increasing interest in using situational judgment tests (SJTs) to supplement traditional student admission procedures. An important unexplored issue is whether students can intentionally distort or fake their responses on SJTs. This study examined the fakability of an SJT of college students’ performance. Two hundred ninety-three psychology students completed a cognitive test, a personality measure, and an SJT. Only for the SJT, the students were assigned to either an honest or a fake condition. The scores of students in the fake condition were significantly higher than those of students in the honest condition (d = .89). Furthermore, faking had a negative effect on the criterion-related validity (there was a significant drop from r = .33 to r = .09) and the incremental validity of the SJT over cognitive ability and personality. These results are discussed in terms of the use of SJTs in high-stakes testing programs.
Abrahams, N. M., Neumann, I., & Githens, W. H. (1971). Faking vocational interests: Simulated versus real life motivation. Personnel Psychology, 24, 5-12.
2.
Alliger, G. M., & Dwight, S. A. (2000). A meta-analytic investigation of the susceptibility of integrity tests to faking and coaching. Educational and Psychological Measurement, 60, 59-72.
3.
Anderson, C. D., Warner, J. L., & Spencer, C. C. (1984). Inflation bias in self-assessment examinations: Implications for valid employee selection. Journal of Applied Psychology, 69, 574-580.
4.
Anderson, E. S., & Keith, T. Z. (1997). A longitudinal test of a model of academic success for at-risk high school students. Journal of Educational Research, 90, 259-268.
5.
Atkinson, R. C. (2001, April). Standardized tests and access to American universities. Lecture delivered at the 83rd annual meeting of the American Council on Education, Washington, DC.
6.
Becker, T. E., & Colquitt, A. L. (1992). Potential versus actual faking on a biodata form: An analysis along several dimensions of item type. Personnel Psychology, 45, 389-406.
7.
Bess, T. L., & Mullins, M. E. (2002, April). Exploring a dimensionality of situational judgment: Task and contextual knowledge. Paper presented at the 17th annual conference of the Society for Industrial and Organizational Psychology, Toronto, Canada.
8.
Boekaerts, M. (1996). Personality and the psychology of learning. European Journal of Personality, 10, 377-404.
9.
Busato, V. V., Prins, F. J., Elshout, J. J., & Hamaker, C. (2000). Intellectual ability, learning style, personality, achievement motivation and academic success of psychology students in higher education. Personality and Individual Differences, 29, 1057-1068.
10.
Chan, D., & Schmitt, N. (1997). Video-based versus paper-and-pencil method of assessment in situational judgment tests: Subgroup differences in test performance and face validity perceptions. Journal of Applied Psychology, 82, 143-159.
11.
Chan, D., & Schmitt, N. (2002). Situational judgment and job performance. Human Performance, 15, 233-254.
12.
Clause, C. S., Delbridge, K., Schmitt, N., Chan, D., & Jennings, D. (2001). Test preparation activities and employment test performance. Human Performance, 14, 149-167.
13.
Clause, C. S., Mullins, M. E., Nee, M. T., Pulakos, E., & Schmitt, N. (1998). Parallel test form development: A procedure for alternative predictors and an example. Personnel Psychology, 51, 193-208.
14.
Clevenger, J., Pereira, G. M., Wiechmann, D., Schmitt, N., & Harvey, V. S. (2001). Incremental validity of situational judgment tests. Journal of Applied Psychology, 86, 410-417.
15.
Cook, T. D., & Campbell, D. T. (1979). Quasi-experimentation: Design and analysis issues for field settings. Boston: Houghton Mifflin.
16.
Costa, P. T., & McCrae, R. R. (1992). Revised NEO Personality Inventory (NEO-PI-R) and NEO Five-Factor Inventory (NEO-FFI) professional manual. Odessa, FL: Psycholocical Assessment Resources.
17.
Cowles, M., Darling, M., & Skanes, A. (1992). Some characteristics of the simulated self. Personality and Individual Differences, 13, 501-510.
18.
Cunningham, M. R., Wong, D. T., & Barbee, A. P. (1994). Self-presentation dynamics on overt integrity tests: Experimental studies of the Reid report. Journal of Applied Psychology, 79, 643-658.
19.
Dalen, L. H., Stanton, N. A., & Roberts, A. D. (2001). Faking personality questionnaires in personnel selection. Journal of Management Development, 20, 729-742.
20.
De Raad, B., & Schouwenburg, H. C. (1996). Personality in learning and education: A review. European Journal of Personality, 10, 303-336.
21.
Douglas, E. F., McDaniel, M. A., & Snell, A. F. (1996, August). The validity of non-cognitive measures decays when applicants fake. Paper presented at the annual conference of the Academy of Management, Cincinnati, OH.
22.
Dunnette, M. D., McCartney, J., Carlson, H. C., & Kirchner, W. K. (1962). A study of faking behavior on a forced-choice self-description checklist. Personnel Psychology, 15, 13-24.
23.
Dwight, S. A. (1999). An assessment of the different effects of warning applicants not to fake. Unpublished doctoral dissertation, State University of New York at Albany.
24.
Edens, P. S., & Arthur, W. (2000, April). A meta-analysis investigating the susceptibility of self-report inventories to distortion. Paper presented at the 15th annual conference of the Society for Industrial and Organizational Psychology, New Orleans, LA.
25.
Ellingson, J. E., Smith, D. B., & Sackett, P. R. (2001). Investigating the influence of social desirability on personality factor structure. Journal of Applied Psychology, 86, 122-133.
26.
Feldhusen, J., & Jarwan, F. (1995). Predictors of academic success at state-supported residential schools for mathematics and science: A validity study. Educational and Psychological Measurement, 55, 505-512.
27.
Furnham, A. (1986). Response bias, social desirability & dissimulation. Personality and Individual Differences, 7, 385-340.
28.
Graham, K. E., McDaniel, M. A., Douglas, E. F., & Snell, A. F. (2002). Biodata validity decay and score inflation with faking: Do item attributes explain variance across items?Journal of Business and Psychology, 16, 573-592.
29.
Griffith, R. L., Frei, R. L., Snell, A. F., Hamill, L. S., & Wheeler, J. K. (1997, April). Warning versus no-warnings: Differential effect of method bias. Paper presented at the 12th annual conference of the Society for Industrial and Organizational Psychology, St. Louis, MO.
30.
Haas, A., & McDaniel, M. A. (1999, April). Faking strategies: Effects on a situational judgment test. Paper presented at the 14th annual conference of the Society for Industrial and Organizational Psychology, Atlanta, GA.
31.
Hezlett, S. A., Kuncel, N. R., Vey, M. A., Ahart, A. M., Ones, D. S., Campbell, J. P., et al. (2001, April). The predictive validity of the SAT: A meta-analysis. Paper presented at the 16th annual conference of the Society for Industrial and Organizational Psychology, San Diego, CA.
32.
Hoekstra, H. A., Ormel, J., & De Fruyt, F. (1996). Handleiding NEO persoonlijkheidsvragenlijsten NEO-PI-R en NEO-FFI [Manual for NEO personality inventories NEO-PI-R and NEO-FFI]. Lisse, the Netherlands: Swets & Zeitlinger.
33.
Hough, L. M. (1998). Effects of intentional distortion in personality measurement and evaluation of suggested palliatives. Human Performance, 11, 209-244.
34.
Hough, L. M., Eaton, N. K., Dunnette, M. D., Kamp, J. D., & McCloy, R. A. (1990). Criterion-related validities of personality constructs and the effect of response distortion on those validities. Journal of Applied Psychology, 75, 581-595.
35.
Hough, L. M., Oswald, F. L., & Ployhart, R. E. (2001). Determinants, detection and amelioration of adverse impact in personnel selection procedures: Issues, evidence and lessons learned. International Journal of Selection and Assessment, 9, 152-194.
36.
Hough, L. M., & Schneider, R. J. (1996). The frontiers of I/O personality research. In K. R. Murphy (Ed.), Individual differences and behavior in organizations (pp. 31-88). San Francisco: Jossey-Bass.
37.
Juraska, S. E., & Drasgow, F. (2001, April). Faking situational judgment: A test of the Conflict Resolution Skills Assessment. Paper presented at the 16th annual conference of the Society for Industrial and Organizational Psychology, San Diego, CA.
38.
Kluger, A. N., & Collela, A. (1993). Beyond the mean bias: The effect of warning against faking on biodata item variances. Personnel Psychology, 46, 763-780.
39.
Kluger, A. N., Reilly, R. R., & Russell, C. J. (1991). Faking biodata tests: Are option-keyed instruments more resistant?Journal of Applied Psychology, 76, 889-896.
40.
Kluger, A. N., & Rothstein, H. R. (1993). The influence of selection test type on applicant reactions to employment testing. Journal of Business and Psychology, 76, 698-707.
41.
Kuncel, N. R., Hezlett, S. A., & Ones, D. S. (2001). A comprehensive meta-analysis of the predictive validity of the graduate record examinations: Implications for graduate student selection and performance. Psychological Bulletin, 127, 162-181.
42.
Lievens, F., Buyse, T., & Sackett, P. R. (in press). The operational validity of a video-based situational judgment test for medical college admissions: Illustrating the importance of matching predictor and criterion construct domains. Journal of Applied Psychology.
43.
Lievens, F., & Coetsier, P. (2002). Situational tests in student selection: An examination of predictive validity, adverse impact, and construct validity. International Journal of Selection and Assessment, 10, 245-257.
44.
Martin, B. A., Bowen, C. C., & Hunt, S. T. (2002). How effective are people at faking on personality questionnaires?Personality and Individual Differences, 32, 247-256.
45.
McCrae, R. R., & Costa, P. T. (1983). Social desirability scales: More substance than style. Journal of Consulting and Clinical Psychology, 51, 882-888.
46.
McDaniel, M. A., Morgeson, F. P., Finnegan, E. B., Campion, M. A., & Braverman, E. P. (2001). Use of situational judgment tests to predict job performance: A clarification of the literature. Journal of Applied Psychology, 86, 730-740.
47.
McFarland, L. A., & Ryan, A. M. (2000). Variance in faking across non-cognitive measures. Journal of Applied Psychology, 85, 812-821.
48.
McFarland, L. A., Ryan, A. M., & Aleksander, E. (2002). Item placement on a personality measure: Effects on faking behavior and test measurement properties. Journal of Personality Assessment, 78, 348-369.
49.
Motowidlo, S. J., Dunnette, M. D., & Carter, G. W. (1990). An alternative selection procedure: The low-fidelity simulation. Journal of Applied Psychology, 75, 640-647.
50.
Motowidlo, S. J., Hanson, M. A., & Crafts, J. L. (1997). Applied measurement methods in industrial psychology. In D. L. Whetzel & G. R. Wheaton (Eds.), Applied measurement methods in industrial psychology (pp. 241-260). Palo Alto, CA: Davies-Black.
51.
Motowidlo, S. J., & Tippins, N. (1993). Further studies of the low-fidelity simulation in the form of a situational inventory. Journal of Occupational and Organizational Psychology, 66, 337-344.
52.
Nguyen, N. T., & McDaniel, M. A. (2001). Constructs assessed in situational judgment tests: A meta-analysis. Paper presented at the 16th annual conference of the Society for Industrial and Organizational Psychology, San Diego, CA.
53.
Nguyen, N. T., McDaniel, M. A., & Biderman, M. D. (2002). Response instructions in situational judgment tests: Effects on faking and construct validity. Paper presented at the 17th annual conference of the Society for Industrial and Organizational Psychology, Toronto, Canada.
54.
Nicholson, R. A., & Hogan, R. T. (1990). The construct validity of social desirability. American Psychologist, 45, 290-292.
55.
Ones, D. S., & Viswesvaran, C. (1998). The effects of social desirability and faking on personality and integrity assessment for personnel selection. Human Performance, 11, 245-269.
56.
Ones, D. S., Viswesvaran, C., & Korbin, W. (1995, May). Meta-analyses of fakability estimates: Between-subjects versus within-subjects designs. Paper presented at the 10th annual conference of the Society for Industrial and Organizational Psychology, Orlando, FL.
57.
Ones, D. S., Viswesvaran, C., & Reiss, A. D. (1996). Role of social desirability in personality testing for personnel selection: The red herring. Journal of Applied Psychology, 81, 660-679.
58.
Oswald, F. L., Schmitt, N., Kim, B. H., Ramsay, L. J., & Gillespie, M. A. (2004). Developing a biodata measure and situational judgment inventory as predictors of college student performance. Journal of Applied Psychology, 89, 187-207.
59.
Paulhus, D. L. (1986). Self-deception and impression management in test responses. In A. Angleiter & J. S. Wiggins (Eds.), Personality assessment via questionnaire: Current issues in theory and measurement (pp. 143-165). Berlin, Germany: Springer-Verlag.
60.
Paulhus, D. L. (1991). Measurement and control of response bias. In J. P. Robinson, P. R. Shaver, & L. S. Wrightsman (Eds.), Measures of personality and social psychological attitudes (pp. 17-59). San Diego, CA: Academic Press.
61.
Ployhart, R. E., & Ehrhart, M. G. (2001, April). Effects of response instructions on the criterion-related validity, construct validity, and reliability of situational judgment tests. Paper presented at the 16th annual conference of the Society for Industrial and Organizational Psychology, San Diego, CA.
62.
Ployhart, R. E., & Ryan, A. M. (1998). The relative importance of procedural and distributive justice in determining applicants’ reactions. Journal of Applied Psychology, 83, 3-16.
63.
Raven, J., Raven, J. C., & Court, J. H. (1998). Advanced progressive matrices manual (1998 ed.). Oxford, UK: Oxford Psychologists Press.
64.
Rosse, J. G., Stechner, M. D., Levin, R. A., & Miller, J. L. (1998). The impact of response distortion on preemployment personality testing and hiring decisions. Journal of Applied Psychology, 83, 634-644.
65.
Roth, P. L., & Bobko, P. (2000). College grade point average as a personnel selection device: Ethnic group differences and potential adverse impact. Journal of Applied Psychology, 85, 399-406.
66.
Ryan, A. M., Ployhart, R. E., Greguras, G. J., & Schmit, M. J. (1998). Test preparation programs in selection contexts: Self-selection and program effectiveness. Personnel Psychology, 51, 599-621.
67.
Sackett, P. R. (2002, January). New measures: Issues of validity and diversity. Presentation at the College Board Conference on New Tools for Admission to Higher Education, Washington, DC.
68.
Sackett, P. R., Schmitt, N., Ellingson, J. E., & Kabin, M. B. (2001). High-stakes testing in employment, credentialing, and higher education: Prospects in a post-affirmative action world. American Psychologist, 56, 302-318.
69.
Schmidt, F. L. (2002). The role of general cognitive ability and job performance: Why there cannot be a debate. Human Performance, 15, 187-211.
70.
Schmidt, F. L., & Hunter, J. E. (1998). The validity and utility of selection methods in personnel psychology: Practical and theoretical implications of 85 years of research findings. Psychological Bulletin, 124, 262-274.
71.
Schmit, M. J., & Ryan, A. M. (1992). Test-taking dispositions: A missing link?Journal of Applied Psychology, 77, 629-637.
72.
Schmitt, N., Clause, C. S., & Pulakos, E. D. (1996). Subgroup differences associated with different measures of some common job-relevant constructs. International Review of Industrial and Organizational Psychology, 11, 116-139.
73.
Schmitt, N., Sackett, P. R., & Ellingson, J. E. (2002). No easy solution to subgroup differences. American Psychologist, 57, 305-306.
74.
Smith, D. B., & Ellingson, J. E. (2002). Substance versus style: A new look at social desirability in motivating contexts. Journal of Applied Psychology, 87, 211-219.
75.
Snell, A. F., Sydell, E. J., & Lueke, S. B. (1999). Towards a theory of applicant faking: Integrating studies of deception. Human Resource Management Review, 9, 219-242
76.
Sternberg, R. J., Wagner, R. K., Williams, W. M., & Horvath, J. A. (1995). Testing common sense. American Psychologist, 50, 912-927.
77.
Vey, M. A., Ones, D. S., Hezlett, S. A., Kuncel, N. R., Vannelli, J. R., Briggs, K. H., et al. (2003, April). Relationships among college grade indices: A meta-analysis examining temporal influences. Paper presented at the 18th annual conference of the Society for Industrial and Organizational Psychology, Orlando, FL.
78.
Viswesvaran, C., & Ones, D. S. (1999). Meta-analyses of fakability estimates: Implications for personality measurement. Educational and Psychological Measurement, 59, 197-210.
79.
Weekley, J. A., & Jones, C. (1999). Further studies of situational tests. Personnel Psychology, 52, 679-700.
80.
Weiner, J. A., & Gibson, W. M. (2000, April). Practical effects of faking on job applicant attitude test scores. Paper presented at 15th annual conference of the Society for Industrial and Organizational Psychology, New Orleans, LA.
81.
Wolfe, R. N., & Johnson, S. D. (1995). Personality as a predictor of college performance. Educational and Psychological Measurement, 55, 177-185.
82.
Worthington, D. L., & Schlottmann, R. S. (1986). The predictive validity of subtle and obvious empirically derived psychology test items under faking conditions. Journal of Personality Assessment, 50, 171-181.
83.
Zickar, M. J. (1997, April). Computer simulation of faking on a personality test. Paper presented at the 12th annual conference of the Society for Industrial and Organizational Psychology, St. Louis, MO.
84.
Zickar, M. J., & Robie, C. (1999). Modeling faking good on personality items: An item-level analysis. Journal of Applied Psychology, 84, 551-563.
