Comparing the Utility of Two Procedures for Admitting Students to Liberal Arts: An Application of Decision-Theoretic Models

Abstract

The present study applied a threshold utility model to assess the utility of adding a scholastic aptitude test to a selection system, based on achievement tests, designed to screen applicants for university studies in liberal arts. The threshold model was chosen because admitting students to liberal arts is characterized by an undefined quota. The present analysis avoided the exact estimation of the decision-maker's utility function and estimated the range of utility ratios (i.e., the ratio between the costs of false positives and false negatives) for which the additional predictor is useful. A nonselected sample of applicants, including a random group sampled from the applicants who would have been rejected under regular circumstances, was used to obtain the ranges of utility ratios for which the additional test would add to the expected utility under three definitions of success in academic studies. The results revealed that the gains associated with the additional information depend on the definition of success one wishes to adopt. If the goal is to recruit students who can achieve minimal passing grades, no gains are expected. If, however, the goal is to identify potentially excellent students, then the gains associated with the scholastic aptitude test are significant.

Get full access to this article

View all access options for this article.

References

Arnold, J. D. , Rauschenberger, J. M. , Soubel, W. G. , & Guion, R. M. (1982). Validation and utility of a strength test for selecting steel workers. Journal of Applied Psychology, 67, 588-604.

Beller, M. (1994). Psychometric and social issues in admissions to Israeli universities. Educational Measurement: Issues and Practice, 13, 12-20.

Beller, M. , & Ben-Shakhar, G. (1981). An evaluation study of the students' selection process at the Hebrew University of Jerusalem. Megamot-Behavioral Sciences Quarterly, 27, 22-36 (Hebrew).

Ben-Shakhar, G. , & Beller, M. (1983). An application of a decision-theoretic model to a quota-free selection problem. Journal of Applied Psychology, 68, 137-146.

Ben-Shakhar, G. , Kiderman, I. , & Beller, M. (1993). Comparing the utility of two procedures for admitting students to liberal arts: An application of decision-theoretic models. NITE Report No. 183, Jerusalem, Israel.

Ben-Shakhar, G. , Lieblich, I. , & Bar-Hillel, M. (1982). An evaluation of polygrapher's judgments: A review from a decision theoretic perspective. Journal of Applied Psychology, 67, 701-713.

Brogden, H. E. (1949). When testing pays off. Personnel Psychology, 2, 171-183.

Cascio, W. F. , & Silbey, V. (1979). Utility of the assessment center as a selection device. Journal of Applied Psychology, 64, 107-118.

Cronbach, L. J. , & Gleser, G. C. (1965). Psychological tests and personnel decisions (2nd ed.). Urbana: University of Illinois Press.

10.

Crouse, J. (1985). Does the SAT help colleges make better selection decisions? Harvard Educational Review, 55, 195-219.

11.

de Gruijter, D.N.M. , & Hambleton, R. K. (1984). On problems encountered using decision theory to set cutoff scores. Applied Psychological Measurement, 8, 1-8.

12.

Glass, G. V (1978). Standards and criteria. Journal of Educational Measurement, 15, 237-261.

13.

Gottfredson, L. S. , & Crouse, J. (1986). Validity versus utility of mental tests: Example of the SAT. Journal of Vocational Behavior, 29, 363-378.

14.

Gross, A. L. , & Su, W (1975). Defining fair or unbiased selection model: A question of utilities. Journal of Applied Psychology, 60, 345-351.

15.

Hanford, H. G. (1985). Yes, the SAT does help colleges. Harvard Educational Review, 55, 324-331.

16.

Linn, R. L. (1986). Comments on the g factor in employment testing. Journal of Vocational Behavior, 29, 438-444.

17.

Linn, R. L. (1990). Admissions testing: Recommended uses, validity, differential prediction, and coaching. Applied Measurement in Education, 3, 297-318.

18.

Mellenbergh, G. J. , & Van der Linden, W. J. (1981). The linear utility model for optimal selection. Psychometrika, 46, 283-293.

19.

Nevo, B. , & Oren, C. (1986). Concurrent validity of the American Scholastic Aptitude Test (SAT) and the Israeli inter-university Psychometric Entrance Test (PET). Educational and Psychological Measurement, 46, 235-239.

20.

Petersen, N. S. (1976). An expected utility model for "optimal selection." Journal of Educational Statistics, 2, 333-358.

21.

Petersen, N. S. , & Novick, M. R. (1976). An evaluation of some models for culture-fair selection. Journal of Educational Measurement, 13, 3-29.

22.

Ramist, L. (1984). Predictive validity of the ATP tests. In T. F. Donlon (Ed.), The College Board technical handbook for the Scholastic Aptitude Test and achievement tests (pp. 141-170). New York: College Entrance Examination Board.

23.

Schmidt, F. L. , Hunter, J. E. , McKenzie, R. C. , & Muldrow, T. W. (1979). Impact of valid selection procedures on work-force productivity. Journal of Applied Psychology, 64, 609-626.