Sage Journals: Discover world-class research

Abstract

In educational and industrial psychology, utility theory has been used for determining optimal decision- theoretic procedures such as optimal test cutting scores for Pass/Fail and Accept/Reject decisions. Three meth ods are described for empirically assessing utility functions: (1) a method for scaling utility mixtures, consisting of a true achievement or criterion level combined with the probability of passing the test or being accepted, which is applicable for determining optimal decision procedures; (2) a method for scaling the utility as a function of the true achievement or cri terion level; and (3) a graphical procedure for choos ing a utility function. These methods are useful for in vestigating the utility structure. The three methods are investigated using 30 students in a hypothetical educa tional Pass/Fail situation and appear to yield reliable information. Moreover, an overview of the students' utility structures is reported.

Get full access to this article

View all access options for this article.

References

Alf, E.F. , & Dorfman, D.D. The classification of individuals into two criterion groups on the basis of a discontinuous payoff function. Psychometrika, 1967, 32, 115-123.

Bechtel, G.G. Multidimensional preference scaling. 's-Gravenhage, The Netherlands: Mouton, 1976.

Chen, J.J. , & Novick, M.R. On the use of a cumulative distribution as a utility function in educational or employment selection. Journal of Educational Statistics , 1982, 7, 19-35.

Chuang, D.T. , Chen, J.J. , & Novick, M.R. Theory and practice for the use of cut scores for personnel decisions. Journal of Educational Statistics, 1981, 6, 129-152.

Coombs, C.H. , Dawes, R.M. , & Tversky, A. Mathematical psychology: An elementary introduction. Englewood Cliffs NJ: Prentice-Hall Inc., 1970 .

Cronbach, L.J. , & Gleser, G.C. Psychological tests and personnel decisions. Urbana: University of Illinois Press, 1957.

Ferguson, T.S. Mathematical statistics: A decision-theoretic approach. New York: Academic Press, 1967.

Gross, A.L. , & Su, W. Defining a "fair" or "unbiased" selection model: A question of utilities. Journal of Applied Psychology, 1975, 60, 345-351.

Hambleton, R.K. , & Novick, M.R. Toward an integration of theory and method for criterion-referenced tests . Journal of Educational Measurement, 1973 , 10, 159-170.

10.

Hull, J.C. , Moore, P.G. , & Thomas, H. Utility and its measurement. Journal of the Royal Statistical Society, Series A, 1973, 136, 226-247.

11.

Huyhn, H. Statistical considerations of mastery scores. Psychometrika , 1976, 41, 65-79.

12.

Huyhn, H. A nonrandomized minimax solution for passing scores in the binomial error model. Psychometrika, 1980, 45, 167-182.

13.

Keeney, R.L. , & Raiffa, H. Decisions with multiple objectives: Preferences and value tradeoffs. New York: Wiley, 1976.

14.

Lord, F.M. , & Novick, M.R. Statistical theories of mental test scores. Reading MA : Addison-Wesley, 1968.

15.

Mellenbergh, G.J. , & van der Linden, W.J. The internal and external optimality of decisions based on tests. Applied Psychological Measurement , 1979, 3, 257-273.

16.

Mellenbergh, G.J. , & van der Linden, W.J. The linear utility model for optimal selection . Psychometrika, 1981, 46, 283-293.

17.

Mellenbergh, G.J. , & van der Linden, W.J. Selecting items for criterion-referenced tests. Evaluation in education: An international review series , 1982, 6, 171-183.

18.

Novick, M. R. Statistics as psychometrics . Psychometrika, 1980, 45, 411-424.

19.

Novick, M.R. , & Lindley, D.V. The use of more realistic utility functions in educational applications. Journal of Educational Measurement, 1978, 15, 181-191.

20.

Novick, M.R. , & Lindley, D.V. Fixed-state assessment of utility functions. Journal of the American Statistical Association, 1979, 74, 306-311.

21.

Petersen, N.S. An expected utility model for "optimal" selection. Journal of Educational Statistics, 1976, 1, 333-358.

22.

Ross, R.T. Optimum orders for the presentation of pairs in the method of paired comparisons . Journal of Educational Psychology, 1934, 25, 375-382.

23.

Torgerson, W.S. Theory and methods of scaling. New York: Wiley, 1958.

24.

Van den Brink, W.P. Binomial test models for domain-referenced testing. Evaluation in education: An international review series, 1982, 6, 160-170.

25.

Van der Linden, W.J. Using aptitude measurements for the optimal assignment of subjects to treatments with and without mastery scores. Psychometrika, 1981, 46, 257-274.

26.

Van der Linden, W.J. , & Mellenbergh, G.J. Optimal cutting scores using a linear loss function . Applied Psychological Measurement, 1977, 1, 593-599.

27.

Van der Linden, W.J. , & Mellenbergh, G.J.

Coefficients for tests from a decision-theoretic point of view

Applied Psychological Measurement, 1978, 2, 119-134.

Assessing and Studying Utility Functions in Psychometric Decision Theory

Abstract

Get full access to this article

References