Four measurement models for use in performance evaluation (norm, criterion, domain, and objectives-referenced measurement) are presented and the contingent relationships among them are discussed. An argument is made for the use of criterion-referenced measurement in performance testing. Literature on two major problem areas in criterion-referenced measurement, reliability and validity, is briefly reviewed; and a recent example of criterion-referenced performance test development in an applied training context is described.
Get full access to this article
View all access options for this article.
References
1.
BlockJ. H. (Ed.) Mastery learning: Theory and practice.New York: Holt, Rinehart and Winston,1971.
EdmonstonL. P.RandallR. S.OaklandT. D.A model for estimating the reliability and validity of criterion-referenced measures. Paper presented at the annual meeting of the American Educational Research Association, Chicago, 1972.
4.
FrederiksenNProficiency tests for training evaluation. In GlaserR. (Ed.) Training research and education.Pittsburgh: University of Pittsburgh Press,1962.
5.
GlaserR.NitkoA. J.Measurement in learning and instruction. In ThorndikeR. L. (Ed.) Educational measurement. Washington: American Council on Education, 1971, 625–670.
6.
GoodmanL. A.KruskalW. H.Measures of association for cross classification. American Statistical Association Journal, 1954, 49, 732–764.
7.
GuionR. M.Open a new window: Validities and values in psychological measurement. American Psychologist, 1974, 29, 287–296.
8.
GuionR. M.Content validity—The source of my discontent. Applied Psychological Measurement, 1977, 1, 1–10.
9.
HambletonR. K.Testing and decision-making procedures for selected individualized instructional programs. Review of Educational Research, 1974, 44, 371–400.
10.
HambletonR. K.NovickM. R.Toward an integration of theory and method for criterion-referenced tests. Journal of Educational Measurement, 1973, 10, 159–170.
11.
HarrisC. W.An interpretation of Livingston's reliability coefficient for criterion-referenced tests. Journal of Educational Measurement, 1972, 9, 27–29.
HolmgrenJ.SwezeyR. W.EakinsR.HilligossR.Performance testing in a field evaluation of a military training system. Paper presented at the convention of the American Psychological Association. San Francisco, California, August, 1977.
14.
KleinS. P.KosecoffJ.Issues and Procedures in the development of criterion-referenced tests. Princeton, New Jersey: Educational Testing Service, ERIC TM Report 26, September, 1973.
15.
LivingstonS. A.The reliability of criterion-referenced measures. Paper presented at the annual meeting of the American Educational Research Association, 1971.
16.
LivingstonS. A.A classical test-theory approach to criterion-referenced tests. Paper presented at the annual meeting of the American Educational Research Association, Chicago, 1972. (a)
17.
LivingstonS. A.A reply to Harris' an interpretation of Livingston's reliability coefficient for criterion-referenced tests. Journal of Educational Measurement, 1972, 9, 3. (b)
18.
LivingstonS. A.Criterion-referenced applications of classical test theory. Journal of Educational Measurement, 1972, 9(I), 13–26. (c)
19.
LordF. M.NovickM. R.Statistical theories of mental test scores.Reading, Mass.: Addison-Wesley,1968.
20.
LovettH. T.Criterion-referenced reliability estimated by ANOVA. Educational and Psychological Measurement, 1977, 37, 21–29.
21.
McClellandDTesting for competence rather than intelligence. American Psychologist, 1973, 28, 1–14.
22.
MeredithK. E.SabersD. L.Using item data for evaluating criterion-referenced measures with an empirical investigation of index consistency. Paper presented at the annual meeting of the Rocky Mountain Psychological Association, Albuquerque, 1972.
23.
MeskauskasJ. A.Evaluation models for criterion-referenced testing: Views regarding mastery and standard-setting. Review of Educational Research, 1976, 46, 133–158.
24.
MessickSThe standard problem: Meaning and values in measurement and evaluation. American Psychologist, 1975, 30, 955–966.
25.
MillmanJSampling plans for domain-referenced tests. In HivelyW. (Ed.). Domain-referenced testing.Englewood Cliffs, N.J.: Educational Technology Publications,1974.
26.
NitkoA. J.HsuT.Using domain-referenced tests for student placement, diagnosis, and attainment in a system of adaptive, individualized instruction. In HivelyW., (Ed.) Domain-referenced testing.Englewood Cliffs, N.J.: Educational Technology Publications,1974.
OaklandTAn evaluation of available models for estimating the reliability and validity of criterion-referenced measures. Paper presented at the annual meeting of the American Educational Research Association, Chicago, 1972.
29.
OsbornW. C.An approach to the development of synthetic performance tests for use in training evaluation, Alexandria, Virginia: HumRRO Professional Paper 30–70, 1970.
30.
OsbornW. C.Developing performance tests for training evaluation. Alexandria, Virginia: HumRRO Professional Paper 3–73, 1973 (a).
31.
OsbornW. C.Process versus product measures in performance testing. Paper presented at the annual conference of Military Testing Association, San Antonio, October, 1973 (b).
32.
PieperW. J.CatrowE. J.SwezeyR. W.SmithE. A.Automated apprenticeship training (AAT): A systematized audio-visual approach to self-paced job training. JSAS Catalog of Selected Documents in Psychology, Winter 1973, 3, 21, Ms. 315.
33.
PophamW. J.HusekT. R.Implication of criterion-referenced measures. Journal of Educational Measurement, 1969, 6, 1–9.
34.
RonanW.PrienE.Toward a criterion theory: A review and analysis of research and opinion. JSAS Catalog of Selected Documents in Psychology, 1973, 3, 68.
35.
SandersJ. R.MurrayS. L.Alternatives for achievement testing. Educational Technology, 1976, 17–23.
36.
SchoenfeldtL. F.SchoenfeldtB. B.AckerS. R.PerlsonM. R.Content validity revisited: The development of a content-oriented test of industrial reading. Journal of Applied Psychology, 1976, 61, 581–588.
37.
ShoemakerD. M.Toward a framework for achievement testing. Review of Educational Research, 1975, 45, 127–147.
38.
StanleyJ. C.Reliability. In ThorndikeR. L. (Ed.) Educational measurement.Washington, D.C.: American Council of Education,1971.
39.
SwezeyR. W.Toward the development of realistic measures of performance effectiveness. Journal of Educational Technology Systems, 1977, 5(4), 355–367.
40.
SwezeyR. W.PearlsteinR. B.Developing criterion-referenced tests. JSAS Catalog of Selected Documents in Psychology, Spring, 1975, 5, 227.
41.
SwezeyR. W.PearlsteinR. B.TonW. H.Criterion-referenced testing: A discussion of theory and of practice in the Army. Arlington, Va.: U.S. Army Research Institute Research Memorandum 75–11, December 1975.
VinebergR.TaylorE. N.Performance in four Army jobs by men at different aptitude levels: 4. Relationships between performance criteria. Alexandria, Virginia: HumRRO Technical Report 72–23, 1972.
44.
WoodsonM. I.C. E.The issue of item and test variance for criterion-referenced tests. Journal of Educational Measurement.1974, 11, 63–64. (a).
45.
WoodsonM. I. C. E.The issue of item and test variance for criterion-referenced tests: A reply. Journal of Educational Measurement.1974, 11, 139–140. (b)
