There is a robust tradition of program evaluation in the field of testing and measurement, but within the TESOL community, pro gram evaluation has only recently begun to be considered as a vital component. To identify some of the major concepts involved, we provide an overview of the better known approaches in program evaluation and, where applicable, focus attention on related measurement issues. We attempt to relate the following to ESL pro grams: formative and summative evaluation, criterion-, growth-, and norm-referenced evaluation, and related psychometric considera tions.
Get full access to this article
View all access options for this article.
References
1.
Airasian, Peter W.1974. Designing summative evaluation studies at the local level . In Evaluation in education: current applications. W. James Popham (Ed.), 147-199. Berkeley: McCutchan.
2.
Akst, G. and M. Hecht.1980. Program evaluation. In Teaching basic skills in college, Alice Stewart Trillin and Associates , 261-296. San Francisco: Jossey-Bass Publishers.
3.
Andrews, Frank M., Laura Klem, Terrence N. Davidson, Patrick M. O'Malley and Williard L. Rodgers .1978. A guide for selecting statistical techniques for analyzing social science data. Ann Arbor : Institute for Social Research, University of Michigan .
4.
Bachman, Lyle F.1981. Formative evaluation in specific purpose program development . In Languages for specific purposes: program design and evaluation, Ronald Mackay and JoeDarwinPalmer (Eds.), 106-116. Rowley, MA: Newbury House Publishers, Inc.
5.
Baker, Eva L.1974. Formative evaluation of instruction. In Evaluation in education: current applications, W. James Popham (Ed.), 531-585. Berkeley: McCutchan Publishing Corp.
6.
Birnbaum, A.1968. Some latent trait models and their use in inferring an examinee's ability. In Statistical theories of mental test scores, Frederick M. Lord and Melvin R. Novick (Eds.), Reading, MA: Addison-Wesley.
7.
Bloom, Benjamin S., John T. Hastings, and George F. Madaus .1971. Handbook on formative and summative evaluation of student learning. New York : McGraw-Hill.
8.
Campbell, Donald T. and Donald W. Fiske .1959. Convergent and discriminate validation by the multitrait-multimethod matrix. Psychological Bulletin56(2):81-105.
9.
Campbell, Donald T. and John C. Stanley .1963. Experimental and quasi-experimental designs for research. Chicago: Rand McNally.
10.
Cowell, William R.1982. Item response theory preequating in the TOESL testing program. In Paul W. Holland and Donald B. Rubin (Eds.), Test equating. New York: Academic Press.
11.
Cronbach, Lee J. and Paul E. Meehl.1955. Construct validity in psychological tests. Psychological Bulletin52(4):281-302.
12.
Davis, Frederick B. and James J. Diamond .1974. The preparation of criterion-referenced tests. In Problems in criterion-referenced measurement, Chester W. Harris, Marvin C. Alkin, and W. James Popham (Eds.), Los Angeles: UCLA Graduate School of Education, Center for the Study of Evaluation.
13.
Ebel, Robert L.1972. Essentials of educational measurement. Englewood Cliffs, N.J.: Prentice-Hall.
14.
Emrick, John A.1971. An evaluation model or mastery testing. Journal of Educational Measurement8(4):321-326.
15.
Fremer, John, Michael Zieky, Gary Marco and Susan Valentine.1974. Development of handbook for conducting task analyses and developing criterion-referenced tests of language skills. Princeton: Educational Testing Service .
16.
Glaser, G. Robert.1963. Instructional technology and the measurement of learning outcomes. American Psychologist18(8):519-521.
17.
Glass, Gene V.1978. Standards and criteria. Journal of Educational Measurement. 15(4):237-261.
18.
Hambleton, Ronald K.1978. On the use of cut-off scores with criterion-referenced tests in instructional settings. Journal of Educational Measurement . 15(4):277-290.
19.
Hambleton, Ronald K. and Daniel Eignor.1978a. A practitioner's guide to criterion-referenced test development, validation, and test sc ore usage, (Laboratory of Psychometric and Evaluative Research Report No. 70). Amherst, MA: School of Education, University of Massachusetts.
20.
Hambleton, Ronald K. and Daniel Eignor.1978b. Determining minimum competency levels: a consideration of issues, methods, and implementation strategies . A paper presented at the AERA Conference on Minimal Competency Testing, Washington, D.C.
21.
Hambleton, Ronald K. and Daniel Eignor.1978c. Guidelines for evaluating criterion-referenced tests and test manuals. Journal of Educational Measurement15(4):321-327.
22.
Hambleton, Ronald K., Hariharan Swaminathan, James Algina and Douglas Bill Coulson .1978. Criterion-referenced testing and measurement: a review of technical issues and developments. Review of Educational Research48(1):1-47.
23.
Henning, Grant.1982. Growth-referenced evaluation of foreign language instructional program. TESOL Quarterly16(4):467-477.
24.
Henning, Grant.In press. Language test development. Rowley, MA : Newbury House.
25.
Jaeger, Richard M.1976. Measurement consequences of selected standard-setting models. Florida Journal of Educational Research18:22-27.
26.
Lachica, G.M. and G. Brooks.1976. A study of factors related to success in ESL program, Fall 1975. Unpublished report, Office of Instructional Testing and Research, Borough of Manhattan Community College.
27.
Livingston, Samuel A.1978. Setting standards of speaking proficiency. In Direct testing of speaking proficiency: theory and application , John L.D. Clark(Ed.), Princeton: Educational Testing Service .
28.
Lord, Frederick M. and Melvin R. Novick. 1968. Statistical theories of mental test scores . Reading, MA: Addison-Wesley .
29.
Magnusson, David.1966. Test theory. Reading, MA: Addison-Wesley.
30.
Mehrens, William A. and Irvin J. Lehmann .1978. Measurement and evaluation in education and psychology. Second edition. New York: Holt, Rinehart and Winston .
31.
Meskauskas, John A.1976. Evaluation models for criterion-referenced testing: views regarding mastery and standard-setting. Review of Educational Research46(1):133-158.
32.
Millman, Jason.1972. Tables for determining number of items needed on domain-referenced tests and number of students to be tested, (Technical Paper No. 5). Los Angeles: Instructional Objective Exchange.
33.
Millman, Jason.1973. Passing scores and test lengths for domain-referenced measures. Review of Educational Research43(2):205-216.
34.
Nedelsky, Leo.1954. Absolute grading standards for objective tests . Educational and Psychololgical Measurement.14(1):3-19.
35.
Popham, W. James.1978. Criterion-referenced measurement. Englewood Cliffs, NJ: Prentice-Hall.
36.
Powers, Donald E. and Charles W. Stansfield .1983. The test of spoken English as a measure of communicative ability in the health professions: validation and standard setting. Princeton: Educational Testing Service.
37.
Rasch, Georg.1960. Probabilistic models for some intelligence and attainment tests. Copenhagen: Danmarks Paedagogiske Institut.
38.
Robertson, Donald L.1982. Toward a model for the evaluation of programs in English as a second language. In TESL studies. Yamuna Kachru and J. Ronayne Cowan (Eds.), 155-171. Urbana-Champaign: University of Illinois.
39.
Roudabush, E.1974. Models for a beginning theory of criterion-referenced tests. Paper presented at the meeting of the National Council for Measurement in Education, Chicago.
40.
Sanders, James R. and Donald J. Cunningham .1973. A structure for formative evaluation in product development. Review of Educational Research43(2):217-236
41.
Scriven, Michael.1967. The methodology of evaluation. In Curriculum evaluation, Robert E. Stake (Ed.), 39-83. AERA Monograph Series on Evaluation, No. 1. Chicago : Rand McNally.
42.
Shepard, Lorrie A.1976. Setting standards and living with them. Florida Journal of Educational Research18:23-32.
43.
Swinton, Spenser.1983. A manual for assessing language growth in instructional settings. Princeton: Educational Testing Service.
44.
Tallmadge, G. Kasten.1985. Rumours regarding the death of the equi-percentile assumption may have been greatly exaggerated. Journal of Educational Measurement22(1):33-39.
45.
Valette, Rebecca M.1971. Evaluation of learning in a second language. In Handbook on formative and summative evaluation of student learning , Benjamin S. Bloom, John T. Hastings, and George F. Madaus, (Eds.), 815-853. New York: McGraw-Hill.
46.
Wright, Benjamin D. and Mark H. Stone.1979. Best test design. Rasch measurement . Chicago: Mesa Press.
