In this paper we describe and illustrate a procedure for selecting items from a large pool for a certification test. The proposed procedure, which is intended to improve the alignment of the certification test with on-the-job performance, is based on an expertise sensitive index. This index for an item is the difference between the item's p values for experts and novices. An example is provided of the application of the index for selecting items to be used in certifying bakers.
Get full access to this article
View all access options for this article.
References
1.
CoxR. C.VargasJ. (1966) A comparison of item selection techniques for norm-referenced and criterion-referenced tests. Paper presented at the annual meeting of the National Council on Measurement in Education, Chicago. (ERIC ED 010 517)
2.
CronbachL. J. (1950) Further evidence on response sets and test design. Educational and Psychological Measurement, 10, 3–31.
3.
Employment and Immigration Canada. (1991) Occupational analyses series: Baker. Ottawa: Occupational and Career Information Branch, Employment and Immigration Canada.
4.
GronlundN. E. (1998) How to make achievement tests and assessments. (6th ed.) Boston, MA: Allyn & Bacon.
5.
HaladynaT.RoidG. (1981) The role of instructional sensitivity in the empirical review of criterion-referenced test items. Journal of Educational Measurement, 18, 39–52.
6.
HambletonR. K.SwaminathanH.AlginaJ.CoulsonD. B. (1978) Criterion-referenced testing and measurement: A review of technical issues and developments. Review of Educational Research, 48, 1–47.
7.
HannaG. S.BennettJ. A. (1984) Instructional sensitivity expanded. Educational and Psychological Measurement, 44, 583–596.
8.
JaegerR. M. (1994) The psychometric demands of testing for licensure and certification. In LaveaultD.ZumboB. D.GessaroliM. E.BossM. W. (Eds.), Modern theories of measurement: Problems and issues. Ottawa, Canada: Edumetrics Research Group, Univer. of Ottawa. Pp. 305–351.
9.
KaneM. T. (1994) Validating interpretive arguments for licensure and certification examinations. Evaluation and the Health Professions, 17, 133–159.
10.
KellyT. L. (1939) The selection of upper and lower groups for the validation of test items. Journal of Educational Psychology30, 17–24.
11.
MillmanJ.GreeneJ. (1993) The specification and development of tests of achievement and ability. In LinnR. L. (Ed.), Educational measurement. (3rd ed.) New York: Macmillan. Pp. 335–366.
12.
OsterlindS. J. (1989) Constructing test items. Boston, MAKluwer Academic Publ.
13.
PophamJ. W. (2000) Modern educational measurement: Practical guidelines for educational leaders. (3rd ed.) Boston, MA: Allyn & Bacon.
14.
SireciS. G. (1998) The construct of content validity. In ZumboB. D. (Ed.), Validity theory and the methods used in validation: Perspectives from social and behavioral sciences. Boston, MA: Kluwer Academic Publ. Pp. 83–117.
15.
TraubR. E. (1994) Reliability for the social sciences: Theory and applications. Thousand Oaks, CA: Sage.
