Sage Journals: Discover world-class research

Abstract

A practice analysis, in which an appropriate sample of professionals is asked to describe what they do in their practices, is considered an essentialpart of the content-related validity evidence for licensure and certification tests. This article examines the precision of estimates of the mean frequencies for categories of activities in an activity inventory administered on two occasions about 6 weeks apart. In particular, it explores the variability in estimates of the meanfrequenciesfor categories due to the sampling of respondents, activity statements, and occasions. The results suggest that samples of 200 respondents, about 10 to 20 activity statements per category, and one occasion are adequatefor estimating the mean frequency for each category.

Get full access to this article

View all access options for this article.

References

American Educational Research Association , American Psychological Association , and National Council on Measurement in Education . (1985). Standards for educational and psychological testing. Washington, DC: American Psychological Association.

Brennan, R. L. (1992). Elements of generalizability theory. Iowa City, IA: ACT Publications.

Cardinet, J. , Toumeur, Y. , & Allal, L. (1976). The symmetry of generalizability theory: Applications to educational measurement. Journal of Educational Measurement, 13, 119-135.

Crick, J. , & Brennan, R. (1992). GENOVA: A generalized analysis of variance system (Computer program and manual). Dorchester, MA: Computer Facilities, University of Massachusetts at Boston.

Cronbach, L. J. , Gleser, G. C. , Nanda, H. , & Rajaratnam, N. (1972). The dependability of behavioral measurement. New York: Wiley.

Gael, S. (1983). Job analysis: A guide to assessing work activities. San Francisco: Jossey-Bass.

Kane, M. (1982). The validity of licensure examinations. American Psychologist, 37, 911-918.

Kane, M. , Estes, C. , Colton, D. , & Eltoft, C. (1989). Role delineation study for entry-level registered dietitians, entry-level dietetic technicians, and beyond-entry-level registered dietitians (Vols. 1-3). Chicago: American Dietetic Association.

Kane, M. , Kingsbury, C. , Colton, D. , & Estes, C. (1989). Combining data on criticality and frequency in developing test plans for licensure and certification examinations. Journal of Educational Measurement, 26, 17-27.

10.

McCormick, E. (1979). Job analysis: Methods and applications. New York: American Management Association.

11.

Shavelson, R. J. , & Webb, N. M. (1991). Generalizability theory: A primer. London: Sage.

12.

Shimberg, B. (1981). Testing for licensure and certification. American Psychologist, 36, 1138-1146.

13.

Valiga, M. (1990). Assessing the reliability of questionnaires and survey instruments using generalizability theory. Unpublished doctoral dissertation, University of Iowa.

The Precision of Practice Analysis Results in the Professions

Abstract

Get full access to this article

References