Ethnic Group Representation in Test Construction Samples and Test Bias: The Standardization Fallacy Revisited

Abstract

The purpose of the present study is to test empirically the tenability of the hypothesis that tests tend to be systematically biased against ethnic groups with smaller or no representation in the test standardization sample. Two test construction models were developed under tight experimental controls: one with differential representation of ethnic groups (White, African American, Hispanic, and Asian) in the test development sample, and the other with maximum representation of one ethnic group (100%) in the test development sample. The findings revealed that underboth test construction models, consistently there is no systematic bias against the group(s) with smaller or no representation in the test construction samples. The empirical results support the integrity of the sampling and item selection procedures widely used in psychometric practice.

Get full access to this article

View all access options for this article.

References

Berk, R. A. (Ed.). (1982). Introduction. In R. A. Berk (Ed.), Handbook of methods for detecting test bias (pp. 1-8). Baltimore: Johns Hopkins University Press.

Crocker, L. , & Algina, J. (1986). Introduction to classical and modern test theory. San Francisco: Holt Rinehart & Winston.

Educational Testing Service. (1985). Legal issues in testing. Princeton, NJ: Educational Testing Service. (ERIC Document Reproduction Service No. ED 289 884)

Green, B. F. (1981). Issues in testing: Coaching, disclosure, and ethnic bias. San Francisco: Jossey-Bass.

Green, D. R. , & Draper, J. F. (1972, September). Exploratory studies of bias in achievement tests. Paper presented at the Annual Meeting of the American Psychological Association, Honolulu, HI. (ERIC Document Reproduction Service No. ED 070 794)

Hanington, G. M. (1975). Intelligence tests may favor the majority groups in a population. Nature, 258, 708-709.

Harrington, G. M. (1984). An experimental model of bias in mental testing. In C. R. Reynolds & R. T. Brown (Eds.), Perspectives on bias in mental testing (pp. 101-138). New York: Plenum.

Harrington, G. M. (1988). Two forms of minority-group test bias as psychometric artifacts with an animal model (Rattus norvegius). Journal of Comparative Psychology, 102, 400-407.

Hickman, J. A. , & Reynolds, C. R. (1986-1987). Are race differences in mental test scores an artifact of psychometric methods? A test of Harrington's experimental model. Journal of Special Education, 20, 409-430.

10.

Hilliard, A. G. (1984). IQ testing as the emperor's clothes: A critique of Jensen's Bias in Mental Testing. In C. R. Reynolds & R. T. Brown (Eds.), Perspectives on bias in mental testing (pp. 139-169). New York: Plenum.

11.

Hirsch, J. , & Tully, T. P. (1982). The challenge is unmet. The Behavioral and Brain Sciences, 5, 324-327.

12.

Jensen, A. R. (1980). Bias in mental testing. New York: Free Press.

13.

Jensen, A. R. (1984). Testbias: Concepts and criticisms. In C. R. Reynolds & R. T. Brown (Eds.), Perspectives on bias in mental testing (pp. 507-586). New York: Plenum.

14.

Mehrens, W. A. , & Popham, W. J. (1992). How to evaluate the legal defensibility of high-stakes tests. Applied Measurement in Education, 5, 265-283.

15.

Nunnally, J. C. (1978). Psychometric theory (2nd ed.). New York: McGraw-Hill.

16.

Page, E. B. (1963). Ordered hypotheses for multiple treatments: A significance test for linear ranks. Journal of the American Statistical Association, 58, 216-230.

17.

Reynolds, C. R. , & Brown, R. T. (Eds.). (1984). Bias in testing: Introduction to the issues. In C. R. Reynolds & R. T. Brown (Eds.), Perspectives on bias in mental testing (pp. 1-39). New York: Plenum.