The Degree of Congruence between Test Standards and Test Documentation with in Journal Publications

Abstract

The present study examined the extent to which testing practices comply with professional guidelines regarding examination of relevant reliability and validity evidence. The focus of testing use was limited to published research investigations appearing in 1992 American Psychological Association journals. Data regarding the use of paper and pencil tests, type of instrument, and reliability and validity information were abstracted from each article. Reliability and validity evidence supporting the use of a particular test for a specific contextual application was often lacking. Documentation of both of these key psychometric properties was reported for 20% of the 2,167 instruments used. About half(49%) of instrumentation was supported with evidence of one type or the other. When reported, test use was generally supported by internal consistency reliability and construct validity evidence.

Get full access to this article

View all access options for this article.

References

American Educational Research Association , American Psychological Association , National Council on Measurement in Education. (1985). Standards for educational and psychological testing. Washington, DC: American Psychological Association.

American Psychological Association. (1992). Ethical principles of psychologists and code of conduct. American Psychologist, 47, 1597-1611.

Dawis, R. V. (1987). Scale construction. Journal of Counseling Psychology, 34, 481-489.

Joint Committee on Standards for Educational Evaluation. (1988). The personnel evaluation standards: How to assess systems for evaluating educators. Newbury Park, CA: Sage.

Joint Committee on Standards for Educational Evaluation. (1994). The program evaluation standards: How to assess evaluations of educational programs (2nd ed.). Newbury Park, CA: Sage.

Messick, S. (1989). Validity. In R. Linn (Ed.), Educational measurement (3rd ed., pp. 13-103). Washington, DC: American Psychological Association.

Willson, V. L. (1980). Research techniques in AERJ articles: 1969 to 1978. Educational Researcher, 9(6), 5-10.