The present study explored gender differences in answers to constructed-response mathematics items. Features relevant and irrelevant to the scoring rubric but possibly related to gender differences were identified by a content expert after a review of the literature. Raters were trained to score the identified features for approximately 500 papers evenly divided across two grade levels and between genders. The papers rated for these features had holistic scores assigned by local teachers using a state-provided rubric. The relationship between these holistic scores, studied features, objective scores, and gender differences was explored through standardized mean differences, correlations, and covariance analyses. The results indicate that rubric-relevant variables were highly predictive of holistic scores and accounted for some of the gender differences, especially at seventh grade. However, the possibly rubric-irrelevant feature of number of words written was statistically significant even when the rubric-relevant features were controlled.
Get full access to this article
View all access options for this article.
References
1.
Breland, H. M. , Bonner, M. W., & Kubota, M. Y. (1996). Factors in performance on brief, impromptu essay examinations (College Board Report No. 95-4; ETS Research Report No. 95-41). New York: College Entrance Examination Board.
2.
Breland, H. , Danos, D., Kahn, H., Kubota, M., & Bonner, M. (1994). Performance versus objective testing and gender: An exploratory study of an advanced placement history examination. Journal of Educational Measurement, 31, 275-293.
3.
Breland, H. , & Griswold, P. A. (1982). Use of a performance test as a criterion in a differential validity study. Journal of Educational Psychology, 74, 713-721.
4.
Brookhart, S. (1993). Teachers’grading practices: Meaning and values. Journal of Educational Measurement, 30, 123-142.
5.
Burton, N. W. , Lewis, C., & Robertson, N. (1988). Sex differences in SAT scores (College Board Report No. 88-9). Princeton, NJ: Educational Testing Service.
6.
Clements, D. H. , & Ballista, D. H. (1992). Geometry and spatial reasoning. In D. A. Grouws (Ed.), Handbook of research on mathematics teaching and learning (pp. 420-464). New York: Macmillan.
7.
Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum.
8.
Cohen, J. , & Cohen, P. (1983). Applied multiple regression/correlation analysis for the behavioral sciences. Hillsdale, NJ: Lawrence Erlbaum.
9.
Halpern, D. (1992). Sex differences in cognitive abilities. Hillside, NJ: Lawrence Erlbaum.
10.
Hill, R. (1993, September). Why educational assessment must be reformed not just revised. Paper presented at the Secretary’s Third Conference on Mathematics and Science, Washington, DC.
11.
Hyde, J. S. , & Linn, M. C. (1988). Gender differences in verbal ability: A meta-analysis. Psychological Bulletin, 104, 53-69.
12.
Kansas State Board of Education . (1993). Kansas mathematics curriculum standards. Topeka: Author.
13.
Kentucky Department of Education . (1995). KRIS accountability cycle I technical manual.Lexington: Author.
14.
Lane, S. , Wang, N., & Magone, M. (1996). Gender-related differential item functioning on a middle-school mathematics performance assessment. Educational Measurement: Issues and Practices, 15, 21-27.
15.
Langer, J. , Campbell, J., Neuman, S., Mullis, I., Persky, H., & Donahue, P. (1995). Reading assessment redesigned: Authentic texts and innovative instruments in NAEP’s 1992 survey. Washington, DC: U.S. Department of Education, Office of Educational Research and Improvement.
16.
Mazzeo, J. , Schmitt, A., & Bleistein, C. (1992). Sex-related differences on constructed -response and multiple choice sections of the advanced placement examinations: Three exploratory studies. Princeton, NJ: Educational Testing Service.
17.
Messick, S. (1991). Trait equivalence as construct validity of score interpretation across multiple methods of measurement (RM-91-1). Princeton, NJ: Educational Testing Service.
18.
Murphy, R. J. (1977, April). Sex differences in examination performance: Do these reflect differences in ability or sex-role stereotypes? Paper presented at the International Conference on Sex-Role Stereotyping, Cardiff, Wales.
19.
National Council of Teachers of Mathematics . (1989). Curriculum and evaluation standards for school mathematics. Reston, VA: Author.
20.
Poggio, J. , Glasnapp, D., Pomplun, M., & Capps, L. (1996). Kansas Mathematics Assessment: Scoring guidelines and criteria for open-ended questions for all grades. Lawrence: Center for Educational Testing and Evaluation, University of Kansas.
21.
Pomplun, M. , Capps, L., & Sundbye, N. (1998). Criteria teachers use to score performance items. Educational Assessment, 5(2), 95-110.
22.
Pomplun, M. , Glasnapp, D., & Poggio, J. (1996). Technical manual: 1993, 1994, and 1995 Kansas assessments in mathematics, reading, science, social studies, and writing. Lawrence: Center for Educational Testing and Evaluation, University of Kansas.
23.
Pomplun, M. , & Sundbye, N. (in press). Gender differences in constructed-response reading items. Applied Measurement in Education.
24.
Pomplun, M. , Sundbye, N., & Kelley, J. (1995, April). Exploring gender differences in constructed-response reading items. Paper presented at the annual meeting of the American Educational Research Association, San Francisco.
25.
Pomplun, M. , Wright, D., Oleka, N., & Sudlow, M. (1992). An analysis of English composition test essay prompts for differential difficulty. New York: College Entrance Examination Board.
26.
Sincoff, J. B. , & Sternberg, R. J. (1988). Development of verbal fluency abilities and strategies in elementary-school-age children. Developmental Psychology, 24, 646-653.
27.
Volkoff, J. , & Hocevar, D. (1995, April). Multiple-choice versus performance-based testing and gender: Eighth graders in the California Assessment Program. Paper presented at the annual meeting of the American Educational Research Association, San Francisco.
28.
Willingham, W. W. , & Cole, N. S. (1997). Gender and fair assessment. Mahwah, NJ: Lawrence Erlbaum.
