This article describes two related studies that began to explore the validity of scores from select-and-fill-in (SAFI) concept map assessments as measures of students’ connected understanding of science. Scores from SAFI maps created for this purpose and used with middle school students and undergraduate astronomy students possessed high internal consistency and exhibited large mean increases with increased domain exposure. SAFI scores were strongly related to scores from a standardized multiple-choice (MC) achievement measure for middle school students; work with individual students suggested that they used strategies requiring connected understanding to successfully complete the maps. SAFI scores from undergraduate students exhibited large relationships with scores from direct-instruction MC exams and scores from a relatedness ratings measure, taken together and separately. Results provide initial evidence of the validity of scores from SAFI maps as measures of connected understanding of science in middle school and undergraduate introductory science students.
Get full access to this article
View all access options for this article.
References
1.
American Association for the Advancement of Science . (1993). Benchmarks for science literacy. New York: Oxford University Press.
2.
Anderson, J. R. (1995). Learning and memory: An integrated approach. New York: John Wiley.
3.
Anderson, T. H. , & Huang, S.C.C. (1989). On using concept maps to assess the comprehension effects of reading expository text (Report No. 483). Champaign: University of Illinois. (ERIC Document Reproduction Service No. ED 310 368)
4.
Barenholz, H. , & Tamir, P. (1992). A comprehensive use of concept mapping in design instruction and assessment. Research in Science and Technological Education, 10, 39-52.
5.
Cawley, J. M. , Zimmaro, D. M., Van Meter, P., & Theodorou, E. S. (1999, April). Validation of concept maps as a tool to predict performance on course exams. Paper presented at the annual meeting of the American Educational Research Association, Montreal, Canada.
6.
Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum.
7.
Diekhoff, G. M. (1983). Testing through relationship judgments. Journal of Educational Psychology, 75, 227-233.
8.
Farnham-Diggory, S. (1992). Cognitive processes in education (2nd ed.). New York: HarperCollins.
9.
Goldsmith, T. E. , Johnson, P. J., & Acton, W. H. (1991). Assessing structural knowledge. Journal of Educational Psychology, 83, 88-96.
10.
Johnson, P. J. , Goldsmith, T. E., & Teague, K. W. (1995). Similarity, structure, and knowledge: A representational approach to assessment. In P. D. Nichols, S. F. Chipman, & R. L. Brennan (Eds.), Cognitively diagnostic assessment (pp. 221-249). Hillsdale, NJ: Lawrence Erlbaum.
11.
Jonassen, D. H. , Beissner, K., & Yacci, M. (1993). Structural knowledge: Techniques for representing, conveying, and acquiring structural knowledge. Hillsdale, NJ: Lawrence Erlbaum.
12.
Jones, L. R. , Mullis, I.V.S., Raizen, S. A., Weiss, I. R., & Weston, E. A. (1992). The 1990 science report card: NAEP’s assessment of fourth, eighth, and twelfth graders. Washington, DC: U.S. Department of Education.
13.
Liu, X. (1994, April). The validity and reliability of concept mapping as an alternative science assessment when item response theory is used for scoring. Paper presented at the meeting of the American Educational Research Association, New Orleans, LA.
14.
Marshall, S. (1995). Schemas in problem solving. Cambridge, UK: Cambridge University Press.
15.
National Research Council . (1996). National science education standards. Washington, DC: National Academy Press.
16.
Naveh-Benjamin, M. , Lin, Y., & McKeachie, W. J. (1995). Inferring students’ cognitive structures and their development using the “fill-in-the-structure” (FITS) technique. In P. D. Nichols, S. F. Chipman, & R. L. Brennan (Eds.), Cognitively diagnostic assessment (pp. 279-304). Hillsdale, NJ: Lawrence Erlbaum.
17.
Novak, J. D. , & Gowin, D. B. (1984). Learning how to learn. New York: Cambridge University Press.
18.
Novak, J. D. , & Musonda, D. (1991). A twelve year longitudinal study of science concept learning. American Educational Research Journal, 28, 117-153.
19.
Rice, D. C. , Ryan, J. M., & Samson, S. M. (1998). Using concept maps to assess student learning in the science classroom: Must different methods compete?Journal of Research in Science Teaching, 35, 1103-1127.
20.
Ruiz-Primo, M. A. , Schultz, S. E., Li, M., & Shavelson, R. J. (1998, April). Comparison of the reliability and validity of scores from two concept-mapping techniques. Paper presented at the annual meeting of the American Educational Research Association, San Diego, CA.
21.
Ruiz-Primo, M. A. , Schultz, S. E., Li, M., & Shavelson, R. J. (1999, April). On the cognitive validity of interpretations of scores from alternative concept-mapping techniques. Paper presented at the annual meeting of the American Educational Research Association, Montreal, Canada.
22.
Ruiz-Primo, M. A. , & Shavelson, R. J. (1996). Problems and issues in the use of concept maps in science assessment. Journal of Research in Science Teaching, 33, 569-600.
23.
Ruiz-Primo, M. A. , Shavelson, R. J., & Schultz, S. E. (1997, April). On the validity of concept map-based assessment interpretations: An experiment testing the assumption of hierarchical concept maps in science. Paper presented at the annual meeting of the American Educational Research Association, Chicago.
24.
Schau, C. , & Mattern, N. (1997). Use of map techniques in teaching applied statistics courses. American Statistician, 51, 171-175.
25.
Schau, C. , Mattern, N., Weber, R. W., Minnick, K., & Witt, C. (1997, April). Use of fill-in concept maps to assess middle school students’ connected understanding of science. Paper presented at the annual meeting of the American Educational Research Association, Chicago.
26.
Schvaneveldt, R. W. (Ed.). (1990). Pathfinder associative networks: Studies in knowledge organization. Norwood, NJ: Ablex.
27.
Shavelson, R. J. , Lang, H., & Lewin, B. (1994). On concept maps as potential “authentic” assessments in science (CSE Tech. Rep. No. 388). Los Angeles: University of California, Center for Research on Evaluation, Standards, and Student Testing (CRESST).
28.
Surber, J. R. (1984). Mapping as a testing and diagnostic device. In C. D. Holley & D. F. Dansereau (Eds.), Spatial learning strategies: Techniques, applications, and related issues (pp. 213-233). Orlando, FL: Academic Press.
29.
Tabachnick, B. G. , & Fidell, L. S. (1996). Using multivariate statistics (3rd ed.). New York: HarperCollins.
