This article reports the results of an investigation of the convergent criterion-related validity of two computer-based tools for scoring concept maps and essays as part of the ongoing formative evaluation of these tools. In pairs, participants researched a science topic online and created a concept map of the topic. Later, participants individually wrote a short essay from their concept map. The concept maps and essays were scored by the computer-based tools and by human raters using rubrics. Computer-based concept map scores were a very good measure of the qualitative aspects of the concept maps (r = 0.84) and were an adequate measure of the quantitative aspects (r = 0.65). Also, the computer-based essay scores were an adequate measure of essay content (r = 0.71). If computer-based approaches for scoring concept maps and essays can provide a valid, low-cost, easy to use, and easy to interpret measure of students' content knowledge, then these approaches will likely gain rapid acceptance by teachers at all levels.
ClarianaR. B., & KoulR. (2004). A computer-based approach for translating text into concept map-like representations. In CanasA. J.NovakJ. D., and GonzalesF. M. (Eds.), Concept maps: Theory, methodology, technology, vol. 2, in the Proceedings of the First International Conference on Concept Mapping, Pamplona, Spain, Sept 14–17, pp. 131–134. Retrieved October 3, 2004, from http://cmc.ihmc.us/papers/cmc2004–045.pdf
4.
ClarianaR. B.KoulR., & SalehiR. (2005). The criterion-related validity of a computer-based approach for scoring concept maps. International Journal of Instructional Media, 33(3), in press.
5.
GoldsmithT. E., & JohnsonP. J. (1990). A structural assessment of classroom learning. In SchvaneveldtR. W. (Ed.), Pathfinder associative networks: Studies in knowledge organization, pp. 241–253. Norwood, NJ: Ablex Publishing Corporation.
6.
GoldsmithT. E.JohnsonP. J., & ActonW. H. (1991). Assessing structural knowledge. Journal of Educational Psychology, 83, 88–96.
7.
GonzalvoP.CanasJ. J., & BajoM. (1994). Structural representations in knowledge acquisition. Journal of Educational Psychology, 86, 601–616.
8.
JonassenD. H.BeissnerK., & YacciM. (1993). Structural knowledge: Techniques for representing, conveying, and acquiring structural knowledge. Hillsdale, NJ: Lawrence Erlbaum and Associates.
9.
KinchinI. M. (2001). If concept mapping is so helpful to learning biology, why aren't we all doing it?International Journal of Science Education, 23, 1257–1269.
10.
KinchinI. M., & HayD. B. (2000). How a qualitative approach to concept map analysis can be used to aid learning by illustrating patterns of conceptual development. Educational Research, 42, 43–57.
11.
LandauerT. K.FoltzP. W., & LahamD. (1998). Introduction to Latent Semantic Analysis. Discourse Processes, 25, 259–284.
12.
LandauerT. K.LahamD.RehderR., & SchreiderM. E. (1997). How well can passage meaning be derived without using word order? A comparison of latent semantic analysis and humans. In ShaftoM. G. & LangleyP. (Eds.), Proceedings of the 19th annual meeting of the Cognitive Science Society, 412–417. Mahwah, NJ: Erlbaum.
13.
LomaskM.BaronJ. B.GreigJ., & HarrisonC. (1992, March). ConnMap: Connecticut's use of concept mapping to assess the structure of students' knowledge of science. Paper presented at the annual meeting of the National Association of Research in Science Teaching, in Cambridge, MA.
14.
MarkhamK. M.MintzesJ. J., & JonesM. G. (1994). The concept map as a research and evaluation tool: Further evidence of validity. Journal of Research in Science Teaching, 31, 91–101.
15.
National Research Council (2001). Knowing what students know: The science and design of educational assessment. Washington, DC: National Academies Press.
16.
NitkoA. J. (1996). Educational assessment of students. Englewood Cliffs, NJ: Merrill.
17.
NovakJ. D., & GowinD. B. (1984). Learning how to learn. Cambridge, MA: Cambridge University Press.
18.
PowersD. E.BursteinJ. C.ChodorowM. S.FowlesM. E., & KukichK. (2002). Comparing the validity of automated and human scoring of essays. Journal of Educational Computing Research, 25, 407–425.
19.
Ruiz-PrimoM. A. (2000). On the use of concept maps as an assessment tool in science: What we have learned so far. Revista Electronica de Investigacion Educative, 2 (1). Retrieved May 3, 2004, from http://www.redie.ens.uabc.mx/vol2no1/contenido-ruizpri.html
20.
Ruiz-PrimoM. A.SchultzS. E.LiM., & ShavelsonR. J. (2000). Comparison of the reliability and validity of scores from two concept mapping techniques. Journal of Research in Science Teaching, 38, 260–278.
21.
Ruiz-PrimoM. A.ShavelsonR. J.LiM., & SchultzS. E. (2001). On the validity of cognitive interpretations of scores from alternative concept-mapping techniques. Educational Assessment, 7, 99–141.
SchvaneveldtR. W. (1990). Pathfinder associative networks: Studies in knowledge organization. Norwood, NJ: Ablex Publishing Corporation.
24.
ShavelsonR. J.LangH., & LewinB. (1994). On concept maps as potential “authentic” assessments in science (CSE Tech. Rep. No. 388). Los Angeles, CA: University of California, Center for Research on Evaluation, Standards, and Student Testing (CREST).
25.
StoddartT.AbramsR.GasperE., & CanadayD. (2000). Concept maps as assessment in science inquiry learning—A report of methodology. International Journal of Science Education, 22, 1221–1246.
26.
TaricaniE. M., & ClarianaR. B. (in press). A technique for automatically scoring open-ended concept maps. Educational Technology Research and Development, 53(4), in press.
27.
TurnsJ.AtmanC. J., & AdamsR. (2000). Concept maps for engineering education: A cognitively motivated tool supporting varied assessment functions. IEEE Transactions on Education, 43, 164–173.
