This paper describes the empirical validation of a knowledge acquisition methodology using conceptual graph analysis. Conceptual graphs are a representational medium used to integrate and organize knowledge obtained from documents, verbal protocols, question probes, and observation of task performance. The method was validated by presenting two groups of subjects with text and graphics on a topic in engineering dynamics. One set of materials was written by a recognized subject matter expert. The other set was developed by subjecting the expert generated text to conceptual graph analysis. The text was first translated into a conceptual graph structure, then the graph was revised via question probes and observation/induction methods. The final graph was returned to a standard text format. Students receiving the knowledge-engineered text solved significantly more problems than students receiving the original text. Conceptual graph analysis is a generalized method that can be used for a broad range of training domains, providing a highly structured means for making explicit the knowledge base to be incorporated into instructional design.
Get full access to this article
View all access options for this article.
References
1.
Amell. L. (1990). Expert systems for silviculture: A task analysis. Unpublished master's thesis, University of Idaho, Moscow, ID.
2.
Blackman, H. S. (1988). The use of think-aloud verbal protocols for the identification of mental models. In Proceedings of the Human Factors Society 32nd Annual Meeting (pp. 872–874). Santa Monica, CA: Human Factors and Ergonomics Society.
3.
Bonner, J. (1988). Implications of cognitive theory for instructional design: Revisited. Educational Communication and Technology, 36, 3–14.
4.
Boose, J. H. (1988). Uses of repertory grid-centered knowledge acquisition tools for knowledge-based systems. International Journal of Man-Machine Studies, 29, 287-310.
5.
Broadbent, D. E., Fitzgerald, P., and Broadbent, M. H. P. (1986). Implicit and explicit knowledge in the control of complex systems. British Journal of Psychology, 77, 33–50.
6.
Carr, T. H., McCauley, C., Sperber, R. D., and Parmalee, C. M. (1982). Words, pictures, and priming: On semantic activation, conscious identification, and the automaticity of information processing. Journal of Experimental Psychology: Human Perception and Performance, 8, 757–777.
7.
Cooke, N. M., and McDonald, J. E. (1987). The application of psychological scaling techniques to knowledge elicitation for knowledge-based systems. International Journal of Man-Machine Studies, 26, 533–550.
8.
Ericsson, K. A., and Simon, H. A. (1984). Protocol analysis.Cambridge, MA: MIT.
9.
Fairchild, K. M., Poltrock, S. E., and Furnas, G. W. (1988). SemNet: Three-dimensional graphic representations of large knowledge bases. In R. Guindon (Ed.), Cognitive science and its application for human-computer interaction.Hillsdale, NJ: Erlbaum.
10.
Fisher, K. N., Saletti, J., Patterson, H., Thornton, R., Lipson, J., and Spring, C. (1990). Computer-based concept mapping. Journal of College Science Teaching, 19, 347-352.
11.
Fisk, A. D. (1987). High-performance cognitive skill acquisition: Perceptual/rule learning. In Proceedings of the Human Factors Society 31st Annual Meeting (pp. 652-656). Santa Monica, CA: Human Factors and Ergonomics Society.
12.
Fisk, A. D., and Eggemeier, F. T. (1988). Application of automatic/controlled processing theory to training tactical command and control skills: 1. Background and analytic methodology. In Proceedings of the Human Factors Society 32nd Annual Meeting (pp. 1227–1231). Santa Monica, CA: Human Factors and Ergonomics Society.
13.
Flanagan, J. C. (1954). The critical incident technique. Psychological Bulletin, 51, 327–358.
14.
Gagne, E. D. (1985). The cognitive psychology of school learning.Boston: Little, Brown.
15.
Gill, R. T., and Dingus, T. (1987). A structural approach to teaching relative motion. In Proceedings of the 1987 American Society for Engineering Education Annual Conference (pp. 1806–1808). Washington, DC: American Society for Engineering Education.
16.
Gill, R. T., Gordon, S. E., Moore, J., and Barbera, C. (1988). The role of knowledge structures in problem solving. In Proceedings of the 1987 American Society for Engineering Education Annual Conference (pp. 583-590). Washington, DC: American Society for Engineering Education.
17.
Goldstein, I. L. (1986). Training in organizations: Needs assessment, development, and evaluation.Monterey, CA: Brooks/Cole.
18.
Gordon, S. E. (1989). Theory and methods for knowledge acquisition. Al Applications in Natural Resource Management, 3, 19–30.
19.
Gordon, S. E. (1992). Implications of cognitive theory for knowledge acquisition. In R. Hoffman (Ed.), The psychology of expertise: Cognitive research and empirical AI (pp. 99–120). New York: Springer-Verlag.
20.
Gordon, S. E., and Gill, R. T. (1989a). Question probes: A structured method for eliciting declarative knowledge. AI Applications in Natural Resource Management, 3, 13–20.
21.
Gordon, S. E., and Gill, R. T. (1989b). The formation and use of conceptual structures in problem-solving domains (Tech. Report AFOSR-88-0063). Moscow, ID: University of Idaho for the Air Force Office of Scientific Research.
22.
Gordon, S. E., and Gill, R. T. (1992). Knowledge acquisition with question probes and conceptual graph structures. In T. Lauer, E. Peacock, and A. Graesser (Eds.), Questions and information systems (pp. 29–46). Hillsdale, NJ: Erlbaum.
23.
Gordon, S. E., Kinghorn, R., and Schmierer, K. A. (1991). Representing expert knowledge for instructional system design: A case study. In Proceedings of the Human Factors Society 35th Annual Meeting (pp. 1412–1416). Santa Monica, CA: Human Factors and Ergonomics Society.
24.
Gordon, S. E., and Lewis, V. (1990). Knowledge engineering for hypertext instructional systems. In Proceedings of the Human Factors Society 34th Annual Meeting (pp.1412–1416). Santa Monica, CA: Human Factors and Ergonomics Society.
25.
Graesser, A. C., and Clark, L. C. (1985). Structures and procedures of implicit knowledge.Norwood, NJ: Ablex.
26.
Graesser, A. C., and Franklin, S. P. (1990). QUEST: A cognitive model of question answering. Discourse Processes, 13, 279–304.
27.
Graesser, A. C., and Gordon, S. E. (1991). Question answering and the organization of world knowledge. In G. Craik, A. Ortony, and W. Kessen (Eds.), Essays in honor of George Mandler (pp. 227–243). Hillsdale, NJ: Erlbaum.
28.
Graesser, A. C., Gordon, S. E., and Brainerd, L. E. (1992). Quest: A model of question answering. Computers and Mathematics with Applications, 23, 733–745.
29.
Graesser, A. C., Hemphill, D. H., and Brainerd, L. E. (1989). Question answering in the context of causal mechanisms. In Proceedings of the Cognitive Science Society (pp. 621–626). Pittsburgh, PA: Cognitive Science Society.
30.
Greeno, J. G. (1977). Process of understanding in problem solving. In N. J. Castellan, D. B. Pisoni, and G. R. Potts (Eds.), Cognitive theory (pp. 43–83). Hillsdale, NJ: Erlbaum.
31.
Hayes, N. A., and Broadbent, D. E. (1988). Two modes of learning for interactive tasks. Cognition, 28, 80–108.
32.
Kieras, D. E. (1988). What mental model should be taught: Choosing instructional content for complex engineering systems. In J. Psotka, L. D. Massey, and S. A. Mutter (Eds.), Intelligent tutoring systems: Lessons learned (pp. 85–111). Hillsdale, NJ: Erlbaum.
33.
Klein, G. A., Calderwood, R., and Clinton-Scirocco, A. (1986). Rapid decision making on the fire ground. In Proceedings of the Human Factors Society 30th Annual Meeting (pp. 576–580). Santa Monica, CA: Human Factors and Ergonomics Society.
34.
Klein, G. A., Calderwood, R., and MacGregor, D. (1989). Critical decision method for eliciting knowledge. IEEE Transactions on Systems, Man, and Cybernetics, Special Issue, 462–472.
35.
Lang, K. L., Graesser, A. C., and Hemphill, D. D. (1990). The role of questioning in knowledge engineering and the interface of expert systems. Poetics, 19, 143–166.
36.
Lassiter, D. L., Vaughn, J. S., Smaltz, V. E., and Morgan, B. B. (1990). A comparison of two types of training interventions on team communication performance. In Proceedings of the Human Factors Society 34th Annual Meeting (pp. 1372–1376). Santa Monica, CA: Human Factors and Ergonomics Society.
37.
Lewicki, P., Czyzewska, M., and Hoffman, H. (1987). Unconscious acquisition of complex procedural knowledge. Journal of Experimental Psychology: Learning, Memory, and Cognition, 13, 523–530.
38.
Marcel, A. T. (1983). Conscious and unconscious perception: An approach to the relations between phenomenal experience and perceptual processes. Cognitive Psychology, 15, 238–300.
39.
McDonald, J. E., and Schvaneveldt, R. W. (1988). The application of user knowledge to interface design. In R. Guindon (Ed.), Cognitive science and its applications for human-computer interaction (pp. 289–338). Hillsdale, NJ: Erlbaum.
40.
Means, B., and Gott, S. P. (1988). Cognitive task analysis as a basis for tutor development: Articulating abstract knowledge representations. In J. Psotka, L. D. Massey, and S. A. Mutter (Eds.), Intelligent tutoring systems: Lessons learned (pp. 35–57). Hillsdale, NJ: Erlbaum.
41.
Moore, J., and Gordon, S. E. (1988). Conceptual graphs as instructional tools. In Proceedings of the Human Factors Society 32nd Annual Meeting (pp. 1289–1293). Santa Monica, CA: Human Factors and Ergonomics Society.
42.
Rasmussen, J. (1986). Information processing and human-machine interaction: An approach to cognitive engineering.New York: North-Holland.
43.
Redding, R. E. (1989). Perspectives on cognitive task analysis: The state of the state of the art. In Proceedings of the Human Factors Society 33rd Annual Meeting (pp. 1348–1352). Santa Monica, CA: Human Factors and Ergonomics Society.
44.
Redding, R. E. (1990). Taking cognitive task analysis into the field: Bridging the gap from research to application. In Proceedings of the Human Factors Society 34th Annual Meeting (pp. 1304–1308). Santa Monica, CA: Human Factors and Ergonomics Society.
45.
Redding, R. E., and Lierman, B. (1990). Development of a part-task, CBI trainer based upon a cognitive task analysis. In Proceedings of the Human Factors Society 34th Annual Meeting (pp. 1337–1341). Santa Monica, CA: Human Factors and Ergonomics Society.
46.
Roth, E. M., and Woods, D. D. (1990). Analyzing the cognitive demands of problem-solving environments: An approach to cognitive task analysis. In Proceedings of the Human Factors Society 34th Annual Meeting (pp. 1314–1317). Santa Monica, CA: Human Factors and Ergonomics Society.
47.
Ryder, J. M., Redding, R. E., and Beckshi, P. F. (1987). Training development for complex cognitive tasks. In Proceedings of the Human Factors Society 31st Annual Meeting (pp. 1261–1265). Santa Monica, CA: Human Factors and Ergonomics Society.
48.
Sanderson, P. M. (1989). Verbalizable knowledge and skilled task performance: Association, dissociation, and mental models. Journal of Experimental Psychology: Learning, Memory, and Cognition, 15, 729–747.
49.
Schachter, D. L. (1989). On the relation between memory and consciousness: Dissociable interactions and conscious experience. In H. L. Roediger and F. I. M. Craik (Eds.), Varieties of memory and consciousness: Essays in honour of Endel Tulving (pp. 355–389). Hillsdale, NJ: Erlbaum.
50.
Schlager, M. S., Means, B., and Roth, C. (1990). Cognitive task analysis for the real (-time) world. In Proceedings of the Human Factors Society 34th Annual Meeting (pp. 1309–1313). Santa Monica, CA: Human Factors and Ergonomics Society.
51.
Schneider, W., and Detweiler, M. (1987). Connectionist/control architecture for working memory. In G. H. Bower (Ed.), The psychology of learning and motivation (Vol. 21, pp. 53–119). San Diego, CA: Academic.
52.
Schneider, W., and Fisk, A. D. (1982). Concurrent automatic and controlled visual search: Can processing occur without resource cost?Journal of Experimental Psychology: Learning, Memory, and Cognition, 8, 261–278.
53.
Schneider, W., and Regian, J. W. (1989). Developing automatic component skills for air intercept control (Final Tech. Report). San Diego, CA: Office of Naval Research.
54.
Schneider, W., and Shiffrin, R. M. (1977). Controlled and automatic human information processing: I. Detection, search, attention. Psychological Review, 84, 1–66.
55.
Schvaneveldt, R. W. (1990). Pathfinder associative networks: Studies in knowledge organization.Norwood, NJ: Ablex.
56.
Simon, H. A. (1974). How big is a chunk?Science, 183, 482–488.
57.
Sowa, J. F. (1984). Conceptual structures: Information processing in mind and machine.Reading, MA: Addison-Wesley.
58.
Squire, L. M. (1987). Memory and brain.New York, NY: Oxford University.
59.
Zubritsky, M. C., and Coury, B. G. (1987). Multidimensional scaling as a method for probing the conceptual structure of state categories: An individual differences analysis. In Proceedings of the Human Factors Society 31st Annual Meeting (pp. 107–111). Santa Monica, CA: Human Factors and Ergonomics Society
