Cognitive Task Analysis and Innovation of Training: The Case of Structured Troubleshooting

Abstract

Troubleshooting is often a time-consuming and difficult activity. The question of how the training of novice technicians can be improved was the starting point of the research described in this article. A cognitive task analysis was carried out consisting of two preliminary observational studies on troubleshooting in naturalistic settings, combined with an interpretation of the data obtained in the context of the existing literature. On the basis of this cognitive task analysis, a new method for the training of troubleshooting was developed (structured troubleshooting), which combines a domain-independent strategy for troubleshooting with a context-dependent, multiple-level, functional decomposition of systems. This method has been systematically evaluated for its use in training. The results show that technicians trained in structured troubleshooting solve twice as many malfunctions, in less time, than those trained in the traditional way. Moreover, structured troubleshooting can be taught in less time than can traditional troubleshooting. Finally, technicians learn to troubleshoot in an explicit and uniform way. These advantages of structured troubleshooting ultimately lead to a reduction in training and troubleshooting costs.

Get full access to this article

View all access options for this article.

References

Ball, L. J. , Evans, J. , Saint, B. T. , Dennis, I. , & Ormerod, T. C. (1997). Problem-solving strategies and expertise in engineering design. Thinking and Reasoning, 3, 247–270.

Bereiter, S. R. , & Miller, S. M. (1989). A field-based study of troubleshooting in computer-controlled manufacturing systems. IEEE Transactions on Systems, Man, and Cybernetics, 19, 205–219.

Bower, G. H. , Clark, M. C. , Lesgold, A. M. , & Winzenz, D. (1969). Hierarchical retrieval schemes in recall of categorical word lists. Journal of Verbal Learning and Verbal Behavior, 8, 323–343.

Duncker, K. (1945). On problem solving. Psychological Monographs, 58(Whole No. 270).

Egan, D. E. , & Schwartz, B. J. (1979). Chunking in recall of symbolic drawings. Memory and Cognition, 7, 149–158.

Frederiksen, J. R. , & White, B. Y. (1993, August). The avionics job-family tutor: An approach to developing generic cognitive skills within a job-situated context. In P. Brna , S. Ohlsson , & H. Pain (Eds.), Proceedings of the World Conference on Artificial Intelligence in Education (pp. 513–520). Charlottesville, VA: Association for the Advancement of Computing in Education.

Glaser, R. , & Chi, M. T. H. (1988). Overview. In M. T. H. Chi , R. Glaser , & M. J. Farr (Eds.), The nature of expertise (pp. xvxxviii). Hillsdale, NJ: Erlbaum.

Hall, E. P. , Gott, S. P. , & Pokorny, R. A. (1995). A procedural guide to cognitive task analysis: The PARI methodology (Tech. Rep. No. AL/HR-TR-1995-0108). Brooks Air Force Base, TX: AFMC.

Klein, G. A. (1989). Recognition-primed decisions. In W. B. Rouse (Ed.), Advances in man-machine systems research (pp. 47–92. Greenwich, CT: JAI.

10.

Kolodner, J. L. (1984). Retrieval and organizational strategies in conceptual memory: A computer model. Hillsdale, NJ: Erlbaum.

11.

Kurland, L. C. , & Tenney, Y. J. (1988). Issues in developing an intelligent tutor for a real-world domain: Training in radar mechanics. In J. Psotka , L. D. Massey , & S. A. Mutter (Eds.), Intelligent tutoring systems: Lessons learned (pp. 119–180. Hillsdale, NJ: Erlbaum.

12.

Merkelbach, E. J. H. M. , & Schraagen, J. M. C. (1994). A framework for the analysis of cognitive tasks (Tech. Report TNO-TM 1994 B-13). Soesterberg, Netherlands: TNO Human Factors Research Institute.

13.

Potter, S. S. , Roth, E. M. , Woods, D. D. , & Elm, W. C. (in press). Bootstrapping multiple converging cognitive task analysis techniques for system design. In J. M. C. Schraagen , S. F. Chipman , & V. L. Shalin (Eds.), Cognitive task analysis. Mahwah, NJ: Erlbaum.

14.

Rasmussen, J. (1986). Information processing and human-machine interaction: An approach to cognitive engineering. Amsterdam: Elsevier.

15.

Reed, N. E. , & Johnson, P. E. (1993). Analysis of expert reasoning in hardware diagnosis. International Journal of Man-Machine Studies, 38, 251–280.

16.

Roth, E. M. , Bennett, K. B. , & Woods, D. D. (1987). Human interaction with an “intelligent” machine. International Journal of Man-Machine Studies, 27, 479–525.

17.

Schaafstal, A. M. (1991). Diagnostic skill in process operation: A comparison between experts and novices. Unpublished doctoral dissertation, Groningen University, Netherlands.

18.

Schaafstal, A. M. (1993). Knowledge and strategies in diagnostic skill. Ergonomics, 36, 1305–1316.

19.

Schaafstal, A. M. (1998). Innovatie van de functie-opleiding Oto Melara [Innovation of the function course Oto Melara] (Tech. Report TNO-TM 1998 A-31). Soesterberg, Netherlands: TNO Human Factors Research Institute.

20.

Schaafstal, A. M. , & Schraagen, J. M. C. (1992). A method for cognitive task analysis (Tech. Report IZF 1992 B-5). Soesterberg, Netherlands: TNO Human Factors Research Institute.

21.

Schaafstal, A. M. , & Schraagen, J. M. C. (1993). Training of systematic diagnosis: A case study in electronics troubleshooting. In Proceedings of AI-ED 1993: World conference on artificial intelligence in education (pp. 529–533). Amsterdam: IOS Press.

22.

Schraagen, J. M. C. , & Schaafstal, A. M. (1991). Diagnose van storingen in een Sewaco-deelsysteem: De LW08 radar [Diagnosis of a Sewaco-subsystem: The LW08-radar] (Tech. Report IZF 1991 A-36). Soesterberg, Netherlands: TNO Human Factors Research Institute.

23.

Schraagen, J. M. C. , & Schaafstal, A. M. (1995). Evaluatie van een nieuwe methode voor gestructureerd storingzoeken in de SMRμ [Evaluation of a new method for structured troubleshooting in the SMRμ] (Tech. Report TNO-TM 1995 A-45). Soesterberg, Netherlands: TNO Human Factors Research Institute.

24.

Schraagen, J. M. C. , & Schaafstal, A. M. (1996). Training of systematic diagnosis: A case study in electronics troubleshooting. Le Travail Humain, 59, 5–21.

25.

Simon, H. A. (1981). The sciences of the artificial (2nd ed.). Cambridge: MIT Press.