Sage Journals: Discover world-class research

Abstract

Procedures play an important role in domains where humans interact with critical, complex systems. In such environments, the operator’s ability to correctly follow a given set of procedures can directly impact system safety. A quantitative measure of procedural adherence during training for complex system operation would be useful to assess trainee performance and evaluate a training program. This paper presents a novel model-based objective metric for quantifying procedural adherence in training. This metric is sensitive to both the number and nature of procedural deviations, and can be used with cluster analysis to classify trainee performance based on adherence. The metric was tested on an experimental data set gathered from volunteers using aircraft maintenance computer-based training (CBT). The properties of the metric are discussed, along with future possibilities.

Get full access to this article

View all access options for this article.

References

Alessi

(2000). Simulation design for training and assessment Aircrew training and assessment (pp. 199-224). New Jersey: Lawrence Erlbaum Associates, Inc.

Bjork

R. A.

Bjork

E. L.

(2006). Optimizing treatment and instruction: Implications of a new theory of disuse Memory and Society: Psychological Perspectives (pp. 109-133): Psychology Press.

Blessing

S. B. A.

J.R. (1996). How People Learn to Skip Steps. Journal of Experimental Psychology: Learning, Memory, and Cognition, 22, 576-598.

García-García

Hernández

E. P.

Diaz

Maria

(2009). A new distance measure for model-based sequence clustering. Pattern Analysis and Machine Intelligence, IEEE Transactions on, 31(7), 1325-1331.

Gerbaud

S. M. N.

Ganier

Arnaldi

Tisseau

(2008). GVT: a platform to create virtual environments for procedural training. Paper presented at the IEEE Virtual Reality, Reno, Nevada.

Ghodsian

D. B. R.A.

Benjamin

A.S.

(1997). Evaluating Training During Training: Obstacles and Opportunities. In Ehrenstein

M. A. Q. a. A. A.

(Ed.), Training for 21st Century Technology: Applications of Psychological Research (pp. 63-88). Washington D.C.: American Psychological Association.

Govaerts

M. J. B.

van der Vleuten

C. P. M.

Schuwirth

L. W. T.

Muijtjens

A. M. M.

(2007). Broadening perspectives on clinical performance assessment: rethinking the nature of in-training assessment. Advances in health sciences education, 12(2), 239-260.

Hamman

(2004). The complexity of team training: what we have learned from aviation and its applications to medicine. Quality and Safety in Health Care, 13(suppl 1), i72-i79.

Lammers

R. L.

Davenport

Korley

Griswold Theodorson

Fitch

M. T.

Narang

A. T.

(2008). Teaching and assessing procedural skills using simulation: metrics and methodology. Academic Emergency Medicine, 15(11), 1079-1087.

10.

Levenshtein

V. I.

(1966). Binary codes capable of correcting deletions, insertions, and reversals.

11.

Manber

Myers

(1990). Suffix arrays: a new method for on-line string searches. Paper presented at the Proc. of the first annual ACM symposium on Discrete algorithms.

12.

Nählinder

Oskarsson

P. A.

Lindahl

Hedström

Berggren

(2009). Effects of simulator training: motivating factors: Information Systems, Swedish Defence Research Agency (FOI).

13.

Owen

Mugford

Follows

Plummer

J. L.

(2006). Comparison of three simulation-based training methods for management of medical emergencies. Resuscitation, 71(2), 204-211.

14.

Rogovin

(1979). Three Mile Island: a report to the commissioners and the public. Washington, DC: Nuclear Regulatory Commission.

15.

Vora

Nair

Gramopadhye

A. K.

Duchowski

A. T.

Melloy

B. J.

Kanki

(2002). Using virtual reality technology for aircraft visual inspection training: presence and comparison studies. Applied Ergonomics, 33(6), 559-570.

A Model-Based Measure to Assess Operator Adherence to Procedures

Abstract

Get full access to this article

References