Applying Quality Control Charts to the Analysis of Single-Subject Data Sequences

Abstract

Techniques from the field of quality control can be used to classify the quality of individual samples of physical or cognitive performance. After stable baselines have been established for an individual, deviations in performance can be evaluated using control charts. The effectiveness of this approach in evaluating cognitive performance was tested using databases collected under a variety of risk factors. The sensitivity and specificity characteristics of Shewhart, cumulativesum (CUSUM), and exponentially weighted moving average (EWMA) control charts were determined for a total of 174 trials involving 10 participants and 23 cognitive performance assessment measures. The most effective technique in each case was typically a function of the specific performance measure and the type of performance change being evaluated. Sensitivity and specificity for the best techniques were as high as 100%. This study demonstrated the usefulness of quality control charts as a tool to evaluate individual participant performance over time. Actual or potential applications of this research include readiness-to-perform screening of industrial workers in order to improve the health and safety of the workforce.

Get full access to this article

View all access options for this article.

References

Breyfogle, F. W., III. , (1992). Statistical methods for testing, development, and manufacturing. New York: Wiley.

Gawron, V. J. , & Ranney, T. A. (1988). The effects of alcohol dosing on driving performance on a closed course and in a driving simulator. Ergonomics, 31, 1219–1244.

Gilliland, K. , & Schlegel, R. E. (1993). Readiness to perform testing: A critical analysis of the concept and current practices (Final Report DOT/FAA/AM-93--13). Washington, DC: Federal Aviation Administration, Office of Aviation Medicine.

Gilliland, K. , & Schlegel, R. E. (1994, May). Development of a laboratory model of readiness-to-perform testing. Paper presented at the Aerospace Medical Association 65th Annual Scientific Meeting, San Antonio, TX.

Gilliland, K. , & Schlegel, R. E. (1995). Readiness-to-perform testing and the worker. Ergonomics in Design, 3(1), 14–19.

Hays, W. L. (1988). Statistics (4th ed.). Fort Worth, TX: Holt, Rinehart, and Winston.

Hurst, P. M. (1976). Amphetamines and driving behavior. Accident Analysis and Prevention, 8, 9–13.

Kennedy, R. S. , Turnage, J. J. , & Dunlap, W. P. (1992). The use of dose equivalency as a risk assessment index in behavioral neurotoxicology. Neurotoxicology and Teratology, 14, 167–175.

Kennedy, R. S. , Turnage, J. J. , & Dunlap, W. P. (1993). Diagnosis of alcohol intoxication: Effectiveness of cognitive and neurovestibular field sobriety tests. In Proceedings of the Human Factors Society 37th Annual Meeting (pp. 964–967). Santa Monica, CA: Human Factors and Ergonomics Society.

10.

Kennedy, R. S. , Turnage, J. J. , & Jones, M. B. (1995, May). Fitness-for-duty using multiple cut-offs to reduce false positives and retain sensitivity in computerized performance testing. Paper presented at the Aerospace Medical Association 66th Annual Scientific Meeting, Anaheim, CA.

11.

Maylor, E. A. , & Rabbitt, P. M. A. (1987). Effects of alcohol and practice on choice reaction time. Perception and Psychophysics, 42, 465–475.

12.

Mertens, H. W. , & Collins, W. E. (1986). The effects of age, sleep deprivation, and altitude on complex performance. Human Factors, 28, 541–551.

13.

Miller, J. C. (1994). Industrial fitness-for-duty testing (Final Report DTFH61-93-C-00088). Washington, DC: Federal Highway Administration, Trucking Research Institute of the American Trucking Association.

14.

Miller, J. C. , Kim, H. T. , & Parseghian, Z. (1995, May). Feasibility of in-cab fitness-for-duty testing of commercial drivers. Paper presented at the Aerospace Medical Association 66th Annual Scientific Meeting, Anaheim, CA.

15.

Montgomery, D. C. (1997). Introduction to statistical quality control (3rd ed.). New York: Wiley.

16.

O'Donnell, R. D. (1991). Scientific validation of the NOVASCAN™ tests: Theoretical basis and initial validation studies. (Available from Nova Technology, Inc., 19460 Shenango Drive, Tarzana, CA 91356)

17.

O'Donnell, R. D. , Fix, E. , & Morton, P. (1995, May). Non-linear analysis techniques in readiness-to-perform testing. Paper presented at the Aerospace Medical Association 66th Annual Scientific Meeting, Anaheim, CA.

18.

Schiflett, S. G. , Eddy, D. R. , Schlegel, R. E. , French, J. , & Shehab, R. L. (1995, May). Astronaut performance during preflight, inorbit, and recovery. Paper presented at the Aerospace Medical Association 66th Annual Scientific Meeting, Anaheim, CA.

19.

Schlegel, R. E. , Shehab, R. L. , Gilliland, K. , Eddy, D. R. , & Schiflett, S. G. (1995, May). Astronaut baseline practice schedules for the NASA performance assessment workstation (PAWS). Paper presented at the Aerospace Medical Association 66th Annual Scientific Meeting, Anaheim, CA.

20.

Starmer, G. (1985). Antihistamines and highway safety. Accident Analysis and Prevention, 17, 311–317.

21.

Steyvers, F. J. J. M. (1987). The influence of sleep deprivation and knowledge of results on perceptual encoding. Acta Psychologica, 6, 173–187.

22.

Vardeman, S. B. , & Jobe, J. M. (1999). Statistical quality assurance methods for engineers. New York: Wiley.

23.

Wheeler, D. J. , & Chambers, D. S. (1992). Understanding statistical process control. Knoxville, TN: SPC.