Readiness for Duty: Tuning False Positives by Simulation from Empirical Data

Abstract

For applications such as the assessment of environmental stress or toxic agents, the metric requirements of performance test batteries include stability, reliability and sensitivity. However, fitness-for-duty applications present additional organizational and management laboratory conditions where the sensitivities of these test batteries are evaluated, the ratio of “treated” to “untreated” subjects is usually 50/50. However, in the workplace, the percentage of persons who are expected NOT to be impaired may be <5% and unless the accuracy of the psychological tests exceeds one minus the percentage NOT impaired (1−.05 = 95%) the percentage of false positives will exceed the percentage of impaired persons identified - thus false positives become a primary management focus. Data from four different empirical data sets (N>100), with multiple repeated measures (15−40 sessions) and a battery of six computerized tests, were implemented into an interactive computerized algorithm. By varying: a) multiple cut-offs; b) number trials in baseline; and c) select decrement criteria, we were able to tune false positive rates to levels lower than three percent.