To understand the impact of time pressure and automated decision support systems (DSS) in a simulated medical visual search task.
Background
Time pressure usually impairs manual performance in visual search tasks, but DSS support might neutralize this negative effect. Moreover, understanding the impact of time pressure and DSS support seems relevant for many real-world applications of visual search.
Method
We used a visual search paradigm where participants had to search for target letters in a simulated medical image. Participants performed the task either manually or with support of a highly reliable DSS. Time pressure was varied within-subjects by either a trialwise time–pressure manipulation (Experiment 1) or a blockwise manipulation (Experiment 2). Performance was assessed based on signal detection measures. To further analyze visual search behavior, a mouse-over approach was used.
Results
In both experiments, results showed impaired sensitivity under high compared to low time pressure in the manual condition, but no negative effect of time pressure when working with a highly reliable DSS. Moreover, participants searched less under time pressure and when receiving DSS support, indicating participants followed the automation without thoroughly checking recommendations. However, the human-DSS team’s sensitivity was always worse than that of the DSS alone, independent of the strength of time pressure.
Conclusion
Negative effects of time pressure can be ameliorated when receiving support by a DSS, but joint overall performance remains below DSS-alone performance.
Application
Highly reliable DSS seem capable of ameliorating the negative impact of time pressure in complex detection tasks.
AlberdiE.PovyakaloA.StriginiL.AytonP. (2004). Effects of incorrect computer-aided detection (CAD) output on human decision-making in mammography. Academic Radiology, 11, 909–918. https://doi.org/10.1016/j.acra.2004.05.012
2.
BartlettM. L.McCarleyJ. S. (2017). Benchmarking aided decision making in a signal detection task. Human Factors, 59, 881–900. https://doi.org/10.1177/0018720817700258
3.
BejnordiB. E.VetaM.van DiestP. J., et al. (2017). Diagnostic assessment of deep learning algorithms for detection of lymph node metastases in women with breast cancer. JAMA, 318, 2199. https://doi.org/10.1001/jama.2017.14585
4.
BlancaM. J.AlarcónR.ArnauJ.BonoR.BendayanR. (2018). Effect of variance ratio on ANOVA robustness: Might 1.5 be the limit?Behavior Research Methods, 50, 937–962. https://doi.org/10.3758/s13428-017-0918-2
5.
BoskemperM. M.BartlettM. L.McCarleyJ. S. (2021). Measuring the efficiency of automation-aided performance in a simulated baggage screening task. Human Factors, 1-17. https://doi.org/10.1177/0018720820983632.
6.
BricoloE.GianesiniT.FaniniA.BundesenC.ChelazziL. (2002). Serial attention mechanisms in visual search: A direct behavioral demonstration. Journal of Cognitive Neuroscience, 14, 980–993. https://doi.org/10.1162/089892902320474454
7.
BrysonJ. J.DiamantisM. E.GrantT. D. (2017). Of, for, and by the people: The legal lacuna of synthetic persons. Artificial Intelligence and Law, 25, 273–291. https://doi.org/10.1007/s10506-017-9214-9
8.
BurgessA. E.JacobsonF. L.JudyP. F. (2001). Human observer detection experiments with mammograms and power-law noise. Medical Physics, 28, 419–437. https://doi.org/10.1118/1.1355308
de LeeuwJ. R. (2015). jsPsych: A JavaScript library for creating behavioral experiments in a web browser. Behavior Research Methods, 47, 1–12. https://doi.org/10.3758/s13428-014-0458-y
11.
DorriusM. D.der WeideM. C. J.-v.van OoijenP. M. A.PijnappelR. M.OudkerkM. (2011). Computer-aided detection in breast MRI: A systematic review and meta-analysis. European Radiology, 21, 1600–1608. https://doi.org/10.1007/s00330-011-2091-9
12.
DrewT.CunninghamC.WolfeJ. M. (2012). When and why might a computer-aided detection (CAD) system interfere with visual search? An eye-tracking study. Academic Radiology, 19, 1260–1267. https://doi.org/10.1016/j.acra.2012.05.013
13.
FaulF.ErdfelderE.LangA.-G.BuchnerA. (2007). G*Power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behavior Research Methods, 39, 175–191. https://doi.org/10.3758/BF03193146
GaleA. G.MugglestoneM. D.PurdyK. J.McClumphaA. (2000). Is airport baggage inspection just another medical image? In KrupinskiE. A. (Ed.), Medical imaging 2000: Image perception and performance. SPIE. https://doi.org/10.1117/12.383105
16.
GlassG. V.PeckhamP. D.SandersJ. R. (1972). Consequences of failure to meet assumptions underlying the fixed effects analyses of variance and covariance. Review of Educational Research, 42, 237–288. https://doi.org/10.3102/00346543042003237
17.
GreenD. M.SwetsJ. A. (1966). Signal detection theory and psychophysics. Wiley.
18.
HautusM. J. (1995). Corrections for extreme proportions and their biasing effects on estimated values of d’. Behavior Research Methods, Instruments, & Computers, 27, 46–51. https://doi.org/10.3758/bf03203619
19.
HebertC. R.ShaL. Z.RemingtonR. W.JiangY. V. (2020). Redundancy gain in visual search of simulated x-ray images. Attention, Perception, & Psychophysics, 82, 1669–1681. https://doi.org/10.3758/s13414-019-01934-x
20.
HuegliD.MerksS.SchwaningerA. (2020). Automation reliability, human–machine system performance, and operator compliance: A study with airport security screeners supported by automated explosives detection systems for cabin baggage screening. Applied Ergonomics, 86, 103094. https://doi.org/10.1016/j.apergo.2020.103094
21.
JiangF.JiangY.ZhiH.DongY.LiH.MaS.WangY.DongQ.ShenH.WangY. (2017). Artificial intelligence in healthcare: Past, present and future. Stroke and Vascular Neurology, 2, 230–243. https://doi.org/10.1136/svn-2017-000101
22.
LangeK.KühnS.FilevichE. (2015). "Just Another Tool for Online Studies” (JATOS): An easy solution for setup and management of web servers supporting online studies. PLOS ONE, 10, Article e0130834. https://doi.org/10.1371/journal.pone.0130834
23.
LuzM.StraussG.ManzeyD. (2016). Impact of image-guided surgery on surgeons’ performance: A literature review. International Journal of Human Factors and Ergonomics, 4, 229. https://doi.org/10.1504/ijhfe.2016.083516
MadhavanP.WiegmannD. A. (2007). Effects of information source, pedigree, and reliability on operator interaction with decision support systems. Human Factors, 49, 773–785. https://doi.org/10.1518/001872007x230154
MatzenL. E.StitesM. C.GastelumZ. N. (2021). Studying visual search without an eye tracker: An assessment of artificial foveation. Cognitive Research: Principles and Implications, 6, 1–22. https://doi.org/10.1186/s41235-021-00304-2
29.
McCarleyJ. S. (2009). Effects of speed–accuracy instructions on oculomotor scanning and target recognition in a simulated baggage x-ray screening task. Ergonomics, 52, 325–333. https://doi.org/10.1080/00140130802376059
30.
McKinneyS. M.SieniekM.GodboleV., et al. (2020). International evaluation of an AI system for breast cancer screening. Nature, 577, 89–94. https://doi.org/10.1038/s41586-019-1799-6
MeyerJ.KucharJ. K. (2021). Maximal benefits and possible detrimental effects of binary decision aids. 2021. IEEE 2nd International Conference on Human-Machine Systems (ICHMS), Magdeburg, Germany, 8–10 September 2021. https://doi.org/10.1109/ichms53169.2021.9582632
34.
MeyerJ.WiczorekR.GünzlerT. (2014). Measures of reliance and compliance in aided visual scanning. Human Factors, 56, 840–849. https://doi.org/10.1177/0018720813512865
35.
MosierK. L.ManzeyD. (2020). Humans and automated decision aids: A match madein heaven? In MoulouaM.HancockP. A. (Eds.), Human performance in automated and autonomous systems: Current theory and methods (pp. 19–42). CRC Press.
36.
ParasuramanR.ManzeyD. (2010). Complacency and bias in human use of automation: An attentional integration. Human Factors, 52, 381–410. https://doi.org/10.1177/0018720810376055
PashlerH. (1987). Detecting conjunctions of color and form: Reassessing the serial search hypothesis. Perception & Psychophysics, 41, 191–201. https://doi.org/10.3758/bf03208218
39.
PayneJ. W.BettmanJ. R.JohnsonE. J. (1988). Adaptive strategy selection in decision making. Journal of Experimental Psychology: Learning, Memory, and Cognition, 14, 534–552. https://doi.org/10.1037/0278-7393.14.3.534
40.
RiceS.HughesJ.McCarleyJ. S.KellerD. (2008). Automation dependency and performance gains under time pressure. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 52, 1326–1329. https://doi.org/10.1177/154193120805201905
RiceS.KellerD.TrafimowD.SandryJ. (2010). Retention of a time pressure heuristic in a target identification task. The Journal of General Psychology, 137, 239–255. https://doi.org/10.1080/00221309.2010.484447
43.
RiceS.TrafimowD. (2012). Time pressure heuristics can improve performance due to increased consistency. The Journal of General Psychology, 139, 273–288. https://doi.org/10.1080/00221309.2012.705187
44.
RiegerT.HeilmannL.ManzeyD. (2021). Visual search behavior and performance in luggage screening: Effects of time pressure, automation aid, and target expectancy. Cognitive Research: Principles and Implications, 6, 1–12.https://doi.org/10.1186/s41235-021-00280-7
45.
RiegerT.ManzeyD. (2022). Human performance consequences of automated decision aids: The impact of time pressure. Human Factors, 64, 617–634. https://doi.org/10.1177/0018720820965019
46.
RieskampJ.HoffrageU. (2008). Inferences under time pressure: How opportunity costs affect strategy selection. Acta Psychologica, 127, 258–276. https://doi.org/10.1016/j.actpsy.2007.05.004
47.
ShaL. Z.RemingtonR. W.JiangY. V. (2018). Statistical learning of anomalous regions in complex faux x-ray images does not transfer between detection and discrimination. Cognitive Research: Principles and Implications, 3, 1–16. https://doi.org/10.1186/s41235-018-0144-1
SnodgrassJ. G. (1972). Reaction times for comparisons of successively presented visual patterns: Evidence for serial self-terminating search. Perception & Psychophysics, 12, 364–372. https://doi.org/10.3758/bf03207223
50.
StanislawH.TodorovN. (1999). Calculation of signal detection theory measures. Behavior Research Methods, Instruments, & Computers, 31, 137–149. https://doi.org/10.3758/bf03207704
51.
SterchiY.HättenschwilerN.SchwaningerA. (2019). Detection measures for visual inspection of x-ray images of passenger baggage. Attention, Perception, & Psychophysics, 81, 1297–1311. https://doi.org/10.3758/s13414-018-01654-8
van ZandtT.TownsendJ. T. (1993). Self-terminating versus exhaustive processes in rapid visual and memory search: An evaluative review. Perception & Psychophysics, 53, 563–580. https://doi.org/10.3758/bf03205204
