In augmented cognition (AugCog), mitigation strategies are used as real-time intervention techniques that are triggered by the outcome of cognitive state assessment and context to significantly improve human-systems performance. Yet, no common ground has been established regarding best practices and what aspects to consider during implementation. This paper discusses mitigation strategies currently used in AugCog systems and provides insights into their strengths and weaknesses. An event-based conceptual framework is introduced that aids real-time mitigation strategy selection by linking system events to real-time cognitive state indicators, which together determine when, what, and how to mitigate. Insights from the implementation of this framework in an AugCog system designed to optimize situation awareness are presented, which support the architecture of the framework and identify further challenges to mitigation. Future work should focus on further validating the proposed framework and leveraging techniques from other domains (e.g., film, theater) to create more effective mitigation concepts in AugCog systems.
Get full access to this article
View all access options for this article.
References
1.
ArringtonC. M.AltmannE. M.CarrT. H. (2003). Tasks of a feather flock together: Similarity effects in task switching. Memory & Cognition, 31, 781–789.
2.
ArroyoE.SelkerT.StouffsA. (2002). Interruptions as multimodal outputs: Which are the less disruptive? ICMI′02, Pittsburgh, PA. (pp. 479–483).
3.
BarkerR. A.EdwardsR. E.O'NeillK. R.TollarJ. R. (2004). DARPA Improving Warfighter Information Intake Under Stress — Augmented cognition concept validation experiment (CVE) analysis report for the Boeing team. Arlington, VA: DARPA/IPTO.
4.
BarsalouL. W.SimmonsW. K.BarbeyA. K.WilsonC. D. (2003). Grounding conceptual knowledge in modality-specific systems. TRENDS in Cognitive Sciences, 7(2), 84–91.
ClamannM. P.WrightM. C.KaberD. B. (2002). Comparison of performance effects of adaptive automation applied to various stages of human-machine system information processing. Proceedings of the Human Factors and Ergonomics Society 46th Annual Meeting (pp. 342–346). Santa Monica, CA: SAGE Publications.
7.
De KeyserV. (1998). Introduction to dynamic interfaces. In DeKeyserV.D'YdewalleG.VandierendonckA. (Eds.), Time and dynamic control of behavior (pp. 191–200). Berne, Switzerland: Hogrefe and Huber.
8.
DeyA. K.AbowdG. D. (1999). Towards a better understanding of context and context—Awareness (Technical Report GIT-GVU-99–22), Atlanta: Georgia Institute of Technology.
9.
DorneichM.WhitlowS.VerversP. M.MathanS.RajA.MuthE., et al. (2004). DARPA Improving Warfighter Information Intake Under Stress—Augmented cognition: Concept validation experiment (CVE) analysis report for the Honeywell team. Arlington, VA: DARPA/IPTO.
10.
DorseyJ. O.SillionF. X.GreenbergD. P. (1991). Design and simulation of opera lighting and projection effects. Computer Graphics, 25(4), 41–50.
11.
DufresneA.MartialO.RamsteinC.MabilleauP. (1996). Sound, space, and metaphor: Multimodal access to Windows for blind users. Proceedings of ICAD 96: International Conference on Auditory Display (pp. 51–58). Palo Alto, CA: ICAD.
EndsleyM. R. (1995). Toward a theory of situation awareness. Human Factors, 37, 32–64.
14.
EndsleyM. R. (1999). Situation awareness and human error: Designing to support human performance. Proceedings of the High Consequence Systems Surety Conference. Albuquerque, NM: Sandia National Laboratory.
15.
EndsleyM. R.KaberD. B. (1999). Level of automation effects on performance, situation awareness and workload in a dynamic control task. Ergonomics, 42, 462–492.
16.
GopherD.ArmonyL.GreenspanY. (2000). Switching tasks and attention policies. Journal of Experimental Psychology: General, 129, 308–229.
17.
HaleK. S.StanneyK. M. (2004). Deriving haptic design guidelines from human physiological, psychophysical, and neurological foundation. IEEE Computer Graphics and Applications, 24(2), 33–39.
18.
HaleK. S.FuchsS.AxelssonP.BaskinA.JonesD. L.BerkaC., et al. (2006). Information delivery and display for shared awareness in the net-centric battlespace (SBIR Phase I final report contract No. W31P4Q-06-C-0041). Arlington, VA: DARPA/IXO.
19.
HapeshiK.JonesD. (1992). Interactive multimedia for instruction: A cognitive analysis of the role of audition and vision. International Journal of Human-Computer Interaction, 41(1), 79–99.
20.
HildebrandtM.DixA.MeyerH. A. (2004). Time design. In Dykstra-EricksonE.TscheligiM. (Eds.), CHI 2004 Conference on Human Factors in Computing Systems (pp. 1737–1738). New York: ACM Press.
21.
KaberD. B.RileyJ. M.TanK.-W.EndsleyM. R. (2001). On the design of adaptive automation for complex systems. International Journal of Cognitive Ergonomics, 5, 37–57.
22.
KawaharaJ.-I.MiyataniM. (2001). The effect of informative and uninformative cueing of attention on feature integration. Journal of General Psychology, 128, 57–75.
23.
KincsesW. E. (2005). Daimler Chrysler's closed loop integrated prototype: Current status and outlook. In SchmorrowD. D. (Ed.), Foundations of augmented cognition (pp. 477–478). Mahwah, NJ: Erlbaum.
24.
KirshD. (2000). A few thoughts on cognitive overload. Intellectica, 30, 19–51.
25.
KroftP.WickensC. D. (2001). The display of multiple geographical data bases: Implications of visual attention (Report No. ARL-01-2/NASA-01-2). Savoy, IL: University of Illinois, Aviation Research Laboratory. Retrieved October 31, 2007, from http://www.humanfactors.uiuc.edu/ReportsPapersPDFs/TechReport/01–02.pdf.
26.
KruseA. A.SchmorrowD. D. (2005). Session overview. In SchmorrowD. D. (Ed.), Foundations of augmented cognition (pp. 441–445). Mahwah, NJ: Erlbaum.
27.
LindemanR. W.YanagidaY.SibertJ. L.LavineR. (2003). Effective vibrotactile cueing in a visual search task. Proceedings of the Ninth IFIO TC13 International Conference on Human-Computer Interaction (INTERACT 2003), Amsterdam: IOS Press.
28.
LiuY.FeyenR.TsimhoniO. (2006). Queuing Network-Model Human Processor (QN-MHP): A computational architecture for multitask performance in human-machine systems. ACM Transactions on Computer-Human Interaction, 13, 37–70.
29.
MamykinaL.MynattE.TerryM. A. (2001). Time aura: Interfaces for pacing. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp. 144–151). New York: ACM Press.
30.
MankoffJ.DeyA.HsiehG.KientzJ.LedererS.AmesM. (2003). Heuristic evaluation of ambient displays. Proceedings of the ACM Conference on Human Factors in Computing Systems (pp. 169–176). New York: ACM Press.
31.
McFarlaneD.C. (2002). Comparison of four primary methods for coordinating the interruption of people in human-computer interaction. Human-Computer Interaction, 17, 63–139.
32.
MillerC.DorneichM. C. (2006). From associate systems to augmented cognition: 25 years of user adaptation in high criticality systems. In SchmorrowD. D.StanneyK. M.ReevesL. M. (Eds.), Foundations of augmented cognition (2nd ed., pp. 344–353). Arlington, VA: Strategic Analysis, Inc.
33.
MiyataY.NormanD. (1986). Psychological issues in support of multiple activities. In NormanD. A.DraperS. W. (Eds.), User centered systems design: New perspectives on human-computer interaction (pp. 265–284). Mahwah, NJ: Erlbaum.
34.
PascoeJ. (1998). Adding generic contextual capabilities to wearable computers. 2nd International Symposium on Wearable Computers (pp. 92–99). Los Alamitos, CA: IEEE Computer Society.
35.
PecherD.ZeelenbergR.BarsalouL. W. (2003). Verifying different-modality properties for concepts produces switching costs. Psychological Science, 14, 119–124.
36.
PopescuV.BurdeaG.TrefftzH. (2002). Multimodal interaction modeling. In StanneyK. M. (Ed.), Handbook of virtual environments: Design implementation, and applications (pp. 435–454). Mahwah, NJ: Erlbaum.
37.
PosnerM. I. (1980). Orienting of attention. Quarterly Journal of Experimental Psychology, 32, 3–25.
38.
PosnerM. I.SnyderC. R. R.DavidsonB. J. (1980). Attention and the detection of signals. Journal of Experimental Psychology: General, 109, 160–174.
39.
PrinzelL. J.ParasuramanR.FreemanF. G.ScerboM. W.MikulkaP. J.PopeA. T. (2003). Three experiments examining the use of electroencephalogram, event-related potentials, and heart rate variability for real-time human-centered adaptive automation design. NASA Technical Report. NASA STI Program Office, Hanover, MD. Retrieved October 31, 2007, from http://techreports.larc.nasa.gov/ltrs/PDF/2003/tp/NASA-2003-tp212442.pdf.
40.
RothrockL.KoubekR.FuchsF.HaasM.SalvendyG. (2002). Review and reappraisal of adaptive interfaces: Towards biologically inspired paradigms. Theoretical Issues in Ergonomics Science, 3, 47–84.
41.
ScallenS. F.HancockP. A. (2001). Implementing adaptive function allocation. International Journal of Aviation Psychology, 11, 197–221.
42.
ScallenS. F.HancockP. A.DuleyJ. A. (1995). Pilot performance and preference for short cycles of automation in adaptive function allocation. Applied Ergonomics, 26, 397–403.
43.
ScerboM. W. (2001). Adaptive automation. In KarwowskiW. (Ed.), International encyclopedia of ergonomics and human factors (pp. 1077–1079). London: Taylor & Francis.
44.
SchmorrowD.StanneyK. M.WilsonG.YoungP. (2005). Augmented cognition in human-system interaction. In SalvendyG. (Ed.), Handbook of human factors and ergonomics (3rd ed., pp. 1364–1384). New York: Wiley.
45.
Smith-JentschK. A.ZeisigR. L.ActionB.McPhersonJ. A. (1998). A strategy for guided team self-correction. In Cannon-BowersJ. A.SalasE. (Eds.), Making decisions under stress: Implications for individual and team training (pp. 271–297). Washington, DC: American Psychological Association.
46.
JohnM.SmallmanH. S.ManesD. I.FeherB. A.MorrisonJ. G. (2005). Heuristic automation for decluttering tactical displays. Human Factors, 47, 509–523.
47.
SukaviriyaP.FoleyJ. D. (1990). Coupling a UI-framework with automatic generation of context-sensitive animated help. Proceedings of the 3rd annual ACM SIGGRAPH Symposium on User Interface Software and Technology (pp. 152–166), New York: ACM Press.
TollarJ. T. (2005). Statistical process control as a triggering mechanism for augmented cognition mitigations. In SchmorrowD. D. (Ed.), Foundations of augmented cognition (pp. 414–420). Mahwah, NJ: Erlbaum.
50.
TownsendJ. T. (1971). A note on the identifiability of parallel and serial processes. Perception and Psychophysics, 10, 161–163.
51.
TremouletP.BartonP.CravenP.CorradoC.MayerG.StiblerK., et al. (2005). DARPA Improving Warfighter Information Intake Under Stress — Augmented cognition phase 3 concept validation experiment (CVE) analysis report for the Lockheed-Martin ATL team (Contract No. NBCH030032). Arlington, VA: DARPA/IPTO.
52.
WatzmanS. (2003). Visual design principles for usable interfaces. In JackoJ. A.SearsA. (Eds.), The human-computer interaction handbook: Fundamentals, evolving technologies and emerging applications (pp. 263–285). Mahwah, NJ: Erlbaum.
53.
WickensC. D. (1984). Engineering psychology and human performance. New York: Harper Collins.
54.
WickensC. D.LiuY. (1988). Codes and modalities in multiple resources: A success and a qualification. Human Factors, 30, 599–616.
55.
WickensC. D.ConejoR.GemplerK. (1999). Unreliable automated attention cueing for air-ground targeting and traffic maneuvering. Proceedings of the Human Factors and Ergonomics Society 43rd Annual Meeting (pp. 21–25). Santa Monica, CA: SAGE Publications.
56.
WickensC. D. (2002). Multiple resources and performance prediction. Theoretical Issues in Ergonomics Science, 3, 159–177.
57.
WinterH.ChampigneuxG.ReisingJ.StrohalM. (1997). Intelligent decision aids for human operators. AGARD Conference Proceedings, Vol. 2, p. B3). Neuilly Sur Seine, France: NATO-AGARD.
58.
YehM.WickensC. D. (2001). Attentional filtering in the design of electronic map displays: A comparison of color coding, intensity coding, and decluttering techniques. Human Factors, 43, 543–562.
