The expected air traffic growth will introduce new tasks and automation technologies. As a result, the amount of mostly visual cockpit information will increase significantly, leading to more interruptions and risk of data overload. One promising means of addressing this challenge is through the use of multimodal interfaces which distribute information across sensory channels. To inform the design of such interfaces, a meta-analysis was conducted on the effectiveness and performance effects of auditory versus tactile interruption signals. From the 23 studies, ratio scores were computed to compare performance between the two modalities. The impact of 6 moderator variables was also examined. Overall, this analysis shows faster responses to tactile interruptions. However, more complex and very urgent interruption signals are better presented via the auditory modality. The findings add to our knowledge base in multimodal information processing and can inform modality choices in display design for complex data-rich domains.
Get full access to this article
View all access options for this article.
References
1.
BrewsterS.A.KingA.J. (2005). The Design and Evaluation of a Vibrotactile Progress Bar. Proc. of WorldHaptics 2005, Pisa, Italy.
2.
DuncanJ.MartensS.WardR. (1997). Restricted attentional capacity within but not between sensory modalities. Nature, 387, 808-810.
3.
DriverJ.SpenceC. (2004). Crossmodal spatial attention: Evidence from human performance. In SpenceC.DriverJ. (Eds.), Crossmodal Space and Crossmodal Attention. Oxford Univ. Press, New York, NY. 179-220.
4.
FerrisT.SarterN. (2010). When Content Matters. IEEE Transactions on Haptics, 3(3), 199-210.
5.
GlucksbergS. (1963). Rotary pursuit tracing with divided attention to cutaneous, visual, and auditory signals. Journal of Engineering Psychology, 3, 119-125.
6.
HaleK. S.StanneyK. M. (2004). Deriving haptic design guidelines from human physiological, psychophysical, and neurological foundations. IEEE Computer Graphics and Applications. 24(2), 33-39.
7.
HoC-YNikolicM.WatersM.SarterN.B. (2004). Not Now: Supporting Interruption Management By Indicating the Modality and Urgency of Pending Tasks. Human Factors, 46(3), 399-409.
JonesL.A.LockyerB.PiateskiE. (2006). Tactile display and vibrotactile recognition on the torso. Advanced Robotics, 20, 1359-1374.
10.
JonesL.A.SarterN. (2008). Tactile Displays: Guidance for Their Design and Application. Human Factors, 50(1), 90-111.
11.
KaczmarekK. A.Bach-y-RitaP. (1995).Tactile displays. In BarfieldW.FurnessT.III (Eds.), Virtual environments and advanced interface design. Oxford, England: Oxford Univ. Press. 349-414.
12.
KontarinisD. A.HoweR. D. (1995). Tactile display of vibratory information in teleoperation and virtual environments. Presence, 4(4), 387-402.
13.
LatorellaK. A. (1999). Investigating interruptions: Implications for flightdeck performance (NASA/TM-1999-209707). NASA Langley Research Center, Hampton, VA.
14.
MacLeanK. E.HaywardV. (2008). Do It Yourself Haptics, Part II. IEEE Robotics and Automation Magazine, 15(1), 104-119.
15.
OviattS. (2003). Multimodal interfaces. In JackoJ.SearsA. (Eds.), The human-computer interaction handbook: Fundamentals, evolving technologies and emerging applications. Lawrence Erlbaum Assoc., Mahwah, NJ. 286-304.
16.
RollinsH.HendricksR. (1980). Processing of words presented simultaneously to eye and ear. Journal of Experimental Psychology: Human Perception and Performance, 6, 99-109.
17.
RosenthalR. (1991). Meta-analytic procedures for social research (rev. ed.). Newbury Park, CA: Sage.
18.
SarterN.B. (2006). Multimodal Information Presentation. Int’l Journal of Industrial Ergonomics (Invited contribution to Special Issue on “New Insights in Human Performance and Decision Making”), 36(5), 439-445.
19.
SarterN. (2002). Multimodal Information Presentation In Support of Human-Automation Communication and Coordination. In SalasEduardo (Ed.), Advances in Human Performance and Cognitive Engineering Research. New York, NY: JAI Press. 13-36.
20.
SarterN.B.WoodsD.D. (2000). Teamplay with a Powerful and Independent Agent. Human Factors, 42(3), 390-402.
21.
SklarA.E.SarterN.B. (1999). Good Vibrations: Tactile Feedback in Support of Attention Allocation and Human-Automation Coordination in Event-Driven Domains. Human Factors, 41(4), 543-552.
22.
StanneyK.SammanS.ReevesL.HaleK.BuffW.BowersC.GoldiezB.NicholsonD.LackeyS. (2004). A paradigm shift in interactive computing: deriving multimodal design principles from behavioral and neurological foundations. Int’l Journal of Human-Computer Interaction, 17(2), 229-257.
23.
TreismanA.DaviesA. (1973). Dividing attention to ear and eye. In KornblulmS. (Ed.), Attention and Performance IV (pp. 101-117). New York: Academic Press.
24.
Van ErpJ.KooiF.BronkhorstA.van LeeuwenD.van EschM.van WijngaardenS.Multimodal interfaces: a framework based on modality appropriateness. HFES Proc.—2006.
25.
WickensC. D. (1980). The structure of attentional resources. In NickersonR. S. (Ed.), Attention and performance VIII. Hillsdale, NJ: Erlbaum. 239-258.
26.
WickensC.D.PrinetJ.HutchinsS.SarterN.B.SebokA. (2011, submitted). Auditory-Visual Redundancy in Vehicle Control Interruptions: Two Meta-Analyses. To appear in the Proceedings of the Human Factors and Ergonomics Society 55th Annual Meeting.
27.
WickensC. D.SebokA.BzostekJ.Steelman-AllenK.McCarleyJ.SarterN. (2009). NT-SEEV: A model of attention capture and noticing on the flight deck. In Proc. of the 15th Int’l Symposium on Aviation Psychology [CD-ROM]. Dayton, OH: Wright State University.
28.
BlissJ.LiebmanR.BrillJ.Alert Modality and Behavioral Compliance During Virtual Combat. HFES Proc.—2010.
29.
CalhounG.RuffH.DraperM.GuilfoosB.Tactile and Aural Alerts in High Auditory Load UAV Control Environments. HFES Proc.—2005.
30.
CalhounG.FontejonJ.DraperM.RuffH.GuilfoosB. (2004). Tactile Versus Aural Redundant Alert Cues for UAV Control Applications. HFES Proc.—2004.
31.
FerrisT.SarterN.Cross-Modal Links Among Vision, Audition, and Touch in Complex Environments. Human Factors, 50, 17-26.
32.
FerrisT.PenfoldR.HameedS.SarterN. (2006). The Implications of Crossmodal Links in Attention for the Design of Multimodal Interfaces: A Driving Simulation Study. HFES Proc.—2006.
33.
GlummM.KehringK.WhiteT. (2006). Effects of Tactile, Visual, and Auditory Cues About Threat Location on Target Acquisition and Attention to Visual and Auditory Communications. ARL-TR-3863, U.S. Army Research Laboratory.
34.
HoC.SpenceC. (2009). Using Peripersonal Warning Signals to Orient a Driver?s Gaze. Human Factors, 51, 539-556.
35.
HoC.ReedN.SpenceC. (2009). Multisensory In-Car Warning Signals for Collision Avoidance. Human Factors, 49, 1107-1114.
36.
HoC.HongT.SpenceC. (2006). The Differential Effect of Vibrotactile and Auditory Cues on Visual Spatial Attention. Ergonomics, 49, 724-738.
37.
HoC.HongT.SpenceC. (2005). Warning Signals Go Multisensory. In Proc. of HCI Int’l—2005.
38.
KrausmanA.ElliottL.PettittR. (2005). Effects of Visual, Auditory, and Tactile Alerts on Platoon Leader Performance and Decision Making. ARL-TR-3633, U.S. Army Research Laboratory
39.
LarkinJoseph. (1983). A Comparison of Audio, Visual, and Tactile Warning Devices in a Simulated Flight Environment. Master?s Thesis, Naval Postgraduate School.
40.
LeeJ.McGeheeD.BrownT.MarshallD. (2006). Effects of Adaptive Cruise Control and Alert Modality on Driver Performance. Transportation Research Record, 49-56.
41.
MeratN.JamsonA. (2008). The Effect of Stimulus Modality on Signal Detection: Implications for Assessing the Safety of In-Vehicle Technology. Human Factors, 50, 145-158.
42.
MohebbiR.GrayR.TanH. (2009). Driver Reaction Time to Tactile and Auditory Rear-End Collision Warnings While Talking on a Cell Phone. Human Factors, 51, 102-110.
43.
MortimerC. (2006). Affects of Task Difficulty on Target Guidance Using Auditory and Tactile Cues. HFES Proc.—2006.
44.
SavickD.ElliottL.ZubalO.StachowiakC. (2006). The Effect of Audio and Tactile Cues on Soldier Decision Making and Navigation in Complex Simulation Scenarios. ARL-TR-4413, U.S. Army Research Laboratory.
45.
ScottJ.GrayR. (2008). A Comparison of Tactile, Visual, and Auditory Warnings for Rear-End Collision Prevention in Simulated Driving. Human Factors, 50, 264-275.
46.
SmithC.CleggB.HeggestadE.Hopp-LevineP. (2009). Interruption management: A comparison of auditory and tactile cues for both alerting and orienting. Int’l Journal of Human-Computer Studies, 67, 777-786.
47.
StanleyL. (2006). Haptic and Auditory Cues for Lane Departures Warnings. HFES Proc.—2006.
48.
StraughnS.GrayR.TanH. (2009). To Go or Not to Go: Stimulus-Response Compatibility for Tactile and Auditory Pedestrian Collision Warnings. IEEE Transactions on Haptics, 2, 111-117.
49.
TilakR.XholiI.SchowalterD.FerrisT.HameedS.SarterN. (2008). Crossmodal Links in Attention in the Driving Environment: The Roles of Cueing Modality, Signal Timing, and Workload. HFES Proc.—2008.
50.
WickensC.SmallR.AndreT.BagnallT.BrenamanC. (2008). Multisensory Enhancement of Command Displays for Unusual Attitude Recovery. The Int’l Journal of Aviation Psychology, 18, 255-267.
