Recent developments in computer technology make it possible to use sound more extensively in the computer interface than has hitherto been possible. This article describes research on uses of computer sound and suggests how sound might be used effectively by instructional and interface designers. After a review of some general principles of interface design and of basic research into auditory perception, the article examines two uses of sound: Earcons, whose purpose is to inform the user about the state of the system itself, and sonitization, whose purpose is to represent data generated by computer programs as sound. Both symbolic and iconic sounds are examined. The role of sound in virtual environments is also discussed with particular attention to its use in aircraft. Design guidelines are listed, and conclusions about needed research are offered.
Get full access to this article
View all access options for this article.
References
1.
LoyG., Musicians Make a Standard: The MIDI Phenomenon, The Computer Music Journal, 9: 4, pp. 8–26, 1985.
2.
ScalettiC. A. and CraigA. B., Using Sound to Extract Meaning from Complex Data, Proceedings of the SPIE—The International Society for Optical Engineering, 1459, pp. 207–219, 1991.
WenzelE. M., Localization in Virtual Acoustic Displays, Presence, 1, pp. 80–107, 1992.
5.
GaverW., The SonicFinder: An Interface that Uses Auditory Icons, Human-Computer Interaction, 4, pp. 67–94, 1989.
6.
WenzelE. M. and FosterS. H., Realtime Digital Synthesis of Virtual Acoustic Environments, Proceedings of Visualization '90, IEEE Computing Society Press, Los Alamitos, 1990.
7.
BlyC., Sound and Human-computer Information Presentation, unpublished doctoral dissertation, Lawrence Livermore National Laboratory and University of California, Davis, California, 1982.
8.
BlattnerM. M.SumikawaD. A., and GreenbergR. M., Icons and Earcons: Their Structure and Common Design Principles, Human-Computer Interactions, 4, pp. 11–44, 1989.
9.
NormanD., The Psychology of Everyday Things, Basic Books, New York, 1988.
10.
NelsonT., The Right Way to Think About Software Design, in The Art of Human-Computer Interface Design, LaurelB. and MountfordS. (eds.), Addison-Wesley, New York, 1990.
11.
GaverW., Auditory Icons: Using Sound in Computer Interfaces, Human-Computer Interaction, 2, pp. 167–177, 1986.
12.
BlattnerM. M.GlinertE. P.JorgeJ. A., and OrmsbyG. R., Metawidgets: Toward a Theory of Multimodal Interface Design, Proceedings of the Sixteenth Annual International Computer Software and Applications Conference, IEEE Computing Society Press, Los Alamitos, 1992.
13.
GlinertE. P.BlattnerM. M.FrerkingC. J., Visual Tools and Languages: Directions for the '90s, Proceedings of the 1991 IEEE Workshop on Visual Languages, IEEE Computing Society Press, Los Alamitos, 1991.
14.
BlattnerM. M., Messages, Models and Media, Multimedia Review, 3: 3, pp. 15–21, 1992.
15.
KoffkaK., Principles of Gestalt Psychology, Harcourt, Brace, & World, New York, 1935.
16.
MooreB. C. J., Introduction to the Psychology of Hearing, Academic Press, New York, 1982.
17.
BregmanA. S., Auditory Scene Analysis, MIT Press, Cambridge, Massachusetts, 1990.
18.
HandelS., Listening, MIT Press, Cambridge, Massachusetts, 1989.
19.
WarrenR., Auditory Perception, A New Synthesis, Pergamon, New York, 1982.
20.
Espinoza-VarasB. and WatsonC., Perception of Complex Auditory Patterns by Humans, in The Comparative Psychology of Audition: Perceiving Complex Sounds, DoolingR. and HulseS. (eds.), Lawrence Erlbaum Associates, Hillsdale, New Jersey, 1989.
21.
HirschI., Temporal Order and Auditory Perception, in Sensation and Measurement, MoskowitzH. (ed.), Reidal, Dordrecht, The Netherlands, 1974.
22.
JonesM., Auditory Patterns: Studies in the Perception of Structure, in Handbook of Perception, Vol. VIII, Perceptual Coding, CarteretteE. and FriedmanM. (eds.), Academic Pres, New York, 1978.
23.
HirschI., Auditory Perception of Temporal Order, Journal of the Acoustical Society in America, 31, pp. 759–767, 1959.
24.
RockI., The Description and Analysis of Object and Event Perception, in The Handbook of Perception and Human Performance, Volume 2, BoffK. R.KaufmanL., and ThomasJ. P. (eds.), Wiley, New York, 1986.
25.
DeutschD. (ed.), The Psychology of Music, Academic Press, New York, 1982.
26.
CarteretteE. and KendallR., Perceiving Complex Sound, in The Comparative Psychology of Audition, DoolingR. and HulseS. (eds.), Lawrence Erlbaum Associates, Hillsdale, New Jersey, 1989.
27.
DowlingW. and HarwoodD., Music Cognition, Academic Press, New York, 1986.
28.
DowlingW. and FujitaniD., Contour, Interval, and Pitch Recognition in Memory for Melodies, Journal of the Acoustical Society of America, 49, pp. 534–541, 1971.
29.
BuxtonW., Introduction to this Special Issue on Nonspeech Audio, Human Computer Interaction, 4, pp. 1–9, 1989.
30.
CrockfordD., Integrating Computers and Television, in The Art of Human-Computer Interaction, LaurelB. and MountfordS. (eds.), Addison-Wesley, New York, 1990.
31.
BarfieldW.RosenbergC., and LevasseurG., The Use of Icons, Earcons, and Commands in the Design of an Online Hierarchical Menu, IEEE Transactions on Professional Communication, 34, pp. 101–109, 1991.
32.
EdwardsA. D. N., Adapting User Interfaces for Visually Disabled Users, doctoral dissertation, Open University, Milton Keynes, United Kingdom, 1987.
33.
EdwardsA. D. N., Soundtrack: An Auditory Interface for Blind Users, Human-Computer Interaction, 4, pp. 45–66, 1989.
34.
EdwardsA. D. N., Extending Communication for Users with Restricted Channels, Proceedings of the IEE Colloquium on Multimedia: The Future of User Interfaces, Digest No. 164, IEE, London, England, 1990.
35.
PittI. J. and EdwardsA. D. N., Navigating the Interface by Sound for Blind Users, Proceedings of the HCI '91 Conference, People and Computers VI, Cambridge University Press, Cambridge, 1991.
36.
EdwardsA. D. N., Modelling Blind Users' Interactions with an Auditory Interface, International Journal of Man-Machine Studies, 30, pp. 575–589, 1989.
37.
KermanJ., Listen, Worth, New York, 1980.
38.
GaverW.SmithR. B., and O'SheaT., Effective Sounds in Complex Systems: The Arkola Simulation, Proceedings of the Conference on Human Factors in Computing Systems. Reaching Through Technology. (CHI 1991), Association for Computing Machinery, New York, 1991.
39.
GaverW. W., Sound Support for Collaboration, Proceedings of the Second European Conference on Computer-Supported Cooperative Work, Kluwer Academic Publications, Dordrecht, The Netherlands, 1991.
40.
GaverW. W. and SmithR. B., Auditory Icons in Large-scale Collaborative Environments, Proceedings of the Third International Conference of the IFIP TC, North Holland, Amsterdam, 1990.
41.
SonnenwaldD. H.GopinathB.HabermanG. O.KeesoW. M., and MyersJ. S., InfoSound: An Audio Aid to Program Comprehension, Proceedings of the Twenty-Third Annual Hawaii International Conference on System Sciences, IEEE Computing Society Press, Los Alamitos, 1990.
42.
MezrichJ.FrysingerS., and SlivjanovskiR., Dynamic Representation of Multivariate Time Series Data, Journal of the American Statistical Association, 79, pp. 34–40, 1984.
43.
BlyS., Communication with Sound, in Readings in Human-Computer Interactions: A Multidisciplinary Approach, BaeckerR. and BuxtonW. (eds.), Morgan Kaufman, Los Altos, California, 1987.
44.
MansurD., Communication with Sound, in Readings in Human-Computer Interactions, BaeckerR. and BuxtonW. (eds.), Morgan Kaufman, Los Altos, California, 1987.
45.
HardinJ., Collaborative Tools for Scientific Communication and Understanding, paper presented at the annual meeting of the American Educational Research Association, San Francisco, April 1992.
46.
MorrisonR. and LunneyD., Communication with Sound, in Readings in Human-Computer Interactions: A Multidisciplinary Approach, BaeckerR. and BuxtonW. (eds.), Morgan Kaufman, Los Altos, California, 1987.
47.
LunneyD.MorrisonR. C.CeteraM. M.HartnessR. V.MillsR. T.SaltA. D., and SowellD. C., A Microcomputer-based Laboratory Aid for Visually Impaired Students, IEEE Micro, 3: 4, pp. 19–31, 1983.
48.
MansurD., Graphs in Sound: A Numerical Data Analysis Method for the Blind, Masters of Science thesis, Lawrence Livermore National Laboratory and University of California, Davis, 1984.
49.
ScalettiC. A., The Kyma/Platypus Computer Music Workstation, Computer Music Journal, 13: 2, pp. 23–38, 1989.
50.
SmithS.BergeronR. D., and GrinsteinG. G., Stereophonic and Surface Sound Generation for Exploratory Data Analysis, in Empowering People: CHI '90 Conference Proceedings, ChewJ. C. and WhitesideJ. (eds.), Association for Computing Machinery, New York, 1990.
51.
GrinsteinG.PickettR., and WilliamsM., EXVIS: An Exploratory Visualization Environment, Proceedings of Graphics Interface '89, National Research Council of Canada, London, Ontario, 1989.
52.
WenzelE. M.StoneP. K.FisherS. S., and FosterS. H., A System for Three-dimensional Acoustic “Viewing” in a Virtual Environment Workstation, Proceedings of the First IEEE Conference on Visualization. Visualization '90, IEEE Computing Society Press, Los Alamitos, 1990.
53.
WenzelE. M.WightmanF. L., and KistlerD. J., Localization with Non-individualized Virtual Acoustic Display Cues, Proceedings of the Conference Human Factors in Computing Systems. Reaching through Technology. (CHI 1991), Association for Computing Machinery, New York, 1991.
54.
WenzelE. M.ArrudaM.KistlerD. J., and WightmanF. L., Localization using Non-individualized Head-related Transfer Functions, Journal of the Acoustical Society of America, 94, pp. 111–123, 1993.
55.
BegaultD. R. and WenzelE. M., Headphone Localization of Speech Stimuli, Proceedings of the 35th Annual Meeting of the Human Factors Society, Bay Area Chapter, Human Factors Society, Santa Monica, California, 1991.
56.
WenzelE. M.WightmanF. L., and FosterS. F., Development of a Three-dimensional Auditory Display System, SIGCHI Bulletin, 20: 2, pp. 52–57, 1988.
57.
FurnessT. A., The Super Cockpit and Human Factors Challenges, Technical Report HITL-M-86-1, Human Interface Technology Laboratory, Seattle, Washington, 1986.
58.
FumessT. A., Configuring Virtual Space for the Super Cockpit, Technical Report HITL-M-89-2, Human Interface Technology Laboratory, Seattle, Washington, 1989.
59.
HallP. S. and CampbellB. L., Helmet-mounted Systems Technology Planning for the Future, Proceedings of the SPIE—The International Society for Optical Engineering, 1695, pp. 2–7, 1992.
60.
SmallR. L. and HowardC. W., A Real-time Approach to Information Management in a Pilot's Associate, Proceedings of the IEEE/AIAA 10th Digital Avionics System Conference, IEEE, New York, 1991.
61.
UlrichE. and WaltersC., Controls and Displays of Douglas Aircraft for the 1990s, Proceedings of the SPIE—The International Society for Optical Engineering, 1664, pp. 96–107, 1992.
62.
DurlachN., Auditory Localization in Teleoperator and Virtual Environment Systems: Ideas, Issues and Problems, Perception, 20, pp. 543–545, 1991.
63.
MorganD. P. and GehringB., Applications of Binaural Sound in the Cockpit, Speech Technology, 5: 2, pp. 46–51, 1990.
64.
NaishP. L. N., Simulating Directionality in Airborne Auditory Warnings and Messages, Proceedings of the Conference Simulation and the User Interface, Taylor and Francis, London, England, 1990.
65.
BrickenM., Virtual Worlds: No Interface to Design, in Cyberspace: The First Steps, BenediktM. (ed.), MIT Press, Cambridge, Massachusetts, 1991.
66.
LaurelB., Interface as Nemesis, in User-Centered System Design: New Perspectives on Human-Computer Interaction, NormanD. and DraperS. (eds.), Lawrence Erlbaum Associates, Hillsdale, New Jersey, 1986.
67.
SimonB., Mathematica for Windows: Reshaping Math Software with Graphics and Sound, PC Magazine, 11: 6, p. 37, 1992.
68.
DeatherageB. H., Auditory and Other Sensory Forms of Information Presentation, in Human Engineering Guide to Equipment Design, Van CottH. P. and KinkadeR. G. (eds.), U.S. Government Printing Office, Washington, D.C., 1972.
69.
WinnW. D. and SolomonC., The Effect of the Spatial Arrangement of Simple Diagrams on the Interpretation of English and Nonsense Sentences, Educational Technology Research and Development, 41, pp. 29–41, 1993.
70.
MillerG. A., The Magical Number Seven Plus or Minus Two: Some Limits on our Capacity for Processing Information, Psychological Review, 63, pp. 81–97, 1956.
