A substantial literature has been developed that deals with the use of visual displays to support human problem solving. In particular, the cognitive systems engineering literature has emphasized the use of visual displays to improve performance on complex real-world tasks such as process control with the labels direct perception, ecological interface design, representational design, and semantic mapping, focusing on the use of representations that take advantage of powerful perceptual processes to support problem solving. Although the theoretical orientations and details of each approach are slightly different, they all share a fundamental core belief: The effectiveness of a graphical decision aid depends on relationships between the representation, the domain and associated task(s), and the characteristics of the agent (person). This review begins with a discussion of strategies for representation aiding, assuming that the appropriate domain semantics have been determined. It then discusses a core assumption in this literature that, in the design of complex real-world systems, designers cannot anticipate all the possible scenarios that could arise and must therefore design displays that support effective problem solving even when novel or unanticipated scenarios are encountered. Finally, the results of empirical studies of designs based on representation aiding are reviewed.
Get full access to this article
View all access options for this article.
References
1.
BadreA. N. (2002). Shaping web usability, interaction design in context. Boston: Pearson Education.
2.
BennettK. (1992). Representation aiding: Complementary decision support for a complex, dynamic control task. IEEE Control Systems, August, 19–24.
3.
BennettK.FlachJ. (1992). Graphical displays: Implications for divided attention, focused attention, and problem solving. Human Factors, 34, 513–533.
4.
BennettK.NagyA.FlachJ. (1997). Visual displays. In SalvendyG. (Ed.), Handbook of human factors and ergonomics (pp. 659–696). New York: Wiley.
5.
BennettK.TomsM.WoodsD. (1993). Emergent features and graphical elements: Designing more effective configural displays. Human Factors, 35, 71–97.
6.
BennettK. B.FritzH. I. (2005). Objects and mappings: Incompatible principles of display design—A critique of Marino and Mahan. Human Factors, 47, 131–137.
7.
BennettK. B.PayneM.CalcaterraJ.NittoliB. (2000). An empirical comparison of alternative methodologies for the evaluation of configural displays. Human Factors, 42, 287–298.
8.
BennettK. B.WaltersB. (2001). Configural display design techniques considered at multiple levels of evaluation. Human Factors, 43, 415–434.
9.
BrunswickE. (1956). Perception and the representative design of psychological experiments. Berkeley: University of California Press.
BurnsC.KuoJ.NgS. (2003). Ecological interface design: A new approach for visualizing network management. Computer Networks, 43, 369–388.
12.
BurnsC.VicenteK.ChristoffersonK.PawlakW. (1997). Towards viable, useful and usable human factors design guidance. Applied Ergonomics, 28, 311–322.
13.
CarswellC. M.WickensC. D. (1987). Information integration and the object display. Ergonomics, 30, 511–527.
14.
ClevelandW. (1985). The elements of graphing data. Monterey, CA: Wadsworth.
15.
ClevelandW. S.McGillR. (1985). Graphical perception and graphical methods for analyzing scientific data. Science, 229, 828–833.
16.
ColeW. (1986). Medical cognitive graphics. Proceedings of the 1986 ACM Conference on Computer-Human Interaction (pp. 91–95). New York: Association for Computing Machinery.
17.
FlachJ. M.BennettK. B. (1992). Graphical interfaces to complex systems: Separating the wheat from the chaff. Proceedings of the Human Factors Society 36th Annual Meeting (pp. 470–474). Santa Monica, CA: SAGE Publications.
18.
GoettlB.WickensC.KramerA. (1991). Integrated displays and the perception of graphical data. Ergonomics, 34, 1047–1063.
19.
GoldsmithT. E.SchvaneveldtR. W. (1984). Facilitating multiple-cue judgments with integral information displays. In ThomasJ. C.SchneiderM. L. (Eds.), Human factors in computer systems (pp. 243–270). Mahwah, NJ: Erlbaum.
20.
HegartyM. (2004). Diagrams in the mind and in the world: Relations between internal and external visualizations. Lecture Notes in Artificial Intelligence, 2980, 1–13.
21.
HutchinsE. (1995). Cognition in the wild. Cambridge: MIT Press.
22.
JagacinskiR. J.FlachJ. M. (2003). Control theory for humans: Quantitative approaches to modeling performance. Mahwah, NJ: Erlbaum.
23.
JamiesonG. (2002a). Empirical evaluation of an industrial application of ecological interface design. Proceedings of the Human Factors and Ergonomics Society 46th Annual Meeting (pp. 536–540). Santa Monica, CA: SAGE Publications.
24.
JamiesonG. (2002b). Ecological interface design for petrochemical process control: Integrating task and system-based approaches (CEL-02–01). Cognitive Engineering Laboratory, University of Toronto.
25.
KeodigerK.NathanM. (2004). The real story behind story problems: Effects of representations on quantitative reasoning. Journal of the Learning Sciences, 13, 129–164.
26.
KoffkaK. (1935). Principles of Gestalt psychology. New York: Harcourt, Brace and World.
27.
KotovskyK.SimonH. (1990). Why some problems are really hard: Explorations in the problem space of difficulty. Cognitive Psychology, 22, 143–183.
28.
KrugS. (2000). Don't make me think—Common sense approach to web usability. Indianapolis: New Riders.
29.
LarsonA.HayesC. (2005). An assessment of WEASEL: A decision support system to assist in military planning. Proceedings of the Human Factors and Ergonomics Society 48th Annual Meeting (pp. 287–291). Santa Monica, CA: SAGE Publications.
MillerC.VicenteK. (1998). Abstraction decomposition space analysis for NOVA's E1 acetylene hydrogenation reactor (CEL-98–09). Cognitive Engineering Laboratory, University of Toronto.
32.
MitchellC. M.MillerR. A. (1983). Design strategies for computer-based information displays in real-time control systems. Human Factors, 25, 353–369.
33.
MitchellC. M.MillerR. A. (1986). A discrete control model of operator function: A methodology for information display design. IEEE Transactions on Systems, Man and Cybernetics, 16, 343–357.
34.
MorayN.LeeJ.VicenteK. J.JonesB. G.RasmussenJ. (1994). A direct perception interface for nuclear power plants. Proceedings of the Human Factors and Ergonomics Society 38th Annual Meeting (pp. 481–485). Santa Monica, CA: SAGE Publications.
35.
MorayN.LootsteenP.PajakJ. (1986). Acquisition of process control skills. IEEE Transactions on Systems, Man and Cybernetics, 16, 497–504.
36.
NewellA.SimonH. A. (1972). Human problem solving. Englewood Cliffs, NJ: Prentice Hall.
37.
NielsenJ. (1994). Usability engineering. San Francisco: Morgan Kaufmann.
38.
NielsenJ. (1997). Usability testing. In SalvendyG. (Ed.), Handbook of human factors and ergonomics (pp. 1543–1567). New York: Wiley.
39.
NielsenJ. (2000). Designing web usability: The practice of simplicity. Indianapolis: New Riders.
40.
NormanD. A. (1993). Things that make us smart: Defending human attributes in the age of the machine. New York: Pegasus.
41.
NormanD. A. (2002). The design of everyday things. New York: Doubleday.
42.
O'MalleyC.DraperS. (1992). Representation and interaction: Are mental models all in the mind? In RogersY.RutherfordA.BibbyP. (Eds.), Models in the mind (pp. 73–92). London: Academic.
43.
OrmerodT. (2002). Dynamic constraints in insight problem solving. Journal of Experimental Psychology—Learning, Memory and Cognition, 28, 791–799.
44.
PerkinsD. (2000). Archimedes bathtub: The art and logic of breakthrough thinking. New York: W. W. Norton.
45.
PomerantzJ. (1986). Visual form perception: An overview. In NusbaumH. C.SchwabE. C. (Eds.), Pattern recognition by humans and machines Vol. 2: Visual perception (pp. 1–30). Orlando, FL: Academic.
46.
RasmussenJ. (1983). Skills, rules, knowledge, signals, signs and symbols, and other distinctions in human performance models. IEEE Transactions on Systems, Man and Cybernetics, 13, 257–266.
47.
RasmussenJ.PejtersenA. M.GoodsteinL. P. (1994). Cognitive systems engineering. New York: Wiley.
48.
RasmussenJ.VicenteK. J. (1989). Coping with human errors through system design—Implications for ecological interface design. International Journal of Man-Machine Studies, 31, 517–534.
49.
ReisingD.SandersonP. (2002). Ecological interface design for Pasteurizer II: A process description of semantic mapping. Human Factors, 44, 222–247.
50.
RoscoeS. (1968). Airborne displays for flight and navigation. Human Factors, 10, 321–332.
51.
ScaifeM.RogersY. (1996). External cognition: How do graphical representations work? International Journal of Human-Computer Studies, 45, 185–213.
52.
ShiffrinR.SchneiderW. (1977). Controlled and automatic information processing: II. Perceptual learning, automatic attending, and a general theory. Psychological Review, 84, 127–190.
53.
SilverM. (2005). Exploring interface design. Clifton Park, NY: Delmar Learning.
54.
SimonH.HayesJ. (1976). The understanding process: Problem isomorphs. Cognitive Psychology, 8, 165–190.
55.
SkirkaL.MosierK.BurdickM. (1999). Does automation bias decision making? International Journal of Human-Computer Systems, 51, 991–1006.
56.
SmithP. J.GeddesN.BeattyR. (2006). Human-centered design of decision support systems. In SearsA.JackoJ. (Eds.), Handbook of human-computer interaction (2nd ed., pp. 656–675). Mahwah, NJ: Erlbaum.
57.
SmithP. J.McCoyE.LaytonC. (1997). Brittleness in the design of cooperative problem-solving systems: The effects on user performance. IEEE Transactions on Systems, Man and Cybernetics, 27, 360–370.
58.
SmithP. J.StoneR. B.SpencerA. (2006). Design as a prediction task: Applying cognitive psychology to system development. In MarrasW. S.KarwowskiW. (Eds.), Handbook of industrial ergonomics (2nd ed., pp. 24. 1–24.18). New York: Marcel Dekker.
59.
SpoolJ.ScanlonT.SnyderC.DeAngeloT.SchroederW. (1998). Web site usability: A designer's guide. San Francisco: Morgan Kaufmann.
60.
SternbergR. J.DavidsonJ. (Eds.) (1995). The nature of insight. Cambridge: Bradford Books/MIT Press.
61.
TufteE. (1983). The visual display of quantitative information. Cheshire, CT: Graphics Press.
TufteE. R. (1997). Visual explanations. Cheshire, CT: Graphics Press.
64.
Van Der VeerG.MelguizoM. (2003). Mental models. In SearsA.JackoJ. (Eds.), Handbook of human-computer interaction (pp. 52–80). Mahwah, NJ: Erlbaum.
65.
VicenteK. (1991). Supporting knowledge-based behavior through ecological interface design (Tech. Report EPRL-91–1). Urbana-Champaign, IL: Engineering Psychology Research Laboratory and Aviation Research Laboratory, University of Illinois.
66.
VicenteK. (1999). Cognitive work analysis: Toward safe, productive, and healthy computer-based work. Mahwah, NJ: Erlbaum.
67.
VicenteK.RasmussenJ. (1990). The ecology of human-machine systems II: Mediating “direct perception” in complex work domains. Ecological Psychology, 2, 207–249.
68.
VicenteK.RasmussenJ. (1992). Ecological interface design: Theoretical foundations. IEEE Transactions on Systems, Man and Cybernetics, 22, 589–606.
69.
VicenteK. J. (2000). Work domain analysis and task analysis: A difference that matters. In SchraagenJ.ChipmanS.ShalinV. (Eds.), Cognitive task analysis (pp. 101–118). Mahwah, NJ: Erlbaum.
70.
VicenteK. J. (2002). Ecological interface design: Progress and challenges. Human Factors, 44, 62–78.
71.
WickensC.AndreA. (1990). Proximity compatibility and information display: Effects of color, space, and objectness on information integration. Proceedings of the Human Factors Society 32nd Annual Meeting (pp. 1335–1339). Santa Monica, CA: SAGE Publications.
72.
WickensC. D.LeeJ.LiuY.Gordon-BeckerS. (2004). An introduction to human factors engineering (2nd ed.). Upper Saddle River, NJ: Pearson/Prentice Hall.
73.
WoodsD. D. (1991). The cognitive engineering of problem representations. In WeirG. R. S.AltyJ. L. (Eds.), Human-computer interaction and complex systems (pp. 169–188). London: Academic Press.
74.
WoodsD. D.RothE. M. (1988). Cognitive systems engineering. In HelanderM. (Ed.), Handbook of human-computer interaction (pp. 1–41). Amsterdam: Elsevier.
75.
ZacharyW. (1986). A cognitively based functional taxonomy of decision support techniques. Human-Computer Interaction, 2, 25–63.
76.
ZhangJ. (1997). The nature of external representations in problem solving. Cognitive Science, 21, 179–217.
