An Empirical Comparison of Alternative Methodologies for the Evaluation of Configural Displays

Abstract

Two different methodologies (visual, memory) were used to evaluate alternative versions of the same configural display. One version (composite display) had several graphical design techniques applied, whereas the other version (baseline display) did not. Two types of information probes (high-level property, low-level data) were administered. When the displays were visible during completion of the probes (visual methodology), the display manipulation had the largest impact on performance (composite display associated with better performance); when the displays were not visible (memory methodology) the probe manipulation had the largest impact on performance (high-level probes associated with better performance). These results are interpreted in light of the mutually interacting constraints introduced by factors in display design, task requirements, and the participants' cognitive and perceptual capabilities/limitations. Implications for both the design and the evaluation of displays and interfaces in general are discussed. Actual or potential applications of this research include design techniques for improving the quality of graphic displays and methodological insights for interpreting previous research and guiding future experimentation.

Get full access to this article

View all access options for this article.

References

Barnett, V. , & Lewis, T. (1984). Outliers in statistical data Chichester, NY: Wiley.

Barnett, B. J. , & Wickens, C. D. (1988). Display proximity in multicue information integration: The benefits of boxes. Human Factors, 30, 15–24.

Bartlett, F. C. (1932). Remembering: A study in experimental and social psychology Cambridge, UK: Cambridge University Press.

Bennett, K. B. , & Flach, J. M. (1992). Graphical displays: Implications for divided attention, focused attention, and problem solving. Human Factors, 34, 513–533.

Bennett, K. B. , Nagy, A. L. , & Flach, J. M. (1997). Visual displays. In G. Salvendy (Ed.), Handbook of human factors and ergonomics (2nd ed., pp. 659–696). New York: Wiley.

Bennett, K. B. , Toms, M. L. , & Woods, D. D. (1993). Emergent features and configural elements: Designing more effective configural displays. Human Factors, 35, 71–97.

Bennett, K. B., & Walters, B. (in press). Configural Display Design Techniques Considered at multiple Levels of Evaluation. Human Factors.

Bransford, J. D. , & Franks, J. J. (1972). The abstraction of linguistic ideas: A review. Cognition, 1, 211–250.

Cleveland, W. S. (1985). The elements of graphing data Belmont, CA: Wadsworth.

10.

Cleveland, W. S. , & McGill, R. (1985). Graphical perception and graphical methods for analyzing scientific data. Science, 229, 828–833.

11.

Craik, F. I. M. , & Lockhart, R. S. (1972). Levels of processing: A framework for memory research. Journal of Verbal Learning and Verbal Behavior, 11, 671–684.

12.

Gillan, D. J. , & Lewis, R. (1994). A componential model of human interaction with graphs: I. Linear regression modeling. Human Factors, 36, 419–440.

13.

Gillan, D. J. , & Richman, E. H. (1994). Minimalism and the syntax of graphs. Human Factors, 36, 619–644.

14.

Howard, J. H. , & Kerst, S. M. (1981). Memory and perception of cartographic information for familiar and unfamiliar environments. Human Factors, 23, 495–504.

15.

Keppel, G. , & Underwood, B. J. (1962). Proactive inhibition in short-term retention of single items. Journal of Verbal Learning and Verbal Behavior, 1, 153–161.

16.

Klatzky, R. L. (1980). Human memory: Structures and processes San Francisco: Freeman.

17.

Lovie, P. (1986). Identifying outliers. In A. D. Lovie (Ed.), New developments in statistics for psychology and the social sciences (pp. 44–69. London: British Psychological Society and Methuen.

18.

Meyer, J. , Shinar, D. , & Leiser, D. (1997). Multiple factors that determine performance with tables and graphs. Human Factors, 39, 268–286.

19.

Moray, N. , Jones, B. J. , Rasmussen, J. , Lee, J. D. , Vicente, K. J. , Brock, R. , & Djemil, T. (1993). A performance indicator of the effectiveness of human-machine interfaces for nuclear power plants (Report No. NUREG/CR-5977). Washington, DC: U.S. Nuclear Regulatory Commission.

20.

Moyer, R. S. , Bradley, D. R. , Sorensen, M. H. , Whiting, J. C. , & Mansfield, D. P. (1978). Psychophysical functions for perceived and remembered size. Science, 200, 330–332.

21.

Parkin, A. J. (1984). Levels of processing, context, and facilitation of pronunciation. Acta Psychologica, 55, 19–29.

22.

Rasmussen, J. , Pejtersen, A. M. , & Goodstein, L. P. (1994). Cognitive systems engineering. New York: Wiley.

23.

Ratcliff, R. (1993). Methods for dealing with reaction time outliers. Psychological Bulletin, 114, 510–532.

24.

Scott, B. , & Wickens, C. D. (1983). Spatial and verbal displays in a C3 probablistic information integration task. In Proceedings of the Human Factors Society 27th Annual Meeting (pp. 355–358). Santa Monica, CA: Human Factors and Ergonomics Society.

25.

Sperling, G. (1960). The information available in brief visual presentations. Psychological Monographs, 74 (Whole No. 498).

26.

Tulving, E. (1972). Episodic and semantic memory. In E. Tulving & W. Donaldson (Eds.), Organization of memory (pp. 381–403. New York: Academic.

27.

Vicente, K. J. (1988). Adapting the memory recall paradigm to evaluate interfaces. Acta Psychologica, 69, 249–278.

28.

Vicente, K. J. (1992). Memory recall in a process control system: A measure of expertise and display effectiveness. Memory and Cognition, 20, 356–373.

29.

Wickens, C. D. , & Andre, A. D. (1990). Proximity compatibility and information display: Effects of color, space, and objectness on information integration. Human Factors, 32, 61–78.

30.

Wickens, C. D. , & Carswell, C. M. (1995). The proximity compatibility principle: Its psychological foundation and relevance to display design. Human Factors, 37, 473–494.

31.

Wickens, C. D. , Merwin, D. H. , & Lin, E. L. (1994). Implications of graphics enhancements for the visualization of scientific data: Dimensional integrality, stereopsis, motion, and mesh. Human Factors, 36, 44–61.

32.

Woods, D. D. , Wise, J. A. , & Hanes, L. F. (1981). An evaluation of nuclear power plant safety parameter display systems. In Proceedings of the Human Factors Society 25th Annual Meeting (pp. 110–114). Santa Monica, CA: Human Factors and Ergonomics Society.