Image database (IDB) systems are at present often designed to test technology and the efficacy of retrieval algorithms, rather than being oriented towards delivering functionality to users. Research is necessary to design interfaces geared towards human usage of images. The starting point of this research needs to be consideration at a fundamental, user-centred level of how people perceive and interpret images. This article considers literature from many disciplines to describe a taxonomy of image content, from direct sensory elements to high-level abstractions. The nine categories derived will later be validated and used to direct the design of visual query interfaces for IDB systems.
Get full access to this article
View all access options for this article.
References
1.
Bar, M. (2000) `Conscious and Nonconscious Processing of Visual Object Identity', in Y. Rossetti and A. Revonsuo (eds) Beyond Dissociation: Interaction between Dissociated Implicit and Explicit Processing. Advances in Consciousness Research. Amsterdam: John Benjamins .
2.
Berger, J. (1972) Ways of Seeing. Harmondsworth: Penguin .
3.
Biederman, I. (1987) `Recognition by Components: A Theory of Human Image Understanding' , Psychological Review94: 115-147 .
4.
Biederman, I. and Cooper, E.E. (1991) `Priming Contour-Deleted Images - Evidence for Intermediate Representations in Visual Object Recognition' , Cognitive Psychology23(3): 393-419 .
5.
Biederman, I. and Gerhardstein, P.C. (1993) `Recognising Depth-Rotated Objects: Evidence and Conditions for Three-Dimensional Viewpoint Invariance' , Journal of Experimental Psychology: Human Perception and Performance19(6): 1162-1182 .
6.
Boynton, D.M. (1993) `Relativism in Gibson's Theory of Picture Perception' , Journal of Mind and Behavior14(1): 51-70 .
7.
Bremner, J.G. and Moore, S. (1984) `Prior Visual Inspection and Object Naming - 2 Factors that Enhance Hidden Feature Inclusion in Young Children's Drawings' , British Journal of Developmental Psychology2: 371-376 .
8.
Brown, J.M. and Koch, C. (1993) `Influences of Closure, Occlusion, and Size on the Perception of Fragmented Pictures' , Perception & Psychophysics53(4): 436-442 .
9.
Bruce, V., Green, P.R. and Georgeson, M.A. (1996) Visual Perception: Physiology, Psychology & Ecology, 3rd edn. Hove: Psychology Press .
10.
Burt, P. and Adelson, E.H. (1983) `The Laplacian Pyramid as a Compact Image Code' , IEEE Transactions on CommunicationCOM-31: 532-540 .
11.
Cox, M.V. (1986) `Cubes Are Difficult Things to Draw' , British Journal of Developmental Psychology4: 341-345 .
12.
D'Arcais, G.B.F. and Schreuder, R. (1987) `Semantic Activation during Object Naming' , Psychological Research-Psychologische Forschung49(2-3): 153-159 .
13.
Davidoff, J. and De Bleser, R. (1994) `Impaired Picture-Recognition with Preserved Object Naming and Reading' , Brain and Cognition24(1): 1-23 .
14.
Davidoff, J.B. and Ostergaard, A.L. (1988) `The Role of Color in Categorical Judgements' , Quarterly Journal of Experimental Psychology Section A- Human Experimental Psychology40(3): 533-544 .
15.
Dixon, M.J., Piskopos, M. and Schweizer, T.A. (2000) `Musical Instrument Naming Impairments: The Crucial Exception to the Living/Nonliving Dichotomy in Category-Specific Agnosia' , Brain and Cognition43(1-3): 158-164 .
16.
Dyson, M.C. and Box, H. (1997) `Retrieving Symbols from a Database by their Graphic Characteristics: Are Users Consistent?' , Journal of Visual Languages and Computing8(1): 85-107 .
17.
Eakins, J.P. (1998) Techniques for Image Retrieval (Library and Information Briefings 85, October). London: British Library and South Bank University .
18.
Eakins, J.P., Edwards, J.D., Riley, J. and Rosin, P.L. (2001) `A Comparison of the Effectiveness of Alternative Feature Sets in Shape Retrieval of Multi-component Images' , in M.M. Yeung, C.-S. Li and R.W. Lienhart (eds) Storage and Retrieval for Media Databases 2001 , Proceedings of Society of Photo-optical Instrumentation Engineers (SPIE) 4315: 196-207 .
19.
Enser, P.G.B. (1995) `Pictorial Information Retrieval' , Journal of Documentation51: 126-170 .
20.
Fidel, R. (1997) `The Image Retrieval Task: Implications for the Design and Evaluation of Image Databases' , New Review of Hypermedia and Multimedia3: 181-199 .
21.
Firschein, O. and Fischler, M.A. (1971) `Describing and Abstracting Pictorial Structure' , Pattern Recognition3: 421-443 .
22.
Firschein, O. and Fischler, M.A. (1972) `A Study in Descriptive Representation of Pictorial Data' , Pattern Recognition4: 361-377 .
23.
Frazier, L. and Hoyer, W.J. (1992) `Object Recognition by Component Features - Are There Age Differences?' , Experimental Aging Research18(1-2): 9-14 .
24.
Furmanski, C.S. and Engel, S.A. (2000) `Perceptual Learning in Object Recognition: Object Specificity and Size Invariance' , Vision Research40(5): 473-484 .
25.
Garnier, F. (1984) Thesaurus iconographique: Systeme descriptif des representations. Paris: Leopard d'or .
26.
Gentner, D. (1978) `On Relational Meaning: The Acquisition of Verb Meaning' , Child Development49(4): 988-998 .
27.
Gibson, J.J. (1950) The Perception of the Visual World. Boston, MA: Houghton Mifflin .
28.
Gibson, J.J. (1979) The Ecological Approach to Visual Perception. Boston, MA: Houghton Mifflin .
29.
Goodale, M.A. and Humphrey, G.K. (1998) `The Objects of Action and Perception' , Cognition67: 181-207 .
30.
Goodman, N. (1984) Of Mind and Other Matters. Cambridge, MA: Harvard University Press .
31.
Gracia, J.J.E. (1994) `Can There Be Text without Audiences? The Identity and Function of Audiences' , Review of Metaphysics47: 711-734 .
32.
Gross, M.D. and Do, E. Yi-Leun (1995) `Diagram Query and Image Retrieval in Design' , in International Conference on Image Processing Proceedings, Vol. 0-8186-7310-9/95, pp. 308-311 . New York: Institute of Electrical and Electronics Engineers (IEEE).
33.
Grossberg, S., Mingolla, E. and Ross, W.D. (1997) `Visual Brain and Visual Perception: How Does the Cortex Do Perceptual Grouping?' , Neuroscience20: 106-111 .
34.
Grund, A. (1993) `ICONCLASS: On Subject Analysis of Iconographic Representations of Works of Art' , Knowledge Organization20(1): 20-29 .
35.
Hart, J. (1993) `Erwin Panofksy and Karl Mannheim: A Dialogue on Interpretation' , Critical Inquiry19: 534-566 .
36.
Hastings, S.K. (1995) `Query Categories in a Study of Intellectual Access to Digitized Art Images' , ASIS Annual Meeting Proceedings, 9-12 October, pp. 3-8 . Silver Springs, MD: American Society for Information Science (ASIS).
37.
Heehs, P. (1995) `Narrative Painting and Narrative about Paintings: Poussin among the Philosophers' , Narrative3(3): 211-231 .
38.
Heitger, F., Rosenthaler, L., Vonderheydt, R., Peterhans, E. and Kubler, O. (1992) `Simulation of Neural Contour Mechanisms - from Simple to End-Stopped Cells' , Vision Research32(5): 963-981 .
39.
Heller, M.A. (1989) `Picture and Pattern Perception in the Sighted and the Blind - the Advantage of the Late Blind' , Perception18(3): 379-389 .
40.
Hoffmann, J. and Kaempf, U. (1985) `Mechanismen der Objektbenennung-parallele Verarbeitungskaskaden/The Mechanisms of Object Naming: Parallel Processing Cascades' , Sprache und Kognition4(4): 217-230 .
41.
Hummel, J.E. (2000) `Where View-Based Theories Break Down: The Role of Structure in Human Shape Perception', in E. Dietrich and A.B. Markham (eds) Cognitive Dynamics: Conceptual and Representational Change in Humans and Machines. Mahwah, NJ: Erlbaum .
42.
Humphrey, G.K., Goodale, M.A., Jakobson, L.S. and Servos, P. (1994) `The Role of Surface Information in Object Recognition: Studies of a Visual Form Agnosic and Normal Subjects' , Perception23(12): 1457-1481 .
43.
Humphrey, G.K., James, T.W., Gati, J.S., Menon, R.S. and Goodale, M. (1999) `Perception of the McCollough Effect Correlates with Activity in Extrastriate Cortex: A Functional Magnetic Resonance Imaging Study' , Psychological Science10(5): 444-448 .
44.
Ingwersen, P. (1982) `Search Procedures in the Library Analysed from the Cognitive Point of View' , Journal of Documentation38: 165-191 .
45.
Ingwersen, P. (1996) `Cognitive Perspectives of Information Retrieval Interaction: Elements of a Cognitive IR Theory' , Journal of Documentation52(1): 3-50 .
46.
Iser, W. (1978) The Act of Reading. Baltimore, MD: Johns Hopkins University Press .
47.
Jauss, H.R. (1982) Toward an Aesthetic of Reception. Minneapolis: University of Minnesota Press .
48.
Jorgensen, C. (1995) `Image Attributes: An Investigation', PhD thesis, Syracuse University, New York.
49.
Jorgensen, C. (1999) `Access to Pictorial Material: A Review of Current Research and Future Prospects' , Computers and the Humanities33: 293-318
50.
Keister, L.H. (1994) `User Types and Queries: Impact on Image Access Systems', in R. Fidel, T. Bellardo Hahn, E.M. Rasmussen and P.J. Smith (eds) Challenges in Indexing Electronic Text and Images, pp. 7-22. Medford, NJ: Learned Information, Inc.
51.
Kimia, B.B., Tannenbaum, A.R. and Zucker, S.W. (1992) `The Shape Triangle: Parts, Protrusions, and Bends', Technical Report TR-92-15, McGill University Research Center for Intelligent Machines.
52.
Koch, C. and Abbey, L. (1999) `Role of Geons in Object Recognition across Ages' , Perceptual and Motor Skills88(3): 983-991 .
53.
Koffka, K. (1935) Principles of Gestalt Psychology. New York: Harcourt Brace .
54.
Landau, B. and Jackendoff, R. (1993) “`What” and “Where” in Spatial Language and Spatial Cognition' , Behavioral and Brain Sciences16(2): 217-265 .
55.
Landau, B. and Leyton, M. (1999) `Perception, Object Kind, and Object Naming' , Spatial Cognition and Computation1(1): 1-29 .
56.
Landau, B. and Shipley, E. (2001) `Labelling Patterns and Object Naming' , Developmental Science4(1): 109-118 .
57.
Landau, B., Smith, L. and Jones, S. (1998a) `Object Perception and Object Naming in Early Development' , Trends in Cognitive Sciences2(1): 19-24 .
58.
Landau, B., Smith, L. and Jones, S. (1998b) `Object Shape, Object Function, and Object Name' , Journal of Memory and Language38(1): 1-27 .
Larsgaard, M.L. and Carver, L. (1995) `Accessing Spatial Data Online - Project Alexandria' , Information Technology and Libraries14(2): 93-97 .
61.
Levelt, W.J.M., Roelofs, A. and Meyer, A.S. (1999) `A Theory of Lexical Access in Language Production' , Behavioral and Brain Sciences22: 1-38 .
62.
Liter, J.C. (1998) `The Contribution of Qualitative and Quantitative Shape Features to Object Recognition across Change of View' , Memory & Cognition26(5): 1056-1067 .
63.
Malinas, G. (1991) `A Semantics for Pictures' , Canadian Journal of Philosophy21(3): 275-298 .
64.
Mannheim, K. (1952[1923]) Essays on the Sociology of Knowledge. London: Routledge & Kegan Paul .
65.
Mannheim, K. (1997) Essays on the Sociology of Culture. London: Routledge & Kegan Paul .
66.
Marr, D. (1982) Vision. San Francisco: WH Freeman .
67.
Marr, D. and Nishihara, H.K. (1978) `Representation and Recognition of the Spatial Organisation of Three-Dimensional Shapes' , Proceedings of the Royal Society of London Bulletin200: 269-294 .
68.
Merriman W.E., Scott, P.D. and Marazita, J. (1993) `An Appearance Function Shift in Children's Object Naming' , Journal of Child Language20(1): 101-118 .
69.
Meyer, A.S., Sleiderink, A.M. and Levelt, W.J.M. (1998) `Viewing and Naming Objects: Eye Movements during Noun Phrase Production' , Cognition66(2): B25-B33 .
70.
Meyer, A.S. and Van der Meulen, F.F. (2000) `Phonological Priming Effects on Speech Onset Latencies and Viewing Times in Object Naming' , Psychonomic Bulletin and Review7(2): 314-319 .
71.
Mitchell, P., Davidoff, J. and Brown, C. (1996) `Young Children's Ability to Process Object Colour: Coloured Pictogens and Verbal Mediation' , British Journal of Developmental Psychology14(3): 339-354 .
72.
Moore, C. and Cavanagh, P. (1998) `Recovery of 3D Volume from 2-Tone Images of Novel Objects' , Cognition67: 45-71 .
73.
Natsoulas, T. (1999) `Virtual Objects' , Journal of Mind and Behavior20(4): 357-378 .
74.
Neisser, U. (1967) Cognitive Psychology. New York: Appleton-Century-Crofts .
75.
Ornager, S. (1997) `Image Retrieval: Theoretical Analysis and Empirical User Studies on Accessing Information in Images' , ASIS `97: Proceedings of the 60th ASIS Annual Meeting, Vol. 34. Silver Springs, MD: American Society for Information Science (ASIS).
76.
Ostergaard, A.L. and Davidoff, J.B. (1985) `Some Effects of Color on Naming and Recognition of Objects' , Journal of Experimental Psychology: Learning, Memory, and Cognition11(3): 579-587 .
77.
Panofsky, E. (1970[1955]) Meaning in the Visual Arts. London: Penguin .
78.
Persson, P., Höök, K. and Simsarian, K. (2000) `Human-Computer Interaction versus Reception Studies: Objectives, Methods and Ontologies' , paper presented at NorFA Research Seminar on `Reception: Film, TV, Digital Culture', Stockholm University, 4-7 June. http://www.sics.se/~perp/AComparisonofHuman.doc 4th March 2002
79.
Price, C.J. and Humphreys, G.W. (1989) `The Effects of Surface Detail on Object Categorization and Naming' , Quarterly Journal of Experimental Psychology: Human Experimental Psychology41(4-A): 797-827 .
80.
Rasmussen, E. (1997) `Indexing Images' , Annual Review of Information Science and Technology32: 169-196 .
81.
Rollins, M. (1999) `Pictorial Representation: When Cognitive Science Meets Aesthetics' , Philosophical Psychology12(4): 387-413 .
82.
Rosch, E. and Lloyd, B.B. (eds) (1978) Cognition and Categorization. London: Erlbaum .
83.
Rosch, E., Mervis, C., Gray, W., Johnson, D. and Boyes-Braem, P. (1976) `Basic Objects in Natural Categories' , Cognitive Psychology8(3): 382-439 .
84.
Santini, S. and Jain, R. (1996) `The Graphical Specification of Similarity Queries' , Journal of Visual Languages and Computing7: 403-421 .
85.
Sharrock, W. and Coulter, J. (1998) `On J.J. Gibson: A Response to Our Commentators' , Theory & Psychology8(2): 177-181 .
86.
Sharrock, W. and Coulter, J. (1999) `Elucidation vs. Pseudo-Explanation in Psychology: A Response to Professor Wetherick' , Theory & Psychology9(4): 557-563 .
87.
Shatford Layne, S. (1994) `Some Issues in the Indexing of Images' , Journal of the American Society for Information Science45(8): 583-588 .
88.
Siddiqi, K., Kimia, B.B., Tannenbaum, A. and Zucker, S.W. (2001) `On the Psychophysics of the Shape Triangle' , Vision Research41(9): 1153-1178 .
89.
Siddiqi, K., Tresness, K. and Kimia, B.B. (1996) `Parts of Visual Form: Psychophysical Aspects' , Perception25: 399-424 .
90.
Slater, A. (1998) `Visual Organization and Perceptual Constancies in Early Infancy', in V. Walsh and J. Kulikowski (eds) Perceptual Constancy: Why Things Look as They Do. Cambridge: Cambridge University Press .
91.
Smith, B. (2000) `Truth and the Visual Field', in J. Petitot, F.J. Varela, B. Pachoud and J-M. Roy (eds) Naturalizing Phenomenology: Issues in Contemporary Phenomenology and Cognitive Science. Stanford, CA: Stanford University Press .
92.
Tagare, H.D., Jaffe, C.C. and Duncan, J. (1997) `Medical Image Databases: A Content Based Retrieval Approach' , Journal of the American Medical Informatics Association4(3): 184-198 .
93.
Takano, Y. (1989) `Perception of Rotated Forms - A Theory of Information Types' , Cognitive Psychology21(1): 1-59 .
94.
Tanaka, J.W. and Taylor, M. (1991) `Object Categories and Expertise: Is the Basic Level in the Eye of the Beholder?' , Cognitive Psychology23(3): 457-482 .
95.
Tarr, M.J. and Bölthoff, H.H. (1998) `Image-Based Object Recognition in Man, Monkey and Machine' , Cognition67: 1-20 .
96.
Taylor, J.G. (1962) The Behavioural Basis of Perception. London: Yale University Press .
97.
Wetherick, N.E. (1999) `On What Gibson (and the Rest of Us) Cannot Do Without: Comment on Sharrock and Coulter (1998)' , Theory & Psychology9(4): 551-556 .
98.
Williams, L.R. and Hanson, A.R. (1996) `Perceptual Completion of Occluded Surfaces' , Computer Vision and Image Understanding64(1): 1-20 .
