Humans can categorize complex natural scenes quickly and accurately. Which scene properties enable people to do this with such apparent ease? We extracted structural properties of contours (orientation, length, curvature) and contour junctions (types and angles) from line drawings of natural scenes. All of these properties contain information about scene categories that can be exploited computationally. However, when we compared error patterns from computational scene categorization with those from a six-alternative forced-choice scene-categorization experiment, we found that only junctions and curvature made significant contributions to human behavior. To further test the critical role of these properties, we perturbed junctions in line drawings by randomly shifting contours and found a significant decrease in human categorization accuracy. We conclude that scene categorization by humans relies on curvature as well as the same nonaccidental junction properties used for object recognition. These properties correspond to the visual features represented in area V2.
AlvarezG. A.OlivaA. (2008). The representation of simple ensemble visual features outside the focus of attention. Psychological Science, 19, 392–398. doi:10.1111/j.1467-9280.2008.02098.x
BalasB.NakanoL.RosenholtzR. (2009). A summary-statistic representation in peripheral vision explains visual crowding. Journal of Vision, 9(12), Article 13. Retrieved from http://www.journalofvision.org/content/9/12/13
4.
BaldassiS.BurrD. C. (2000). Feature-based integration of orientation signals in visual search. Vision Research, 40, 1293–1300.
BauerB. (2009). Does Steven’s power law for brightness extend to perceptual brightness averaging?Psychological Record, 59, 171–186.
7.
BellA. J.SejnowskiT. J. (1997). The “independent components” of natural scenes are edge filters. Vision Research, 37, 3327–3338. doi:S0042-6989(97)00121-1
8.
BiedermanI. (1987). Recognition-by-components: A theory of human image understanding. Psychological Review, 94, 115–147.
9.
BurrD.RossJ. (2008). A visual sense of number. Current Biology, 18, 425–428. doi:10.1016/j.cub.2008.02.052
ChooH.LevinthalB. R.FranconeriS. L. (2012). Average orientation is more accessible through object boundaries than surface features. Journal of Experimental Psychology: Human Perception and Performance, 38, 585–588. doi:10.1037/a0026284
EvansK. K.TreismanA. (2005). Perception of objects in natural scenes: Is it really attention free?Journal of Experimental Psychology: Human Perception and Performance, 31, 1476–1492. doi:10.1037/0096-1523.31.6.1476
14.
Fei-FeiL.IyerA.KochC.PeronaP. (2007). What do we perceive in a glance of a real-world scene?Journal of Vision, 7(1), Article 10. Retrieved from http://www.journalofvision.org/content/7/1/10
15.
Fei-FeiL.PeronaP. (2005, June). A Bayesian hierarchical model for learning natural scene categories. Paper presented at the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, San Diego, CA.
16.
FreemanJ.SimoncelliE. P. (2011). Metamers of the ventral stream. Nature Neuroscience, 14, 1195–1201. doi:10.1038/nn.2889
17.
GoffauxV.JacquesC.MourauxA.OlivaA.SchynsP. G.RossionB. (2005). Diagnostic colours contribute to the early stages of scene categorization: Behavioural and neurophysiological evidence. Visual Cognition, 12, 878–892.
18.
GreeneM. R.OlivaA. (2009). The briefest of glances: The time course of natural scene understanding. Psychological Science, 20, 464–472.
KarklinY.LewickiM. S. (2009). Emergence of complex cell properties by learning to generalize in natural scenes. Nature, 457, 83–86.
21.
KimJ. G.BiedermanI. (2011). Where do objects become scenes?Cerebral Cortex, 21, 1738–1746. doi:10.1093/cercor/bhq240
22.
KoenderinkJ. J.van DoornA. J. (1987). Representation of local geometry in the visual system. Biological Cybernetics, 55, 367–375.
23.
LarsonA. M.LoschkyL. C. (2009). The contributions of central versus peripheral vision to scene gist recognition. Journal of Vision, 9(10), Article 6. Retrieved from http://www.journalofvision.org/content/9/10/6
24.
LiF. F.VanRullenR.KochC.PeronaP. (2002). Rapid natural scene categorization in the near absence of attention. Proceedings of the National Academy of Sciences, USA, 99, 9596–9601.
25.
LoschkyL. C.HansenB. C.SethiA.PydimarriT. N. (2010). The role of higher order image statistics in masking scene gist recognition. Attention, Perception, & Psychophysics, 72, 427–444.
26.
MarrD. (1982). Vision. New York, NY: W. H. Freeman.
27.
MarrD.NishaharaH. K. (1978). Representation and recognition of the spatial organization of three-dimensional shapes. Proceedings of the Royal Society B: Biological Sciences, 200, 269–294.
28.
OlivaA.SchynsP. G. (2000). Diagnostic colors mediate scene recognition. Cognitive Psychology, 41, 176–210. doi:10.1006/cogp.1999.0728
29.
OlivaA.TorralbaA. (2001). Modeling the shape of the scene: A holistic representation of the spatial envelope. International Journal of Computer Vision, 42, 145–175.
30.
OlshausenB. A.FieldD. J. (1996). Emergence of simple-cell receptive field properties by learning a sparse code for natural images. Nature, 381, 607–609.
31.
ParkS.ChunM. M. (2009). Different roles of the parahippocampal place area (PPA) and retrosplenial cortex (RSC) in panoramic scene perception. NeuroImage, 47, 1747–1756. doi:10.1016/j.neuroimage.2009.04.058
32.
PasupathyA.ConnorC. E. (2002). Population coding of shape in area V4. Nature Neuroscience, 5, 1332–1338. doi:10.1038/nn972
33.
PelliD. G.PalomaresM.MajajN. J. (2004). Crowding is unlike ordinary masking: Distinguishing feature integration from detection. Journal of Vision, 4(12), Article 12. Retrieved from http://www.journalofvision.org/content/4/12/12
34.
PeterhansE.von der HeydtR. (1989). Mechanisms of contour perception in monkey visual cortex. II. Contours bridging gaps. Journal of Neuroscience, 9, 1749–1763.
35.
PortillaJ.SimoncelliE. P. (2000). A parametric texture model based on joint statistics of complex wavelet coefficients. International Journal of Computer Vision, 40, 49–71.
36.
PotterM. C.LevyE. I. (1969). Recognition memory for a rapid sequence of pictures. Journal of Experimental Psychology, 81, 10–15.
37.
RenningerL. W.MalikJ. (2004). When is scene identification just texture recognition?Vision Research, 44, 2301–2311. doi:10.1016/j.visres.2004.04.006
38.
RubinN. (2001). The role of junctions in surface completion and contour matching. Perception, 30, 339–366.
TorralbaA.OlivaA. (2003). Statistics of natural image categories. Network, 14, 391–412.
41.
TorralboA.WaltherD. B.ChaiB.CaddiganE.Fei-FeiL.BeckD. M. (2013). Good exemplars of natural scene categories elicit clearer patterns than bad exemplars but not greater BOLD activity. PLoS ONE, 8(3), Article e58594. Retrieved from http://www.plosone.org/article/info:doi/10.1371/journal.pone.0058594
42.
TverskyB.HemenwayK. (1983). Categories of environmental scenes. Cognitive Psychology, 15, 121–149.
43.
WaltherD. B.CaddiganE.Fei-FeiL.BeckD. M. (2009). Natural scene categories revealed in distributed patterns of activity in the human brain. Journal of Neuroscience, 29, 10573–10581. doi:10.1523/JNEUROSCI.0559-09.2009
44.
WaltherD. B.ChaiB.CaddiganE.BeckD. M.Fei-FeiL. (2011). Simple line drawings suffice for functional MRI decoding of natural scene categories. Proceedings of the National Academy of Sciences, USA, 108, 9661–9666. doi:10.1073/pnas.1015666108
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
