Lightness Discrimination Depends More on Bright Rather Than Shaded Regions of Three-Dimensional Objects

Abstract

The brighter portions of a shaded complex object are in principle more informative about its lightness and are preferentially fixated during lightness judgments. In this study, we investigate whether preventing this strategy also has measurable detrimental effects on performance. Observers were presented with a reference and a comparison three-dimensional rendered object and had to choose which one was “painted with a lighter gray.” The comparison was rendered with different diffuse reflectance values. We compared precision between three different conditions: full image, 20% of the lightest pixels removed, or 20% of the darkest pixels removed. Removing the bright pixels maximally impaired performance. The results confirm that the strategy of relying on the brightest areas of a complex object in order to estimate lightness is functionally optimal, yielding more precise representations.

Introduction

The amount of light coming to the eye from a matte object depends on the intensity of the illumination and on the proportion of light that is reflected (i.e., the surface albedo). Albedo is a specific property of the surface’s material and is thus of great importance for material perception. Lightness is defined as the visual counterpart of albedo (Adelson, 2000).

To perceive lightness, the visual system is faced with the problem that it is impossible to distinguish between illumination and reflectance solely based on the light originating from one point of the surface. Different factors contribute to solve this ambiguity. Lateral inhibition between retinal neurons filters out shallow intensity gradients, which are prevalently due to illumination rather than reflectance differences within a surface (Cornsweet, 1970; Land & McCann, 1971; Mach, 1866). Lightness perception also depends on higher order inferences on the scene. The geometrical interpretation of the shape of a surface in its context dramatically changes its lightness (Gilchrist, 1977; Knill & Kersten, 1991; Purves, Shimpi, & Lotto, 1999). The visual system is also able to parse the scene in depth layers and account for the effects of semitransparent media on the reflected light (Anderson, 1997, 2003; Anderson & Winawer, 2005).

As a general theoretical framework, it was proposed that the visual system recovers albedo by explicitly estimating and discounting the contributions of illumination and geometry to luminance (e.g., Marr, 1982; Pizlo, 2001; Poggio & Koch, 1985; Poggio, Torre, & Koch, 1987). This is computationally complex and poses a “chicken and egg” problem, because in order to estimate and discount illumination and geometry, the visual system would need to estimate the surface’s albedo (Fleming, 2014). Alternatively, the system could use simple heuristics to bypass this problem and estimate surface albedo directly from image statistics (see for review: Fleming, 2014; Thompson, Fleming, Creem-Regehr, & Stefanucci, 2011). For instance, it can exploit that albedo is negatively correlated to the skewness of the histogram of the surface luminance distribution and positively correlated to its standard deviation and to its 90th percentile (Motoyoshi, Nishida, Sharan, & Adelson, 2007; Sharan, Li, Motoyoshi, Nishida, & Adelson, 2008).

Our previous research (Toscani, Valsecchi, & Gegenfurtner, 2013a, 2017) indicated that the brightest portions of the luminance histogram are diagnostic for surface albedo. We showed this using a physically based rendering system (Heasly, Cottaris, Lichtman, Xiao, & Brainard, 2014; Ward, 1994) to simulate reflections in a large number of three-dimensional realistic shapes. This approach allows to generate a large dataset of object images to statistically relate reflected light to surface properties (e.g., Singh, Cottaris, Heasly, Brainard, & Burge, 2018; Wiebel, Toscani, & Gegenfurtner, 2015). Specifically, we rendered images of different objects, placed in different complex realistic light fields (Debevec, 1998). The images were rendered with different albedos under different positions and viewing angles. By means of receiver operating characteristic (ROC) analysis (Toscani et al., 2013a) and linear classifiers (Toscani, Valsecchi, et al., 2017), we demonstrated that the highest percentiles of the luminance distributions of matte objects are particularly diagnostic for their albedo. This property is exploited by human observers, who tend to base their lightness judgments on the surfaces’ brightest portions (Toscani et al., 2013a, 2015; Toscani, Valsecchi, & Gegenfurtner, 2013b; Toscani, Valsecchi, et al., 2017; Toscani, Zdravković, & Gegenfurtner, 2016).

When we selectively manipulated different bands of the luminance histograms of three-dimensional simulated objects while keeping their mean luminance constant, we observed that for matte surfaces, increasing or decreasing luminance of the brightest band maximally affected lightness appearance (Toscani, Valsecchi, et al., 2017).

The strategy of using the most diagnostic luminance values as a proxy for lightness appearance would help the visual system to create a perceptual representation which maximally tells surfaces of a different albedo apart, and that is relatively immune to changes in illumination and geometry. In fact, in a complex three-dimensional object, the maximum luminance would be expressed where the local surface is oriented perpendicular to the light source, the rest would be affected by shading and thus depend on the local orientation. This strategy constitutes a possible heuristic to achieve a stable lightness appearance, without requiring knowledge of the scene geometry, shape, or illumination.

While we know that this strategy influences lightness appearance, the direct prediction of our simulations is that lightness discrimination judgments are mostly dependent on the brightest percentiles and was never tested on human perception. In fact, the simulations are based on luminance, but perceived brightness is known to be nonlinearly related to luminance, and, whether this relationship is logarithmic (Fechner, 1860) or roughly a cube-root curve (Stevens, 1957), this implies that differences within the low luminance range might be amplified compared to those in the high range. Hence, at the perceptual level, the dark portion of an object’s luminance distribution might impact discrimination more than the light one, despite being in principle less informative.

Here, we test the relative importance of dark and light portions of the luminance distribution of shaded complex objects in a lightness discrimination task. Removing the lightest portions impaired discrimination more than removing the dark ones.

Methods

Participants

Five naive observers took part in the experiment. They all had normal or corrected visual acuity. All gave written informed consent in accordance with the Code of Ethics of the World Medical Association (Declaration of Helsinki) for experiments involving humans. The experiments were approved by the local ethics committee (approval number LEK 2009-0008).

Apparatus

We used the psychtoolbox-3 software (Kleiner, Brainad, & Pelli, 2007) working on MATLAB (http://www.mathworks.com), to display the rendered movies on Samsung SyncMaster 1100 MB monitor. The procedure for monitor linearization has been described elsewhere (e.g., Ennis, Toscani, & Gegenfurtner, 2017; Milojevic, Ennis, Toscani, & Gegenfurtner, 2018).

Stimuli

We used two three-dimensional models (Figure 1) to render realistic matte objects under different views and orientations. The objects were positioned with random orientation in a complex light field (Debevec, 1998); the viewing position was randomly sampled from a spherical perimeter surrounding the objects (see for details: Toscani et al., 2013a). Reflections were simulated with the physically based rendering system RADIANCE (Ward, 1994) interfaced with a MATLAB toolbox (Heasly et al., 1994). The background was masked by a black and white grid pattern, to standardize local contrasts. Each shape was rendered with the following diffuse reflectance parameters (0.29, 0.35, 0.38, 0.41, 0.44, 0.47, 0.5, 0.53, 0.56, 0.5916, 0.65) and 30 different random orientation and viewing conditions. The central reflectance value (i.e., 0.47) was chosen as reference and the other ones were comparisons, in a constant stimuli paradigm.

Figure 1.

Stimuli. (a) Reference and comparison stimulus, one on the left and the right side of the display. The horse shape is shown here; the other one was the three-dimensional model of a whale; see example in (b). (b) Experimental conditions. In the Full condition, the whole shape was presented. Here, for illustration purposes, the 20% lightest and darkest pixels are highlighted in red and blue, respectively. In the Dark Cut condition, the 20% dark pixels were substituted with the background pattern; the 20% lightest in the Light Cut condition.

Procedure

In a dark room, participants had their heads stabilized by a chinrest with 58 cm distance between forehead and the center of the screen. They were presented with the reference and comparison stimuli next to each other, randomly left and right or vice versa (Figure 1(a)). The task was to indicate which one was “painted with a lighter gray”. In the Full condition, the comparison and the reference were presented as they were rendered, with no manipulation. In the Dark Cut condition, the 20% darkest pixels (Figure 1(b), blue area -- left panel) were substituted with the background (Figure 1(b) -- central panel); in the Light Cut condition, the 20% lightest pixels (Figure 1(b), red area -- left panel) were substituted (Figure 1(b) -- right panel). For each of the three conditions and each of the two shapes, each reflectance comparison was shown 30 times, at random orientations and viewing angles, for a total of 1980 experimental trials. Presentation order was randomized.

Analyses

For each condition and each participant, we modeled the probability of the comparison to be reported as lighter than the reference as a function of the reflectance difference. To do that, we used the psignifit 4 (Schütt, Harmeling, Macke, & Wichmann, 2016) MATLAB toolbox to fit a psychometric function to the observers’ responses (with a cumulative Gaussian function). To stabilize the fitting procedure, data were binned into five intervals. The slope of the psychometric function is a measure of JND (just noticeable difference), that is, the albedo difference at which the participants perform discrimination with an 84% success rate. Steeper curves correspond to better ability to tell apart objects with different albedos. We used a one-way repeated measure analysis of variance (ANOVA) on the JNDs to test for overall differences between the three conditions (Full, Light Cut, and Dark Cut) and Bonferroni-corrected t tests to assess difference between all condition pairs.

Results

Figure 2(a) shows an example of the psychometric functions for the three conditions of a participant. The slope is steepest in the Full condition (dashed gray line), indicating that both replacing the dark and the light portions impaired performance. Crucially, the slope is shallowest in the Light Cut condition, suggesting that replacing the light portions of the objects impairs performance the most.

Figure 2.

Results. (a) Example of psychometric functions for one observer for the three conditions. y-axis denotes the probability of judging the comparisons as lighter than the reference; x-axis denotes albedo difference between the comparison and the reference. Albedo is expressed in percentage from black (0%) to white (100%). Black circles indicate the measured probabilities in the Dark Cut condition, gray in the Full and light gray in the Light Cut condition. The black solid line represents the fitted psychometric function for the Dark Cut condition, the solid light gray line for the Light Cut condition, and the dashed gray line for the Full condition. (b) JND averaged across observers (y-axis), for the three conditions (x-axis). Error bars represent the standard error of the mean. JND = just noticeable difference. (*p < 0.017).

Figure 2(b) represents the average JND in the three conditions. ANOVA reveals a significant difference between conditions, F(2,14) = 27.312, p < .001.

JNDs are lowest in the Full condition, indicating better performed when all the information was available. Both removing the light and dark portions impaired performance—t(4) = 4.21 and t(4) = 6.12, respectively; p values < α, with α = .017 because of the Bonferroni correction. However, the negative effect on performance was bigger in the Light Cut than in the Dark Cut condition, t(4) = 4.25, p < .017, indicating the light portions have an higher influence on performance than the dark portions.

Discussion

We tested how lightness discrimination performance depends on the light and the dark portions of shaded three-dimensional objects. Our results indicate that the light portions are particularly important for lightness discrimination. This is in agreement with our previous results from physically based rendering simulations (Toscani et al., 2013a, 2015; Toscani, Valsecchi, et al., 2017), which could predict perception without incorporating the nonlinear relationship between luminance and perceived brightness (Fechner, 1860; Stevens, 1957). Also cutting the darkest portions impaired performance, suggesting that, although discrimination mostly depends on the lightest portions, the whole distribution is taken into account.

Discrimination and appearance are different perceptual processes characterized by specific but also shared common mechanisms. For instance, color appearance and discrimination are both influenced by a slow color adaptation process, but extremely rapid adaptation exclusively influences appearance (Rinner & Gegenfurtner, 2000).

We argue that lightness appearance of shaded objects is built by selecting the brightest areas of an object’s surface, which are the most informative for lightness discrimination. This strategy supports the discrimination of objects with different albedos and at the same time stabilizes lightness perception when viewing conditions change. In fact, if an object exhibits enough geometrical complexity, some portion of it is surface will be under direct illumination, thus reflecting the largest amount of light possible, given the illuminant and the surface reflectance. This portion would be the brightest one, while other portions would be in shadow.

Using the maximum luminance within the boundaries of an object to estimate lightness is an adaptive strategy because the maximum luminance value is maximally diagnostic, based on the fact that, given a complex enough geometry, one portion of the surface will always be maximally illuminated. This offers an interesting parallel with the problem of estimating the relative lightness of objects within a scene. Again, assuming that there is a complex enough assortment of lightness values within the scene, the highest brightness will correspond to a white or close to white object. Indeed, among the strategies that the visual system uses achieve lightness constancy, one is to assume that the highest luminance in an image is assumed to be white, “anchoring” the perceived albedos of the other surfaces. The lightness of each surface is proportional to its luminance, but dependent on the other surfaces in the scene, being the scale of variability normalized to a fixed range (Gilchrist et al., 1999). These phenomena could be the results of Bayesian integration of prior knowledge on illumination and reflectances in the word with the current sensory evidence (Brainard & Freeman, 1997; Murray, 2013). In general, the visual system seems to estimate the illuminant from the brightest light in the scene (Gilchrist & Soranzo, 2019; Kozaki, 1973; Noguchi & Masuda, 1971), consistent with the “anchoring” rule, since a white surface reflects almost all the light that received from the illumination.

Nevertheless, constancy is far from perfect: When observers are asked to judge lightness, their behavior is often—at last in part—driven by perceived luminance (i.e., brightness; Ripamonti et al., 2004; Robilotto & Zaidi, 2004, 2006; Toscani et al., 2016; Zdravković, 2008). When an object moves through an environment in which the illumination changes across space, its lightness dramatically depends on the intensity of the illumination (Toscani et al., 2016; Zdravković, 2008). Furthermore, some participants systematically base their lightness judgments on albedo, others on brightness (Robilotto & Zaidi, 2004). Thus, lightness and brightness are not trivial to dissociate and it is possible that in the present study, instead of discriminating reflectance differences, observers based their judgments on brightness. However, whether they used a simple (e.g., mean perceived luminance), a more complex brightness measure (e.g., Robilotto & Zaidi, 2004), or measure of perceived surface albedo (i.e., lightness), our results show that in a discrimination task this measure depends more on the brightest than on the darkest portions of the surfaces’ luminance distribution.

Exactly what mechanism implements the heuristic of relying more on the brightest part of an object for recovering albedo is not yet clear. One possibility is that observers only consider the lightest portions of objects for their judgments and ignore the rest. Alternatively, they might compute albedo integrating the perceived luminance distribution, after this has been shifted towards higher values through fixation behavior. In fact, there is evidence that luminance at fixation is extrapolated to peripheral portions of the rest of the surface (Toscani, Gegenfurtner, & Valsecchi, 2017). Both possibilities are consistent with the finding that while judging the lightness of real three-dimensional objects, observers tend to look at their brightest portions and when forced to a different fixation pattern, their lightness judgments change depending on the luminance at fixation (Toscani et al., 2013a, 2013b, 2015, 2016).

Extrapolating perceived luminance based on fixation and fixating the most diagnostic areas within an object could contribute to compress differences within an objects surface and enhancing differences between different objects and their parts, ultimately helping to discriminate different objects. This is similar to what happens for color. Chromatic discrimination, as inferred from the shape of the MacAdam ellipses (MacAdam, 1942), is worse along the direction of elongation of the color distribution of colored natural objects (Ennis, Schiller, Toscani, & Gegenfurtner, 2018; Hansen, Giesel, & Gegenfurtner, 2008). This implies, for instance, humans are poorly sensitive to the different shades of yellow within the surface of a lemon, but can tell apart the color of the lemon from the one of an orange.

The brightest areas within the object surface determine the lightness of matte objects (Toscani, Valsecchi, et al., 2017). This is however not the case for glossy objects, presumably because the visual system ignores specular highlights when judging lightness (Toscani, Valsecchi, et al., 2017). Specular highlights tend to be colocated with the areas that are best illuminated (Koenderink & van Doorn, 1980); therefore, they cover the areas that would be brightest if the object was matte. Based on the results of the present study, ignoring specular highlights should impair lightness discrimination similar to removing the brightest regions of a matte object. Indeed, lightness discrimination is more difficult for glossy surfaces (Toscani, Valsecchi, et al., 2017).

To summarize, object regions which are relied upon for lightness appearance are also more important for lightness discrimination. We argue that the strategy of basing appearance on the most discriminable regions helps to build useful perceptual representations for object recognition, coherent with the idea that the main purpose of lightness and color perception is to recover the properties of objects and materials in our environment, rather than precisely representing the light that reaches our eyes (Witzel & Gegenfurtner, 2018).