Designing for Sensory Adaptation: What You See Depends on What You’ve Been Looking at - Recommendations,Guidelines and Standards Should Reflect This

Abstract

Sensory systems continuously recalibrate their responses according to the current stimulus environment. As a result, perception is strongly affected by the current and recent context. These adaptative changes affect both sensitivity (e.g., habituating to noise, seeing better in the dark) and appearance (e.g., how things look, what catches attention) and adjust to many perceptual properties (e.g., from light level to the characteristics of someone's face). They therefore have a profound effect on most perceptual experiences, and on how well the senses work in different settings. Characterizing the properties of adaptation, how it manifests, and when it influences perception in modern environments can provide insights into the diversity of human experience. Adaptation could also be leveraged both to optimize perceptual abilities (e.g., in visual inspection tasks like radiology) and to mitigate unwanted consequences (e.g., exposure to potentially unhealthy stimulus environments).

Social Media Text

The human senses continuously adapt to the current context, discounting prevailing stimuli and highlighting new ones. Adaptation affects what and how well humans perceive and whether they see the same thing, and could be incorporated into policies to optimize perceptual experience and abilities.

Highlights

• The world is experienced through the senses, yet all sensory systems adapt to the stimuli they are exposed to. This normalizes perception for the prevailing stimulus, so that percepts are relative rather than absolute (e.g., whether a light appears bright or dim depends on the adapted light level).

• Adaptation affects all perceptual processing and thus most if not all perceptual experiences (from how loud a voice appears to whose voice it appears to be).

• Adaptation helps the senses work better – by tuning them for the current setting. It promotes more stable percepts and heightens the salience of novel information.

• Because of adaptation, different individuals will tend to perceive the world in similar ways if they are adapted to the same world, but should have diverging percepts if they are adapted to different stimulus environments.

• Policies and standards for designing stimulus environments (e.g., lighting) and how people interact with them (e.g., radiology, security screening, food inspection) should account for sensory adaptation to optimize perceptual experience and performance and to mitigate negative consequences.

Sensory Adaptation

Most people imagine that what they see or hear is what is there – that the senses objectively record the reality before them, and that (barring sensory deficit) all humans should perceive the same world in similar ways. However, this commonsense view is wrong. Individual differences are in fact a hallmark of perception and suggest that each person lives in a unique perceptual world (Mollon et al., 2017; Wilmer, 2008; Wilmer et al., 2012). This essay focuses on individual differences that arise from differences in the worlds people are exposed to.

All sensory systems adjust how they operate based on experience and in response to changes in the environment or the organism. These adjustments take many forms (Bosten et al., 2022):

During development, experience is established and fine-tuning neural processing (Kiorpes, 2016).

During normal aging, sensory processing also compensates for changes so that perception tends to remain more stable than the sensory losses predict (Werner, 1996).

Throughout life, perceptual learning enhances perceptual judgments, leading to “expertise” in specific judgments (e.g., (Seitz et al., 2023).

This review considers one aspect of this plasticity – adaptation – and illustrates its role in the context of seeing. Visual adaptation refers to changes in visual sensitivity or perception that result from changes in stimulation (Clifford et al., 2007; Webster, 2015). For example, in dim light, vision becomes more sensitive, while in bright light sensitivity is reduced, similar to adjusting a camera's exposure level. Adaptation is necessary because vision must operate over a wide range of light environments, and this is not possible without adjusting (Rieke & Rudd, 2009). However, as a consequence, we are only sensitive to the relative rather than absolute light level – to how bright or dark things are relative to the adapting light level.

Adaptation Across the Visual System

Light adaptation begins at the earliest steps of seeing, adjusting sensitivity of the receptors in the eye. People with typical color vision have three types of receptors (trichromacy) that differ in wavelength sensitivity. Because each receptor adapts independently, changes in the light spectrum alters the balance of receptor responses (chromatic adaptation) leading to color afterimages. For example, after adapting to blue, a gray looks like the opposite color (yellow). The red also looks less red over time and eventually could appear gray. This is because adaptation tends to neutralize or discount the perception of the adapting stimulus itself, a process referred to as normalization. Normalization maintains homeostatic balance, both within neurons (resetting the baseline response for whatever the average stimulus is) and across neurons (rebalancing the relative responses). Thus adaptation sets the norm (e.g., what looks gray) relative to which other percepts are judged. Thus, what is perceived as a norm or neutral stimulus is not absolute, but continuously recalibrated for the current situation.

Visual processing involves many stages in the brain that also adapt to the current characteristics of the visual world. Consequently, adaptation occurs for and affects most if not all things we see. For example, if you look at tilted or moving lines you become less sensitive to their particular orientation or direction of motion. These changes arise later in the visual pathway (in visual cortex), but reflect the same adaptive process. Adaptation also affects “high level” perceptions, such as the attributes of someone's face (Webster & MacLeod, 2011). After looking at one type of face (e.g., with male, old, or sad features), a neutral face appears biased in the opposite direction (e.g., so that it looks more female, young, or happy, respectively). That adaptation is so similar across diverse stimuli and levels of coding suggests it is a generic and core mechanism in sensory processing.

The Functions of Adaptation

Unlike portrayals in textbooks or even some theoretical accounts, adaptation is not a consequence of over-taxing or fatiguing the visual system under artificial conditions, but is instead a beneficial process controlling perception at every moment. Adaptation aids sensory processing in many ways (Clifford et al., 2007; Kohn, 2007; Webster, 2015). In addition to sensitivity regulation (as in light adaptation), adaptation helps build more efficient neural codes, correct for coding errors, and promote perceptual constancy.

Efficiency refers to how well the limited capacity of the brain can be allocated for carrying information. Adaptation reduces redundancies, for example by recalibrating neural signals so that different response levels or neurons are given equal weight (Atick et al., 1993; Wainwright, 1999).

Constancy is the tendency for percepts to remain stable despite variations in the stimulus. For example, in color vision, objects appear largely the same color even if the lighting changes. One mechanism contributing to this is adaptation to the average color (e.g., when the scene gets redder, vision becomes less sensitive to the red, so that a gray surface under red light continues to appear gray) (Foster, 2011).

Error correction compensates for variations within or between observers (e.g., so that the color seen is not affected by individual differences in the sensitivity to wavelength). Because of this, normal variations in wavelength sensitivity have little effect on color appearance (e.g., (Brainard et al., 2000; Werner & Schefrin, 1993)).

These functions are also tied to the idea that adaptation helps build a prediction about the world (Huang & Rao, 2011), by calibrating for the expected or average properties of the environment. This allows the mean state of the world to be encoded implicitly, freeing neural resources to encode deviations or errors in the prediction. Interestingly, the observer is not aware of the adaptation state(s) they are in (Seriès et al., 2009). Thus visual aftereffects are experienced as changes in the stimulus rather than the observer. By this account, observers also do not “see” the average state of the world, but only how the current stimulus differs from the stimuli they are adapted to. In this sense, most or all perceptual experiences are adaptation aftereffects. Consider again the perception of gray. This is the absence of hue, and all hue percepts are relative to this norm (Webster & Leonard, 2008). But what appears gray is not an inherent property of the light or the eye, but rather adaptation of the eye to the light spectrum.

Again, these consequences apply to most percepts. For example, our ability to recognize the characteristics of faces (e.g., the identity or expression) may depend on “correcting” coding for the average face to represent the deviations from the average. People rapidly adapt to the faces they are seeing, so that the prevailing face literally looks more average, and recalibrate many of the judgments made about the face (Leopold et al., 2001; Webster et al., 2004). Indeed, prominent models of face perception are based on similar principles to color perception, where identity (like hue) is represented by how an individual face differs from the average (Valentine et al., 2016). Both color and faces are therefore anchored by a central norm which itself appears neutral, and this norm is in turn anchored by and synonymous with the observer's state of adaptation (Webster & MacLeod, 2011).

Adaptation and the Problem of Other Minds

Adapting different observers to the same environments. The consequences of adaptation determine whether two individuals experience the world in the same or different ways. To draw again from color, the lens of the eye progressively yellows with age, blocking the “blue” short-wavelength energy entering the eye. Without adaptation, the world should therefore look yellower with age. Yet old and young adults describe their color percepts in very similar ways because they tend to be adapted to the same spectral environment (Werner & Schefrin, 1993). The lens and corneal surface also differ between observers in their refractive properties, so that some people have better acuity than others. Yet except when trying to see fine details like print, most people may feel that the world looks focused through their eyes, because they are adapted to their own optical errors (Sawides et al., 2011; Webster et al., 2002).

A more extreme example is individuals with color deficiencies, which affect 8% of the US male population. Errors in the genes coding the light-sensitive receptor pigments result in a loss in one type (dichromacy) or a shift in wavelength sensitivity (anomalous trichromacy). The latter weakens the comparisons available for signaling redish versus greennish, but the color experience of some anomalous trichromats nevertheless approach normal trichromats (Boehm et al., 2014; Isherwood et al., 2020). This could occur if neurons that receive the inputs from the receptors adapt their responses to match the range of their inputs. All of these adjustments could be accomplished by simple recalibrations that compensate perception for the particular sensitivity of the observer. In fact, it would be surprising if the visual system did not exhibit some level of autocorrection for individual variations in sensitivity.

Adapting the same observers to different environments. Conversely, these recalibrations will lead to divergent percepts when people are adapted to different environments. For example, what appears gray may vary for people living in a forest or desert. As more fanciful examples, in the recent film Barbie, viewers saw Barbie and Ken immersed in a pink world, but it likely would not have appeared very pink to them; similarly, while earthlings describe Mars as the red planet, it may not seem red to future colonists living there, but will be normalized for the same gray percept (but different spectral average) that people experience on earth (Webster, 2014; Webster & Tregillus, 2017). Again, these effects are not unique to color but are a general property of perception. For example, if people inhabit different social environments they should be normalized to different facial features. People working in a daycare or senior center might literally see the age of a face differently. This partly explains why people are better at discriminating among the types of faces (e.g., ages or ethnicities) they are used to seeing compared to faces seen less often (Rhodes et al., 2010).

Adaptation to Unnatural Environments – The Case of Wide Color Gamuts

The environments humans now inhabit are very different from the natural habitats the senses evolved in. Relatively little is known about how and how well sensory systems can adjust to these novel worlds, or to what extent technology and culture or pastimes and preferences are literally changing how humans see. What, for example, are the perceptual consequences of spending hours looking at a computer screen or cell phone? When monitors shifted from bulky CRT tubes to flat-panel displays, many suddenly noticed that the old TVs flickered, while the new flat screens looked concave. These aftereffects arose because people were adapted to the faster refresh and curved screens of the CRTs.

A growing development in color technology is wide gamut lighting and displays. These use primaries with narrow wavelength spectra to increase the range (gamut) of colors, enabling more saturated colors. For example, wide gamut lighting might boost the red and green contrasts by 30% compared to broadband illuminants. Adaptation to stronger color differences leads to corresponding loss in perceived color contrast. Thus over time colors under wide gamut systems might start to look similar to the colors previously perceived in natural lighting (Ilic et al., 2022). In turn the world under natural illumination might start to look less colorful, and it could become harder to discriminate small color differences.

The spectral shaping in wide gamut lighting has similar effects to optical filters designed to enhance color differences to aid color deficiencies. These filters block selected wavelength bands, while the lights emit selected bands. The renormalization for a wider color gamut is opposite to the compensation predicted for color deficiencies (i.e., observers are becoming less sensitive to color because the contrasts are higher, rather than more sensitive because the contrasts are lower). A recent study of these filter aids found that after a week of wear colors looked weaker, consistent with adaptation (Bosten & Somers) (though the filters can also enhanced color salience over time, perhaps reflecting perceptual learning (Werner et al., 2020)).

Adaptation to Specialized Environments – The Case of Medical Image Perception

Many individuals work in highly specialized visual contexts to which they might be uniquely adapted. One example is medical image perception. Despite rapid progress in image processing and machine learning, the interpretations and diagnoses from medical images still relies on visual judgments by radiologists and other medical professionals. How the visual properties of medical images and characteristics of the observers (e.g., training and expertise) influence these judgments have therefore been intensely studied, with implications for policies and protocols (e.g., (Taylor-Phillips & Stinton, 2020). Radiologists spend extended periods inspecting images that have unique visual characteristics, and might therefore be placed in unique states of adaptation that could shape their percepts in positive or negative ways.

For example, breast-cancer screening with x-ray mammography is a high-volume reading task that allows for early detection of asymptomatic malignancies. While mammographic screening finds the majority of malignancies with relatively few false-positive findings, there is still considerable room for improvement, including in the reading process. Multiple studies have shown that batch reading improves screening performance, specifically a reduction in radiologists’ false-positive rates, compared to “interrupted” reading that includes other clinical duties (Burnside et al., 2005; Cohen et al., 2021; Ram et al., 2022). This improvement is robust to changes in the technology used to acquire images (screen-film, digital, and digital breast tomosynthesis mammography systems), indicating that the source of these effects lies within the reader.

Laboratory studies have demonstrated that breast images induce robust adaptation, including selective adaptation effects based on mammographic breast density (Kompaniez et al., 2013; Kompaniez-Dunigan et al., 2015, 2018; Parthasarathy et al., 2023). This is consistent with clinical observation studies showing improved reading performance over the course of a batch (Abbey et al., 2020; Backmann et al., 2021; Taylor-Phillips et al., 2016). This provides convergent evidence for adaptation as a mechanism that impacts and can potentially improve performance in batch reading of screening mammograms. The adaptation hypothesis posits that as a reader proceeds through a batch, they adapt to the image characteristics, and that this allows them to more clearly distinguish normal and abnormal (novel) features in the images. The density-selective findings suggest that this benefit could be optimized by ordering cases within a batch on the basis of breast density or other textural features of breast images.

Beyond Seeing

As noted, adaptation is not unique to vision but characteristic of all the senses. A striking example is in olfaction. When entering a room, an initially prominent odor becomes inconspicuous after a few moments. Again adaptation filters the ambient stimulus from awareness, so that the senses are poised to register changes in the environment. Such effects pervade the gamut of sensory processing.

What about beyond the senses? An unresolved question is the distinction between perceiving and thinking, or percepts vs. concepts (e.g., (Block, 2023; Wagemann, 2017)). These differ on many grounds, but also share properties in how they are structured and calibrated. “Adaptation level theory” was developed based on perceptual experiments, but served as a general principle for how psychological phenomena are anchored by context (Helson, 1964). Similarly, cultural, social, or aesthetic norms and how they are calibrated (e.g., (Robson et al., 2023)) share parallels with adaptations woven into the earliest stages of sensory processing. Knowledge about adaptation at early visual stages (e.g., (Graham, 1989) was critical for understanding perception at higher stages (e.g., for face perception (Webster & MacLeod, 2011)). Could sensory adaptation also serve as a model system for understanding the dynamics of more conceptual mental states? Do the consequences of adaptation – e.g., for what people are aware of or blind to– inform how an understanding of cultural and individual diversity in knowledge and experience?

Policy Implications

Recognizing adaptation and understanding its impact. An important first step in incorporating perceptual adaptation into policies is to increase recognition among stakeholders that perceptual experience is in fact highly malleable and constantly recalibrating based on experience. This includes recognition that many forms of plasticity can (and probably must) remain functional throughout the lifespan. This is well understood for simple phenomena like light adaptation, but it is only within the last few decades that scientists have recognized the extent to which adaptation pervades even high-level percepts (e.g., the sense of space (Greene & Oliva, 2010), the direction of someone's gaze (Jenkins et al., 2006), or the identity of someone's face or voice (Yovel & Belin, 2013)). Concomitantly there is growing recognition that adaptation reflects normal and advantageous processes actively engaged in everyday vision. Increased understanding that people can literally see differently because of their different past and present visual environments could give more insights into differences in “perspective” and should lead to policies that are more responsive to these differences. As an example, standards for color technology are based on a single “standard observer” based on measurements from a small number of individuals. It is well known that color vision varies widely, even among those classified as color “normal,” and the field has recently begun to explore models and standards that better account for this diversity (Asano et al., 2016; Smet et al., 2021; Stockman & Rider, 2023). Modern color appearance metrics also recognize the need for specifying the observer's state of adaptation, and routinely incorporate chromatic adaptation (to the average color) as necessary to predict appearance (Fairchild, 2013). The pervasiveness of adaptation suggests that there are similarly large and quantifiable variations in many other aspects of human perceptions that arise because of adaptation to quantifiable properties of the world. These effects could similarly be built into plans and predictions for how, for example, a change to the visual environment would be experienced within different individuals or contexts, or for developing algorithms that “adapt” the properties of stimuli to match them to the observer (Webster, 2014).

Mitigating negative consequences of adaptation. In some cases, adaptation might lead to unintended or unforeseen consequences. In lighting the principle that vision adapts to the average color is widely understood, while the fact that it also adapts to the range of colors is much-less appreciated. As described above, adaptation to wide gamut lighting could place the visual system in uncharacteristic operating states, for which the long-term consequences are not understood. As another example, we noted that if the visual system changes (e.g., either with normal aging or the onset of disease) then the system will naturally compensate perceptions for these changes. This could mask the onset of a deficit and may be why problems can sometimes progress substantially before they are noticed. Better understanding of this compensation could potentially help inform how vision and hearing should be monitored. Adaptation might also mask environmental conditions that are potentially unhealthy, simply because people become perceptually dulled to them. For example, high flicker rates or exposures to certain light regimens can be uncomfortable or have health consequences (Spitschan & Joyce, 2023; Wilkins, 2016), and adapting to such stimuli could affect how noticeable they are without necessarily changing their physiological consequences (Yoshimoto et al., 2019).

Without appropriate guidelines, new technologies may find it challenging to optimize both product quality and value, especially when there are trade-offs between factors that attract customers (such as low cost and immediate visual appeal) and more subtle factors such as ergonomics, safety, and reliability. Often, manufacturers rely on guidance from scientific and technical organizations to which they also contribute, to develop recommendations and standards to assist with design. For illumination and vision-related products, there are numerous excellent bodies, including the International Electrotechnical Commission (IEC), the International Lighting Commission (CIE), the Illuminating Engineering Society (ES), and the Society for Information Displays (SID). Such organizations could help develop recommendations that incorporate how vision adapts to the visual environment and potential long-term consequences of this adaptation.

Harnessing positive consequences. Policies or protocols could also be developed to take advantage of adaptation. Currently in radiology there are no standards based on adaptation for how images are ordered within a reading session, or what constitutes an optimal session length. Given the potential for adaptation to influence assessments (e.g., (Abbey et al., 2020; Taylor-Phillips & Stinton, 2020)), sequences could be structured to facilitate performance. The potential impact of such changes would at best be subtle, and control only one of very many factors that influence how and how well a reader can assess images. However, given the enormous consequences of medical assessments for both patients and the healthcare system, and the very high rates of screening, even very slight improvements could have substantial public impact. Moreover, the “cost” of controlling or enhancing the prospects for beneficial adaptation could be minimal in this context, since they could involve as little as developing strategies for ordering the sequence of images within a batch.

There are many other professions and tasks that depend on interpreting visual images that might be impacted by sequential context effects like adaptation (e.g., face identification, quality inspection, security screening or visual surveillance). How observers adapt to such contexts could be tailored to optimize the rate or consequences for performance. For example, in some cases it is the number rather than the length of exposures that matter (Aagten-Murphy & Burr, 2016), and some studies have found that adaptation is itself adaptable, or becomes faster with repeated exposures (Li et al., 2020; Yehezkel et al., 2010). When the goal is to facilitate adjustments, for example to new visual devices, then regimens could be developed to increase the rate or magnitude of adapting, or at least better predict the time course of the expected adjustments.

As a concrete example, consider again the example of breast-cancer screening by mammography. There are many policies and protocols in place for addressing the widely-recognized problem of viewer fatigue (Taylor-Phillips & Stinton, 2019). Similar efforts could be developed to address viewer adaptation, for which there is currently little recognition, despite clinical evidence for improved performance from batch reading supported by laboratory studies identifying adaptation as a specific mechanism. Policies encouraging batch reading – and also recommendations for optimal batch sizes or how images are ordered within a batch with regard to their adaptive characteristics – therefore appear to be warranted in much the same way that ambient lighting and high-quality monitors are currently specified.

Remaining Questions

The properties of adaptation remain in many ways poorly understood. For example, we lack a good understanding of the scope and limits of adaptation or of how the processes are engaged in real-world contexts, or of their temporal dynamics. There is evidence that adaptation can unfold over many timescales that depend on the nature of the adjustments (Kording et al., 2007; Li et al., 2020), and as noted these timescales can themselves be experience-dependent. Another major challenge is to clarify how adaptation interacts with other sensory and cognitive processes, as well as how it fits within the broader scope of sensory plasticity. Prioritizing basic science programs to answer these questions will lead to better insights into the role that adaptation and experience play in shaping the nature, variety, and capacities of human perceptual experience.

Conclusions

Sensory adaptation is a basic and pervasive property of sensory processing that affects all percepts. Perception is better when observers are in the appropriate state of adaptation for the judgments they are trying to make, and worse when the state is inappropriate. For the many professions that require perceptual evaluations (e.g., radiology), policies (e.g., for how and how long images are inspected) could be developed that help ensure that the observer is optimally adapted for the task at hand. For technological developments that alter the stimulus environment (e.g., new lighting designs), policies should consider the effects and consequences of adaptation to those environments.