Colour for Behavioural Success

Abstract

Colour information not only helps sustain the survival of animal species by guiding sexual selection and foraging behaviour but also is an important factor in the cultural and technological development of our own species. This is illustrated by examples from the visual arts and from state-of-the-art imaging technology, where the strategic use of colour has become a powerful tool for guiding the planning and execution of interventional procedures. The functional role of colour information in terms of its potential benefits to behavioural success across the species is addressed in the introduction here to clarify why colour perception may have evolved to generate behavioural success. It is argued that evolutionary and environmental pressures influence not only colour trait production in the different species but also their ability to process and exploit colour information for goal-specific purposes. We then leap straight to the human primate with insight from current research on the facilitating role of colour cues on performance training with precision technology for image-guided surgical planning and intervention. It is shown that local colour cues in two-dimensional images generated by a surgical fisheye camera help individuals become more precise rapidly across a limited number of trial sets in simulator training for specific manual gestures with a tool. This facilitating effect of a local colour cue on performance evolution in a video-controlled simulator (pick-and-place) task can be explained in terms of colour-based figure-ground segregation facilitating attention to local image parts when more than two layers of subjective surface depth are present, as in all natural and surgical images.

Introduction

In order for colour to have a functional meaning, in terms of a driving factor for behavioural success, colour must be detected and encoded by the visual receptors of a living organism, be subsequently processed by its nervous system, and affect action. Such processing enables what is commonly called colour perception (e.g. Hurvich & Jameson, 1957). As will be made clear in this review, the ability to detect, process, and act in response to colour information may be a direct consequence of evolutionary pressure (e.g. Mollon & Jordan, 1988). It is argued that such ability has evolved to a greater or lesser extent indifferent species and is determined by the functional anatomy and epigenetic development of their visual systems. At least 40 different types of visual system exist in the animal world. The simplest is found in aplysiae (e.g., Goelet, Castellucci, Schacher, & Kandel, 1986), a species that has been extensively studied by Eric R. Kandel and his team and only seems able to differentiate light from dark. The visual brains of higher order species are able to discriminate not only light intensities but also shape and colour. An enormous diversity in animal retinal structures and visual neuronal mechanisms is found, with a corresponding diversity in the functional role of species specific colour vision, perception, and behaviour (e.g., Land, Horwood, Lim, & Li, 2007; Land & Nilsson, 2002; Jacobs, 2008). This diversity can be explained by the fact that separate evolutionary processes have acted on the different species. We will first start out with a brief overview of how anatomically and functionally different visual systems have enabled colour vision in different species.

A living organism possessing a single retinal pigment would most likely see the world in monochrome, as we do at night when only our rod photoreceptors are active. In addition to rods, most species have cones, the photoreceptors that provide colour perception during the day. We can define the input to these photoreceptors in physical terms as the overall level of light provided by the sun or moon (Osorio & Vorobyev, 2005) between about 360 nm and 700 nm and colour as the distribution of wavelengths within this range (e.g., McLeod & Boynton, 1979). The number of retinal pigments in many animal species is not, as it is in the human (e.g., Bagnara, 1966, 1998; Bagnara, Fernandez, & Fujii, 2007), limited to three (V. C. Smith & Pokorny, 1977). Of vertebrate species studied so far, the most developed ability to discriminate stimuli on the basis of their colour is found in birds (S. Andersson & Amundsen, 1997). Birds vary according to species in their capacity to discriminate colour. Diurnal birds tend to have increased ultraviolet (UV) sensitivity, either UVA or UVB (S. Andersson & Amundsen, 1997; Cuthill, Bennett, Partridge, & Maier, 1999; Cuthill et al., 2000) and have more cones, while nocturnal species have a much higher proportion of rods (Ruggeri et al., 2010). UV wavelength sensitivity improves the temporal resolution of the avian visual system by enabling the detection of rapid flicker movements (Rubene, Hastad, Tauson, Wall, & Odeen, 2010), which may be critically important to ensure their behavioural adjustment when they fly, or have to catch prey. Microspectrophotometry has been employed to test the spectral sensitivity properties of animal retinas (Bowmaker, Dartnall, & Mollon, 1980). With this technique and through behavioural studies, it has been deduced that four cone classes exist in approximately 30 species of birds. Some even have five types of photoreceptors: four single cone classes and, for about half of all cones, a double cone class with two different photopigments (Bowmaker, Heath, Wilkie, & Hunt, 1997). In addition, the spectral sensitivities of certain cones are narrowed by a coloured oil droplet filtering light signals (Bowmaker et al., 1997; Partridge, 1989), with the double cone class having a different oil droplet filter and broader spectral tuning (Bowmaker et al., 1997). By sequencing the part of the gene coding for avian retinal opsins, Odeen and Hastad (2003) have shown that the evolution of avian colour vision is more complex than had previously been thought. Their data support the theory that sensitivity biases toward violet (short wavelengths) has evolved independently at least four times, which may explain why some species such as the chicken (Osorio, Miklosi, & Gonda, 1999) process colour inputs with at least three opponent mechanisms: one comparing the outputs of UV- and short-wavelength-sensitive receptors, one comparing the outputs of medium- and long-wavelength receptors, and a third comparing outputs of short- and long- or medium-wavelength receptors. Diurnal birds tend to be tetrachromatic rather than trichromatic and are able to detect chromatic variations in skylight that are invisible to us (as we are insensitive to UV) and provide information about updrafts or other atmospheric conditions that are critical to their survival. Another advantage is related to sexual selection, as colour discrimination is enhanced by seeing UV reflections from the wings of potential mates in many birds (e.g., Boulcott, Walton, & Braithwaite, 2005; E. J. Smith et al., 2002), and in particular starlings and zebra finches (Bennett, Cuthill, Partridge, & Lunau, 1997; Bennett, Cuthill, Partridge, & Maier, 1996).

Retinas with four classes of cones involved in colour discrimination (tetrachromatic vision) have also been reported in fish. Most of the latter have photoreceptors with peak sensitivities in the UV range; 95% of all known fish species seem to discriminate red, yellow or green or blue, violet, and UV up to 365 nm. However, since sea water selectively absorbs longer wavelengths, that is, red light, many fish living below 10 m detect poorly in the red region. Nevertheless, many reef fish species living at this depth emit red fluorescence, the origin of which are guanine crystals, and seem able to detect this colour. Fluorescence is widespread in marine organisms and was recently also discovered in some terrestrial chameleon species, which have evolved fluorescing bony tubercles protruding from their skull. The fluorescence becomes visible in males only, shining through their scales under UV light at 365 nm (Prötzel et al., 2018). The fluorescent signal is likely to play a role in sexual selection by helping to attract females.

Poralla and Neumeyer (2006) showed in behavioural experiments that there is no abrupt transition between green and red but that there is yellow in-between. Kelber, Vorobyev, and Osorio (2003) more recently have discussed how photoreceptor signals are combined and compared to allow for the discrimination of biologically relevant stimuli. Five retinal pigments have been found in butterflies and many more in sea organisms such as the mantis shrimp and the sea manta, a giant ray (Cronin, Caldwell, & Marshall, 2001). While the human eye cannot detect all of the electromagnetic spectrum emitted by the sun and UV being in any case absorbed by the human lens and cornea, lobsters, shrimp, goldfish, trout, bees, and lizards seem to be able to detect light in the UV region (e.g., Brink, van der Berg, & Botha, 2002; Cronin et al., 2001; Fleishman, Loew, & Leal, 1993). Comparative psychophysics have shown that the goldfish displays the phenomenon of colour constancy, previously found only in primates and avian species.

Research data from genetics suggest that the variations in colour vision across species may result from the fact that the capacity to detect colour has evolved more than once, with gene duplication for visual opsin pigments (Bowmaker, 1998). In addition, probably because of their nocturnal behaviour, certain mammals have rod-dominated retinas (see e.g., Ruggeri et al., 2010) and many selectively lack cones with the red-sensitive pigment and, therefore, display only dichromatic colour vision. The dichromate cat is incapable of discriminating red, with cones only sensitive to blue or indigo and to yellow or green, as found in the ferret (Calderone & Jacobs, 2003). Horses also have only dichromatic vision detecting long wavelengths and the colours based on them (Carroll, Murphy, Neitz, Ver Hoeve, & Neitz, 2001). A different system has evolved in rats and mice which have excellent night vision, due to a higher number of rods than cones but see poorly in colour although both are dichromatic. Both rat and mouse cones coexpress two photopigments, one sensitive to wavelengths around 510 nm and another sensitive to UV, and their visual systems exploit these differences to enable them to discriminate certain colours (Jacobs, Fenwick, & Williams, 2001; Jacobs & Williams, 2007; Jacobs, Williams, & Fenwick, 2004). Diurnal rodents and rodents which live in almost lightless conditions have been found to have similar colour vision (Jacobs, Calderone, Fenwick, Krogh, & Williams, 2003; Williams, Calderone, & Jacobs, 2005). Genetic data have also confirmed the long-held suspicion that colour vision in primates including humans, apes, and Old World monkeys is better developed than in other mammals (Jacobs, 1993). Despite data lacking on many species, primates are largely considered to be trichromatic, although variations exist (Jacobs, 1993, 1996; Jacobs & Deegan, 1999). Variations amongst New World monkeys are even greater, some species being trichromatic while others are only dichromatic (Jacobs & Williams, 2006). In addition, evidence predicts that all male New World monkeys are dichromatic while, depending on their opsin gene arrays, individual females can be either dichromatic or trichromatic (Jacobs & Deegan, 2003a, 2003b, 2005; Rowe & Jacobs, 2004). Some nocturnal species appear to be monochromatic (Jacobs, 1996). The situation for aquatic mammals is quite different as many species, and in particular mammals that live in deep water, are monochromatic and tend to have blue shifted vision compared with that of many terrestrial mammals (Fasick, Cronin, Hunt, & Robinson, 1998). This is considered to result from the absence of evolutionary pressure to maintain colour in a dark monochromatic oceanic environment (Newman & Robinson, 2005). On the other hand, behavioural data on the Bottlenose Dolphin (Tursiops Truncatus), which loves to stick their head out of the water and seems to enjoy daylight, suggest some colour vision based on rods and a single class of cones (Griebel & Schmid, 2002).

According to Darwin (1871), the choice of a sexual partner represents an important step toward the most successful propagation of a given species by a sexual process. Partner choice in animals for sexual selection is therefore a major factor in the survival of the species (M. Andersson, 1994; Houde, 2001). The choice of a sexual partner in the animal kingdom is determined in one of three distinct ways. In polygamous animals, the male usually eliminates potential rivals by chasing them away aggressively. After elimination of competitors, the female, and often a harem of females, is left with one alpha male as sexual partner by default. In addition to leaving little freedom of choice to the female of the species, the successful aggressive displays can be viewed by the female as a reflection of the most successful genes (Darwin, 1871). Alternatively, a male seduces a potential female by parading his colourful attributes as ornaments, which correspond to his secondary sexual characteristics. Such nuptial behaviour frequently leads to exaggerated display of the ornaments, which include shimmering of peacock feathers or lifting the bright blue webbed feet of the blue-footed boobies of the Galapagos archipelago, well known for having inspired Darwin's theory of evolution during his studies there in 1835. The female thus has a selective choice, based on her estimation of the best genes, which may be considered to reflect how impressive her potential partner's ornaments are in terms of size and colour intensity (Price, 2006), signalling the physical condition and immune competence of the male. This process is frequently employed in monogamous animals and in particular in monogamous birds. The female preference to mate with males with exaggerated external ornaments may have arisen from evolutionary pressure (Cornwallis & Uller, 2010; Qvarnstrom, Brommer, & Gustafsson, 2006), when a female preference for some aspect of male morphology provoked selection for males with the appropriate ornament, a possibility generally known as the sexy son hypothesis (Weatherhead & Robertson, 1979). Plasticity (Danielson-Francois, Kelly, & Greenfield, 2006) in sexual mate choice behaviour has been observed in birds during the course of the breeding season, which is accompanied by a reduced variation in the size and brilliance (brightness) of colourful ornaments. Later in the season, the choice based on male ornaments is replaced by a choice based on genetic complementation (Badyaev, Hill, Dunn, & Glen, 2001). Although most studies have focused exclusively on the role of female mate preferences in maintaining or promoting colour variation, a recent study (Pryke & Griffith, 2007) highlights that both female and male components of mate choice should be taken into account. Females showed a strong preference for mates with the most elaborate sexually dimorphic traits, while males were particularly choosy, associating and pairing only with females of their own morph-type. Sometimes, even in highly coloured male and female fish (Sköld et al., 2008), the males rather than the females make the choice of a sexual partner (Houde & Endler, 1990), which suggests that female ornamentation male also be sexually selected (Amundsen & Forsgren, 2001).

In vertebrates such as birds, the success of sexual selection is critically determined by both the successful production of colour traits (G. E. Hill & McGraw, 2006) and by their successful visual detection and perceptual processing (Cuthill et al., 1999). Successful reproductive sex in animals constitutes a process of combining and mixing genetic traits, often resulting in the specialization of organisms and to form offspring that inherit traits from both parents (M. Andersson, 1994). Genetic traits, contained within the DNA of chromosomes, are passed on from one parent to another in this process. Because of their motility, animals often engage in coercive sex. In a large number of essentially monogamous species, which include humans, the selection of a sexual partner is most often mutual. In species which are not monogamous, such as lions for example, male domination is often determinant. Since certain inherited characteristics may be linked to specifically sex-associated chromosomes, physical differences are frequently observed between the different sexes of an organism. These have been particularly well studied in birds, where major differences in feather colours of males and females may be observed (Brush & Seyfried, 1968; Pryke & Griffith, 2006, 2007). Over recent years an overwhelming bulk of evidence has shown that these differences play a significant role in mate determination, with males frequently displaying these attributes with the aim of attracting a female partner (G. E. Hill & McGraw, 2006, for review). Ornamental colours are critical secondary traits in this context, as secondary sexual traits are classically considered to be derived by sexual selection for traits which give an individual an advantage over rivals in courtship or aggressive interactions (M. Andersson, 1994; Darwin, 1871). The evolution of secondary sexual traits is not a recent phenomenon in evolution (Tomkins, LeBas, Witton, Martill, & Humphries, 2010) but harks back to Dimetrodon and Edaphosaurus, the earliest examples of sexual dimorphism known in the animal kingdom.

Both pigment-based colourations and UV reflectance, which is structural colour and not perceptible by humans, play an important role in communicating critical information to potential mates. The biological significance of structural colour was first investigated mainly in birds (for reviews, see Auber, 1957; Dyck, 1976). More recent reports have emphasized the potential biological role of pigment-based colour, structural colour (Vukusic & Sambles, 2003) including UV reflectance, and fluorescence in avian communication (S. Andersson & Amundsen, 1997; Hunt, Cuthill, Bennett, Church, & Partridge, 2001; Örnborg, Andersson, Griffiths, & Sheldon, 2002; Siitari, Honhavaara, Huhta, & Viitala, 2002; for recent reviews, see G. E. Hill & McGraw, 2006). There is agreement that multilayer reflectors (Ghiradella, 1994; Ghiradella, Aneshansley, Eisner, Silberglied, & Hinton, 1972) at the origin of structural colourations in butterflies (Vértesy et al., 2006) are found in other species (Parker, 2000). They were present early in evolution and have even been found in beetle fossils, for example (Parker, 1998; Parker & McKenzie, 2003). Coherent scattering is the optical mechanism that explains colour production in all structurally coloured butterfly scales (Prum, Quinn, & Torres, 2006). Male individuals of various avian species exhibit conspicuous colours on their feathers evolved by sexual selection driven by mating preferences (M. Andersson, 1994; Darwin, 1871; Solis, Aviles, de la Cruz, Valencia, & Sorci, 2008), such as the males of the cooperative breeding azure-winged magpies (Cyanopica cyanus), where only a fraction access the breeding status. These birds display conspicuous blue plumage colourations, and all males that become breeders have a more brilliant and more saturated blue colouration, with a more intense violet hue compared with the nonbreeders (Solis et al., 2008). The additional capacity of avian ornaments to reflect UV plays an important role during sexual displays (S. Andersson & Amundsen, 1997; Bennett et al., 1997; Finger & Burckhardt, 1994; Finger, Burkhardt, & Dyck, 1992; Hausmann, Arnold, Marshall, & Owens, 2003; Hunt et al., 2001; Parker, 1998; Siitari et al., 2002). Experimental alterations of the UV component in plumage have been shown to significantly affect sexual signals in many bird species (S. Andersson & Amundsen 1997; Bennett et al., 1996, 1997; Maier & Bowmaker, 1993).

King Penguins (Aptenodytes patagonicus) display highly coloured ornaments, notably yellow or orange breast and auricular feathers and the two orange or pink UV reflecting beak horns (Dresp, Jouventin, & Langley, 2005; Dresp & Langley, 2006; Dresp-Langley & Langley, 2010) on each side of the beak. These colourations have been suggested to signal the health status and immunocompetence of individuals, seem to be detected by individuals of both sexes (Nolan, Dobson, Dresp, & Jouventin, 2006), and are therefore likely to ensure the successful mating of the fittest individuals. During courtship, King Penguins flaunt their beak ornaments by raising their heads high in the air when calling out for a potential partner. The iridescence of their beaks is reminiscent of the shimmering iridescence of peacock feathers, which may explain why the King Penguin is also quite often perceived as a rather sexy beast by our own species (Carmichael, 2007). The fact that the beak horn also displays an orange-to-pink colour tone increases the signal function, as more than a single type of photoreceptor in the observing individual would be activated. The perception of this colour signal may also be enhanced by a contrast effect, since the tissue surrounding the beak horn is deep black. In addition, the multiplicity of microstructures with slightly different orientations producing the UV reflectance, which was found to obey Braggs law (Bragg & Bragg, 1915; Dresp & Langley, 2006) spreads both the wavelength and also the angle over which it is reflected, thereby producing a more easily perceptible signal. Such UV reflecting ornaments are absent in sexually immature individuals (Massaro, Lloyd, & Darby, 2003). Apart from the function of visually attracting potential mates, ornamental colours communicate information about the general fitness of individuals, in particular whether an individual is well nourished and healthy, providing additional criteria particularly for males in attracting female partners. In addition to pure survival, mate selection, breeding performance, fecundity, and growth, colour traits appear to be associated in various ways with immunity (Blount, Metcalfe, Birkhead, & Surai, 2003; Hamilton & Zuk, 1982; G. E. Hill, 2001; G. E. Hill & Montgomerie, 1994; Lochmiller & Deerenberg, 2000; Nolan et al., 2006; Svensson, Raberg, Koch, & Hasselquist, 1998).

Another important step in ensuring the survival of a species is foraging (Dall & Johnstone, 2002). After mating and reproductive sex, it becomes necessary to find food sources to ensure the survival of the offspring, and learning to discriminate between the colours of fruits and foliage acquires is a critical function to successful foraging behaviour, especially in birds. Forage success is determined by whether a food source is recognized at the right time by individuals of a species or not. Under conditions of abundance, this does not represent a problem, but in natural environments, food sources are often scarce or irregularly distributed, and their availability varies over time. Thus, the foragers need to identify profitable food sources and to retain information relative to their spatial location (Lehrer, 1994) to be able to return to them for meeting nutritional requirements. One and the same food source rarely produces the sought after nutrient all year long, and foraging in a dynamic environment therefore requires a balance between exploitation of known resources and the exploratory sampling of new resources (e.g., Dall & Johnstone, 2002; A. T. Hurly, 1996), two types of foraging behaviours referred to as win-stay and win-shift strategies (Sulikowski & Burke, 2011). Associating a visual cue such as colour with a given food reward increases the rate of discovery of further food sources and thereby helps individuals of a species cope with the spatial heterogeneity of food resource distribution in a natural environment (Edwards, Newman, Parsons, & Krebs, 1996). Examples of increased foraging efficiency brought about by visual cues such as colour in a variety of species, including vertebrates (Durier & Rivault, 2000; Hsu & Yang, 2012; T. A. Hurly & Healy, 1996; R. Johnson, 1991), suggests that animals learn to identify food sources on the basis of such cues. Colour is one of these, and combinations of cues include other visual, olfactory, auditory, tactile, and gustatory ones (Arenas & Farina, 2012; Croney, Adams, Washington, & Stricklin, 2003; Goyret & Raguso, 2006; P. S. M. Hill, Hollis, & Wells, 2001; Ishii & Shimada, 2010; Montgomerie & Weatherhead, 1997). The use of colour cues for successful foraging may be particularly important under conditions of temporal and spatial variation in food availability. Pollinators forage on irregularly distributed plants and flowers, and fruit or nectar availability varies over time depending on flowering times. The availability as well as the quality of a given food reward varies between flowers on the same plant, depending on the age of the flowers, the time of day, and other environmental conditions (Corbet, Unwin, & Prys-Jones, 1979). To locate and remember high-quality food sources, birds often use colour cues to direct and to optimize their foraging (Gill & Wolf, 1977). Bird-pollinated flowers (Castellanos, Wilson, & Thomson, 2002) are precious food sources for bees. They commonly display bright yellow or red colourations (Rodríguez-Gironés & Santamaría, 2004), and there is evidence that angiosperm flowers have evolved chromatic cues that suit the colour discrimination abilities of their pollinators, most likely to reinforce visits by the most efficient of them (Shrestha, Dyer, Boyd-Gerny, Wong, & Burd, 2013).

Many animals can form associations between colour and reward (Kelber et al., 2003; Raine & Chittka, 2008; Vallortigara, 1996; Weiss, 1997). Studies on the foraging behavior of free-living rufous hummingbirds (Selasphorus rufus) have shown that these birds retain information relative to high-quality food sources on the basis of learnt colour cues (Healy & Hurly, 2013; Henderson, Hurly, Bateson, & Healy, 2006; T. A. Hurly & Healy, 1996, 2002). They predominantly use spatial information when returning to repeatedly sampled flowers (T. A. Hurly & Healy, 1996; Marshall, Hurly, & Healy, 2012) and rely on learnt colour cues for foraging in unfamiliar places and the exploratory sampling of new flowers. Goldsmith and Goldsmith (1979) found that hummingbirds could learn to associate colour with nectar reward within less than 20 visits to the food source. They learn rapidly to associate colour with the presence or absence of reward (Goldsmith & Goldsmith, 1979; T. A. Hurly & Healy, 2002; Meléndez-Ackerman, Campbell, & Waser, 1997), or to associate colour with either food volume or concentration (Bacon, Hurly, & Healy, 2010; Bateson, Healy, & Hurly, 2003). Colour cues also enhance their ability to return to target flowers in the presence of distracting flowers (T. A. Hurly & Healy, 1996). Coexisting species of hummingbirds were found to forage on the basis of learnt colour cues, selecting almost exclusively high-quality food sources and ignoring poor-quality ones (Sandlin, 2000). Other avian species learn to combine spatial cues and colour to detect and recognize rewarding flowers or feeders as, for example, rainbow lorikeets (Sulikowski & Burke, 2011). Although the nomadic lifestyle of sunbirds does not facilitate studying their perceptual abilities in the wild, a recent study in seminatural enclosures (Whitfield, Köhler, & Nicolson, 2014) has shown colour-learning and colour-cue-use abilities in the captive amethyst sunbird (C. amethystina). Following training to colour cues, amethyst sunbirds more effectively selected high-quality food sources at the first attempt and throughout the feeding period, thereby decreasing the total time spent on feeding and the number of visits to the food source while increasing the rate of energy gain per meal. These findings are in agreement with earlier studies on foraging behaviour of sunbird species in natural environments (Gill & Wolf, 1977). A higher number of visits to abundant food sources in the presence of colour would lead to suggest that the colour cues facilitate the return of sunbirds to rich food sources (Scoble & Clarke, 2006) in the natural environment. Sunbirds forage over large distances, on plant species with characteristically red or orange flowers (S. D. Johnson & Nicolson, 2008) with variable nectar concentrations. The birds rapidly adjust their feeding patterns to maintain an energy balance (Köhler, Verburgt, Fleming, & Nicolson, 2008). Sunbirds foraging without colour cues are likely to sample more frequently from a larger number of poor feeders compared with birds that have learnt to use colour cues. As a consequence, the latter return to the food source less frequently and display shorter times spent there (Whitfield et al., 2014).

It takes a giant leap up the phylogenetic scale, from the avian species to the human primate, to unravel the functional role of colour in the behavioural success of our own species. In humans, the colour quality of hair and skin may to a given extent contribute to whether a first date between individuals will be successful or not. Similarly, the colour of a meat, fruit, or vegetable displayed in a supermarket may influence humans when choosing from several available options, and recent research data have revealed that goods with fully saturated colours are instantly perceived as larger in size than poorly saturated equivalents (Hagtvedt & Brasel, 2017). However, the functional significance of colour for man (Dresp-Langley & Langley, 2010; Pinna & Reeves, 2013) has shifted toward other, more sophisticated aspects of a far more complex process of visual communication. Pinna and Reeves suggest that, in addition to the discrimination of light and surface chromaticity, the visual purpose of color in humans is threefold to interrelate each chromatic component of an object, thus favoring the emergence of the whole, to support a part–whole organization in which components reciprocally enhance each other by amodal completion, and, paradoxically, to reveal fragments and hide the whole, as in camouflage—in sum, there is a chromatic parceling-out process of separation, division, and fragmentation of the whole in order to best specify the important figure-ground relations in the scene. The evidence they provide is psychophysical, principally based on illusions such as the watercolor effect, and it remains to be seen to what extent these different facets of colour perception can be verified in animals. Although these aspects may seem remote from what we call evolutionary pressure they are, based on what has been discussed earlier, likely grounded in evolution. In fact, colour plays a critical role in the cultural development of human society, and well before the effects of the colour of objects on our perception were investigated more or less systematically by the French philosopher Michel Eugène Chevreul (1839), artists had already begun to use colour selectively and effectively to create novel visual sensations in order to refine their techniques and promote the success of their work. In modern art and design, colour not only plays a critical role in generating moods and emotions but also communicates information relative to what is supposed to be seen as figure and what as ground in complex planar creations. Finally, colour provides visual cues to object qualities that have the power to directly guide human decision-making and action, as will be illustrated here on the example of image-guided surgical planning and intervention.

The Renaissance painters had preferentially resorted to chiaroscuro and geometric cues to aerial perspective, using a very limited chromatic range, to create landscape depth and figure-ground effects in their works. In the 19th century, at the dawn of abstract expressionism, painters such as Turner, especially in his later work, effectively used a larger variety of colours in combination with aerial perspective to suggest what should be seen as nearer and what as further away to the observer in a painting. He also most successfully combined chromatic brightness and saturation to express and balance figure and ground, and to visually communicate specific moods, and other qualia. Later in the evolution of the visual arts and design, architects and designers like Vasarély most effectively manipulated colour brightness and saturation in combination with planar shape geometry, playing with foreground and background effects to generate powerful and complex three-dimensional (3D) shape effects in highly abstract configurations which he baptized with strange names. His work as a visual artist is probably among the most relevant for perceptual science, illustrating how colour, luminance, saturation, and shape can be combined with local variations in two-dimensional (2D) geometry to elicit powerful visual effects and sensations of 3D structure in the plane. For some time, chromatic and achromatic pathways in the visual brain were considered independent by the visual neurosciences, mostly on the basis of partial evidence from functional neuronal anatomy (e.g., Page & Crognale, 2005). Yet, Leonardo da Vinci in his Trattato della Pittura (1651), and subsequently also Chevreul (1839), had already discussed contrast and colour combinations as powerful cues to pictorial depth and how contextual variations may influence such perceptions. More recently, visual science has confirmed that contrast, colour, and shape effects, together or independently, can determine what will be seen as nearer or further away in planar image configurations (Dresp & Fischer, 2001; Dresp-Langley & Reeves, 2012, 2014; Guibal & Dresp, 2004). Others have shown conditions under which surrounding light, contrasts, shadows, and nearby objects can determine how we perceive the given colour as a function of the context (Devinck, Delahunt, Hardy, Spillmann, & Werner, 2005, 2006; Devinck & Spillmann, 2009). Most contemporary visual artists probably would readily agree that, in addition to planar geometry, colour can be a powerful and potentially self-sufficient medium for creating structural effects. Science has only just begun to unravel some of functional contribution of colour to the wider development of human cultural activities as well as modern technology, as in the health-care domain, where colour is exploited by imaging techniques as a powerful tool for guiding, and thereby promoting the success of complex human interventions.

The use of colour cues has become an important strategy in recent advances in visual interface technology for image-guided surgical planning and intervention. Image-guided surgery uses images taken before or during the procedure to help the surgeon navigate. The goal is to augment the surgeon's capacity for decision-making and action during the procedure (see Dresp-Langley, 2015; Perrin et al., 2009 for reviews). Colour is strategically added to images at various stages of the process. In this kind of augmented reality by colour, guidance is provided directly on the surgeon's view of the patient by mixing real and virtual properties of human tissue and organs. The perceptual quality of the colour tone, brightness, or saturation in the rendered image is essential for making specific regions of interest to the surgeon optimally perceptible. This includes the visual traceability of devices relative to the patient, the registration, and alignment of the preoperative model and optimized rendering and visualization of preoperative data. Visualization by colour in this context means translating image data into a graphic representation that is understandable by the user (the surgeon) and to convey important information for assessing structure and function. In other words, colour is exploited in this context to “make the invisible visible,” and for helping make critical decisions as swift and as safe as possible during interventions. Although the field has evolved dramatically in recent years, the most critical problem is still the one of task-centred user interface design, where the timing of the generation of image data is critical, and to facilitate navigation through large cavities with multiple potential obstacles, such as within the human abdomen or the brain. Complex displays designed to provide navigational aids benefit from additional colour cues to visual field depth and tool-alignment. Interventional navigation techniques use surface renderings of anatomy from preoperative imaging with intraoperative visualization techniques, where colour is a highly effective tool for making the invisible visible to the human operator or surgeon. A common strategy here is representing volumetric data as 2D surfaces in colours of varying opacity. The efficiency of renderings in facilitating human perception-based decisions can be evaluated in terms of the perceptual salience of the surfaces that represent the most critical regions of interest for effective and safe human intervention. Intraoperative imaging often provides further diagnostic information and permits assessing risks as well as perspectives of repair.

Image-guided instrument tracking is a major challenge for image-guided surgery technology (West & Maurer, 2004). In image-guided interventions, the critical problem for the surgeon is eye-hand coordination, which involves keeping track of the relative positions of the surgical tool(s) and of the direction of the tool-movements (e.g., Jiang, Zheng, & Atkins, 2014). The effectiveness of visual displays for tracking tooltips in simulator training is an important factor to skill evolution in trainees (Batmaz, de Mathelin, & Dresp-Langley, 2016a, 2016b, 2017). In a series of psychophysical experiments designed to study image conditions likely to facilitate simulator training for optimally precise and swift manual gestures with a tool of novice surgeon, we have included the strategic use of a colour cue to test whether it would help novice surgeons to become more precise more rapidly in a computer-controlled simulator task guides by a surgical fisheye camera image.

Material and Methods

The experiments were run on a computer-controlled perception-action platform (EXCALIBUR) for image-based analysis of data relative to the time (in milliseconds) and precision (in pixels) of manual operations with a tool in a pick-and-place task. Hard and software components of the simulator platform are described in detail elsewhere (Batmaz et al., 2017).

Ethics

The study was conducted in conformity with the Helsinki Declaration relative to scientific experiments on human individuals with the full approval of the ethics board of the corresponding author's host institution (CNRS). All participants were volunteers and had provided written informed consent.

Subjects

Six surgeons with minimal experience in image-guided interventions, ranging in age between 25 and 35 years, participated in this study. They were all right handed.

Objects Displayed on the 2D Monitor

Video input received from an HD USB fisheye camera was processed by a DELL Precision T5810 model computer equipped with an Intel Xeon CPU E5-1620 with 16 Giga bytes memory (RAM) capacity at 16 bits and an NVidia GForce GTX980 graphics card. Experiments were programmed in Python 2.7 using the Open CV computer vision software library. The computer was connected to a high-resolution color monitor (EIZO LCD Color Edge CG275W 27″) with a 30 Hz refresh rate and a 2560 × 1440 pixel display. The screen has an inbuilt color calibration device (colorimeter) using the Color Navigator 5.4.5 interface for Windows. The video input received by the computer from the HD USB fisheye camera generated the raw image data. These were adjusted to a viewing frame of 640 pixels (width) × 480 pixels (height) and processed to generate 2D visual displays (Figure 1) in a viewing frame of 1280 pixels (width) × 960 pixels (height), the size of a single pixel on the screen being 0.32 mm. Real-world data and visual display data were scaled psychophysically to ensure that the visual display subjectively matched the scale of the real-world action field (RAF) of the simulator as seen in front of the observer as closely as possible. The luminance (L) of the light grey (x = 3.03, y = 3.19, z = 3.47 in CIE color space) RAF visualized on the screen was 33.8 cd/m². The luminance of the medium grey (x = 0.66, y = 0.70, z = 0.76 in CIE color space) target areas was 15.4 cd/m², producing a target or background luminance contrast (Weber contrast: ((L_foreground−L_background)/L_background)) of −0.54. The luminance of the blue (x = 0.15, y = 0.05, z = 0.80 in CIE color space) object surface on the screen was 3.44 cd/m², when shown in colour and when shown in its achromatic version on the screen. This gave Weber contrasts of −0.90 with regard to the RAF and −0.78 with regard to the target areas in both achromatic and colour viewing modes. The luminance (29.9 cd/m²) of the green (x = 0.20, y = 0.70, z = 0.10 in CIE color space) tool-tips produced Weber contrasts of −0.11 with regard to the RAF and 0.94 with regard to the target areas. All luminance values for calculating the object contrasts visualized on the screen were obtained on the basis of standard photometry using an external photometer (Cambridge Research Instruments) and interface software. The luminance of the different objects (object top surface, tool-tips, the square-shaped target areas, and the grey action field) displayed on the screen was photometrically identical in the two display conditions (Figure 1), yielding identical local (Michelson) contrasts in both viewing modes. All subjects stated to perceive the contrast intensities as equivalent in the two displays (Figure 1). The pick-and-place simulator task here requires placing the square-shaped 3D object on the centre of each of the different dark grey 2D target squares. Optimal precision in this task involves, as pointed out earlier (Dresp-Langley, 2015), segregating the blue or grey square top of the 3D object from the grey 2D squares defining the target areas. Psychophysical figure-ground experiments (Guibal & Dresp, 2004) have shown that achromatic patterns with identical Michelson contrasts produce the same perceptual figure-ground response probabilities, however, when local colour cues were added, stronger figure-ground responses were found. Therefore, we are led to assume that any difference in placing precision from the two different display conditions here can confidently be explained by the presence or absence of the local colour cue in the image.

Figure 1.

Screen snapshots of the two viewing conditions for image-guided task execution (five step pick-and-place simulator task controlled by a HD fisheye micro camera equivalent to a surgical camera and connected to a computer with optimal processing capacity) with local colour cueing (left) and without (right).

Experimental Procedure

The experiments were run under conditions of free viewing, with illumination levels that can be assimilated to daylight conditions. The RAF was illuminated by two lamps (40 Watt, 6500 K) which were constantly lit during the whole duration of an experiment. Participants were comfortably seated at a distance of approximately 150 cm away from the screen, looking straight-ahead at the image. Seats were adjusted individually in height at the beginning of a session to ensure that the image displayed on the monitor was slightly higher than the individual's eyes when looking straight at the screen, which is a near-optimal viewing position for laparoscopic simulator training. All participants were given a printout of the targets-on-RAF configuration with white straight lines indicating the ideal object trajectory, and the ordered (red numbers) target positions the small blue cube object had to be placed on in a given trial set of the pick-and-place task, always starting from zero, then going to one, to two, to three, to four, to five, and back to position zero. Participants were informed that they would have to position the cube with the tool, a forceps-type instrument, held in their dominant hand “as precisely as possible on the center of each target area of the RAF, as swiftly as possible, and in the right order, as indicated on the printout.” Before starting the first trial set, the participant could look at the printout of the RAF with the ideal trajectory steps in the right order for as long as he or she wanted. When an individual felt confident to be able do the task, the experiment was started. Each participant was given a blank warm-up trial set, and then trained in 10 repeated trial sets in each of the two viewing conditions (with colour cue and achromatic). The order of conditions was counterbalanced within individual sessions and between subjects in two separate successive sessions. Each time the small object was placed on a target area, the computer software processing the video data (see Batmaz et al., 2016a, 2016b, 2017; Batmaz, de Mathelin, & Dresp-Langley, 2018) calculated the total number of pixels separating the object from the ideal on-target-centre location. At the end of a trial set, the cumulated number of off-target pixels from the five successive task steps (place manoeuvers) and the total time of task completion in milliseconds were written to a labelled excel file.

Results and Discussion

Results were analysed in terms of the means M and their standard errors SEM for dependent variables relative to time in seconds and precision in pixels for with two levels (achromatic vs. with colour cue) of the display factor D₂, 10 levels of the trial rank factor T_10, and six subjects. We therefore have a balanced Cartesian experimental design S₆×D₂×T₁₀ yielding a total of 120 data (observations) for time and precision.

For the dependent variable relative to time, the data analysis returned a shorter average time for the achromatic image condition (M = 15.6 s, SEM = 1.5) compared with the image condition with local colour cue (M = 16.9, SEM = 1.4). Two-way repeated measures analysis of variance signals that the difference in means for time (d = 1.3) between the two levels of the display factor is statistically significant, F(1, 5) = 8.423; p<.05. Average times decrease systematically as the rank number of the trials in a session increases. Means for time as a function of the rank number of trial sets are shown in Figure 2 (top) for each of the two display conditions. The effect of trial rank, F(9, 45) = 10.24; p < .001, and the interaction between the trial rank factor and the display factor on time, F(9, 45) = 11.41; p < .001, are statistically significant. The condition of equality of variance of the distributions for time was satisfied.

Figure 2.

Average data relative to task execution times (top) in seconds and task precision (bottom) in terms of the number of pixels recorded off-target-centre for successive place steps of the pick-and-place task.

For the dependent variable relative to precision, the data analysis returned a higher average off-target score, indicating lesser precision, for the achromatic image (M = 1,682, SEM = 64) condition compared with the image condition with the local colour cue (M = 1,396, SEM = 59), which produced greater precision. Two-way repeated measures analysis of variance signals that the difference in means for precision (d = 286) between the two levels of the display factor is statistically significant, F(1, 5) = 22.60; p<.01. As the rank number of the trials in a session increases, average precision increases significantly, F(9, 45) = 3.500; p<.01, as testified by the decreasing off-target scores, depending on the display condition shown in Figure 2 (bottom). Average off-target scores get smaller as the trial increases only in the image condition with the colour cue. The interaction between the trial rank factor and the display factor on the dependent variable for precision is statistically significant, F(9, 45) = 9.878; p<.001. The condition of equality of variance of the distributions for precision was satisfied.

Beyond considering the population statistics from analysis of variance, there is a complementary and quite revealing way of looking at the average data for time and precision from this experiment. Trend analysis (linear regression analysis) on the average data reveals the correlation coefficients R² from a least-squares fitting process of the variance of the residuals to the variance of the one-sample dependent variable, representing 1 minus the ratio of that variance. R² gives an estimate of the fraction of variance in the average data that is explained by the fitted trend line. The statistical significance of the trend is determined by the statistical probability that the linear trend is different from zero using the Student distribution (t) with n−2 degrees of freedom.

Here (see Figure 2), it is shown that the achromatic image conditions produce horizontal (flat) regression lines indicating an absence of any linear trend indicative of a training effect. In other words, the average performance scores for time and precision do not evolve in training sessions with 10 trial sets when guided by images without a colour cue. On the other hand, the image condition with the colour cue, on average, produces such training effects, for both time and precision. These effects are revealed by the significant linear trends for this image condition, shown in Figure 2 (top and bottom, respectively). Under image guidance with the colour cue, average times and average off-target scores present a significantly decreasing linear trend, t(1, 8) = 17.79; p<.001, and t(1, 8) = 13.27; p<.001, as the rank number of the trial set increases. In other words, subjects are, on average, getting faster and more precise in 10 trial sets when guided by the image with the local colour cue. The one-sample t statistic from the linear trend analysis gives the statistical probability that the trend is real, that is, significantly different from zero.

Earlier work by Batmaz et al. (2016a, 2016b, 2017, 2018) had shown that the evolution of the performance scores of complete novices and more experienced surgeons in simulator tasks as this one here depend on factors relative to individual speed-precision strategy and on the capacity to select information relative to local detail in the visual display guiding the motor performance. Others (e.g., Yantis & Jones, 1991) had found some time ago that colour can act as a powerful priority tag in guiding attention for cognitive performance. The results from this study here show that a local colour cue in the image consistently promotes positive training effects on time and precision in a total of 10 trial sets, while an image without the colour cue (achromatic viewing) fails to do so. We suggest that this can be explained in terms of a colour-facilitated figure-ground segregation in images of complex natural scenes with more than two layers of subjective surface depth.

This interpretation is consistent with data from Dresp-Langley and Reeves (2012, 2014) who found effects of relative brightness contrast and subject depth induced locally by coloured patches in complex configurations (Figure 3) were significantly stronger in images with more than two subjective surface depth levels in the plane when compared with configurations with only two levels of surface depth in the plane, that is, only two alternatives for figure and ground.

Figure 3.

Examples of configurations from Dresp-Langley and Reeves (2012, 2014). In images with more than two subjective surface depth levels in the plane (top), as is the case in most natural scene images including surgical images, effects of relative brightness contrast and subject depth induced by the coloured patches were found to be significantly stronger compared with the same configurations with only two levels of surface depth in the plane, that is, only two alternatives for what is likely to be figure or ground.

Conclusions

Evolutionary and environmental pressures influence not only colour trait production in the different species, but also their perceptual ability to see and process colour in terms of useful information for guiding action. This ability directly ensures the success of survival relevant actions directly related to mating (sexual selection) and feeding (foraging). Moreover, colour not only helps sustain the survival of animal species, but also acts as a key factor in the cultural and technological development of our own species. In the contemporary imaging sciences, the strategic use of colour has become a powerful tool for guiding the planning and execution of interventional procedures (medical precision technology). Here we have shown a facilitating effect of a colour cue on training for task precision in a computer-controlled simulator (pick-and-place) task. This effect can be explained in terms of a colour-cue-based enhancement of figure-ground salience in images with more than two layers of subjective surface depth such as natural surgical images. Colour thus plays an important role in the cognitive development of man by facilitating his adaptation to new and challenging visual worlds in the context of rapidly progressing technological development.