Investigating the chromatic contribution to recognition of facial expression

Abstract

A pedestrian may judge the intentions of another person by their facial expression amongst other cues and aiding such evaluation after dark is one aim of road lighting. Previous studies give mixed conclusions as to whether lamp spectrum affects the ability to make such judgements. An experiment was carried out using conditions better resembling those of pedestrian behaviour, using as targets photographs of actors portraying facial expressions corresponding to the six universally recognised emotions. Responses were sought using a forced-choice procedure, under two types of lamp and with colour and grey scale photographs. Neither lamp type nor image colour was suggested to have a significant effect on the frequency with which the emotion conveyed by facial expression was correctly identified.

1. Introduction

Eye tracking studies involving pedestrians walking outdoors show that a large number of visual fixations at critical moments are directed at other people.^1,2 One reason to examine other people is to understand their intentions, i.e. whether they are friendly, aggressive or indifferent.³ After dark, road lighting should be designed to enhance the performance of this task.

A fundamental variable of lighting is the lamp spectral power distribution (SPD). Current guidance for road lighting in the UK^4–6 accounts for the variety of SPDs among different types of lamp by allowing a reduction in horizontal illuminance when using lamps of CIE general colour rendering index (R_a) equal to or greater than 60. The size of the reduction is scaled by the lamp scotopic-to-photopic luminous flux ratio (S/P ratio).

For the perception of spatial brightness (a proxy for the reassurance or perceived safety an area presents) and the detection with peripheral vision of pavement obstacles, there is some evidence that lighting of higher S/P ratio is of benefit.^7,8 The trade-off between S/P ratio and illuminance for these tasks was characterised using the CIE recommended system for visual performance based mesopic photometry,⁹ although there is evidence that within the mechanism for spatial brightness, the effect of SPD may be better ascribed to the short wavelength sensitive cones and/or the ipRGC than to the rods.^10–13 It has been shown that lighting of higher R_a tends to improve the appearance of a scene:¹⁴ in the absence of a precise metric the threshold of R_a ≥ 60 was retained from previous guidance.¹⁵

However, at the point at which this guidance was developed, research regarding interpersonal judgements was less clear. Lighting is expected to matter because some cues in interpersonal evaluation are visual rather than verbal: Non-verbal behaviour can reveal how a person feels, even when they would rather conceal this.¹⁶ Previously, lighting research focussed on facial recognition as a proxy for interpersonal judgements. Evidence of SPD effects on facial recognition was available from seven studies of which four suggested that SPD was significant,^17–20 while three suggested it was not.^21–23 To resolve these mixed results, consideration was given to the study by Yip and Sinha²⁴ who demonstrated that facial recognition was better using colour photographs than grey scale when these photographs were blurred to reduce resolution, a difference that was absent when the photographs were of higher resolution.

Lin and Fotios²⁵ proposed that variations in methodology would explain the differences in these results and concluded that an effect of SPD on facial recognition would be significant when the task was difficult, defining difficulty as a function of visual size (i.e. distance between a pedestrian and the target), duration of observation and the experimental procedure. In which case, evidence for lighting design should be established in conditions that resemble the behaviour and context of pedestrians walking after dark.

Two common facial recognition procedures are matching and identification.²⁶ In the matching procedure, the test participant is required to identify the target face from amongst a set of reference faces; in the identification procedure, the test participant is required to state the identity of a person by recollection from memory, these studies often using alleged celebrities as the target to be identified. In the two studies using a matching procedure mean recognition distances were reported to be 12 m and 25 m.^22,23 However, in the three studies using an identification procedure, mean recognition distances were between 5.40 m and 8.45 m.^18–20 One explanation for the shorter recognition distances found in those studies using an identification procedure is that this is the more difficult task – the target needed to be closer (a larger visual size) for recognition than in those studies using matching. Note also that the three identification studies concluded that SPD had an effect on recognition performance, whilst the two studies using matching did not suggest SPD to be significant and thus the effect of SPD is seen when the task is more difficult. Dong et al.²⁷ subsequently demonstrated that the facial identification task was more difficult than the facial matching task.

A common approach to measuring facial recognition under different lighting is to ask the test participant to walk towards a target face and to stop at the point at which the target can be correctly identified (the stop-distance method), the interpretation being that recognition at a greater distance implies better lighting. There are two limitations in this method. First, that it measures the distance at which a face was recognised but perhaps not the distance at which it would be desirable to recognise a face when walking after dark. Second, the task requires continuous observation on the target face which is not a realistic representation of real-world interpersonal judgements: Gaze can be used to exert social control²⁸ and prolonged visual fixation on others is avoided in many social situations. Both the visual size of a target (targets at greater distance subtend a smaller angle at the observer’s eye) and the duration of observation affect the ability to detect and/or identify a target and thus the stop-distance method is not appropriate for the task of identifying optimum lighting conditions.

To establish a foundation for the provision of lighting in residential areas Caminada and van Bommel²⁹ suggested a requirement to recognise the face of an approaching pedestrian at a distance of 4 m as the criterion for ‘an overall minimum lighting value’. This was apparently rounded (once converted to metric units) from the minimum public distance proposed by Hall,³⁰ a distance of 3.7 m (12 feet), suggested to be the minimum distance at which an alert subject would be able to take evasive or defensive action if threatened. An ideal facial recognition distance was suggested to be 10 m, this being the transition point between the close and far phases of Hall’s public zone.²⁹ Distance is important because it changes the size of the target subtended at the observer’s eye and in turn that may affect the visual pathway by which the image is processed.³¹

Subsequent review of Hall suggested it was not appropriate evidence for determining the desirable distance for evaluating other people, and that Hall did not intend it to be used for such purposes.³² Instead, a distance of 15 m was established from two sources: Townshend’s³³ after-dark field study in which he asked members of the public to estimate the distance at which they would be comfortable about an approaching person or group of people, and estimation of the distances at which other people were fixated from the records of an outdoor eye tracking study.³² Tests using a celebrity identification task, with size or blurring used to simulate reduced identification at greater distances, suggest that face identification accuracy remains at 100% up to a distance of approximately 8 m, reducing to 75% at 10 m and 25% at 23 m.³⁴ It is therefore reasonable to expect satisfactory evaluations of the face at 15 m.

Evidence that the duration of face observation matters was presented by Lin and Fotios²⁵ who found that an effect of SPD in their facial identification test was significant with a 1 s duration but not significant with a 3 s duration. Dong et al.²⁷ used five durations (0.1, 0.3, 1.0, 3.0 and 10.0 s) in tests using matching and identification procedures and found that the effect of duration was statistically significant, with longer duration leading to a higher frequency of correct recognition. This confirms the discussions reported elsewhere.^35–37 As to what observation duration might be representative of typical behaviour when walking outdoors, the results of two studies using eye tracking to record visual fixation suggest this to be approximately 500 ms.^32,38 Carey³⁹ suggests a familiar face can be recognised in 500 ms, and a face recognition task suggested two fixations to be optimum⁴⁰ which is also approximately 500 ms. Such a brief observation presents a more difficult task than the unlimited durations of previous work.

While past research of lighting has tended to investigate the facial recognition, the ability to determine a person’s identity from their face, the importance of this task has not been established. However, there is evidence that recognition of the emotion conveyed by facial expression may be a more representative task than identity recognition as it appears to contribute to the approach or avoid reaction of pedestrians.^41,42 Two experiments were therefore carried out to examine how luminance and SPD affected the ability to recognise the emotion conveyed by facial expression.^43,44 The targets in these trials were photographs from the FACES database of actors portraying, by their facial expression, the six universally recognised emotions: anger, disgust, fear, happiness, neutrality and sadness.⁴⁵ Image size was varied to simulate different interpersonal distances. The targets were observed under different SPDs and luminances and were presented on a non-self-luminous LCD screen to avoid screen-generated light confounding the test light. Both studies used a forced-choice procedure requiring participants to identify which of the six emotions was being conveyed.

The first study⁴³ presented targets at three luminances (0.01, 0.1 and 1.0 cd/m²), under two types of lamp (high pressure sodium – HPS and metal halide – MH), using target size on screen to simulate three distances (4, 10 and 15 m) and they were observed for one second. Results from the 30 test participants did not suggest a significant effect of SPD on recognition of the emotion conveyed by facial expression. There was however a significant effect of SPD on recognition of emotion portrayed by body posture carried out in parallel using images from the BEAST database.⁴⁶ The second study⁴⁴ used three additional luminances (0.03, 0.33 and 3.33 cd/m²), a third type of lamp, an additional, shorter observation duration (500 ms), and simulated distances of 4 m and 15 m. Each of 20 test participants observed all 24 targets under all 18 conditions. Again, the effect of SPD was not found to be significant.

One possible reason why an effect of SPD was not found to be significant in these experiments is that images displayed on the non-self-luminous LCD screen were reduced to greyscale. This was demonstrated when the screen was illuminated by a D65 fluorescent lamp and target faces were found to have the same chromaticity (x ≈ 0.29, y ≈ 0.33, two degree, measured with a Konika Minolta CS1000a spectroradiometer) as saturated red, green and blue colour patches (RGB values of 255/0/0, 0/255/0, and 0/0/255, respectively). The targets observed by test participants were thus essentially achromatic and for achromatic, foveal tasks, we do not expect an effect of SPD on visual performance.⁴⁷

Consequently, the experiment reported below investigated facial expression recognition using an apparatus designed to display coloured images generated from a specific spectrum at low light levels. Target sizes and observation durations were selected to include those considered to present a difficult task (i.e. short exposure duration and a small size subtended at the eye). The experiment used two lamps of different SPD and the target photographs were used in their original colour format as well as a grey scale format. While both formats can be considered fixed attributes of the target image files, it is important to note that the appearance of colours in the displayed images will depend on how these attributes interact with the light source spectrum. Thus, the experiment investigated colour in terms of lamp SPD as used by past studies of lighting and by target colour as used by Yip and Sinha.²⁴

2. Method

This experiment simulated the effect of changes in road lighting on the ability to discriminate the emotion of another person by their facial expression. For ease of expression replication and randomisation of presentation order, the targets were photographs. The changes in lighting included variations in luminance and SPD but did not include variation in spatial distribution of light.

2.1. Apparatus

The experiment presented target images of actors (photographed close-up, passport-style, against a neutral background) expressing a range of emotions through facial expression, obtained with permission from the FACES database.⁴⁵ In this experiment, as before,^43,44 24 images were used comprising six expressions from each of four target actors: a young female, a young male, an old female and an old male. Figure 1 provides an example of the images used. The FACES database provides colour images. The 24 images used here were also transformed to grey scale and trials were repeated with both coloured and grey scale targets.

Figure 1

Example facial expressions taken from the FACES database⁴⁵ for a young female (http://faces.mpdl.mpg.de/faces/). Emotions conveyed (from left to right): anger, disgust, fear, happiness, neutrality and sadness (available in colour in online version)

The target images were observed from a distance of 0.65 m, this was maintained by using a chin rest with forehead restraint. Image size was manipulated to represent the two target distances, 4 m and 15 m, selected to follow previous work.⁴⁴ The visual sizes of the target images are given in Table 1.

Table 1

Size of targets according to simulated interpersonal distance

Simulated distance (m)	Target size (mm)	Angle subtended at the eye (°)
4	32	2.9
15	9	0.8

Note: Target sizes calculated assuming a face size of 200 mm from chin to top of head.

Target faces together with an expanded neutral background were displayed on a sheet of white paper using an LCD data projector in which the standard lamp had been removed and light provided instead by one of the test lamps (Figure 2). The projector and observer were each about 15° from the normal to the screen. The projected area was therefore slightly trapezoidal, with a constant width of 282 mm and heights of 345 mm and 370 mm on the left and right hand sides, respectively. While the border of the projected area was noticeably not rectangular the target face at its centre appeared to be normal. The paper screen, with an opaque backing, was mounted to coincide exactly with the projected area on a large translucent diffuser panel (1.15 m wide by 0.93 m high) attached to the front of the test booth used in previous studies.^43,44 In this way, the back-lit diffuser provided a large adapting background to the front-projected targets. This set-up presented a face within a large visual field to simulate observation of a person in an outdoor setting, albeit with a uniform background rather than the complex background of a real scene. Separate iris dampers for the projector light source and the diffuser light source allowed the luminances for the target and background fields to be independently adjusted. Spectral power distribution was altered by changing lamp type, with the same lamp type used in the projector and booth for any given trial. There were negligible changes in spatial distribution of luminance between different types of lamp.

Figure 2

Section (top) and plan (bottom) of test apparatus used to observe target faces under different light conditions (not to scale). The projector and observer were each about 15° from the normal to the screen.

2.2. Test variables

Six lighting conditions were used, comprising two SPDs and three luminances. The two types of lamp were HPS (2000K, S/P = 0.57, R_a = 25) and MH (4200K, S/P = 1.77, R_a = 92), these being two types of lamp commonly used in the UK for road lighting and as were used in previous work.^43,44 Figure 3 shows the variation in chromaticity of a target face (measurement centred on the nose) in colour and grey scale versions when projected using the MH and HPS lamps. Each data point is the mean of four measurements made using the neutral expression images of the target actors. It can be seen that target colour has an effect, yielding a different chromaticity for the colour and grey scale images. In contrast, the non-self-luminous screen used in the previous experiments would result in different chromaticities for the two types of lamp but would not reveal different chromaticities for the colour and grey scale targets.

Figure 3

Chromaticities of coloured and grey scale target faces illuminated by the metal halide (MH) and high pressure sodium (HPS) lamps. These data are the mean of measurements of the four target actors, using their neutral expression image, with the spectroradiometer centred on the nose

The three luminances were 0.1, 0.33 and 1.0 cd/m² (Konica-Minolta LS100 luminance meter) as measured on the projected target images. These luminances are in the middle of the six used by Yang and Fotios⁴⁴ with the current apparatus unable to present their lower (0.01 and 0.03 cd/m²) and higher (3.3 cd/m²) luminances. Luminance measurements were made for the whole face area, centred on the bridge of the nose, rather than for specific areas of the face. This is therefore an average luminance over a large area and does not characterise the variation of luminance expected at different parts of a face.

The apparatus was set up so that the target (the projected image in the centre of the white paper), the near background (projected over the remaining area of the white paper) and the wider background (the acrylic diffuser) were of the same luminance. Since both the luminance of the target image and that of the near background depended on the same dimming adjustment, the near background had grey scale RGB settings that were calibrated in order to balance luminance across the white paper screen. Luminance of the wider background was adjusted by varying the amount of light inside the booth using the other iris damper.

2.3. Procedure

Twenty-eight test participants (13 male, 15 female; aged between 18 and 34 years; ages collected in bins with the mode being 19–22 years) were recruited and were paid a small fee for their contribution. For the standard α level of 0.05 this is the minimum sample required to detect a significant effect.⁴⁸ All participants had normal or corrected-to-normal visual acuity as tested using a Landolt-ring test, and all had normal colour vision as examined using the Ishihara test under a D65 daylight-simulating lamp. Each test session started with 20 minutes for adaptation to the low light level.

The responses sought were judgements of emotion conveyed through facial expression (anger, disgust, fear, happiness, neutrality or sadness). Each target face was presented for 500 ms. Responses were given using a button box, with one button for each of the available responses, and there was no time limit for this response.

For each participant, a series of practice trials was used to present and confirm understanding of the available response options. Initially the available options (the six different facial expressions) were shown simultaneously to help the participant discriminate them. Twenty-four example face targets (the six expressions by four actors not used in trials) were shown in random order under office lighting conditions and without time limit, to allow each expression to be linked to the corresponding emotion.

Lamp types and luminances were observed in separate blocks. The three luminances were used in a random order for a given lamp, these trials being completed before moving to the second lamp, and the two lamps were used in a balanced order across the sample. Within a given block the 96 target images (6 facial expressions, 4 actors, 2 sizes, colour and grey scale) were presented in a random order.

3. Results

Table 2 and Figure 4 show the results, these being the frequency of correct responses for the 24 target images in each combination of luminance, target colour and lamp type. Analyses of the data distributions using a range of statistical and graphical measures did not suggest that they were drawn from a normally distributed population and thus statistical analysis was carried out using non-parametric tests. When performing repeated tests there is a possibility of making a Type I error (capitalising on chance). To account for this, conclusions regarding the significance of differences were drawn by looking at the overall pattern of results and with consideration to a threshold of p = 0.01, a conservative approach to Bonferroni correction which otherwise makes it more difficult for any difference to be concluded as being statistically significant, regardless of whether or not it is true.

Table 2

Median frequency of correct recognition of facial expression. In these data the maximum possible frequency of correct identification is 24

			Median frequency of correct recognition of expression
Simulated distance	Lamp	Target colour	1.0 cd/m²	0.33 cd/m²	0.1 cd/m²	Luminance effect (Friedman’s test)
4 m	HPS	Grey scale	22	22	21	p < 0.01
	HPS	Colour	22.5	22	21.5	n.s
	MH	Grey scale	22.5	22	21.5	p < 0.01
	MH	Colour	22	21.5	22	n.s
15 m	HPS	Grey scale	16	15.5	12	p < 0.01
	HPS	Colour	17	16	12.5	p < 0.01
	MH	Grey scale	17	14	12	p < 0.01
	MH	Colour	16.5	15	14	p < 0.01

Figure 4

Median frequency of correct identification plotted against luminance for the eight combinations of lamp SPD, target colour and equivalent distance.

Figure 4 indicates clearly that face size affected performance, with a lower frequency of correct identification for the 15 m target than for the 4 m target in all conditions. The Wilcoxon signed ranks test confirmed that the effect of distance was statistically significant (p < 0.01) in all 12 combinations of lamp type, luminance and target colour.

The Friedman test suggests the effect of luminance to be statistically significant (p < 0.01) in six cases, but not significant for the remaining two cases, these being tests with the HPS and MH lamps with the coloured target at 4 m. In general, this confirms that higher luminances increase the ability to correctly detect emotion from facial expression, although Figure 4 indicates there may be a ceiling to this effect. Further analysis carried out using the Wilcoxon signed ranks test suggests that for the 15 m targets there is a significant difference between 0.1 and 0.33 cd/m² (p < 0.05) but that the difference between 0.33 and 1.0 cd/m² is not significant. With the 4 m tests the effect of luminance was not suggested to be significant. This confirms the expected plateau-escarpment relationship between size, luminance and performance.⁴⁹ With higher luminance and larger visual size, performance reaches a plateau above which increasing luminance gives diminishing returns in terms of increased probability of correct identification. At lower target luminance and smaller size, a change in light level can significantly affect performance.

The aim of this experiment is to determine whether chromatic information enhances the ability to correctly identify facial expressions. The data in Table 2 and Figure 4 do not suggest any consistent trend: There is little difference between the two types of lamp and between the colour and grey scale targets.

Paired comparisons using the Wilcoxon signed ranks test do not suggest differences between MH and HPS lamps to be statistically significant (p > 0.17) in nine of the 12 cases, but the difference is close to significance in three cases:

grey scale targets, 4 m, 0.1 cd/m² (p = 0.015),

grey scale targets, 15 m, 0.33 cd/m² (p = 0.032)

coloured targets, 15 m, 1.0 cd/m² (p = 0.035).

Similarly, the Wilcoxon test does not suggest the effect of target colour to be statistically significant (p > 0.14) in eight of the 12 cases but is close to significance in four cases:

HPS, 4 m, 0.1 cd/m² (p = 0.022)

HPS, 15 m, 1.0 cd/m² (p = 0.038)

MH, 4 m, 1.0 cd/m² (p = 0.039)

MH, 15 m, 0.33 cd/m² (p = 0.037)

To illustrate the variance in these data, Figure 5 shows the median responses averaged across the four combinations of SPD and target colour, with the upper and lower quartiles for these data.

Figure 5

Median frequency of correct identification plotted against luminance. These data averaged across the four combinations of SPD and target colour. Error bars show the interquartile ranges of correct responses.

4. Discussion

This experiment was carried out to examine the influence of colour on the ability to recognise facial expressions, with variation in both image colour and lamp colour rendition. The data did not suggest either to have a statistically significant effect with no cases reaching a p-value of 0.01. This confirms the conclusion drawn in previous work^43,44 that SPD does not have a significant effect on discrimination of emotion conveyed by facial expression, these studies using similar combinations of lamp type, target size, and presentation time.

This conclusion does not mean that SPD is unimportant for pedestrians. There is evidence⁷ that lighting of higher short-wavelength content improves spatial brightness which is associated with judgements of reassurance²³ and aids the detection of trip hazards in peripheral vision.^8,50,51 Lighting of better colour rendering leads to a more preferred environment¹⁴ and to better ability to identify colours^52–54 which may in turn assist with tasks such as witness descriptions of criminal acts. What this means is that if illuminance were to be decreased, this reduction might be offset by changes to lamp SPD for some tasks, but it would not be offset for recognition of facial expression.

4.1. Caveats

This work is reported to better understand the relationship between lighting and facial expression recognition through an understanding of how performance changes with variation in the parameters of the lighting and the task. To apply these findings with confidence, a better understanding is needed of the significance of variations in experimental design.

The targets used in this experiment were photographs of real faces. They were thus 2D representations of the original 3D target and therefore do not capture the directional effect of real lighting such as variations in shadow. Recognition accuracy is expected to increase when 3D information is available.⁵⁵ While Harries et al.⁵⁶ used clay sculptures, the majority of studies use photographs or similar 2D images. An advantage of using photographs is that the targets are consistent in repeated trials and the duration and order of presentation are easy to set up. Bruce et al.²⁶ used a facial recognition matching task in which the targets were images of a 3D head without colour or hair in profile and three quarter views and the matching stimuli were grey scale photographs of four people, including the target, in full face and three quarter views. Correct recognition was approximately 50%, this being above the chance level (25%) but reported to be ‘far from ceiling’. This may be because the target stimuli lacked hair and colour. The recent availability of 3D printing provides a possible means for generating 3D targets in future work.

In this study, we used six facial expressions – anger, disgust, fear, happiness, neutrality and sadness. Different expressions can be discriminated with different degrees of accuracy, although these differences are not consistent. Neath and Itier³⁵ suggest the lowest accuracy for fear and surprise (51–75%), the highest accuracy for happiness (88–99%) and with angry, disgusted and neutral somewhere in between. Ebner et al.⁴⁵ found happy facial expressions in the FACES database were identified with the highest accuracy (96%), disgust with the lowest accuracy (68%), and with sad, fear, angry and neutral in between. Yang and Fotios⁴⁴ found accuracy of recognising expressions in the FACES database under their best conditions (3.33 cd/m²; 4 m) to be similar to that reported by Ebner et al. Some studies exploring different issues have used only three expressions, happy, angry and neutral.⁵⁷ The question is whether these differences would affect conclusions drawn about the effects of changes in lighting, and in particular the effects of chromatic information. Further research is required to answer this question, in particular, whether discrimination between a subset of these expressions is more relevant for lighting than discrimination between the six standard expressions. We note also that there can be significant interaction between emotions conveyed by facial expression and body posture.³⁶

The current study used only young test participants (aged 18 to 34 years) and thus the possible effect of observer age must be considered. Konar et al.⁵⁸ found a significant difference between older (60–82 years) and younger (17–25 years) observers using a four-alternative matching task, with the old group performing less well than the young group. Older adults show an attentional preference towards happy faces and away from angry ones.⁵⁹ However, in a facial emotion recognition experiment, the difference between older (62–76 years) and younger (18–24) observers does not appear to be as strong, being a statistically significant effect only in a minority of cases.⁶⁰ If this latter approach is considered to better represent the critical pedestrian task, that of judging the likely intent of another person by their facial expression, then the effect of age does not appear to be significant.

The photographs used in this study comprised Caucasian faces. This raises questions as to whether skin colour tones and ethnic/ancestral features matter for expression recognition. Eye tracking has demonstrated that other-race faces are scanned differently to own-race faces.⁶¹ Different skin tones offer different levels of chromatic information, and for faces, colour cues are more important for image segmentation (i.e. edge definition) than identification.²⁴ Skin tone may matter if perceived appearance is considered an important factor and lighting of different SPD is being considered.⁶²

One approach to capturing the effect of skin tone on the segmentation provided by chromatic information is facial contrast (C_F), defined (equation (1)) using an adaptation of Michelson contrast where feature luminance is the mean of the eyes and lips and skin luminance is that surrounding them.⁶³ Measurements of young people (aged <30 years) found that female faces had greater facial contrast than male faces, and that East Asian faces had greater contrast than Caucasian faces (Table 3)⁶³

C_{F} = \frac{featureluminance - skinluminance}{featureluminance + skinluminance}

(1)

Table 3

Mean facial contrast for people aged <30 years⁶³

Ethnicity	Facial Contrast according to gender
Ethnicity		Male	Female
Caucasian	0.113	0.123
East Asian	0.15	0.16

4.2. Illumination direction

Uniformity of illuminance is one of the key parameters of road lighting design but has been the focus of little research in lighting associated with interpersonal judgements. Variation in illuminance means a change in the dominant direction of illumination. Changes in illumination direction matter for face evaluations because it can lead to changes in shading gradients and shapes and location of shadows;⁶⁴ shadows have the potential to affect facial recognition in two ways: they might hinder recognition by masking informative features or by introducing spurious contours, or, alternatively, they might improve recognition by providing information about 3D shape.

Liu et al.⁶⁵ used a sequential matching task in which two images were seen in succession, which may present different views or different people, and participants were asked to state if the two images were of the same or different persons. The targets were laser scans of faces, nine males and nine females, in full face and three quarter views, with features above the hairline and below the chin removed. The first image shown for 1.5 s and the second image for an unlimited time until the response was given. Five lighting directions were simulated, +60, +30, 0, −30 and −60° relative to the horizontal meridian. The results (Figure 6) did not suggest a significant difference other than a lower rate of correct responses at the −60° direction.

Figure 6

Percentage accuracy for a same-different face matching task plotted against lighting direction relative to the horizontal, after Liu et al.⁶⁵

Hill and Bruce⁶⁶ compared two lighting directions, either 45° above or below the horizontal and found the effect of lighting direction depended on view and gender. The targets were scanned images of work colleagues, with no hair or colour information, offering front, three quarter and profile views. The task was to identify the person in each image. For male targets, the full and three quarter views led to significant differences between the top and bottom lighting (p < 0.05), with top lighting giving a higher percentage of correct responses, but there was no effect in profile views. For female targets there were no significant effects of lighting direction.

Johnston et al.⁶⁷ used 72 photographs of students with lighting 80° above, 80° below, or straight ahead but 20° to the side. Participants were asked to state which of the targets were their classmates (36/72) following a 750 ms presentation. For upright images, the error rate was similar for front-lit and top-lit images (less than 10%) but increased to over 30% for bottom lit. Braje⁶⁴ used images rendered to give appearance of a face lit from 45° to the left or right of the face (and 45° above in both cases), and using a same:different sequential matching task concluded that face recognition was found to be sensitive to changes in illumination direction.

When walking along a footpath the illuminance direction is likely to vary from around 15° above horizontal (when located midway between two 6 m tall lamp posts at 30 m spacing) to 90° when standing directly underneath a lamp post. Of these studies, the conditions used by Liu et al.⁶⁵ better resemble pedestrian experience than Hill and Bruce⁶⁶ or Johnston et al.⁶⁷ and this does not suggest a significant effect of illuminance direction. Note, however, that Braje⁶⁴ suggests a significant effect between lighting from 45° to the left or right but of the same vertical angle and that neither of these studies appears to account for the variation in vector/scalar ratio that would accompany a change in illuminance direction when walking between consecutive lamp posts. It is clear that further research is required to better understand the implications of illuminance uniformity. One further outcome of variation in illuminance distribution is glare, for which there is some evidence that the face luminance required for pedestrian visibility increases as equivalent veiling luminance increases.⁶⁸

4.3. Task difficulty

It was suggested that colour information is more likely to have a significant effect on facial recognition when the task is difficult.²⁵ In the current study, task difficulty was increased by using a short observation duration (500 ms). Targets were presented at sizes representing a face at 4 m and 15 m: From previous work,⁴⁴ performance at 4 m would approach a ceiling at all three luminances, whilst at 15 m the small size makes the task more difficult, resulting in lower performance than at 4 m and with this performance increasing with increasing luminance. The results confirm these expectations of difficulty. However, despite an apparently difficult task the current experiment did not find an effect of colour to be significant, either with lamp SPD or target colourfulness.

Yip and Sinha²⁴ increased difficulty by degrading the shape cues to recognition through blurring, thereby reducing image resolution. Participants were required to identify the celebrity presented on the image, and these images were presented in colour and grey scale formats. With the high resolution images, recognition performance with the grey scale images was not significantly different from that with colour images. However, with the low resolution images, the more difficult task, then the colour images permitted significantly better recognition than did grey scale.

The current task was designed to represent the need of a pedestrian to evaluate the intention of another person at a distance of 15 m with a brief 500 ms visual fixation.³² For this task, the current data do not suggest colour to be critical. That Yip and Sinha did reveal an effect of colour might be because theirs was a more difficult task due to the low resolution. This level of difficulty may be beyond the needs that road lighting is required to address.

5. Conclusion

An experiment was carried out to investigate how changes in lamp SPD and target colour affected the ability to evaluate the emotion of a person through facial expression. The results provide evidence that lamp SPD does not significantly affect the ability to interpret emotion from facial expression. If the luminance from road lighting is reduced, as might be done for energy saving, it is not possible to offset the reduction in performance by a change in SPD. However, this conclusion must be considered with caution: The light levels of current road lighting guidance have little empirical basis and a reduction in light level may be acceptable if the starting light level was unnecessarily high; also, we do not know for certain that face observation is the primary means for determination of intent, and there is some evidence that SPD does affect recognition of emotion and intent from body posture.⁴³

Footnotes

Acknowledgments

Images used in these trials were taken, with permission, from the FACES database developed by the Max Planck Institute for Human Development.

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was carried out through funding received from the Engineering and Physical Sciences Research Council (EPSRC) grant number EP/H050817.

References

Fotios

Uttley

Cheal

Hara

. Using eye-tracking to identify pedestrians’ critical visual tasks. Part 1. Dual task approach. Lighting Research and Technology 2015; 47: 133–148.

Fotios

Uttley

Yang

. Using eye-tracking to identify pedestrians’ critical visual tasks. Part 2. Fixation on pedestrians. Lighting Research and Technology 2015; 47: 149–160.

British Standards Institution. BS 5489-3: 1992. Road Lighting - Part 3: Code of Practice for Lighting for Subsidiary Roads and Associated Pedestrian Areas. London: BSI, 1992.

Fotios

Goodman

. Proposed UK guidance for lighting in residential roads. Lighting Research and Technology 2012; 44: 69–83.

British Standards Institution BS 5489-1:2013. Code of Practice for the Design of Road Lighting Part 1: Lighting of Roads and Public Amenity Areas. London: BSI, 2012.

Institution of Lighting Professionals. Professional Lighting Guide PLG03:2012. Lighting for Subsidiary Roads: Using White Light Sources to Balance Energy Efficiency and Visual Amenity. Rugby, UK: ILP.

Fotios

Cheal

. Predicting lamp spectrum effects at mesopic levels. Part 1: Spatial brightness. Lighting Research and Technology 2011; 43: 143–157.

Fotios

Cheal

. Obstacle detection: A pilot study investigating the effects of lamp type, illuminance and age. Lighting Research and Technology 2009; 41: 321–342.

Commission International de l’Éclairage. Recommended System for Visual Performance Based Mesopic Photometry. CIE Publication 191:2010. Vienna: CIE.

10.

Fotios

Levermore

. Chromatic effect on apparent brightness in interior spaces, II: SWS lumens model. Lighting Research and Technology 1998; 30: 103–106.

11.

Brown

Tsujimura

Allen

Wynne

Bedford

Vickery

Vugler

Lucas

. Melanopsin-based brightness discrimination in mice and humans. Current Biology 2012; 22: 1–8.

12.

Besenecker UC, Bullough JD. Spectrum and scene brightness perception: Proceedings of the Experiencing Light, Eindhoven, The Netherlands, November 10–11: 2014: 32–35.

13.

Rea

Radetsky

Bullough

. Toward a model of outdoor lighting scene brightness. Lighting Research and Technology 2011; 43: 7–24.

14.

Fotios

Cheal

. Predicting lamp spectrum effects at mesopic levels. Part 2: Preferred appearance and visual acuity. Lighting Research and Technology 2011; 43: 159–172.

15.

British Standards Institution BS5489-1:2003. Code of Practice for the Design of Road Lighting —Part 1: Lighting of Roads and Public Amenity areas. 2003. London: BSI.

16.

Ekman

Friesen

. Detecting deception from the body or face. Journal of Personality and Social Psychology 1974; 29: 288–298.

17.

Raynham

Saksvikrønning

. White light and facial recognition. The Lighting Journal 2003; 68: 29–33.

18.

Knight

. Field surveys investigating the effect of lamp spectrum on the perception of safety and comfort at night. Lighting Research and Technology 2010; 42: 313–330.

19.

Knight C, Van Kemenade J. Wahrnehmung und Wertschätzung der Endkunden in Bezug auf die öffentliche Beleuchtung (Effect of outdoor lighting on perception and appreciation of end-users). Licht, Sep 10–13: 2006: Berne, Switzerland.

20.

Yao

Sun

Lin

. Research on facial recognition and color identification under CMH and HPS lamps for road lighting. Leukos 2009; 6: 169–178.

21.

Alferdinck JWAM, Hogervorst MA, van Eijk AMJ, Kusmierczyk JT. Mesopic vision and public lighting – A literature review and a face recognition experiment. TNO-DV C435. Soesterberg, The Netherlands. 2010.

22.

Boyce

Rea

. Security lighting: effects of illuminance and light source on the capabilities of guards and intruders. Lighting Research and Technology 1990; 22: 57–79.

23.

Rea

Bullough

Akashi

. Several views of metal halide and high pressure sodium lighting for outdoor applications. Lighting Research and Technology 2009; 41: 297–314.

24.

Yip

Sinha

. Contribution of color to face recognition. Perception 2002; 31: 995–1003.

25.

Lin

Fotios

. Investigating methods for measuring facial recognition under different road lighting conditions. Lighting Research and Technology 2015; 47: 221–235.

26.

Bruce

Healey

Burton

Doyle

Coombes

Linney

. Recognising facial surfaces. Perception 1991; 20: 755–769.

27.

Dong

Fotios

Lin

. The influence of observation duration and procedure on luminance required for recognition of pedestrians’ faces. Lighting Research and Technology 2015; 47: 693–704.

28.

Kleinke

. Gaze and eye contact: A research review. Psychological Bulletin 1986; 100: 78–100.

29.

Caminada

van Bommel

. New lighting considerations for residential areas. International Lighting Review 1980; 3: 69–75.

30.

Hall

. The Hidden Dimension, New York: Anchor Books, 1969, pp. 123–124.

31.

Yang

Shafai

Oruc

. Size determines whether specialized expert processes are engaged for recognition of faces. Journal of Vision 2014; 14: 1–12.

32.

Fotios

Yang

Uttley

. Observing other pedestrians: Investigating the typical distance and duration of fixation. Lighting Research and Technology 2015; 47: 548–564.

33.

Townshend

The role of public lighting. In: Oc

Tiesdell

(eds). Safer City Centres: Reviving the Public Realm, London: Paul Chapman Publishing Ltd., 1997, pp. 119–129.

34.

Loftus

Harley

. Why is it easier to identify someone close than far away? Psychonomic Bulletin and Review 2005; 12: 43–65.

35.

Neath

Itier

. Facial expression discrimination varies with presentation time but not with fixation on features: A backward masking study using eye-tracking. Cognition and Emotion 2014; 28: 115–131.

36.

van den Stock

de Gelder

. Face identity matching is influenced by emotions conveyed by face and body. Frontiers in Human Neuroscience 2014; 8: 53–53.

37.

Willis

Todorov

. First impressions: Making your mind up after a 100-ms exposure to a face. Psychological Science 2006; 17: 592–598.

38.

Jovancevic-Misic

Hayhoe

. Adaptive gaze control in natural environments. The Journal of Neuroscience 2009; 29: 6234–6238.

39.

Carey

. Becoming a face expert. Philosophical Transactions of the Royal Society of London 1992; 335: 95–103.

40.

Hsiao

Cottrell

. Two fixations suffice in face recognition. Psychological Science 2008; 19: 998–1006.

41.

Elman

Schulte

Bukoff

. Effects of facial expression and stare duration on walking speed: Two field experiments. Environmental Psychology and Nonverbal Behaviour 1977; 2: 93–99.

42.

Willis

Palermo

Burke

. Judging approachability on the face of it: The influence of face and body expressions on the perception of approachability. Emotion 2011; 11: 514–523.

43.

Fotios

Yang

Cheal

. Effects of outdoor lighting on judgements of emotion and gaze direction. Lighting Research and Technology 2015; 47: 301–315.

44.

Yang B, Fotios S. Lighting and recognition of emotion conveyed by facial expressions. Lighting Research and Technology. doi:10.1177/1477153514547753.

45.

Ebner

Riediger

Lindenberger

. FACES—A database of facial expressions in young, middle-aged, and older women and men: Development and validation. Behavior Research Methods 2010; 42: 351–362.

46.

de Gelder

van den Stock

J.V

. The bodily expressive action stimulus test (BEAST). Construction and validation of a stimulus basis for measuring perception of whole body expression of emotions. Frontiers in Psychology 2011; 2: 1–6.

47.

Fotios

Cheal

Boyce

. Light source spectrum, brightness perception and visual performance in pedestrian environments: A review. Lighting Research and Technology 2005; 37: 271–294.

48.

Field

. Discovering Statistics using SPSS, Sage Publications: London, 2005.

49.

Boyce PR, Rea MS. Plateau and escarpment: the shape of visual performance: Proceedings of the 21st session of the CIE, Venice, CIE Publication 71, Vienna: CIE, 1987.

50.

Fotios

Cheal

. Using obstacle detection to identify appropriate illuminances for lighting in residential roads. Lighting Research and Technology 2013; 45: 362–376.

51.

Uttley J, Fotios S, Cheal C. Effect of illuminance and spectrum on peripheral obstacle detection by pedestrians. Lighting Research and Technology. doi:10.1177/1477153515602954.

52.

Fotios

Cheal

. Lighting for subsidiary streets: investigation of lamps of different SPD. Part 1 – visual performance. Lighting Research and Technology 2007; 39: 215–232.

53.

Boyce

Bruno

. An evaluation of high pressure sodium and metal halide light sources for parking lot lighting. Journal of the Illuminating Engineering Society 1999; 28: 16–32.

54.

Chen L. The accuracy of color naming under different light sources at low photopic to low mesopic conditions. MSc thesis. Troy, New York: Rensselaer Polytechnic Institute, 1998.

55.

Chelnokova

Laeng

. Three-dimensional information in face recognition: An eye tracking study. Journal of Vision 2011; 11: 1–15.

56.

Harries

Perrett

Lavender

. Preferential inspection of views of 3-D model heads. Perception 1991; 20: 669–680.

57.

Kanan

Bseiso

DNF

Ray

Hsiao

Cottrell

. Humans have idiosyncratic and task-specific scanpaths for judging faces. Vision Research 2015; 108: 67–76.

58.

Konar

Bennett

Sekuler

. Effects of aging on face identification and holistic face processing. Vision Research 2013; 88: 38–46.

59.

Isaacowitz

Wadlinger

Goren

Wilson

. Selective preference in visual fixation away from negative images in old age? An eye tracking study. Psychology and Aging 2006; 21: 40–48.

60.

Mienaltowski

Johnson

Wittman

Wilson

A-T

Sturycz

Norman

. The visual discrimination of facial expressions by younger and older adults. Vision Research 2013; 81: 12–17.

61.

Wang

Quinn

Lee

. Both children and adults scan faces of our own and other races differently. Vision Research 2014; 102: 1–10.

62.

Quellman

Boyce

. The light source colour preferences of people of different skin tones. Journal of the Illuminating Engineering Society 2002; 31: 109–118.

63.

Russell

. A sex difference in facial contrast and its exaggeration by cosmetics. Perception 2009; 38: 1211–1219.

64.

Braje

. Illumination encoding in face recognition: effect of position shift. Journal of Vision 2003; 3: 161–170.

65.

Liu

Collin

Burton

Chaudhuri

. Lighting direction affects recognition of untextured faces in photographic positive and negative. Vision Research 1999; 39: 4003–4009.

66.

Hill

Bruce

. Effects of lighting on the perception of facial surfaces. Journal of Experimental Psychology 1996; 22: 986–1004.

67.

Johnston

Hill

Carman

. Recognising faces: Effects of lighting direction, inversion and brightness reversal. Perception 1992; 21: 365–375.

68.

Kohko

Kawakami

Nakamura

. A study on effects of veiling luminance on pedestrian visibility. Journal of Light and Visual Environment 2008; 32: 315–321.