The Lifetime of Salience Extends Beyond the Initial Saccade

Abstract

Several models of selection in search predict that saccades are biased toward conspicuous objects (also referred to as salient objects). Indeed, it has been demonstrated that initial saccades are biased toward the most conspicuous candidate. However, in a recent study, no such bias was found for the second saccade, and it was concluded that the attraction of conspicuous elements is limited to only short-latency initial saccades. This conclusion is based on only a single feature manipulation (orientation contrast) and conflicts with the prediction of influential salience models. Here, we investigate whether this result can be generalized beyond the domain of orientation. In displays containing three luminance annuli (Experiment 1), we find a considerable bias toward the most conspicuous candidate for the second saccade. In Experiment 1, the target could not be discriminated peripherally. When we made the target peripherally discriminable, the second saccade was no longer biased toward the more conspicuous candidate (Experiment 2). Thus, conspicuity plays a role in saccadic selection beyond the initial saccade. Whether second saccades are biased toward conspicuous objects appears to depend on the type of feature contrast underlying the conspicuity and the peripheral discriminability of target properties.

Introduction

When observers search for an object, the limited resolution of the visual periphery and peripheral crowding typically requires them to make saccadic eye movements (e.g., Vlaskamp, Over, & Hooge, 2005). Considering that the locations selected during search will crucially determine the speed at which a target is found, an important question in visual search is what factors determine saccadic selection. Studies on what is selected have revealed both biases toward objects that resemble the target object (De Vries, Hooge, Wiering, & Verstraten, 2011b; Hooge & Erkelens, 1999; Luria & Strauss, 1975) as well as biases toward conspicuous objects (also referred to as salient) ¹ that do not necessarily resemble the target (e.g., Shen & Pare, 2006; Zelinsky, 1996). Saccades toward objects resembling the target are typically referred to as goal-driven saccades, while those attracted toward conspicuous locations are referred to as stimulus-driven saccades. The latter, stimulus-driven saccades, form the basis of the influential salience model (Itti & Koch, 2000; Koch & Ullman, 1987). The salience model predicts fixation locations based on local feature contrast, that is, locations will be fixated in order of their relative conspicuity, starting with the most conspicuous. Recently, however, questions have arisen whether the bias toward more conspicuous locations is directly related to their conspicuity. For instance, it has been argued that a preference for objects in general rather than for conspicuous locations underlies the bias toward areas with higher feature contrast (e.g., Einhäuser, Spain, & Perona, 2008). As locations containing objects will typically be associated with contrasts (due to the edges associated with objects), this could be a confounding factor explaining the predictive power of the salience model.

Natural images are frequently used to study the influence of conspicuous locations on the saccadic landing points. Even though this allows for observing search behavior in scenes similar to those from daily life, the high number of co-varying factors in these images makes it difficult to determine what specific aspects drive selection. To determine such relations, artificial displays can be insightful, as the relation between a specific image manipulation and saccade selection can be observed directly. Studies using artificial displays have found that saccades frequently aim for conspicuous elements (e.g., De Vries, Hooge, Wiering, & Verstraten, 2011a; Donk & Zoest, 2008). Typically, this bias is found to be strongest for short saccade latencies, but disappears for the longer latencies. However, studies using artificial displays for investigating conspicuity-driven biases have almost exclusively focused on initial saccade destinations contingent on stimulus onset. Whether conspicuity plays a role in oculomotor selection beyond the initial saccade has still to be investigated in detail. A recent study did focus on the role of conspicuity in selection beyond the initial saccade by evaluating the second saccade destination (Siebold et al., 2011). Using displays containing three different conspicuous locations, Siebold et al. found no influence of relative conspicuity on the destination of the second saccade. That is, no bias toward the most conspicuous candidate (the most conspicuous element, excluding the currently fixated one) was found for the second saccade. On the basis of these findings, the authors concluded that the role of conspicuity in saccadic selection is short lived and limited to only initial saccades at short latencies. Such a conclusion clearly conflicts with the salience model that predicts elements to be fixated in the order of their conspicuity (starting with the most conspicuous).

Since Siebold et al. only used a single conspicuity manipulation (based on orientation deviations in a grid), the interesting question is now whether their result generalizes beyond the single orientation manipulation. This is an important question as salience models have received wide-spread attention and many models of saccadic selection have incorporated conspicuity as one of the factors influencing selection. If merely the direction of initial saccades at short latencies is influenced by conspicuity, the value of incorporating this measure into models of saccadic selection can be considered limited. In this article, we have set out to evaluate whether the conspicuity of a location can play a role in saccadic selection using a different conspicuity manipulation than Siebold et al.’s orientation manipulation. A replication of Siebold et al. in different feature domains would provide strong converging evidence that the role of conspicuity in saccadic selection in search is limited. Alternatively, a bias toward more conspicuous elements on the second saccade in a different feature domain would argue that conspicuity should be considered a factor in search even for saccades beyond the first.

Orientation manipulations similar to the one in Siebold et al. (2011) are used frequently (Donk & Zoest, 2008; Nothdurft, 2000; Wolfe, Friedman-Hill, Stewart, & O’Connell, 1992). However, orientation does not typically capture the eyes as readily as other features. For example, when an element of a deviating color and one with a deviating orientation are placed on the same grid, observers are biased toward the color deviation over the orientation deviation (van Zoest & Donk, 2006). Therefore, the orientation manipulation used might not be the most effective type of feature contrast manipulation for attracting second saccades. Previous studies manipulating the luminance contrast in natural images only found evidence for an early conspicuity bias (e.g., Anderson, Ort, Kruijne, Meeter, & Donk, 2015; Einhäuser, Rutishauser, & Koch, 2008). Nevertheless, the lack of evidence for a bias beyond this point does not necessarily implicate that conspicuity does not play a role. For instance, while the main finding by Anderson et al. is that the bias toward the higher contrast side of the screen decays over the duration of the initial latency, they also find that for subsequent saccades there is a higher chance of switching from a reduced contrast side to a regular contrast side in an image than vice versa. The latter could be the result of a conspicuity bias beyond the initial saccade.

Here, we extend the study of Siebold et al. (2011) by moving beyond the domain of orientation deviations. In Experiment 1, luminance annuli are used to create conspicuous locations. Previewing these results, we do find a considerable bias toward the more conspicuous candidate not only for the first but also for the second saccade. In Experiment 1, the target could not be discriminated peripherally. This precluded any target bias on the second saccade. Therefore, the question remains whether the second saccade is still driven by conspicuity when targets can be discriminated. In Experiment 2, this is addressed by presenting a target that can be discriminated peripherally.

Experiment 1

The purpose of Experiment 1 was to investigate whether the destination of the second saccade can be affected by the conspicuity of elements. Siebold et al. (2011) demonstrated that relative orientation contrast does not influence the destination of the second saccade. To extend this paradigm to a different feature domain, we used luminance variations to create conspicuous locations in Experiment 1. We chose luminance over the often-used color, because color sensitivity is more prone to differences between individual observers (e.g., Mollon, 1992).

In the study by Siebold et al. (2011), the target identity was either defined by a minor detail (a single black pixel defined the target in their Experiment 1) or could be discriminated peripherally (the orientation used to create conspicuous deviations in Experiment 2) In the current experiment, we created a condition that lies in between these two extremes. That is, the properties of the target are chosen such that the limited resolution of the visual periphery does not allow for rapid target discrimination, but once fixated, the high foveal resolution allows for rapid identification of the target.

Methods

Ethics

The research reported in this article involves healthy human participants and does not utilize any invasive techniques, substance administration, or psychological manipulations. Therefore, compliant with Dutch law, this study only required and received approval from our internal faculty board (Faculty’s Advisory Committee under the Medical Research [Human Subjects] Act [WMO Advisory Committee] at the Utrecht University). Furthermore, this research was conducted, and informed consent of each participant obtained, according to the principles expressed in the Declaration of Helsinki.

Stimuli and apparatus

To compare our results with those of Siebold et al. (2011), we used a similar grid also containing three deviating locations. Each display consisted of a rectangular grid (12 × 12) of diagonally oriented (45°) rectangles with a center-to-center distance of 1.95° between neighboring elements (see Figure 1 for an example of the stimulus). Rectangles had a long side of 0.23° and a short side of 0.16°. Luminance of the rectangles was 99.3 cd/m². In this grid, three conspicuous locations are introduced by adding annuli with a Gaussian luminance profile (the Gaussian profile is centered on a imaginary ring with a diameter of 1.75° and has an SD of 0.17°). To introduce conspicuity differences between the conspicuous locations, three different luminance peaks were used for the annuli. Peak luminance of the first annulus was set to 67 cd/m² and it will be referred to as the high-luminance annulus. The second annulus (peak luminance of 61.5 cd/m²) will be referred to as the medium-luminance annulus, and the third (peak luminance of 56 cd/m²) as the low-luminance annulus. The background was mid-gray at a luminance of 50.5 cd/m².

Figure 1.

An example of the stimulus used in Experiment 1. The vertical target can be found in the upper right quadrant of the stimulus within the low-luminance annulus. The medium- and high-luminance annulus can be found in the lower left and lower right quadrant, respectively.

We used the annuli to create displays with locations varying in conspicuity. Adding targets and distractors may reduce the relative conspicuity difference between the three annuli. To limit the influence of target and distractors on conspicuity, the orientation of the small rectangles was used. The target was oriented vertically, while the two distractors were oriented horizontally compared to the diagonal nontargets. Using pilot data, we selected an element size that (a) does not allow for peripheral discrimination of the target but (b) does allow for rapid foveal identification of the potential targets. This was accomplished by estimating the largest element size that did not result in a target bias for the first and second saccade.

The conspicuous locations with either a target or distractor at its center were selected from a set of eight potential grid locations on an imaginary circle at equal distance (7.4°) from initial fixation. To avoid eye movements landing in between the targets, a phenomenon known as the global effect (Findlay, 1982), targets were never placed in adjacent positions. Also, in order to preempt potential intertrial effects, conspicuous locations were never placed in the same position as in the previous trial.

The stimuli were generated using Matlab on an Apple Macintosh G5 and displayed on a linearized LaCie 22′′ CRT monitor at a resolution of 1,600 by 1,200 pixels and at a refresh rate of 75 Hz.

Observers

Six observers participated in this experiment. All observers had normal or corrected to normal vision and ranged in age from 25 to 31 years. Observer JV is an author on this article; other observers were naive as to the goal of the experiment. All observers either worked or studied at the Utrecht University. They participated on a voluntary basis.

Procedure

Observers were instructed to locate the vertical element as fast as possible. They were also informed that the target would be at the center of one of the three Gaussian annuli, but that the brightness of the annuli was not predictive of the location of the target. Each trial started with a central fixation dot, placed on a gray background. Observers started a trial by pressing the space bar. After a stimulus onset asynchrony (randomly selected in a 600–1,200 ms range), the search display was presented. In an attempt to obtain a large variation in saccade latencies, the fixation dot was not removed directly but after a random period ranging from 0 to 400 ms from the onset of the display. Observers indicated finding the target by pressing the 0 key on the numerical keypad. Each observer ran a total of 399 trials, which lasted approximately 30 minutes.

Eye movement analysis

Eye movements were recorded using an SR-Research EyeLink II system at a sampling frequency of 500 Hz. The observer’s head was placed in a chinrest so that the eyes were at a distance of 64 cm from the screen. Images were viewed binocularly, but eye movements were recorded from the left eye only. Eye movement data were collected for off-line analysis. Saccades were detected at a velocity of 20°/s, after which start and endpoint were found by searching back and forth until the velocity was two standard deviations higher than the velocity during fixation (as in Smeets & Hooge, 2003). Saccades with amplitudes smaller than 3° were removed from the analysis. The range is larger compared to typical search experiments because our specific interest lies with movements from annulus to annulus and we wanted to avoid including corrective saccades in the analysis as a result of hypo- or hypermetric saccades.

To ensure only saccades correctly moving from the central fixation to individual annuli, we applied a number of exclusion criteria. Initial saccade latencies were required to lie between 60 and 600 ms (criterion 1). Initial fixations deviating more than 3° of the fixation dot were rejected (criterion 2). And saccade endpoints should fall within 3° of the center of an annulus (criterion 3). In trials including at least one saccade, criterion 1 was violated on 0.6%, criterion 2 was violated on 5.0%, and criterion 3 was violated on 3.7%. For these trials, the conjunctive exclusion percentage was 8.5%. For trials including at least two saccades, the analysis of the second saccade criterion 1 was violated on 0.7%, criterion 2 was violated on 5.0%, and criterion 3 was violated on 5.5% of these trials. The conjunctive set of excluded trials was 10.2%. Note that the conjunctive set is smaller than adding the percentages together as more than one criterion may be violated within a single trial. Moreover, a total of 71.1% of all trials passed both the criteria and included a second saccade.

Results

Destination of the first saccade

As we used a different conspicuity manipulation than in similar previous studies, we first evaluate whether the effect of the manipulation on the initial saccade destination was comparable. Therefore, the data were analyzed using a cumulative link mixed model using the ordinal package for R (Christensen, 2015). Destinations of the first saccade were supplied as ordered categorical responses. In the model, latency was included as a fixed effect and observers were modeled as random offsets. The link function used was the Aranda-Ordaz. The results can be found in Figure 2 where one can see that most saccades aim for the high-luminance annulus followed by the medium-luminance annulus, and finally the low-luminance annulus. As expected, there is a clear effect of latency as the bias for more conspicuous annuli is strongest at the shortest latencies, while it largely disappears for the longer latencies. In line with this pattern in bias, a significant fixed effect for the latency term was found (β = 0.68, z = 5.52, p < .00001). Moreover, based on the model estimates and confidence intervals, we can clearly see that the bias is strongest at the shortest latencies and disappears for longer latency saccades. Figure 2 also includes the proportion of saccades toward the target. The proportion varies slightly around the expected 33%.

Figure 2.

Proportion of saccades toward high-, medium (med)-, and low-luminance annulus as a function of saccade latency in Experiment 1. A cumulative link mixed model was run with the three destinations as dependent categories. The resulting lines are the fits for the high- (solid orange), medium- (dashed green), and low-luminance annulus (dashdot blue), respectively. A separate line fit is introduced to show selection for the target in dotted gray (not part of the cumulative link mixed model). The shaded region around each line represents the bootstrapped 95% confidence intervals of the model estimates. Data points (filled dots for high-luminance annulus, triangles for medium-luminance annulus, diamonds for low-luminance candidate, and x’s for target) are average proportions for saccades with latencies split over the five quintiles in each observer’s latency distributions. These data averages are purely for illustrative purposes and not the source of any analysis.

Destination of the second saccade

To evaluate whether conspicuity affects saccade destinations beyond the initial saccade, we turn to the second saccade destination. A similar procedure was followed as for the first saccade. However, to keep oculomotor selection insightful, the data were first split on the basis of the destination of the first saccade. Hence, the second saccade destination is evaluated for when the first saccade aims for the high-, medium-, and low-luminance annulus, separately. For these circumstances, the destination of the second saccade is analyzed and shown in Figure 3(a) through (c), respectively. For each condition, separately, a cumulative link mixed model with the logit as link function was implemented with the more and less conspicuous candidate as categorical dependent variable, intersaccade interval as fixed effect, and observers as random effect. Note that considering the two possible outcomes, this is similar to a generalized linear mixed effects model. However, for consistency with the aforementioned analysis, we again applied the cumulative mixed model.

Figure 3.

Proportion of second saccades toward the higher luminance and lower luminance candidate as a function of the intersaccade interval between the first and second saccade in Experiment 1. Proportion fits for the cumulative link mixed model are shown for saccades stemming from each destination (high-, medium-, and low-luminance annulus, from left-to-right). The orange solid line represents the fit for the proportion of saccades toward the higher luminance annulus and the blue dash-dotted line the fit for the proportion toward the lower luminance annulus. A separate fit (dotted gray) is included to show the target selection. The shaded region represents the bootstrapped 95% confidence intervals of the model estimates. Data points (dots for higher luminance candidate, diamonds for lower luminance candidate and x’s for target) are average proportions for saccades with latencies split over the five quintiles in each observer’s latency distributions. These data averages are purely for illustrative purposes and not the source of any analysis.

We find a clear bias toward the most conspicuous candidate for the second saccade. For saccades coming from the high-luminance annulus (Figure 3(a)), the medium-luminance annulus is selected above chance compared to the low-luminance annulus as is evident by running the model intercept-only—leaving out intersaccade interval—(β = 0.74, z = 3.23, p < .005). Running the model including the intersaccade interval term, we found that there was no significant effect of intersaccade interval (β = 0.31, z = 1.76, p = .079).

Similarly, for saccades coming from the medium-luminance annulus (Figure 3(b)), a larger proportion of second saccades land on the high-luminance annulus than the low-luminance annulus (intercept-only: β = 0.96, z = 5.29, p < .00001). Here, we do see a significant effect of intersaccade interval, indicating that the attraction of relative conspicuity may rely on intersaccade interval (β = 0.79, z = 3.47, p < .001).

Finally, for the second saccades originating from the low-luminance annulus (Figure 3(c)), only a trend toward the more conspicuous candidate was found (intercept-only: β = 0.20, z = 1.75, p = .08) and, as for saccades stemming from the high-luminance annulus, no significant effect was found of intersaccade interval (β = 0.25, z = 1.13, p = .26). However, compared to the number of saccades from the high- and medium-luminance annulus, the number of saccades stemming from the low-luminance annulus was much smaller. This was to be expected as only few initial saccades aim for the low-luminance annulus. Therefore, vigilance has to be applied in drawing conclusions from the apparent lack of bias in destination. Hence, we do not want to conclude that conspicuity did not influence selection in this particular case.

While we find a bias toward the more conspicuous annulus on the second saccade, it is somewhat unclear how to interpret the finding that the intersaccade interval term is only significant for saccades stemming from the medium-luminance annulus. Potentially, this is because here the effect is the strongest as the conspicuity difference is the greatest (the two candidates for the second saccade comprise the high- and the low-luminance annulus). Nevertheless, because it is not consistent and we did not find this in our planned comparisons, ² we do not want to attach any strong conclusions to the effect of intersaccade interval in only one case.

Examining the proportion of saccades toward the target, we again find that this proportion barely deviates from the chance level (50% for the remaining two candidates). This suggests that also for the second saccade destination, the location of the target did not play a role.

Equidistant trials

The annuli are divided over locations at the same distance from the central fixation dot. However, the distances between the annuli vary. Thus, the two saccade candidates that are left after the first saccade often stand at different distances from the current fixation. To ensure the current conspicuity bias on the second saccade is not merely a result of different distances, we performed a post hoc follow-up analysis using only trials where the distance of the two potential targets after the first saccade was approximately equal (while the number of cases where distance was exactly equal was limited, often the difference in distance was less than 2%, those trials were also included). All trials that satisfy this distance criterion were collapsed, regardless of the conspicuity of the target selected on the initial saccade. Per observer, this analysis included an average of 93 trials. We evaluated whether the second saccade targeted the more or less conspicuous of the two potential targets for the second saccades: Results are plotted in Figure 4. In line with the notion that conspicuity influences the direction of the second saccade we find that observers are consistently biased toward the more conspicuous element: A t-test comparing the proportion of second saccades against 0.5 subscribes the bias toward the more conspicuous element, t(5) = 4.71, p < .01.

Figure 4.

Second saccade destination for equidistant annuli. (a) Proportion of second saccades toward the more conspicuous of the two potential targets after the initial saccade for each observer, individually. For this analysis, all trials of an observer with at least two saccades were collapsed and those where the potential targets after the first saccade stood at equal distance from the currently fixated element were selected. (b) Mean proportion of second saccades toward the more conspicuous of the two potential targets after the initial saccade. Error bars denote standard errors of the mean.

Discussion Experiment 1

The main goal of the current experiment was to investigate whether the results of Siebold et al. (2011) would apply to feature domains outside of orientation. This is clearly not the case: Experiment 1 demonstrates that the second saccade destination can be influenced by relative luminance. Therefore, a complete rejection of the notion that conspicuity is a factor in saccadic selection beyond the first at best only holds in specific cases.

While the results from the current experiment demonstrate that conspicuity can play a role in saccadic selection beyond the initial saccade, one could argue that the bias toward the more conspicuous annulus on the second saccade is actually a bias against the low-luminance annulus: That is, the bias toward the more conspicuous annulus for the second saccade only reaches significance when one of the two candidates for the second saccade is the low-luminance annulus. Naturally, if the low-luminance annulus was too inconspicuous to be reliably detected as a potential target, it would not be surprising that we find a bias away from this element: Observers cannot program a visually guided saccade toward a target that is not detected. To ensure that the luminance of the low-luminance annulus was sufficient to be a saccade target, we have run an additional experiment evaluating the ability of detecting it in the periphery (see supplemental materials, Control Experiment 1). This control demonstrated that the lack of bias toward the low-luminance annulus is not the result of it being undetectable in the periphery.

Experiment 2

The determining factors of visual selection are still subject to an extensive and unresolved debate (e.g., Folk & Remington, 2010; Leber & Egeth, 2006; Theeuwes, 2010). A saccadic bias toward more conspicuous elements is typically referred to as stimulus driven and explained as evidence of bottom-up processes dominating the selection process. On the other hand, eye movements toward the target are described as goal driven and put forth as evidence for top-down input dominating the process. In light of a model where bottom-up and top-down processes compete, it can be argued that Experiment 1 provides evidence for bottom-up processes dominating selection on the second saccade. However, in Experiment 1, the target could not be discriminated peripherally. Therefore, considering the notion of a competition between bottom-up and top-down processes, it is not surprising that the result can be described in terms of bottom-up processes dominating the outcome of selection. The question then arises whether this also holds when the location of the target can be discriminated peripherally.

In Experiment 2 of Siebold et al. (2011), the target could be discriminated peripherally and a significant bias toward the target was found for the second saccade, leading to the conclusion that oculomotor selection beyond the initial saccade is primarily under top-down control. As the orientation manipulation by Siebold et al. never resulted in a bias toward more conspicuous candidates for the second saccade, it is still an open question whether the conspicuity bias found in the current Experiment 1 will disappear and selection will be goal driven when the target can be discriminated peripherally. We address this question by increasing the target size and evaluating whether the second saccade is still biased toward the more conspicuous candidates or whether it will aim for the target.

Methods

The experimental setup was identical to the setup in Experiment 1 with the exception of the a few differences that are mentioned below.

Observers

Eight observers, selected from the same pool as in Experiment 1, participated in this experiment. All observers had normal or corrected to normal vision and ranged in age from 26 to 47 years. Observers SS, IH, and JV are authors of this article, the other observers were naive as to the goal of the experiment.

Stimulus material

The display properties were identical to those of Experiment 1 except for the size of the filled rectangles. To allow for peripheral discrimination of the target, the size of all rectangles on the grid was increased. As we desired to limit the influence of the deviating rectangle on the conspicuity of the annuli, the size was only increased moderately: rectangles now subtend 0.5° by 0.22°.

This change of size requires an important addition to what is meant here by peripheral discriminability. Peripheral discrimination thresholds are time dependent and will decrease over time (Geisler & Chou, 1995). Increasing element size considerably would ensure that the target can be discriminated peripherally even when available processing time is limited. However, because we did not want to manipulate the ability of the annuli to attract stimulus-driven saccades, increasing the size of the target considerably was not an option. That is, we wanted to avoid adapting the bottom-up attraction of the annuli. Therefore, when we state the target is peripherally discriminable we mean that, given sufficient time, it could be discriminated peripherally, but at the same time we should add that it is possible that on some of the trials processing time was insufficient. Nevertheless, to contrast this larger target to the peripherally indiscriminable target in Experiment 1, we will refer to the target in Experiment 2 as peripherally discriminable.

Eye movement analysis

Eye movement analysis parameters were the same as described in the methods section of Experiment 1. Also, the same criteria were used to exclude trials. Saccade latencies were required to lie between 60 and 600 ms (criterion 1). Initial fixations deviating more than 3° of the fixation dot were rejected (criterion 2). And saccade endpoints should fall within 3° of the center of an annulus (criterion 3). In trials including at least one saccade, criterion 1 was violated on 0.5%, 2 was violated on 1.3%, and 3 was violated on 9.3%. For these trials, the conjunctive exclusion percentage was 10.8%. For trials including at least two saccades, the analysis of the second saccade criterion 1 was violated on 0.2%, criterion 2 was violated on 1.6%, and criterion 3 was violated on 16.4% of these trials. The conjunctive set of excluded trials was 17.6%. Note that the conjunctive set is smaller than adding the percentages together as more than one criterion may be violated within a single trial. Moreover, a total of 55% of the trials passed both the criteria and included a second saccade.

The excluded trial percentage for the second saccade is somewhat high. On the one hand, this number is skewed by a single observer that had 32% of trials excluded on this basis. On the other hand, it is likely that the peripheral detectability of the target led to more averaging saccades and interrupted saccades. Comparing the number of excluded trials over the conditions we find that the largest number of trials excluded in the condition where the target was located in the high-luminance annulus (22%), followed by medium-luminance annulus (16%) and low-luminance annulus (15%). As the detection of the target in the high-luminance annulus may have required slightly more processing time, this may have led to more global effect and interrupted saccades (for a discussion on interrupted saccades see McPeek, Skavenski, & Nakayama, 2000).

Results

Destination of the first saccade

Even though we kept the increase in element size limited, there is a possibility that the increase may have affected the potential of the annuli to attract stimulus-driven saccades. Therefore, it is important to verify that despite the increase in element size, the relative attraction of the three annuli for the initial short-latency saccades is still similar as in Experiment 1. In Figure 5, we evaluated selection as a function of latency. By comparing Figures 2 to 5, it becomes clear that the relative attraction of the annuli is still similar to that of Experiment 1 for the short-latency saccades. The only difference is that the bias toward more conspicuous annuli appears slightly more pronounced, at the short latencies, but this rapidly disappears for longer latency saccades. This appears to be due to the ability to discriminate the target, which likely rises for the longer lasting latencies as there is more time for processing. Again, the cumulative link mixed model with the Aranda-Ordaz link function was applied to compare the destination of the initial saccade targeting the three annuli and any effects of latency as a predictor. As in Experiment 1, we find a significant effect of latency as a predictor underlining the importance of latency in this effect (β = 0.54, z = 10.65, p < .0001). Also, again the model and confidence interval estimates clearly show the bias is strongest for the shortest latency saccades, thus resembling the results from Experiment 1.

Figure 5.

Proportion of saccades toward high-, medium (med)-, and low-luminance annulus as a function of saccade latency in Experiment 2. A cumulative link mixed model was run with the three destinations as dependent categories. The resulting lines are the fits for the high- (solid orange), medium-, (dashed green), and low-luminance annulus (dashdot blue), respectively. A separate line fit is introduced to show selection for the target in dotted gray (not part of the cumulative link mixed model). The shaded region around each line represents the bootstrapped 95% confidence intervals of the model estimates. Data points (filled dots for high-luminance annulus, triangles for medium-luminance annulus, diamonds for low-luminance candidate and x’s for target) are average proportions for saccades with latencies split over the five quintiles in each observer’s latency distributions. These data averages are purely for illustrative purposes and not the source of any analysis.

Destination of the second saccade

To evaluate the effect of peripheral discriminability of the target on the saccade destinations beyond the initial saccade, the second saccade destination is plotted as a function of intersaccade interval. Similar to Experiment 1, the destination of the second saccade is evaluated for when the initial saccade aims for the high-luminance annulus, medium-luminance annulus, and low-luminance annulus, separately. The results are plotted in Figure 6. As in Experiment 1, we ran three separate cumulative link mixed models with a logit as link function. In contrast with Experiment 1, we find no significant increase above chance level for either destination (intercept-only fits return for saccades from the high-luminance annulus: β = −0.11, z = 1.56, p = .12; From the medium-luminance annulus: β = −0.07, z = 0.60, p = .55; From the low-luminance annulus, β = −0.13, z = 1.09, p = .28. Also, in all cases, intersaccade interval as a predictor did not show any significant slope (from the high-luminance annulus, β = −0.04, z = 0.25, p = .80; From the medium-luminance annulus, β = −0.03, z = 0.15, p = .88; From the low-luminance annulus, β = 0.14, z = 0.44, p = .66).

Figure 6.

Proportion of second saccades toward the higher luminance and lower luminance candidate as a function of the intersaccade interval between the first and second saccade in Experiment 2. Proportion fits for the cumulative link mixed model are shown for saccades stemming from each destination (high-, medium-, and low-luminance annulus, from left-to-right). The orange solid line represents the fit for the proportion of saccades toward the higher luminance annulus and the blue dash-dotted line the fit for the proportion toward the lower luminance annulus. A separate fit (dotted gray) is included to show the target selection. The shaded region represents the bootstrapped 95% confidence intervals of the model estimates. Data points (dots for higher luminance candidate, diamonds for lower luminance candidate, and x’s for target) are average proportions for saccades with latencies split over the five quintiles in each observer’s latency distributions. These data averages are purely for illustrative purposes and not the source of any analysis.

As we are now interested in whether a target bias is found, we also compared whether the number of saccades directed at the target was significantly different from chance using separate t-tests. For saccades stemming from the high-luminance annulus, a significant bias toward the target was found, t(7) = 5.5422, p < .005). To account for three multiple comparisons, all p values were adjusted upwards applying Bonferroni correction. For saccades stemming from the medium-luminance annulus, a significant bias toward the target was found, t(7) = 5.4937, p < .005. Similarly, for saccades stemming from the low-luminance annulus, a significant bias toward the target was found, t(7) = 6.2499, p < .005. Thus, in contrast to Experiment 1, relative conspicuity has little effect on the destination of the second saccade. Rather, the main determining factor driving the second saccade seems to be the target location.

Discussion Experiment 2

Although in Experiment 1, the second saccade was biased toward the most-conspicuous candidate, in Experiment 2 such a bias is no longer found. Instead, a strong bias toward the target is found. It appears that once the target can be discriminated peripherally, the bias toward the target is stronger than the relative conspicuity bias. Therefore, whether the second saccade is characterized as stimulus driven or goal driven is dependent on the peripheral discriminability of the target. From a perspective of a competition between bottom-up and top-down processes underlying saccadic selection, in this case one could state that top-down processes dominated selection on the second saccade in the current experiment. Unlike the results under the conditions of Experiment 1, these results are in line with the results of Siebold et al.’s Experiment 2 (Siebold et al., 2011).

General Discussion

Salience models predict locations in a scene to be fixated in the order of their relative conspicuity (e.g., Itti & Koch, 2001). A recent study by Siebold et al. (2011) provides data that are in conflict with this prediction, as they find no bias toward the most conspicuous candidate for the second saccade. However, as only a single conspicuity manipulation was used it was unclear whether this finding actually generalizes to all such conspicuity manipulations. In Experiment 1, we find that the second saccade does more frequently target the most conspicuous candidate for the second saccade. To our knowledge, our study is the first report to show elements being selected in order of their relative conspicuity beyond the initial saccade. Importantly, it demonstrates that conspicuous objects can attract saccades beyond the first.

This conclusion not only deviates from that of the Siebold et al. (2011) study but also appears to conflict with recent studies arguing that visual conspicuity does not drive eye movements in visual search (e.g., Henderson, Brockmole, Castelhano, & Mack, 2007; Stirk & Underwood, 2007). In the study by Henderson et al., it was demonstrated that while observers do more frequently fixate locations associated with high relative contrasts and edge density, fixated areas are also rated higher on semantic informativeness. Thus, the correlation between fixation and conspicuity is not sufficient evidence to claim a conspicuity bias. The findings also conflict with those who argue that the conspicuity bias is confounded by an object bias: When observers are biased toward objects, this may appear as a conspicuity bias, because objects are typically associated with greater conspicuity due to their contrasts and edges. For instance, it has been argued that interesting objects guide attention rather than conspicuous features (Einhäuser et al., 2008). However, a comparison using several newer implementations of salience models on the same data sets demonstrated that salience models can outperform object-based predictions (Borji, Sihite, & Itti, 2013; see also Einhäuser, 2013 for a reply). Thus, the role of conspicuity in selection of saccade targets during search is still a topic of debate. The current results demonstrate that conspicuity can bias saccades, even beyond the first. However, we do not want to argue that conspicuity by itself can explain all saccade landing points in visual search.

Many factors are likely at play in determining saccadic selection in search. A bias toward more conspicuous objects, as in Experiment 1, is often explained as a result of bottom-up processing. However, other potential mechanisms have yet to be excluded. For instance, because observers are required to select a target among several singletons, a bias toward the more conspicuous could also be the result of observers adopting a singleton detection mode (Bacon & Egeth, 1994). That is, even when a target is less conspicuous than a distractor it is often still a singleton and, therefore, a strategy of targeting elements based on their singleton status, rather than their exact identity, may be applied. Important here is that this strategy seems to be an automatic one. The general prevalence of this bias on the short-latency saccade could potentially be explained by the limited processing time, driving selection toward elements that are processed first (e.g., De Vries et al., 2011a). Importantly, the aim of this article is not to uncover why the bias occurs, but rather whether it is present for the second saccade. Moreover, because the bias toward more conspicuous elements on the initial saccades has been found to generalize to natural scenes (Anderson et al., 2015), it is well possible that the current bias on the second saccade also occurs in search in natural scenes.

Studies using natural images have demonstrated that the conspicuity bias primarily affects early saccades in free viewing (e.g., Parkhurst, Law, & Niebur, 2002; Underwood, Foulsham, van Loon, Humphreys, & Bloyce, 2006). When one focuses only on eye movements in search, the bias, however, appears to be more limited, with studies finding little evidence for a conspicuity bias beyond the first saccade. For instance, Einhäuser et al. (2008) manipulated the contrast of natural images in a gradient across the screen and only found a conspicuity bias in some cases: When searching for an unspecified target (oddity target, Experiment 3), a bias was only found for the first eye movement. In contrast, initial eye movements were not biased toward the higher contrast side when the target was a predefined bull’s-eye (Experiment 1). Similarly, Anderson et al. (2015) created images with a higher and a lower contrast side. Again the conspicuity bias does not appear to extend beyond the first saccade: Only initial short-latency saccades show a bias toward the higher contrast side. Nevertheless, an indication that the conspicuity bias may extend beyond the first fixation stems from the fact that the probability of switching the gaze position from a reduced to a regular contrast side was higher than the other way around. Moreover, the lack of a consistent bias in the direction of higher contrast in Einhäuser et al. (2008) could potentially be explained by the fact that multiple saccade destinations may have been decided upon during the initial fixation (e.g., three saccades may be planned during the initial fixation; De Vries, Hooge, & Verstraten, 2014). Thus, while it is clear that the conspicuity bias is more pronounced early in search of natural images, it is still unclear how many saccades are affected.

In contrast to Experiment 1, in Experiment 2, the target could be discriminated peripherally. Under these conditions, no bias toward the most conspicuous candidate is found for the second saccade. This means that while the results from Experiment 1 allow us to conclude that relative conspicuity can be a determining factor in saccadic selection, the extent to which conspicuity influences search (at the least) depends on the ability to peripherally discriminate target properties. Here, we see an important distinction: While in some cases targets by virtue of their conspicuity can be peripherally discriminated without saccades, in many cases identifying a target requires saccades. As multiple saccades are typically required, we refer to this latter category of search as saccadic search. Previous studies have shown that search in natural scenes can require a considerable number of saccades (e.g., Over, Hooge, Vlaskamp, & Erkelens, 2007). The task in Experiment 1 has been explicitly designed to create a situation where saccades are required. Therefore, the results likely extend the selection in many searches in daily life.

While in studies focusing on the initial saccade the bias toward the most conspicuous element is typically only found for short-latency saccades, this does not appear to be the case for the conspicuity bias found for the second saccade. If each saccade was based on a decision made during the directly preceding fixation, it is difficult to explain this discrepancy between selection on the first and the second saccade. However, it is known that second saccades can be prepared concurrently to the first (Becker & Jürgens, 1979; McPeek et al., 2000). Also, we have recently demonstrated that even in a task requiring saccades for locating the target (as is also the case in Experiment 1), up to three saccade destinations can be determined ahead of time (De Vries et al., 2014). When the second saccade destination is already determined during the initial fixation, it is not surprising that we do not find a clear effect of the intersaccade interval that preceded the second saccade

While it is tempting to conclude that the difference between our results and those of Siebold et al. (2011) is due to having conspicuous locations based on deviations in a different feature domain, it should be noted that several factors could be at play. For instance, relative conspicuity differences may play a role. We tried to create similar relative conspicuity differences between our conspicuous elements as those used in the study by Siebold et al., but some differences are unavoidable (Control Experiment 1, e.g., shows that absolute conspicuity of the luminance annuli is higher than that of the orientation deviations). Finally, not just conspicuity varies between the two types of conspicuous locations but also the nature of their conspicuity. While the currently used luminance annuli can be detected purely based on their distinction from the uniform background, an orientation deviation by definition has to be distinguished from neighboring elements. To distinguish an orientation deviation from its neighbors, information on both of their identities is required. In the current experiments, the identity of neighboring elements is not important for the detection of the luminance annuli. It is likely that the detection of the presence of a deviating element occurs earlier when its presence can be deducted purely by segmentation from the background compared with when its conspicuity has to be deduced based on differences between neighboring elements. The faster detection of an element may influence its role in determining of future saccade destinations. Future research is required to reveal the exact contributions.

In conclusion, we find that eye movements beyond the first can be biased by relative conspicuity. When the target of the search cannot be discriminated peripherally and saccades are required for locating the target, we find second saccades to be biased toward the more conspicuous candidate. As this contrasts to a previous finding, it appears that whether second saccades are biased toward more conspicuous objects depends on the type of feature contrast underlying conspicuity and the peripheral discriminability of target properties. While this finding stands in apparent contrast to studies arguing a limited role for conspicuity in search in natural scenes, it is important to note that many factors drive eye movements in search. As such, in natural scenes, the exact role of conspicuity in selection is likely difficult to detect.