A Step to VAR: The Vision Science of Offside Calls by Video Assistant Referees

Abstract

The Video Assistant Referee (VAR) system has had a major impact on decision-making in professional association football. However, offside decisions remain a major area of dispute and debate, with over 34 goals ruled out in the first season of VAR in the Premier League. Evidence in vision science points toward two problems with the application of the offside law in VAR, due to their use of a live TV video feed in reviews. First, due to physical and perceptual limits on spatial resolution, there is a significant probability that the spatial positions of the ball and players as judged by VAR will be several centimetres to one side of their true positions. Second, the 50 Hz TV update rate means that judgements of the time-of-contact between player and ball will on average be 10 ms too late, which translates into an increased likelihood of offside calls in fast-moving play. Suggestions are made for how to compensate for these problems during decision-making.

The video assistant referee (VAR) system was introduced in the Premier League at the beginning of the ill-fated 2019–2020 football season. It has made a major impact on the accuracy of decision-making (Mather & Breivik, 2020; Spitz et al., 2020), but some decisions have been disputed or mocked. Arguably the most controversial decisions made by VAR reviews have been offside decisions. According to the ESPN website (July 2020), 34 goals were ruled out by VAR reviews in its first season of use, and the term armpit offside entered the football lexicon. It was first used after the Premier League tweeted in November 2019 that Roberto Firmino’s armpit was in an offside position during a match. The Liverpool manager, Jurgen Klopp (Firmino plays for Liverpool FC) said afterwards:

When we talk about serious moments, very important moments in football, it's not right to sit here and everyone wants to laugh about it, it is too serious. Managers get sacked for losing football games. They just have to clarify it.

Why are offside decisions so contentious and apparently error-prone? The situation can be understood on the basis of well-established principles and findings is sensory science. In brief, there are two parts to the offside rule:

Space—a player is in an offside position if certain parts of their body are closer to the opposing goal than certain body parts of the last two opponents (even armpits, apparently).

Time—a player is penalised if they are in an offside position at the moment that the ball is played by a team mate.

According to the Premier League website (February 2020), the space rule is applied in VAR reviews by projecting a one-pixel wide line onto the TV image of the pitch at the “exact” positions “of the parts of the body of the attacking and defending players that can be used to score goals.”

The time rule is applied by finding the still-frame in the TV video sequence in which the relevant player first makes contact with the ball: “The broadcast cameras operate with 50 frames per second, so the point of contact with the ball is one of those frames inside the 50 per second.”

Evidence in vision science points toward two problems with this process. Let’s start with the space rule. Under reasonable assumptions (1,920 × 1,080 pixel video, view-angle covering about 50 m), each pixel in the TV image covers an on-field distance of approximately 2.5 cm.

In actual still-frame footage, the images of moving players and balls are smeared over a distance of approximately eight pixels (see Figure 1), so their true positions lie at the centre of roughly Gaussian blur functions covering a distance of perhaps ± 10 cm. There is a significant probability that the position judged by the VAR will be several centimetres to one side of the true position (Mather & Morgan, 1986). Clear offsides beyond 10 cm on either side of the “level” point should not be affected by this spatial uncertainty and bias, and are highly likely to be called correctly. But close offsides are common in Premier League matches in which elite, highly skilled players operate at the finest margins, and these calls will be subject to spatial errors.

Figure 1.

Two close-up frames from a video sequence of a ball being kicked.

The problem with application of the time rule to video still-frames is familiar from psychophysical experiments that measure reaction times. The participant is given a response button and asked to press it as soon as possible after stimulus onset. A computer polls the status of the button at regular intervals to check whether it has been pressed. If the polling interval is 20 ms, measured reaction times will be at multiples of 20 ms. If the participant actually presses the button at, say, 22 ms, the press will be detected at the next polling point, namely, 40 ms. Similarly, a button-press at 38 ms would also be recorded as 40 ms. The polling interval therefore introduces a slight bias in measured reaction times: On average, recorded reaction times will be longer than the true reaction times by half the polling interval (Ulrich & Giray, 1989).

This situation can be applied directly to offside decisions in VAR. The VAR reviewer receives snapshots of the play at 20 ms intervals. When establishing the first point of contact with the ball, there will be two successive frames in the video feed similar to those shown in Figure 1. In the first frame, the foot is approaching the ball, and in the second frame, the foot has made contact with the ball. The VAR review would select the second frame as the best estimate of the moment that the ball was played.

In reality, the moment of contact between ball and foot could have occurred at any time during the 20 ms interval between the contact frame and the frame before and will, on average, have occurred 10 ms before the contact frame. Although 10 ms is very short, it can make a significant difference to decisions about fast-moving play. If the attacking player is moving forward at 20 kph or 5.56 m/s, then their position in the selected contact frame will, on average, be 5.6 cm further forward than their position at the actual point of contact. A roughly two-pixel shift in the player’s position should be large enough to be just detectable in the video frame, and alter the position of the projected line. Consequently, the attacking player is more likely to be called offside by the VAR.

So the use of frame-based video introduces a bias in decision-making in favour of more frequent offside calls. What is the solution—a return to traditional methods involving on-field officials? Typically, one official calls “now” at the moment of contact with the ball, and another (in radio contact with the first) judges the spatial alignment of players at the instant they hear “now.” This process is clearly even more bias-prone; reaction times to motion onset are typically several hundred milliseconds (Gilis et al., 2009; Porciatti et al., 1999).

It is not possible to completely eliminate these biases, but the application of the offside rule in VAR could be changed to acknowledge the problem by projecting a “zone of uncertainty” onto the pitch rather than a single-pixel line, to reflect the fact that within this zone a reliable offside decision cannot be made. Higher frame-rate cameras would also reduce the width of the zone. In a separate development, Arsene Wenger, FIFA’s head of global development, proposed a change to the offside rule itself early in 2020: “You will not be offside if any part of the body that can score a goal is in line with the last defender, even if other parts of the attacker’s body are in front” (Guardian Newspaper, 19 February 2020).

This proposed rule has not been adopted by the sport, but from the perspective of vision science it would make some sense because it would add a bias that favours the attacker, as a way to counteract the bias against the attacker introduced by VAR.

More generally, given that modern officiating in sport relies primarily on sensory data that is often mediated by audio-visual technology, there is plenty of scope for sensory science to contribute to sport science and officiating ( e.g., Gilis et al., 2009; Mather, 2008; Mather & Breivik, 2020; Spitz et al., 2017, 2018, 2020 ).