Abstract
Video assistant referee (VAR) has been introduced to elite football (soccer) to avoid clear and obvious mistakes, especially for goal scoring situations. Some literature has reported on VAR’s impact on the game across particular league competitions. The current study meta-analysed data across two seasons (pre-VAR and post-VAR) from 20 competitions (n = 9076 matches; men’s domestic leagues and both men’s and women’s international tournaments) to examine the overall impact that VAR initially had on number of goals scored, and the extent of inter-competition heterogeneity. A secondary aim was to determine any VAR-associated reduction in home advantage for goals scored, mean result direction and match closeness. Findings demonstrated that there were no overall statistically significant nor meaningful differences between pre and post-VAR seasons/competitions for total number of goals scored per match. Similarly, there were no VAR-associated reductions in home advantage in terms of goals scored, mean result direction and closeness of match outcome. There was moderate heterogeneity for some comparisons, and VAR lessened inter-competition variance for number of goals scored and match score closeness. Implications of findings are discussed. As the largest VAR-focused study and the first to use meta-analytic approach, the current findings are the most comprehensive and definitive to date.
Keywords
Introduction
Video Assistant Refereeing/Referees (VAR) were officially integrated into the ‘Laws of the Game’ across European domestic and international competitions in the 2017–2018 season, to assist and thereby improve decision-making in football officiating. Away from the field-of-play, VARs review incidents on request or communicate proactively with the on-field referee about any ‘match-changing’ events that may have been missed. Specifically, these consist of decisions relating to; goal/no goal, penalty/no penalty, direct red card and mistaken identity (i.e. administrative) scenarios. 1 Since its introduction, the use of VAR has expanded worldwide, to most topflight leagues, European club tournaments and international competitions.
Despite football being the world’s most popular sport, with an approximate fan base of four billion people, 2 VAR-focused research is still in relative infancy given the technology has only recently been integrated across the upper echelons of the game. Studies have reported on the fans’, managers’/coaches’ and referees’ opinions and perceptions of VAR.3–7 Positive findings include high levels of trust in VAR, 3 and that it has both brought justice and fair play to football and improved refereeing performance, allowing the deserving team to win.4,7 Negative findings indicate that implementation of the VAR system detrimentally impacts the fluency of football for spectators, 8 can decrease referees’ confidence in their own profession, 4 and that VAR can hinder fans’ opportunities to debate. 3 Indeed, there are significant complexities between the VAR and the experience of matchday fans, 8 as well as inter-fan variation, such as one study finding that younger English Premier League fans felt more positively than their older counterparts towards VAR. 9 Other studies have reported on a wide range of aspects, such as referees’ perceptions of VAR pre- and post-training, 10 expert opinion about visual science issues of VAR, 11 and philosophical narratives on the use of VAR for upholding rules and standards. 12
An increasing volume of studies have explored how the introduction of VAR has more directly impacted football matches. These quantitative examinations have compared variables of interest between two adjacent seasons, that is, pre- and post-introduction of VAR, within respective competitions.13–22 Using this approach, research has reported on VAR’s influences on refereeing decision-making variables such as decision accuracy17,22 and decision time,15,19 as well as other officiating factors, such as the number of offsides, fouls, cards awarded (yellow and red) and penalties given.15,17,19 Technical play variables have also been investigated, including the number of corners, shots, crosses, dribbles, fouls and passes, average distance run, number of sprints and perhaps most pertinently, goals scored.13,17,19,21 Although two studies found a significant decrease in goals scored in seasons since VAR was introduced,13,14 five found no difference.15,17,19–21 Adopting a slightly different approach, one study split the data set by number of VAR interventions, and found that matches with no VAR interventions had significantly fewer goals than matches with either one or more VAR intervention(s). 13 Interestingly, another study had mixed results whereby there was a pre to post-VAR introduction decrease in goals within Italy’s Serie A league but in Germany’s Bundesliga there was no statistically significant change. 18 Together with sample variation across the aforementioned studies, this finding suggests that inter-competition variation in the implementation of VAR, and differing football cultural norms may manifest within VARs influence on number of goals scored. To this point, the largest study of this kind to date analysed 2195 matches across 13 competitions, 22 albeit it did not specifically examine the number of goals, rather decision accuracy (which increased from 92.1% to 98.3%) and median duration of VAR reviews (22 s). To this end, most work has focused on considerably smaller samples and number of competitions. Research is yet to statistically examine the extent of heterogeneity of VAR’s impact between competitions; and the combination of the statistical approaches used, together with limited geographical or competition(s) focus, means that that no consensus exists regarding the potential impact of VAR on specified match variables. Additionally of note, across the mentioned studies, only men’s football has been investigated.
Beyond raw match variables such as goals scored, potential influences of VAR on the phenomenon of home advantage relating to referees’ decision-making has received some attention (the consistent finding that home teams in sports competitions win a disproportionate number of the games played under a balanced home and away schedule23,24). This is particularly pertinent with football demonstrating one of the largest home advantages in the sporting literature.25,26 With the introduction of VAR, two studies reported a potential reduction in home team bias by analysing several officiating variables. Specifically, referees played a more equitable amount of additional time regardless of match status relative to the home team (i.e. more if they were losing and less if winning) (16). Further, there were no significant differences between the number of fouls, yellow and red cards awarded between the home and away teams with the introduction of VAR (15). However, VAR did not initially change the occurrence of home teams being awarded relatively more penalty kicks than away teams.15,16 To this end, research is yet to examine the initial impact of VAR on home advantage in terms of goals scored and the proportion of matches won by home teams. These metrics might be expected to be indirectly influenced by VAR as the awarding of offside decisions can lead to disallowing of goals; the awarding of red cards often changes the dynamics of match play for the remainder of the match, and the awarding of yellow cards might change the decision-making and behaviour of players in the knowledge that subsequent fouls awarded against a yellow-carded player are more likely to result in them receiving a red card.
The current study therefore collated data, aiming to address these evidence gaps via a meta-analytic approach, to provide the largest academic examination of the overall impact that VAR initially had on number of goals scored, and the extent of inter-competition heterogeneity, including women’s football. As previously identified, studies report mixed findings pertaining to the number of goals scored, but the majority have found negligible effects. To this end, our primary hypotheses were that:
(i) There would be no statistically significant effect nor meaningful difference in the overall impact of VAR on the number of goals scored.
(ii) There would be statistically significant heterogeneity of effects between competitions.
The collation of data to address the primary hypotheses provided additional opportunity to examine hypotheses of interest on the phenomenon of home advantage. The secondary hypotheses were that:
(iii) There would be a VAR-associated reduction in home advantage in terms of goals scored.
(iv) There would be a VAR-associated reduction in home advantage in terms of mean result direction.
(v) There would be a VAR-associated reduction in match score closeness.
(vi) VAR would lessen the observed between-competition variance for each of the dependent variables.
Methods
Data collection and inclusion/exclusion criteria
Across 20 competitions that met our inclusion and exclusion criteria, scores (results) of 9076 football matches were collated from official records across two respective competition seasons or tournament cycles: immediately before the first full competition season/tournament cycle that VAR was implemented (pre-VAR); and the first full competition season/tournament cycle that VAR was implemented throughout the entire competition season/tournament cycle (post-VAR; see Table 1). For domestic competitions, only regular season match data were included, and from tournaments only data from ‘group stage’ matches were included. Data from playoff matches (league) and knock-out matches (tournaments) were excluded as these matches can include extra time, which manifests as potentially confounding between season variation as an opportunity for more (but not fewer) goals to be scored (due to additional time).
Competition seasons included and number of matches.
Final nine matches not included as VAR was introduced.
Season concluded prematurely or was interrupted by the COVID-19 pandemic. Only already-played matches from these seasons included for analysis.
Non-adjacent seasons because VAR was only introduced part-way through 2017 season (K League 1; Major League Soccer) or 2017–2018 season (Primera Liga; Juliper Pro League).
For competitions that phased-in VAR part-way through a season, the pre-VAR season was selected as the last fully non-VAR season preceding that phasing, and the post-VAR season was selected as the first season whereby VAR was implemented throughout the full season. Where respective competition seasons were concluded prematurely or interrupted by the COVID-19 pandemic, played matches from these seasons were still included for analysis, but any postponed games that were played without or with limitations to crowd size were excluded, as crowd presence and size have long been accepted as to influence officiating decisions,24,27 which can indirectly impact scores.
Dependent variables
Additional to the raw ‘Number of Goals per match’ data, three further variables were derived. All dependent variables were obtained or calculated at individual match-level, to enable a mean and standard deviation to be calculated per competition/season for entry into meta-analyses.
To examine whether home advantage was influenced by VAR, two variables were calculated: ‘Directional Difference in Home Versus Away Goals per match’ was determined by number of home goals minus number of away goals; and ‘Result Direction’ was calculated as home team win = 1, away team win = −1, draw = 0. Both variables were calculated within club competitions only, as aside from host nations, there are no true home/away teams in international tournaments.
To provide an angle from which to examine whether closeness of match scores was influenced by VAR, ‘Non-Directional Difference in Home Versus Away Goals per match’ was calculated as the square root of the squared difference in home versus away goals per match. As this variable addresses match closeness rather than home advantage, it was calculated across all 20 competitions.
Data analysis
A series of meta-analyses were conducted using SPSS version 28. For each outcome variable of interest (Number of Goals per match; Directional Difference in Home Versus Away Goals per match; Result Direction; Non-Directional Difference in Home Versus Away Goals per match) mean and standard deviation values from each competition were compared between the two respective competition seasons: pre-VAR; post-VAR. Each meta-analysis was conducted and interpreted in the same way as Moran et al. 28 The inverse-variance restricted maximum likelihood random-effects model was used to examine both the weighted overall effect size and weighted heterogeneity between competitions.29,30 Unstandardised mean differences were calculated as the ‘effect size’, for ease of interpretation as the respective units of measurement (goals) were identical across competitions. Heterogeneity was assessed using the I2 and Q statistics, whereby I2 represents the percentage of variation across competitions that is due to true heterogeneity rather than chance 30 (25% = low heterogeneity; 50% = moderate and 75% = high 31 ) whereas Q assesses the likelihood that observed heterogeneity is due to chance alone.28,32 Where appropriate, two of the meta-analyses were additionally calculated with each of two moderator variables: competition type (club/nations); sex (men’s/women’s) to assess the importance of these factors. For hypothesis (vi), that the VAR would lessen the observed variance between competitions for each of the variables, standard deviation values for pre-VAR and post-VAR seasons were interpreted.
Results
For ‘Total Number of Goals Scored per match’, overall, there was no statistically significant nor meaningful difference between pre-VAR (MCompetitions 2.9, ±0.5) and postVAR (MCompetitions 2.9, ± 0.3) seasons (Z = 0.52, p = 0.60), whereby the mean difference across competitions was 0.03 goals per match. Heterogeneity between competitions was moderate (I2 = 0.61), and statistically significant, thus is unlikely due to chance (Q (19) = 53.54, p < 0.01). When moderator variables were included, these were each not statistically significant (sex: Q (1) = 0.95, p = 0.33; competition type: Q (1) = 0.18, p = 0.67, see Figure 1). Descriptive values for all variables are given in Tables 2 and 3.
Forest plot of unstandardised overall and subgroup effects for total number of goals scored, by competition type.
Season-level mean (± SD) values for goals by competition.
Season-level mean (± SD) values for calculated variables by competition.
For ‘Directional Difference in Home Versus Away Goals per match’, overall, there was no statistically significant difference between pre-VAR (MCompetitions 0.4, ±0.2) and postVAR (MCompetitions 0.3, ±0.2) seasons (Z = −0.51, p = 0.61). The mean difference across competitions was −0.02, indicating that although home teams still tended to score more goals than away teams, this difference became marginally lesser following the introduction of VAR. Heterogeneity between competitions was very low (I2 < 0.01) and statistically non-significant (Q (15) = 6.67, p = 0.97).
For ‘Result Direction’, overall, there was no statistically significant difference between pre-VAR (MCompetitions 0.2, ±0.1) and post-VAR (MCompetitions 0.1, ±0.1) seasons (Z = −1.18, p = 0.24). The mean difference across competitions was −0.021, indicating that results were very marginally less favourable to home teams following the introduction of VAR. Heterogeneity between competitions was very low (I2 < 0.01), and statistically non-significant (Q (15) = 4.87, p = 0.99).
For ‘Non-Directional Difference in Home Versus Away Goals per match (Match Score Closeness)’, overall, there was no statistically significant difference between pre-VAR (MCompetitions 1.5, ±0.4) and post-VAR (MCompetitions 1.5, ±0.3) seasons (Z = 0.62, p = 0.54). The mean difference across competitions was 0.03, indicating that winning scores became marginally greater following the introduction of VAR. Heterogeneity between competitions was moderate (I2 = 0.58), and statistically significant (Q (19) = 49.2, p < 0.01). When moderator variables were included, these were each not statistically significant (sex: Q (1) = 1.99, p = 0.16; competition type: Q (1) = 0.01, p = 0.93).
Standard deviation values for competition-level means decreased slightly for ‘Total Number of Goals Scored per match’ and ‘Non-Directional Difference in Home Versus Away Goals per match (Match Score Closeness)’, but there was no change for ‘Directional Difference in Home Versus Away Goals per match’ or ‘Result Direction’ (Tables 2 and 3).
Discussion
The purpose of this study was to provide the first meta-analytic approach and largest academic examination of the overall impact that VAR initially had on number of goals scored, and the extent of inter-competition heterogeneity. This is the first study to include data from both domestic and international competitions for men and women’s football.
Hypothesis (i) – there would be no statistically significant effect nor meaningful difference in the overall effect of VAR on number of goals scored, was supported. Given that time equates to opportunities to score goals, the current overall finding that the introduction of VAR did not change the number of goals scored supports the conclusion of Lago-Penas et al. 19 who found that playing time lost to VAR checks is offset by increased additional time at the end of the game. Potentially converse to the current findings however, some previous research suggests that certain refereeing decisions, and therefore goalscoring opportunities are influenced by VAR; namely penalties and offside fouls (i.e. match changing events). For example, some,15,17,21 but not all18–20 previous studies have reported more penalties awarded with VAR compared to without, and fewer offside fouls have been awarded across some competitions.15,18,19,21 Additionally for total number of goals scored, it is plausible that with VAR, referees more readily let play continue where previously they may have stopped the game for a potential foul, knowing that VAR will check it – with this difference potentially leading to more goals being scored. That said, these influences did not manifest homogenously in overall number of goals scored within our analysis. Unfortunately, it is beyond the current analysis to examine the possibility that the potential influences discussed underpin our findings. Although taken together with insight from previous literature regarding fans’ perceptions, the current finding that VAR does not influence the number of goals scored suggests that decisions on whether to use or keep the VAR system should be a matter of stakeholder preference.7–10 Other factors that VAR influences, such as number of yellow and red cards given, 15 and discrepancy dynamics in number of foul and offside decisions awarded and added time, 16 also need to be considered.
Hypothesis (ii) – there would be statistically significant heterogeneity of effects between competitions, was supported. While most did not, some competitions did show statistically significant differences between seasons (total goals per game increased in the Eredivisie, Major League Soccer, UAE Pro League, Primera Liga and decreased in Serie A and Super Lig). Perhaps this finding is not overly surprising given the idiosyncrasies that exist between leagues in view of both officiating (i.e. cultural33,34) and playing styles (i.e. tactical 35 ), as well as the governing body responsible for implementing the technology (i.e. procedural). For example, and from an officiating perspective, if VAR officials within a league are more likely to review an incident this may lead to more decisions, and thus potentially goals scored given the nature of incidents reviewed in the VAR process. 13 Nevertheless, and while the specific findings for Serie A and Bundesliga are consistent with those of Carlos et al., 18 our current overall null finding highlights the value of the larger sample size and statistical addressing of inter-competition differences at the holistic sport level.
Hypothesis (iii) – there would be a VAR-associated reduction in home advantage in terms of goals scored, and (iv) mean result direction, were not supported by our statistical analyses. It might be that possible home advantage ingredients such as familiarity, confidence and preparation 22 influence player performance and thereby match scores independently of home advantage phenomena relating more closely to refereeing decisions. 36 Another prospect is that factors such as game management,37–40 team reputation,41,42 crowd noise and density,24,43,44 match timing,37,45 team ranking, 46 and previous in-match decisions 47 may influence VARs similarly to how they are reported to influence on-field referees. If the latter prospect is accurate, our findings imply that it may be beneficial for VARs to be starved of as much contextual information as possible during decision reviews to reduce bias within the decision-making process. Indeed, it is noteworthy that the decision to initiate a VAR check remains a subjective process. Future research should more directly examine this issue, with feasibility to be considered by football’s governing bodies.
Hypothesis (v) – there would be a VAR-associated reduction in match score closeness, was not supported. However, the statistically significant moderate heterogeneity between competitions, akin to what was discussed for hypothesis (ii), indicates that idiosyncrasies in VAR implementation, and cultural differences in rule interpretation and refereeing decision-making manifests in match score closeness.
Hypothesis (vi) – VAR would lessen the observed variance between competitions for each of the variables. Mixed findings were observed here, as the data indicates that VAR may have initially lessened inter-competition differences in total number of goals and match score closeness. However, data collection across a greater number of seasons would be required to interrogate whether the observed changes are beyond regular inter-season fluctuations at the intra- and inter-competition levels. Moreover, alternative explanations may include sampling error, and/or general trends for reduction in inter-competition variance over time as a confounding/extraneous factor. This point is applicable to all findings of the current study and previous examinations of other match variables,13,15,17,19,21 indicating a need for more longitudinal collation of data across seasons. Although the current findings are clear regarding the competition seasons analysed, season-on-season alterations both to ‘laws of the game’ and to VAR’s implementation protocols as all stakeholders have grown accustomed to the technology (1–5), mean that examination of the seasons since VAR’s inception may yield different results. As these factors and their cultural manifestations also vary between competitions, heterogeneity might also be expected to fluctuate. Longitudinal data analyses might be used by future studies to analyse such ongoing changes and examine both officiating and technical/playing variables. Despite this recommendation, the seasons most likely to show VAR-associated differences for match variables would be those immediately preceding and then implementing VAR. To this end, with research demonstrating how VAR increases decision accuracy 22 whilst not initially changing the number of goals scored in matches (i.e. the present study), football governing bodies can focus on developing VAR protocols to reduce the previously reported negatively perceived side effects in relation to fans’ match experience 1, 5.
Omission of competitions due to the comparison seasons including those whereby stadium crowds were absent (due to the COVID-19 pandemic) reduced the sample size and potential global competition coverage. However, this facilitated internal validity of the pre- to post-VAR comparison given the known influences of crowd density and noise on referee bias.24,43,44 Although these extreme differences were avoided, that these factors still varied slightly between seasons but were not measured or statistically controlled, should be acknowledged. Despite these omitted competitions, sample size remained a positive of the study, representing the largest of its kind in the current VAR literature. The meta-analytic approach is also a strength as it statistically addressed inter-competition heterogeneity for the first time.
Although we aimed to include data from every women’s football competition that has introduced VAR, we were only able to access data meeting this criteria for two competitions, thereby limiting our inferences of the impact of VAR on women’s football. Future research should continue to include data from both men’s and women’s football (or focus on each separately) as VAR is introduced to more women’s competitions in the coming years. A further limitation of the current study is that it is not possible to ascertain whether changes in player/official staff within competitions, change in teams involved (due to relegation / qualification), inter-season changes in tactics/playing style, functioned to lessen or magnify the potential influence of VAR. However, the latter issue, which resonates with the earlier-mentioned limitation regarding regular seasonal fluctuations, is somewhat lessened by the fact that different pairs of seasons were analysed across competitions, due to variations in when VAR was introduced. Also, our examination of home advantage is somewhat superficial given it does not account for goal importance – that is, some single goals change the overall result of wins, draws and losses, whereas other goals do not.
Future academic research should include analyses of demographic, eye-tracking and biomarker data, as well as decision meta data (such as time taken to make decisions), so to elucidate indicators and covariates of decisions for both on-field referees and VARs. Similarly, it would also be prudent to examine whether contextual and crowd influences reported in relation to on-field refereeing decisions also function in relation to VAR decision-making.37–47 Finally, qualitative insight both into referees’ perceptions of VAR, and into VARs’ decision-making processes would be valuable additions to this research area to inform further implementation developments, training tools and talent identification within football’s governing bodies. Beyond football, the current meta-analytic approach to match variables might also be useful to address technological refereeing developments within other sports, such as the Touchline Match Official (TMO) in rugby.
Conclusion
The current meta-analysis concludes that overall, across football’s geographical, club and international landscape, the respective initial implementations of VAR did not influence the number of goals scored to a statistically nor meaningful significant extent. This finding was also true when examining each of the home and away goals scored in isolation. Lastly, inter-competition heterogeneity was found to be small.
Footnotes
Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
Data availability statement
All data is and will remain readily available through league / competition records and are publicly accessible.