Abstract
Establishing a causal relationship requires not only the presence of a factor of interest but also the demonstration that the relationship is absent when the factor is absent. Such ideal conditions are rare, especially in observational studies in which creating control conditions is inherently difficult. The COVID-19 pandemic, with its unparalleled disruptions, offers a unique opportunity to isolate causal effects and test previously impossible hypotheses. Here, we examine the home advantage (HA) in sports—a phenomenon in which teams generally perform better in front of their home fans—and how the pandemic-induced absence of fans offered a fortunate yet systematic change to typical conditions, serving as a natural experiment. Using a structural equation modeling approach and building a mediation model encompassing all relevant HA factors, we quantified the reduction in HA and elucidated the specific mechanisms behind it. The theory behind HA and the availability of measures for each factor before and during COVID-19 lockdowns enabled us to estimate all postulated pathways within a natural experimental context. The robust statistical framework used in our study offers a foundational model for integrating naturally occurring events that serve as control conditions into the analysis of various real-life phenomena.
Scientific progress often hinges on our ability to question our most basic assumptions and to conceive of alternative scenarios that might also explain the phenomena we observe. Take, for example, the common experience of thinking of a friend moments before they call you. At first glance, this synchronous occurrence might appear to be more than mere coincidence, perhaps even suggesting telepathic communication. Yet the research method teaches us to consider the counterfactual: all those times you thought of your friend and the phone did not ring, a control condition indispensable for debunking the illusory cause-and-effect relationship (Mill, 1856; Popper, 2005). Sometimes the counterfactual scenarios necessary to test our hypotheses are not available. In observational research, this is a common situation. Assessing the impact of political decisions on voting behaviors is challenging because we cannot know how people would behave if those decisions had not been made. Similarly, it is difficult to understand which factors improve the mental health and well-being of people who go to work when home-office alternatives are rarely offered. Finally, decomposing the effects of multiple socioeconomic factors on health outcomes is a daunting, and perhaps impossible, task.
However, there are occasions when unexpected events force us into new patterns of behavior, providing natural experiments that allow us to test our hypotheses in ways we could not have otherwise imagined. The COVID-19 pandemic is one such event. The restrictions imposed during the pandemic created unique situations that can serve as control conditions, allowing us to gain new insights into complex phenomena. In this article, we outline how COVID-19-induced lockdowns can be used to isolate the causal effects between outcomes and predictors of interest. First, we explain a natural-experiment paradigm and its assumptions. We then describe a theoretical model of the home-advantage (HA) effect in soccer and a statistical approach in which COVID-19-induced lockdowns were used as a natural-experiment variable to decompose the effects of individual factors on the HA effect. More importantly, we situate the model within the broader context of psychological-science research. We then discuss how this modeling approach, with its naturally embedded experiment, can be leveraged to advance our understanding of different phenomena in social science. Finally, we outline the limitations of the approach.
Natural Experiments
Natural events such as pandemics, policy changes, or government decisions offer scientists external variations not influenced by the system being studied. These variations often serve as the counterfactual instances that are typically hard to obtain but that are essential for observing and comparing how the system responds. In natural experiments, the exogenous influence, that is, the influence of factors external to the studied system, functions much like randomization in traditional experiments (Cook et al., 2008). External factors often result in “as-if randomization,” in which exposure provides random allocation to the exposed and nonexposed instances of the same phenomenon. Consequently, natural experiments are generally less prone to confounding than other types of observational studies (De Vocht et al., 2021).
Natural experiments provide a powerful method for exploring complex phenomena, but their value depends on the validity of the randomization introduced by the natural event. If the observed outcome can be attributed to confounding factors rather than the event itself, especially when these influences are difficult to separate from the intervening event, the utility of the natural experiment may be significantly diminished. Similarly, having an explanation of the phenomenon, preferably articulated as a formal model, is essential. The key question then becomes how to integrate natural variation into an existing formal model. Scientists have used various methods to identify potentially causal mechanisms on the basis of external variations. Common approaches compare pre- and postexposure outcomes in a single population, aim to control all variables of interest in regression models when comparing exposed and nonexposed groups, or even model time-varying trends in time-series data through interrupted time-series designs (for an overview of methods, see Craig et al., 2017). Here, we demonstrate the application of structural equation models combined with the changes brought about by the COVID-19 lockdowns. We assume that the natural-event intervention influenced the explanatory factors in our model but had no direct association with the outcome or the residual variance. Residual variance refers to the unexplained variability in the outcome that remains after accounting for the influence of the explanatory factors in the model. In our analysis, we assume that the natural event (absence of fans) influenced the explanatory factors but did not directly affect this unexplained portion of the outcome, meaning we have accounted for all significant pathways by which the event could have influenced game outcomes. We provide a fully commented statistical implementation of the presented model on the OSF at https://osf.io/fxg5w.
Home-Advantage Mediation Model
The HA is a well-documented phenomenon in sports in which teams are more likely to succeed when playing in front of their home fans than when playing away matches (Allen & Jones, 2014). HA is omnipresent. It is independent of the era (Pollard & Gómez, 2014; Pollard & Pollard, 2005) and is found in both collective and individual sports (Jamieson, 2010).
In addition to home players being better rested and more familiar with the quirks of their home ground than away players, the presence of home fans plays an important role (Carron et al., 2005). According to social facilitation theory (Zajonc, 1965), the mere presence of spectators enhances performance on well-learned tasks by elevating arousal levels. The opposite occurs for difficult tasks when the arousal is elevated beyond the optimal level. In the context of sports, the presence of fans should have a positive effect on athletes because they perform well-learned familiar tasks. This is particularly the case for the home team, for which supportive gestures boost arousal beneficially. Conversely, the same gestures can push arousal beyond optimal levels for visiting athletes, potentially impairing their performance because of the perceived threat and heightened pressure (Strauss et al., 2023).
Evolutionary psychology suggests another mechanism for the boost in the home team’s performance: The away team is perceived as intruders threatening a safe space, which evokes protective aggressive behaviors (Furley, 2019; Furley et al., 2018). Although the territorial response may not depend on fans, their vocal support is a continual reminder of the home team’s domain, reinforcing the drive to defend and excel (Furley et al., 2023).
The classical HA framework by Albert Carron (Carron et al., 2005; Courneya & Carron, 1992) assumes that venue factors, such as fans, influence the physiological and psychological responses of the main protagonists, and consequently their performances, and in turn the outcome (for a revised version of the model, see Fig. 1). Although the model features the causal flow from the venue factors through performance indicators to the outcome, it has been used more for categorizing disjoint research on individual factors of HA than as a testable model (see Bilalić et al., 2021, 2024). This mostly theory-devoid approach ignores the interplay between individual factors and may lead to incorrect conclusions.
Home-advantage mediation model. The presence of a crowd influences the physiological and psychological states of players, referees, and coaches, which in turn affects their performance. These performances are interrelated and serve as determinants of the game’s outcome. For example, a team’s performance can influence the frequency of official warnings issued by referees. Unlike the traditional framework by Carron et al. (2005), in which the performances of players, referees, and coaches directly contribute to the home-advantage effect, here they are considered independent factors that mediate the impact of physiological and psychological factors—and, by extension, the influence of the crowd.
We recently reformulated Carron’s model (see Fig. 1) in a way that allows us to answer questions such as whether the HA is a consequence of referees’ behavior independent of the team performance, or to what extent referees’ behavior and/or team performance cause HA (Bilalić et al., 2021, 2023, 2024). The model is based on a combination of direct effects and mediations: The venue factors influence the outcome, but that influence can be direct or indirect as the venue factors such as the crowd first affect the physiological factors of the main players, which in turn affect their psychological factors. The psychological factors are then reflected in the performance of players, referees, and coaches. We call this new framework the “home-advantage mediation (HAM) model” because it is based on the theoretically assumed mediations between the concepts (Hayes, 2017). Because the data on physiological and psychological dimensions are often unavailable, a practical traditional mediation framework, as depicted in Figure 2, has the venue’s influence on the outcome being mediated through performance and referee bias.
Application of the HAM model. The pre-COVID-19 model suggests that the influence of the venue is indirectly mediated by team performance (highlighted in green) and referee bias (highlighted in purple). The dashed line from venue to outcome implies a reduced direct impact of the venue on the outcome when factors such as referee bias and team performance are included in the model. The primary mediations (indirect effects) are listed on the left alongside the direct effect of the venue on the outcome. Collectively, these factors constitute the overall impact of the venue. The COVID-19 factor (red hexagon) that differentiates between conditions before and during COVID (with and without a crowd) acts as an external moderator within the mediation model. COVID-19 directly modifies only the venue factor, which then cascades this change through the model via different pathways. Relationship values within the model differ between the pre-COVID period with crowds and the period during COVID without crowds. HAM = home-advantage mediation.
The HAM model illuminates not only the extent of HA but also how difficult it is to decompose individual pathways because the home fans influence all elements of the model, culminating in a favorable outcome for the home team. By itself, the model does not pinpoint fans as the core driver of the HA process Typically, teams compete with spectators present, whether they are their supporters during home matches or rival fans during away games. This conventional setup was disrupted by the COVID-19 pandemic, leading most sports events to resume without fans. Matches held in such solitude serve as a natural experiment (Dunning, 2012; Grosz et al., 2024) because the lack of spectators presents a contrasting backdrop to the usual scenarios in which fans are present.
Navigating this unique predicament, we can include the fans’ presence or absence in the HAM model, labeled the “COVID-19 factor” here. This new dimension is incorporated as a moderator, augmenting the preexisting pathways. In essence, the crowd’s presence or lack thereof either amplifies or attenuates the established connections (see Fig. 2). This dynamic is frequently termed “moderated mediation” because the existing mediation is further influenced, or “moderated,” by an external element (Hayes, 2017). A crowd-less environment should predominantly impact the paths leading from the venue to team performance and referee bias.
COVID-19 in European Soccer (2019/2020 Season)
Here we present results from the 2019/2020 European soccer season (Bilalić et al., 2021; McCarrick et al., 2021). What sets the 2019/2020 season apart is its abrupt halt in March 2020, with three quarters of the matches already played, in the wake of the COVID-19 pandemic’s surge. Yet the season resumed in most European leagues and concluded by that summer, notably without fan attendance. This distinct scenario serves as an ideal backdrop to assess the influence of fans on the HA.
Utilizing the HAM model, we addressed numerous pivotal inquiries concerning the HA phenomenon. We not only discovered that the HA in European club competitions is larger when the fans are present but also shed light on the mechanism underpinning the HA. The predominant factors contributing to the HA are the enhanced performance of home teams and, even when considering this performance, a noticeable referee bias in favor of the home side. These two routes, both originating from the home venue—one via team performance and the other through referee bias—play a significant role in the overall success of home teams.
Looking at the final quarter of the 2019/2020 season, which was played without fans, the overall HA was substantially reduced from an SD of 0.57 when the fans were present to 0.36 when they were absent, a reduction of 37% (see Fig. 3a). The underlying mechanism remained consistent, with both team performance and referee bias directly affecting the outcome and being influenced by the venue. However, the relationship with the venue factor changed dramatically. When playing without spectators, home teams performed around two thirds as well as when fans were present, in which the advantage was almost a full SD. The treatment of home teams by referees underwent an even more significant transformation. Initially, referees exhibited a bias against away teams, issuing them more official warnings (roughly a third of an SD more). Without a crowd to potentially sway their decisions, the home teams suddenly received more official warnings, not the away teams, by about a fifth of an SD. Similar to team performance, the absence of fans significantly impacted referee bias, amounting to half an SD swing, a considerable effect in psychological research.
Effect of fan absence on HA in European soccer during the 2019/2020 season. The full HAM model shown here (adapted from Bilalić et al., 2021) illustrates (a) the interplay between venue, team performance, referee bias, and outcome and (b) the total effect change (i.e., how the lack of spectators because of COVID lockdowns altered HA). Single-ended arrows indicate the direction of influence, and the numbers along these arrows are path model coefficients. Coefficients for the pre-COVID period are shown in blue, those during COVID are shown in red, and the differences between these periods are denoted in black with a delta (Δ) symbol. The mediation effects of venue on outcome through team performance (green) and referee bias (purple) are formally evaluated in a mediation model (upper left box). Path coefficients that reach statistical significance are marked with an asterisk. HA = home advantage; HAM = home-advantage mediation.
Harnessing Naturally Intervening Variables
Our HAM model offers a primer on how a psychosociological phenomenon can be elucidated using a well-known statistical framework. It not only allows us to quantify the phenomenon but also enables us to clearly distinguish between the proximal causes of HA, such as in-game team performance and referee behavior, and the distal causes, such as the influence of fans. However, it was only with the onset of the COVID-19 pandemic and the ensuing closure of stadiums across Europe to fans that we could firmly ascertain the profound role that fans play in indirectly influencing the outcome.
We believe that the uniquely robust statistical framework used in our research could serve as an introductory example of how naturally occurring control conditions can be formally integrated into the analysis of various real-life phenomena. In the following text, we provide three examples of studies that investigated sociological and psychological theories altered by COVID-19, specifically its related lockdowns. However, these studies utilized natural-experiment scenarios to varying degrees, whereas we argue that our understanding of a phenomenon could benefit from translating theoretical accounts into formal models and falsifying them using a natural-experiment framework. This interconnectedness between theory and statistical models, bolstered by natural experiments, should go a long way toward addressing the current theoretical and generalizability crisis in psychology (Eronen & Bringmann, 2021).
Voting decisions are influenced by a range of factors, from sociodemographic characteristics to specific policies introduced by incumbents. For example, the “rally-round-the-flag hypothesis” (Baum, 2002) suggests that leader trust increases after disasters. The municipal elections in France coincided with the COVID-19 lockdowns, which provided an opportunity to formally test this hypothesis. The first round of voting occurred just before the lockdowns, whereas the second round was delayed until after their conclusion. France also implemented a differential lockdown based on COVID-19 case numbers, dividing the country into areas with “hard” and “soft” lockdowns. Findings suggest that incumbents who implemented more stringent measures received more votes (Giommoni & Loumeau, 2022). One could argue that the voters under the hard lockdown felt safer or more aware of the crisis and rallied around more protective policies, as suggested by the rally-round-the-flag hypothesis.
The rapid shift to remote work during lockdowns offers a rare opportunity to explore the impact of traditional office work on various aspects such as productivity, socialization, and overall well-being (Künn et al., 2022; Mansfield et al., 2021). For instance, individuals who prefer staying up late (owls) tend to struggle more with office work compared with those who enjoy getting up early (larks), who are naturally aligned with social expectations. Owls often experience sleep deprivation, which can lead to deteriorating mental health (Salfi et al., 2022). However, the link between circadian typology (chronotype) and depression through sleep quality has been difficult to examine until the pandemic made remote work a widespread necessity. Salfi et al. (2022) applied a moderated meditation framework to investigate whether remote work during lockdowns influenced the mediation between chronotype and depression through sleep quality. Their findings revealed that sleep quality mediated the relationship between chronotype and depression levels, but only when individuals worked from the office. Late chronotypes indeed experienced depressive symptoms because of their vulnerability to sleep disturbances in an office setting, but these symptoms were alleviated when remote work and its associated flexible schedule were introduced.
In the medical sciences, researchers observed significant reductions in the births of very low birth-weight and extremely low birth-weight infants during COVID-19 lockdowns (Philip et al., 2020). The study identified several possible factors contributing to this reduction, including reduced pregnancy-associated stress, increased family support, environmental changes, and decreased exposure to emissions. Like the previous examples, this research highlights the potential to analyze large observational data from the COVID-19 lockdown periods and compare these data with the same type of data from normal circumstances, which would lead us to better understand the causal factors affecting pregnancy outcomes.
As-If Randomization and Continuum of Validity
Unlike the classical experiment, natural experiments do not offer complete control over confounding factors. It is crucial, therefore, to ensure that the observed changes can be attributed to the (natural) event rather than to other factors. In the case of HA, other factors such as differing team strength, game importance, and travel might contribute to HA alongside, or even instead of, the presence or absence of fans. Our HAM model can accommodate these possible influences on HA (see Bilalić et al., 2021), emphasizing the importance of having a formal model in place before an intervening event occurs. However, even in this ideal scenario, we cannot entirely rule out other confounders that might threaten the causal validity of the natural experiment. These threats could stem from omitted variables, such as differences in team composition or varying responses to the pandemic, which might put additional strain on home teams when hosting away players. Additionally, broader changes in players’ lives, such as concerns about the disease and disruptions to sleep, could also play a role.
The validity of as-if randomization may be even less plausible in the other examples we considered. For instance, voting behavior in France might reflect the rally-round-the-flag hypothesis, in which voters prefer more stringent decisions from incumbents. Yet it is also possible that political preferences were gradually shifting from right to left over time, a common trend in many political systems. The health system poses an even greater challenge because responses to the pandemic, such as lockdowns, simultaneously altered many behaviors within the system. This makes it difficult to account for all factors and causal pathways that could affect outcomes such as depression (Salfi et al., 2022) or infant birth weight (Philip et al., 2020).
Conclusion
There has been a notable surge in the field of psychology in the use of big data and observational studies, marking a transformative shift in research approaches (Vaci & Bilalić, 2017; Vaci, Cocić, et al., 2019; Vaci, Edelsbrunner, et al., 2019). The rapid expansion of digital technologies has allowed psychologists to collect large amounts of data from various sources, including social media activities, online behavior, and sensor data from wearable devices (Rafaeli et al., 2019). This influx of big data has facilitated a more comprehensive and nuanced understanding of human behavior, enabling researchers to identify complex patterns and correlations that were once difficult to detect. However, the methodological settings that enable both support and falsification of the theoretical assumptions have been largely missing in these approaches.
In this article, we demonstrated how practitioners can leverage the natural variations resulting from COVID-19 lockdowns to discern observed from unobserved factors, thereby estimating causal effects within their theoretical frameworks. We utilized the HAM model in sports as our example, which, because of its closed theoretical system and the availability of measures for each factor, allows for the estimation of pathways within a natural experimental context. Research areas that engage with more complex theoretical constructs may find it challenging to achieve the same degree of control and to quantify these pathways. Nevertheless, we believe that the robust statistical framework utilized in our study could provide a foundational model for integrating naturally occurring events that serve as control conditions into the analysis of various real-life phenomena, spanning from remote work and productivity to online-education effectiveness and mental health (Grosz et al., 2024).
Recommended Reading
Bilalić, M., Gula, B., & Vaci, N. (2021). (See References). Introduces the home-advantage (HA) mediation model, a comprehensive framework that interconnects HA factors through a statistically formalized series of mediations and applies it to the 2019/2020 European soccer season.
Bilalić, M., Gula, B., & Vaci, N. (2024). (See References). Theoretical and empirical review that presents an update to the initially classical Carron’s home-advantage model along with a review of studies on individual factors.
Furley, P. (2019). (See References). Offers an evolutionary perspective for understanding human behavior through sports as well as guidelines on how the extensive sports data available can be used to enhance our understanding of human nature in different areas.
Giommoni, T., & Loumeau, G. (2022). (See References). Illustrates how COVID-19-related lockdowns can be treated and incorporated as a natural experiment to answer political and economic questions.
Grosz, M. P., Ayaita, A., Arslan, R. C., Buecker, S., Ebert, T., Hünermund, P., Müller, S. R., Rieger, S., Zapko-Willmes, A., & Rohrer, J. M. (2024). (See References). Appeals to psychologists to incorporate natural experiments into their research and provides theoretical background and empirical examples on the use of natural-experiment design in psychology.
