Sage Journals: Discover world-class research

Abstract

Several sports tournaments contain a round-robin group stage where the teams are assigned to groups subject to some constraints. Hence, the organisers usually use a computer-assisted random draw to avoid any dead end, a situation when the teams still to be drawn cannot be assigned to the remaining empty slots. This procedure is known to be unfair: the feasible allocations are not equally likely, that is, the draw does not have a uniform distribution. We quantify the implied unfairness of the 2018 FIFA World Cup draw and evaluate its effect on the probability of qualification for the knockout stage for each national team. The official draw order of Pot 1, Pot 2, Pot 3, Pot 4 turns out to be a significantly better option than the 23 other draw orders with respect to the unwanted distortions. Nonetheless, the non-uniform draw distorts the probability of qualification by more than one percentage point for two countries. Our results call attention to the non-negligible role of draw order and make it possible for policymakers to decide whether using fairer draw mechanisms is justified.

Keywords

Association football permutation probability simulation tournament design

“Any one who considers arithmetical methods of producing random digits is, of course, in a state of sin.”¹ (John von Neumann: Various techniques used in connection with random digits)

Introduction

The mechanism design literature usually focuses on theoretical requirements like efficiency, fairness, and incentive compatibility.^1–3 On the other hand, institutions—like governing bodies in major sports—often emphasise simplicity and transparency, which calls for a comprehensive review of how the procedures that exist in the real world perform with respect to the above properties.

Many sports tournaments involve a group stage where the teams are assigned to groups subject to some constraints.^4,5 These constraints are imposed “to issue a schedule that is fair for the participating teams, fulfils the expectations of commercial partners and ensures with a high degree of probability that the fixture can take place as scheduled”.⁶ Examples include the FIBA Basketball World Cup,⁷ the FIFA World Cup,⁸ the European Qualifiers for the FIFA World Cup,⁶ the UEFA Euro qualifying,⁹ and the UEFA Nations League.¹⁰ Draw restrictions are also used in club-level tournaments such as the UEFA Champions League and the UEFA Europa League.¹¹

Until 2014, the FIFA World Cup draw divided the teams into pots mainly according to their geographic area, which caused serious unfairness in certain years such as in 1990,¹² 2006,¹³ and 2014.¹⁴ In particular, the top teams had different chances of being placed in a group with difficult opponents. Consequently,¹⁵ has recommended three options to create balanced and geographically diverse groups, which has inspired FIFA to change the draw procedure used in the 2018 and 2022 FIFA World Cups.^16,17

However, this mechanism, based on a computer-assisted random draw to avoid any dead end (a situation when the teams still to be drawn cannot be assigned to the remaining empty slots), is not uniformly distributed: the valid assignments are not equally likely to occur.¹⁸ Hence,¹⁸ propose two unbiased mechanisms that use balls and bowls, making them suitable for a televised draw. On the other hand, they require random simulations, which might threaten transparency.

Therefore, the organiser faces a dilemma: retain the existing but biased method or switch to a correct but less transparent mechanism. In order to understand this trade-off and choose the better option, it is inevitable to explore the extent of the bias, as well as its potential sporting effects. While these issues have been recently analysed in the UEFA Champions League,¹⁹ they have not been addressed in the case of the FIFA World Cup, the most prominent football tournament around the world.

The current work aims to fill this research gap by analysing the unique mechanism used in practice by sports federations to draw groups (containing at least three teams) with constraints through the example of the 2018 FIFA World Cup. This case study has been chosen since the 2022 FIFA World Cup draw has contained an inherent bias due to the uncertainty in the set of teams qualified,¹⁷ and the teams qualifying for the 2026 FIFA World Cup are naturally unknown at the time of writing.

We also call attention to the role of draw order: the order of the pots in the group draw turns out to have a non-negligible sporting effect. This is a somewhat surprising finding because the fairness of the UEFA Champions League Round of 16 draw essentially does not depend on whether the group winners or the runners-up are drawn first.^19,20 On the other hand, no clear recommendation is given for the optimal draw order in general due to the huge complexity of the calculations; according to Csató²¹ Section 5.1, computing the probabilities for six teams drawn from three pots is barely possible—and the 2018 FIFA World Cup contains 32 teams and eight pots. Therefore, several more case studies would be needed to find a general pattern if it exists.

Our topic is hugely relevant to governing bodies in sports. In an ideal environment, the rules governing a sport prevent any player or team from gaining an unfair advantage.^3,22 But the sequential draw method of FIFA may threaten fairness, while the alternative solutions remain more complex and less transparent.¹⁸ Therefore, policymakers could maintain the balance between fairness and integrity only by investigating different fairness indicators.²³

The paper starts with presenting a concise review of related literature, followed by describing the theoretical background of the FIFA World Cup group draw. The numerical results for the 2018 FIFA World Cup draw are discussed in two parts. First, the departure of the draw procedure from a uniformly distributed random choice among all feasible allocations is quantified for all the 24 possible draw orders of the four pots. This is important because the modification of the draw order (relabelling of the pots) does not require any reform in the existing principles of the draw. Second, the bias of the draw procedure is evaluated with respect to the probability of qualification for the knockout stage. That is essential since the ultimate price to pay for the sake of public interest and transparency is the distortion of the final outcome: if the effects on the chances of the teams remain marginal and insignificant, then there is no need to choose a more complex and less transparent draw procedure.

Related literature

Several scientific works analyse the FIFA World Cup draw. Before the 2018 edition, the host nation and the strongest teams were assigned to different groups, while the remaining teams were drawn randomly with maximising geographic separation: countries from the same continent (except for Europe) could not have played in the same group and at most two European teams could have been in any group.

In the case of the 1990 FIFA World Cup,¹² shows that the draw was not mathematically fair. For example, West Germany would have been up against a South American team with a probability of 4/5 instead of 1/2—as it should have been—due to the incorrect consideration of the constraints. Similarly, the host Germany was likely to play in a difficult group in 2006, but other seeded teams, such as Italy, were not.¹³

Guyon¹⁴ identifies severe shortcomings of the mechanism used for the 2014 FIFA World Cup draw such as imbalance (the eight groups are at different competitive levels), unfairness (certain teams have a greater chance of ending up in a tough group), and non-uniform distribution (the feasible allocations are not equally likely). Indeed, there has been a substantial competitive imbalance between the historical FIFA World Cup groups.²⁴

Guyon¹⁵ presents alternative proposals to retain the practicalities of the FIFA World Cup draw but improve its outcome. One of them can be compared to the flawed FIFA rule at https://www.nytimes.com/interactive/2014/06/03/upshot/world-cup-draw-simulation.html.

Laliena and López⁵ develop two uniformly distributed designs for group draw with geographical restrictions that produce groups having similar or equal competitive levels. Cea et al.²⁵ analyse the deficiencies of the 2014 FIFA World Cup draw and provide a mixed integer linear programming model to create the groups. The suggested method takes draw restrictions into account and aims to balance “quality” across the groups.

Roberts and Rosenthal¹⁸ consider the challenge of finding a group draw mechanism that follows the uniform distribution over all valid assignments but is also entertaining, practical, and transparent. The authors suggest two procedures for achieving this aim by using balls and bowls in a way, which is suitable for a nice television show—but, in contrast to the proposals of Guyon,¹⁵ they use computer draws at some stage which may threaten transparency. Both algorithms can be tried interactively at http://probability.ca/fdraw/.

Other studies deal with the UEFA Champions League where a constrained draw mechanism has been used in the Round of 16 between the 2003/04 and 2023/24 seasons. However, this method is distinct from the FIFA World Cup draw procedure.²¹ Wallace and Haigh²⁶ verify that the possible assignments are not equally likely and highlight the connection of the UEFA draw mechanism to Hall’s marriage theorem, see also Haigh²⁷ Section 3.6. Kiesl²⁸ computes the bias in the 2012/13 season and outlines some fair—but uninteresting to watch—methods. According to Klößner and Becker,²⁰ the draw system inherently implies different probabilities for certain assignments, which are translated into more than ten thousand Euros in expected revenue due to the substantial amount of prize money. Finally,¹⁹ reveal how the UEFA draw procedure affects expected assignments and address the normative question of whether a fairer randomisation mechanism exists. The current design is verified to come quantitatively close to a constrained best in fairness terms.

Theoretical background

A permutation of a set is a rearrangement of its elements. In the FIFA World Cup draw, the initial permutation of the teams is provided by a random draw. In an unrestricted group draw, the teams can be assigned to the groups in this permutation. However, in the presence of draw conditions, it is not obvious to find the permutation of the teams that corresponds to the feasible allocation implied by the draw procedure of FIFA.

This mechanism is defined as follows: “when a draw condition applies or is anticipated to apply, the team drawn is allocated to the first available group in alphabetical order”⁶. In other words, the team drawn is assigned to the first empty slot except if all permutations of the remaining teams violate at least one draw condition.

Example 1

Assume that there are $k = 4$ groups A–D and $n = 4$ teams $T 1$ – $T 4$ drawn sequentially from a pot. The sequence of permutations to be checked according to the draw mechanism is shown in Figure 1. Team $T 1$ is assigned to group A in the first six permutations because it can be placed in another group only if either group A is unavailable for team $T 1$ or teams $T 2$ – $T 4$ cannot be assigned to groups B–D.

Figure 1.

The sequence of permutations implied by the draw procedure of FIFA, $n = 4$ .

Let us consider two illustrative cases, where the restrictions are implied by the assignment of teams drawn from the previous pot(s):

If team $T 1$ cannot be placed in group A and team $T 3$ cannot be placed in group C, then the first six permutations are unacceptable due to the first constraint, and permutation 7 is skipped because of the second condition. The outcome of the draw procedure is permutation 8 ( $T 2$ , $T 1$ , $T 4$ , $T 3$ ).

The draw procedure of FIFA works as follows: ⋄

Team $T 1$ cannot be assigned to the first empty slot in group A, hence, it is placed in the first available group, which is group B.

⋄

Team $T 2$ is assigned to the first empty slot in group A.

⋄

Team $T 3$ cannot be assigned to the first empty slot in group C, hence, it is placed in the first available group, which is group D.

⋄

Team $T 4$ is assigned to the first (and last) empty slot in group C.

If teams $T 2$ – $T 4$ cannot be placed in group C and team $T 2$ cannot be placed in group A, then the first 12 permutations are unacceptable due to the first constraint, and the next two are skipped because of the second condition. The outcome of the draw procedure is permutation 15 ( $T 3$ , $T 2$ , $T 1$ , $T 4$ ).

The draw procedure of FIFA works as follows: ⋄

No constraint prohibits directly the assignment of team $T 1$ to group A. However, if team $T 1$ is not assigned to group C, then the three remaining teams $T 2$ – $T 4$ cannot be assigned to groups B–D. A similar argument uncovers that team $T 1$ should be assigned to group C.

⋄

Team $T 2$ cannot be assigned to the first empty slot in group A, hence, it is placed in the first available group, which is group B.

⋄

Team $T 3$ is assigned to the first empty slot in group A.

⋄

Team $T 4$ is assigned to the first (and last) empty slot in group D.

The official video of the 2018 FIFA World Cup group draw is available at https://www.youtube.com/watch?v=jDkn83FwioA.

Generating all permutations of a sequence of values is a famous problem in computer science.²⁹ The classic lexicographic algorithm goes back to Narayaṇa Paṇḍita, an Indian mathematician from the 14th century.³⁰ The sequence corresponding to the FIFA World Cup draw procedure is called representation via swaps³¹ and has been presented first in Myrvold and Ruskey³² according to our knowledge. In particular, Figure 1 is analogous to Arndt³¹ Figure 10.1-E.

The description of the 2018 FIFA World Cup draw³³ does not give an exact algorithm to obtain the implied feasible allocation of the teams into groups for a given random permutation of the teams. The scheme of an appropriate computer program is presented in Figure 2. Pot $j$ , from which team $i$ is drawn, is not a variable, however, the number of available groups can be different for each pot. The algorithm is based on backtracking: if the remaining teams cannot be assigned to the empty group slots in any permutation such that all restrictions are satisfied, then the last team is placed in the next available group in alphabetical order. This process is repeated until the associated feasible allocation is obtained or the non-existence of a valid assignment is verified.

Figure 2.

A backtracking algorithm for restricted group draw that finds the feasible allocation corresponding to a given order of the teams.

Backtracking can be familiar from the problem of scheduling round-robin tournaments, where an unlucky assignment of games to slots can result in a schedule that cannot be completed.^34,35 Backtracking is also widely used to solve puzzles such as the eight queens puzzle, crosswords, or Sudoku. For draw procedures under draw constraints, backtracking has been suggested first in Guyon.¹⁵

The (un)fairness of the 2018 FIFA World Cup draw

For the 2018 FIFA World Cup draw, pots were constructed based on the October 2017 FIFA World Ranking such that Pot $j$ contained the teams ranked between $8 (j - 1) + 1$ and $8 j$ . The only exception was the assignment of the host Russia to Pot 1 despite being the lowest-ranked among all participants.

The draw sequence started with Pot 1 and ended with Pot 4. Each pot was emptied before the next was drawn and some draw conditions were applied³³:

Russia was automatically placed in Group A.

No group could have more than one team from any continental confederation except for UEFA (AFC, CAF, CONMEBOL, CONCACAF).

Each group should have contained at least one but no more than two European teams.

The composition of the pots is shown in Table 1.

Table 1.

Seeding pots in the 2018 FIFA World Cup.

Country	Confederation	Elo	Country	Confederation	Elo
Pot 1			Pot 2
1 Russia	UEFA	1678	9 Spain	UEFA	2044
2 Germany	UEFA	2077	10 Peru	CONMEBOL	1916
3 Brazil	CONMEBOL	2141	11 Switzerland	UEFA	1889
4 Portugal	UEFA	1969	12 England	UEFA	1948
5 Argentina	CONMEBOL	1985	13 Colombia	CONMEBOL	1927
6 Belgium	UEFA	1937	14 Mexico	CONCACAF	1850
7 Poland	UEFA	1831	15 Uruguay	CONMEBOL	1893
8 France	UEFA	1986	16 Croatia	UEFA	1853
Pot 3			Pot 4
17 Denmark	UEFA	1856	25 Serbia	UEFA	1777
18 Iceland	UEFA	1764	26 Nigeria	CAF	1684
19 Costa Rica	CONCACAF	1743	27 Australia	AFC	1741
20 Sweden	UEFA	1795	28 Japan	AFC	1684
21 Tunisia	CAF	1655	29 Morocco	CAF	1733
22 Egypt	CAF	1643	30 Panama	CONCACAF	1658
23 Senegal	CAF	1749	31 South Korea	AFC	1713
24 Iran	AFC	1790	32 Saudi Arabia	AFC	1586

The number before each country indicates its rank among the FIFA World Cup participants according to the October 2017 FIFA World Ranking, except for the host Russia, which automatically occupies the first position.

The column Elo shows the strength of the teams according to the World Football Elo Ratings on 13 June 2018, see https://www.international-football.net/elo-ratings-table?year=2018&month=06&day=13&confed=&. The 2018 FIFA World Cup started on 14 June 2018. This measure quantifies the strengths of the teams in our simulation.

The effect of draw order

According to Roberts and Rosenthal¹⁸ Section 2, the FIFA World Cup draw procedure is unfair since some feasible allocations might occur with a higher probability. In addition, the pre-assignment of Russia to Group A introduces a powerful bias because the draw mechanism is not independent of group labels. Russia has a 12.5% probability of playing against an arbitrarily chosen country from Pot 2 because no draw constraints can apply. However, since there are one CONCACAF, three CONMEBOL, and four UEFA members in Pot 2, the two CONMEBOL teams from Pot 1 (Brazil and Argentina) play against a given European team from Pot 2 with a probability of 0.2 since they have five possible opponents from Pot 2. The remaining five UEFA teams in Pot 1 are identical concerning the draw constraints, thus, they have a chance of $(100 - 12.5) / 5 = 17.5$ % to be assigned to the same group as a given South American team from Pot 2. Analogously, the probability that two given European teams from Pots 1 and 2 play against each other equals $(100 - 2 \times 20 - 12.5) / 5 = 9.5$ %.

However, an appropriate relabelling of the pots may bring the FIFA World Cup draw closer to the uniform distribution. Therefore, all possible draw orders of the pots are examined such that the pre-assignment of Russia to Group A is retained. Since the teams can be drawn in $7! \times (8!)^{3} \approx 3.3 \times 10^{17}$ different draw orders, it is almost impossible to derive exact theoretical results. Consequently, the 25 draw mechanisms—the draw procedure of FIFA with the 24 possible draw orders of the pots and the uniformly distributed rejection mechanism¹⁸—are analysed on the basis of 1 million randomly generated draw orders, analogous to Roberts and Rosenthal.¹⁸

The rejection mechanism (or rejection sampler) $U$ works as follows. The groups are drawn completely randomly disregarding any geographical constraint. Then, it is checked whether all restrictions are satisfied. If yes, the draw is retained as a valid one. If no, the current draw is rejected, and a new random group draw is generated. The rejection mechanism is uniform over all possible valid draws, see, for example, Devroye³⁶ Section II.3. In the case of the 2018 FIFA World Cup, about one of 161 random draws is accepted, which is less than one out of 560 as for the 2022 FIFA World Cup.¹⁸ Consequently, the total number of valid draws is approximately $7! \times (8!)^{3} / 161 \approx$ $2.05 \times 10^{15}$ .

The distortions compared to the rejection mechanism $U$ are quantified through the difference of the probabilities that two given teams are placed in the same group. The bias $Δ_{i j}$ of mechanism $M$ for teams $i$ and $j$ is suggested to be

Δ_{i j} = p_{i j}^{M} - p_{i j}^{U}

where

p_{i j}^{M}

and

p_{i j}^{U}

are the probabilities that teams

i

and

j

are assigned to the same group by a draw mechanism

M

and the rejection mechanism

U

, respectively. When the biases are aggregated for the teams, we will always sum their absolute value

| Δ_{i j} |

The most extreme and aggregated distortions are presented in Table 2. For example, the probability that Denmark (17) (or the equivalent team of Sweden (18) or Iceland (20)) and Serbia (25) play in the same group is more than doubled by the FIFA draw mechanism with the traditional draw order 1-2-3-4. On the other hand, the likelihood of assigning Russia (1) and Serbia to the same group is decreased by 4.68 percentage points, while the likelihood of assigning Mexico (14) and Serbia to the same group is increased by 10.29 percentage points.

Table 2.

The deviations of different draw mechanisms in the 2018 FIFA World Cup.

Draw order	Minimal nonzero probability	Maximal probability	Maximum of positive ( $+$ ) biases	Maximum of negative ( $-$ ) biases	Sum of absolute biases
1-2-3-4	3.26%	32.98%	10.29% (19)	4.68% (7)	3.007 (1)
1-2-4-3	1.84%	29.62%	9.97% (18)	4.18% (1)	3.122 (3)
1-3-2-4	1.18%	28.36%	8.46% (15)	5.17% (11)	3.867 (14)
1-3-4-2	2.86%	28.41%	3.86% (4)	5.19% (13)	3.786 (10)
1-4-2-3	1.05%	29.55%	7.35% (14)	6.33% (19)	3.364 (6)
1-4-3-2	2.12%	28.17%	3.21% (1)	6.33% (18)	3.756 (8)
2-1-3-4	3.26%	33.29%	10.60% (20)	4.67% (6)	3.104 (2)
2-1-4-3	1.75%	29.64%	9.96% (17)	4.23% (2)	3.206 (4)
2-3-1-4	1.00%	28.44%	6.08% (9)	5.69% (16)	5.174 (22)
2-3-4-1	1.67%	28.62%	6.02% (8)	4.87% (8)	4.825 (21)
2-4-1-3	1.11%	29.43%	6.84% (11)	7.45% (23)	3.810 (11)
2-4-3-1	1.48%	28.47%	6.74% (10)	7.53% (24)	4.217 (16)
3-1-2-4	1.18%	28.36%	8.47% (16)	5.17% (12)	3.844 (12)
3-1-4-2	2.85%	28.50%	3.81% (3)	5.17% (10)	3.775 (9)
3-2-1-4	0.95%	28.64%	5.71% (7)	5.29% (15)	4.628 (18)
3-2-4-1	1.37%	28.68%	5.48% (6)	5.07% (9)	4.397 (17)
3-4-1-2	4.13%	39.07%	16.38% (24)	4.52% (4)	4.653 (19)
3-4-2-1	5.72%	35.46%	12.77% (22)	5.26% (14)	5.619 (23)
4-1-2-3	1.05%	29.59%	7.33% (13)	6.27% (17)	3.341 (5)
4-1-3-2	2.12%	28.21%	3.28% (2)	6.38% (20)	3.859 (13)
4-2-1-3	1.09%	29.63%	6.94% (12)	6.83% (22)	3.503 (7)
4-2-3-1	1.35%	28.53%	5.24% (5)	6.76% (21)	4.138 (15)
4-3-1-2	4.11%	37.96%	15.27% (23)	4.24% (3)	4.656 (20)
4-3-2-1	5.47%	34.45%	11.76% (21)	4.67% (5)	5.671 (24)
Uniform	1.56%	28.74%	—	—	—

The numbers in parenthesis indicate the ranks of FIFA draw procedure with the draw order in the row according to the measure of deviation in the column.

The standard error of $Δ_{i j}$ equals

{S E}_{i j} = \sqrt{\frac{p_{i j}^{M} (1 - p_{i j}^{M})}{N}} + \sqrt{\frac{p_{i j}^{U} (1 - p_{i j}^{U})}{N}}

Since any

p_{i j}^{M}

and

p_{i j}^{U}

is at most

0.3907

according to Table 2 and

N = 10^{6}

{S E}_{i j} \leq 0.00098

The last column of Table 2 compares the 24 draw orders by adding the values of $Δ_{i j}$ for the 365 allowed country pairs. The bounds of the 99% confidence intervals are given by

\pm 2.58 \times \sum_{i, j} {S E}_{i j} \leq \pm 2.58 \times \sqrt{365} \times 0.00098 = \pm 0.048

Hence, the official draw order 1-2-3-4 is not only optimal but it is significantly fairer than any other draw order. Furthermore, Table 2 uncovers that there are some low-performing draw orders, dominated by another draw order, such as 3-4-2-1 and 4-3-2-1.

Table 3 presents fairness distortions for two fundamentally different draw orders. Clearly, Russia and Mexico pose challenges for both of them. However, the official mechanism (draw order 1-2-3-4) is clearly better for the country pairs from Pot 1 and Pot 2, as well as from Pot 1 and Pot 3, which is favourable as these teams have the highest chance to qualify for the Round of 16.

Table 3.

Fairness distortions for selected draw orders in the 2018 FIFA World Cup.

The distortions are also worth studying by taking the average of absolute or squared biases for all country pairs that involve a given national team. For the sake of simplicity, four draw orders are analysed in detail: the official 1-2-3-4, one that improves both the minimal nonzero and maximal probabilities with a slight change (1-2-4-3), one that is less biased for Mexico (1-3-2-4), and a fundamentally different draw order (4-2-3-1).

Figure 3 compares the means of the biases for the 32 countries. The official draw procedure is far from the fair rejection mechanism $U$ in the case of Russia (1), Brazil (3), Argentina (5), Mexico (14), and Serbia (25). Note that Brazil and Argentina are interchangeable in the draw, thus, the deviations for these two countries are different only due to the (seemingly low) sampling error. Reversing the draw order of Pots 3 and 4 reduces unfairness for Russia and Serbia but it does not treat the problem for Mexico. Contrarily, the deviation for Costa Rica (19) strongly increases with the draw order 1-2-4-3. Drawing Pot 3 immediately after the best teams in Pot 1 implies a relatively low distortion in the case of Russia, and, especially, Mexico but the situation of the African (21, 22, 23) and European (17, 18, 20) teams in Pot 3 becomes less fair compared to the draw order 1-2-4-3. Finally, the draw order 4-2-3-1 is unfair with respect to all countries in Pot 2 except for Mexico.

Figure 3.

The average biases of different draw procedures for the national teams in the 2018 FIFA World Cup.

The consequences of non-uniform distribution

In the 2018 FIFA World Cup, the top two teams from each group have advanced to the Round of 16. Therefore, the distortions of the draw procedure are important primarily if they affect the probability of qualification for the knockout stage. To that end, the simulation methodology of Football rankings³⁷ is used. This models the number of goals scored in a match by Poisson distribution: the expected number of goals is a quartic polynomial of win expectancy as estimated by a least squares regression based on more than 29 thousand home-away games and almost 10 thousand games played on neutral ground between national football teams.³⁷ Win expectancy depends on the strengths of the teams according to a well-established metric,^38,39 the World Football Elo ratings (http://eloratings.net/about), and the field of the match, which is neutral except for Russia, the host.

In particular, the probability that team $i$ scores $k$ goals against team $j$ equals

P_{i j} (k) = \frac{{(λ_{i j}^{(f)})}^{k} \exp (- λ_{i j}^{(f)})}{k!}

where

λ_{i j}^{(f)}

is the expected number of goals scored by team

i

against team

j

if the match is played on field

f

(home:

f = h

; away:

f = a

; neutral:

f = n

World Football Elo ratings determine the win expectancy $W_{i j}$ of team $i$ against team $j$ as

W_{i j} = \frac{1}{1 + 10^{- (E_{i} - E_{j}) / 400}}

where

E_{i}

and

E_{j}

are the Elo ratings of the two teams, respectively. The rating of the home team (here, Russia) is increased by

100

to reflect home advantage.

Football rankings³⁷ estimates how $λ_{i j}^{(f)}$ depends on $W_{i j}$ by least squares regressions with a regime change at $W_{i j} = 0.9$ due to the excessive number of goals scored in unbalanced matches. Most games of the FIFA World Cup are played on neutral field when

λ_{i j}^{(n)} = {\begin{matrix} 3.90388 \cdot W_{i j}^{4} - \\ - 0.58486 \cdot W_{i j}^{3} - \\ - 2.98315 \cdot W_{i j}^{2} + \\ + 3.13160 \cdot W_{i j} + \\ + 0.33193 & if W_{i j} \leq 0.9 \\ 308097.45501 \cdot (W_{i j} - 0.9)^{4} - \\ - 42803.04696 \cdot (W_{i j} - 0.9)^{3} + \\ + 2116.35304 \cdot (W_{i j} - 0.9)^{2} - \\ - 9.61869 \cdot (W_{i j} - 0.9) + \\ + 2.86899 & if W_{i j} > 0.9 \end{matrix}

In Group A, three home-away matches are played by Russia and another team. Then the expected number of goals scored by the host Russia (denoted by

R

) equals

λ_{i j}^{(h)} = {\begin{matrix} - 5.42301 \cdot W_{R j}^{4} + \\ + 15.49728 \cdot W_{R j}^{3} - \\ - 12.6499 \cdot W_{R j}^{2} + \\ + 5.36198 \cdot W_{R j} + \\ + 0.22863 & if W_{R j} \leq 0.9 \\ 231098.16153 \cdot (W_{R j} - 0.9)^{4} - \\ - 30953.10199 \cdot (W_{R j} - 0.9)^{3} + \\ + 1347.51495 \cdot (W_{R j} - 0.9)^{2} - \\ - 1.63074 \cdot (W_{R j} - 0.9) + \\ + 2.54747 & if W_{R j} > 0.9 \end{matrix}

and the expected number of goals scored by the away team

j

λ_{i j}^{(a)} = {\begin{matrix} 90173.57949 \cdot (W_{R j} - 0.1)^{4} + \\ + 10064.38612 \cdot (W_{R j} - 0.1)^{3} + \\ + 218.6628 \cdot (W_{R j} - 0.1)^{2} - \\ - 11.06198 \cdot (W_{R j} - 0.1) + \\ + 2.28291 & if W_{R j} < 0.1 \\ - 1.25010 \cdot W_{R j}^{4} - \\ - 1.99984 \cdot W_{R j}^{3} + \\ + 6.54946 \cdot W_{R j}^{2} - \\ - 5.83979 \cdot W_{R j} + \\ + 2.80352 & if W_{R j} \geq 0.1 \end{matrix}

The same simulation methodology has recently been applied in some studies on tournament design.^40,17,41 Note that even if this Elo-based approach is not necessarily the best available simulation model, it is only used to compare the outcomes under a uniform and a non-uniform draw. Thus, our assumptions seem to be adequate for comparative purposes as emphasised by Appleton⁴² p. 534 with the following words: “Since our intention is to compare tournament designs, and not to estimate the chance of the best player winning any particular tournament, we may within reason take whatever model of determining winners that we please”.

Last but not least, a ranking should be established based on the results of group matches. According to the rules of the 2018 FIFA World Cup, the first tie-breaking criteria are as follows⁴³ Article 20.6: (1) higher number of points obtained in all group matches; (2) superior goal difference in all group matches; (3) higher number of goals scored in all group matches. In contrast to the official regulation, if some teams are still tied after considering (1)–(3), their ranking is decided by drawing of lots in our simulation.

Thus, a simulation run consists of the following steps for any draw mechanism (uniform or FIFA World Cup draw procedure with a given draw order):

The eight groups are drawn according to the draw mechanism used (1 million times);

The outcomes of all group matches are generated (10 times for each simulated draw);

The group rankings are determined.

The results for any team are the number of times it finishes on the first and the second place in its group, which are at most 10 million, respectively.

Figure 4 shows how the FIFA World Cup draw mechanism distorts the chances of winning the group and being the runner-up. The official rule (draw order 1-2-3-4) increases the probability of winning the group by more than 1 percentage point for the four UEFA teams in Pot 2 (Spain [9], Switzerland [11], England [12], Croatia [16]) mostly at the expense of Mexico (14) and Serbia (25). Fortunately, these effects are somewhat mitigated by taking the likelihood of obtaining the second position into account. While the draw order 1-2-4-3 does not differ much from the traditional order of 1-2-3-4, the draw order 1-3-2-4 strongly reduces the distortions in the case of the above countries except for Serbia. Among the four draw orders, 4-2-3-1 is the less favourable for the weakest European team in Pot 2 (Croatia [16]) and the three UEFA members in Pot 3 (Denmark [17], Iceland [18], Sweden [20]).

Figure 4.

The effect of different draw procedures on being the group winner and the runner-up in the 2018 FIFA World Cup.

The probabilities of being the group winner or the runner-up are aggregated in Figure 5. The draw order 1-2-4-3 is somewhat better than the official 1-2-3-4: despite the increased impact on Costa Rica, it is less unfair for Russia, Croatia, Denmark, Iceland, and Sweden. The draw order 1-3-2-4 can be chosen if the bias for Mexico should be reduced, however, that is achieved at the expense of Serbia. The draw order 4-2-3-1 is not worth implementing because of the high distortions for several countries. These observations are reinforced by the summary statistics provided in Table 4.

Figure 5.

The absolute effect of different draw procedures on the probability of qualification for the knockout stage in the 2018 FIFA World Cup.

Table 4.

Summary statistics on the absolute changes of qualifying probabilities by four draw procedures in the 2018 FIFA World Cup.

Draw order	1-2-3-4			1-2-4-3			1-3-2-4			4-2-3-1
Change (pp)	GW	RU	Q	GW	RU	Q	GW	RU	Q	GW	RU	Q
$> 1$	4	0	1	4	0	0	1	0	1	1	0	1
$0.5$ – $1$	2	2	2	2	2	2	2	0	2	1	3	5
$0.25$ – $0.5$	1	7	6	1	5	4	2	3	4	7	4	8
$- 0.25$ – $0.25$	12	17	14	13	19	19	24	26	15	17	18	8
$- 0.5$ – $- 0.25$	7	0	5	7	0	6	2	3	9	1	4	2
$- 1$ – $- 0.5$	5	5	3	4	5	0	0	0	0	4	3	6
$< - 1$	1	1	1	1	1	1	1	0	1	1	0	2

Column GW/RU/Q shows the number of teams for which the absolute change in percentage points of the probability of winning the group/being the runner-up/qualification by the given draw order (the probability according to the official draw mechanism with the given draw order minus the probability according to a uniform draw) is within the given interval.

Finally, Figure 6 uncovers the relative changes in the probability of qualification for the knockout stage as the same absolute distortions can be more costly for the weaker teams. In this respect, the draw order 1-2-4-3 can be a reasonable alternative to the official 1-2-3-4 as it is less biased for almost all teams except for Costa Rica as shown by Table 5. The other two draw orders are especially unfavourable for Serbia and the three UEFA (draw order 4-2-3-1) or the three CAF (draw order 1-3-2-4) teams in Pot 3.

Figure 6.

The relative effect of different draw procedures on the probability of qualification for the knockout stage in the 2018 FIFA World Cup.

Table 5.

Summary statistics on the relative changes of qualifying probabilities by four draw procedures in the 2018 FIFA World Cup.

Change	1-2-3-4	1-2-4-3	1-3-2-4	4-2-3-1
$> 3 %$	0	0	0	1
$2 %$ – $3 %$	3	1	3	4
$1 %$ – $2 %$	6	4	2	4
$- 1 %$ – $1 %$	16	25	21	16
$- 2 %$ – $- 1 %$	5	1	3	3
$- 3 %$ – $- 2 %$	2	1	2	2
$< - 3 %$	0	0	1	2

The cells show the number of national teams for which the relative change in percentage of the probability of qualification by the given draw order (one minus (the probability according to the official draw mechanism with the given draw order divided by the probability according to a uniform draw)) is within the appropriate interval.

Based on the arguments above, the official draw order of Pot 1, Pot 2, Pot 3, Pot 4 has been a lucky choice for the 2018 FIFA World Cup draw. But a change in the continental allocation of the teams playing in the FIFA World Cup may lead to a highly unfair draw in the future if the current draw procedure is applied.

Discussion

Our paper has analysed the unfairness of the 2018 FIFA World Cup draw. First, the connection of the standard constrained group draw procedure—used by the FIBA, FIFA, and UEFA—to permutation generation has been presented. Second, we have examined how this procedure departs from a random draw among all feasible allocations and considered all alternatives by relabelling the pots. The official draw order (Pot 1, Pot 2, Pot 3, Pot 4) has turned out to be an optimal choice among the 24 alternative rules with respect to natural measures of fairness.

Even though constrained group draw is currently used only in basketball and football, draw constraints offer the unique solution to maximise the number of intercontinental games in a group stage. This can be advantageous in several tournaments; for instance, the 2023 World Men’s Handball Championship contained eight groups, one with two of the three African teams, and another with two of the four South American teams.²² Any governing body will likely adopt the procedure of FIFA if they introduce draw constraints.

We think there is at least some scope to modify the current draw order because the draw of certain sports tournaments has not been started with the strongest teams:

The lowest-ranked teams were drawn first in the 2020/21⁴⁴ and 2022/23⁴⁵ UEFA Nations League, as well as in the 2019 World Men’s Handball Championship⁴⁶;

The runners-up were drawn first in the UEFA Champions League Round of 16,^19,20 and the unseeded teams were drawn first in the UEFA Europa League Round of 32¹¹;

The unusual draw order of Pot 4, Pot 3, Pot 1, Pot 2 was followed in the draw of the 2021 World Men’s Handball Championship.⁴⁷

According to our findings, the non-uniform draw distorts the probability of qualification for eight (two) countries by more than 0.5 (1) percentage points (see Table 4). Although the bias has not exceeded 1.5 percentage points for any national team in the case of the 2018 FIFA World Cup, it has exceeded 2% for Russia, Mexico, Iceland, Tunisia, and Egypt in relative terms. To conclude, it is the responsibility of policymakers to decide whether these values justify the consideration of fairer draw mechanisms such as the algorithms proposed by Guyon¹⁵ and Roberts and Rosenthal.¹⁸

Supplemental Material

sj-py-1-spo-10.1177_17479541241300219 - Supplemental material for The fairness of the group draw for the FIFA World Cup

Supplemental material, sj-py-1-spo-10.1177_17479541241300219 for The fairness of the group draw for the FIFA World Cup by László Csató in International Journal of Sports Science & Coaching

Supplemental Material

sj-py-2-spo-10.1177_17479541241300219 - Supplemental material for The fairness of the group draw for the FIFA World Cup

Supplemental material, sj-py-2-spo-10.1177_17479541241300219 for The fairness of the group draw for the FIFA World Cup by László Csató in International Journal of Sports Science & Coaching

Supplemental Material

sj-py-3-spo-10.1177_17479541241300219 - Supplemental material for The fairness of the group draw for the FIFA World Cup

Supplemental material, sj-py-3-spo-10.1177_17479541241300219 for The fairness of the group draw for the FIFA World Cup by László Csató in International Journal of Sports Science & Coaching

Footnotes

Acknowledgements

This paper could not have been written without my father (also called László Csató), who has primarily coded the simulations in Python.

We are grateful to Julien Guyon for inspiration, and to Julien Guyon, Jeffrey S. Rosenthal, and Lajos Rónyai for useful advice.

The comments of the reviewers Martin Becker, Florian Felice, and Michael A. Lapré have greatly improved the presentation of the results.

Eleven colleagues and anonymous reviewers provided valuable remarks and suggestions on earlier drafts.

We are indebted to the Wikipedia community for summarising important details of the sports competition discussed in the paper.

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship and/or publication of this article.

Funding

The research was supported by the National Research, Development and Innovation Office under Grant FK 145838, and by the János Bolyai Research Scholarship of the Hungarian Academy of Sciences.

ORCID iD

László Csató

Supplemental material

Three Python 3.0 files are provided for the simulation experiment.

The first (FIFA_World_Cup_draw_32_teams_uniform_full.py) implements a uniform draw by the rejection mechanism.

The second (FIFA_World_Cup_draw_32_teams_uniform_full.py) implements the official FIFA World Cup draw. Draw order can be modified by changing the order of the four pots in the function draw_32_teams (lines 374–397).

The third (FIFA_WC_2018_Functions) contains some auxiliary functions that are imported in the other two files but should not be run.

Notes

References

Abdulkadiroğlu

Sönmez

. School choice: A mechanism design approach. Am Econ Rev 2003; 93: 729–747.

Roth

Sönmez

Ünver

. Kidney exchange. Q J Econ 2004; 119: 457–488.

Csató

. Tournament Design: How Operations Research Can Improve Sports Rules. Cham, Switzerland: Palgrave Macmillan, 2021. Palgrave Pivots in Sports Economics.

Kobierecki

. Politics of the group draws in football. The case of restricted team clashes. Int J Sport Policy Politics 2022; 14: 321–336.

Laliena

López

. Fair draws for group rounds in sport tournaments. Int Trans Oper Res 2019; 26: 439–457.

UEFA. FIFA World Cup 2022 qualifying draw procedure, 2020. Available at: https://www.uefa.com/MultimediaFiles/Download/competitions/WorldCup/02/64/22/19/2642219_DOWNLOAD.pdf.

FIBA. The FIBA Basketball World Cup 2023 draw principles explained. Available at: https://www.fiba.basketball/basketballworldcup/2023/news/the-fiba-basketball-world-cup-2023-draw-principles-explained (accessed 21 April 2023).

FIFA. Draw procedures. FIFA World Cup Qatar 2022

^{T M}

, 2022. Available at: https://digitalhub.fifa.com/m/2ef762dcf5f577c6/original/Portrait-Master-Template.pdf.

UEFA. UEFA EURO 2024 Qualifying Draw Procedure, 2022. Available at: https://editorial.uefa.com/resources/0279-1627b29793e1-ffbe5a3c77a1-1000/08.01.00_euro_2024_qualifying_draw_procedure_en_20220920115210.pdf.

10.

UEFA. 2024/25 UEFA Nations League Draw Press Kit, 2024. Available at: https://editorial.uefa.com/resources/028a-1a1e18631263-95c63cec8617-1000/202425_unl_qualifying.pdf.

11.

Csató

. The effects of draw restrictions on knockout tournaments. J Quant Anal Sports 2022; 18: 227–239.

12.

Jones

. The World Cup draw’s flaws. Math Gaz 1990; 74: 335–338.

13.

Rathgeber

. Why Germany was supposed to be drawn in the group of death and why it escaped. Chance 2007; 20: 22–24.

14.

Guyon

. Rethinking the FIFA World Cup

^{T M}

final draw. J Quant Anal Sports 2015; 11: 169–182.

15.

Guyon

. Rethinking the FIFA World Cup

^{T M}

final draw. 2014. Manuscript. DOI: https://doi.org/10.2139/ssrn.2424376.

16.

Guyon

. Pourquoi la Coupe du monde est plus équitable cette année. The Conversation. Available at: https://theconversation.com/pourquoi-la-coupe-du-monde-est-plus-equitable-cette-annee-97948 (accessed 13 June 2018).

17.

Csató

. Group draw with unknown qualified teams: A lesson from the 2022 FIFA World Cup. Int J Sports Sci Coach 2023; 18: 539–551.

18.

Roberts

Rosenthal

. Football group draw probabilities and corrections. Can J Stat 2024; 52: 659–677.

19.

Boczoń

Wilson

. Goals, constraints, and transparently fair assignments: A field study of randomization design in the UEFA Champions League. Manag Sci 2023; 69: 3474–3491.

20.

Klößner

Becker

. Odd odds: The UEFA Champions League round of 16 draw. J Quant Anal Sports 2013; 9: 249–270.

21.

Csató

. Random matching in balanced bipartite graphs: The (un)fairness of draw mechanisms used in sports. 2024. Manuscript. DOI: https://doi.org/10.48550/arXiv.2303.09274.

22.

Devriesere

Csató

Goossens

. Tournament design: A review from an operational research perspective. European J Oper Res, 2024; in press. DOI: https://doi.org/10.1016/j.ejor.2024.10.044.

23.

Scelles

François

Valenti

. Impact of the UEFA Nations League on competitive balance, competitive intensity, and fairness in european men’s national team football. Int J Sport Policy Politics 2024; 16: 519–537.

24.

Lapré

Palazzolo

. The evolution of seeding systems and the impact of imbalanced groups in FIFA Men’s World Cup tournaments 1954–2022. J Quant Anal Sports 2023; 19: 317–332.

25.

Cea

Durán

Guajardo

, et al. An analytics approach to the FIFA ranking procedure and the World Cup final draw. Ann Oper Res 2020; 286: 119–146.

26.

Wallace

Haigh

. Football and marriage – and the UEFA draw. Significance 2013; 10: 47–48.

27.

Haigh

. Mathematics in Everyday Life. 2nd ed. Cham, Switzerland: Springer Nature, 2019.

28.

Kiesl

. Match me if you can. Mathematische Gedanken zur Champions-League-Achtelfinalauslosung. Mitteilungen der Deutschen Mathematiker-Vereinigung 2013; 21: 84–88.

29.

Sedgewick

. Permutation generation methods. ACM Comput Surv (CSUR) 1977; 9: 137–164.

30.

Knuth

. The Art of Computer Programming, volume 4, Fascicle 2: Generating All Tuples and Permutations. Upper Saddle River, New Jersey, USA: Addison-Wesley, 2005.

31.

Arndt

. Matters Computational: Ideas, Algorithms, Source Code. Berlin, Heidelberg, Germany: Springer, 2010.

32.

Myrvold

Ruskey

. Ranking and unranking permutations in linear time. Inf Process Lett 2001; 79: 281–284.

33.

FIFA. Close-up on Final Draw procedures. Available at: https://web.archive.org/web/20171127150059/http://www.fifa.com/worldcup/news/y=2017/m=11/news=close-up-on-final-draw-procedures-2921440.html (accessed 27 November 2017).

34.

Rosa

Wallis

. Premature sets of 1-factors or how not to schedule round robin tournaments. Discrete Appl Math 1982; 4: 291–297.

35.

Schaerf

. Scheduling sport tournaments using constraint logic programming. Constraints 1999; 4: 43–65.

36.

Devroye

. Non-Uniform Random Variate Generation. New York, USA: Springer-Verlag, 1986.

37.

Football rankings. Simulation of scheduled football matches. Available at: http://www.football-rankings.info/2020/12/simulation-of-scheduled-football-matches.html (accessed 28 December 2020).

38.

Lasek

Szlávik

Bhulai

. The predictive power of ranking systems in association football. Int J Appl Pattern Recognit 2013; 1: 27–46.

39.

Gásquez

Royuela

. The determinants of international football success: A panel data analysis of the Elo rating. Soc Sci Q 2016; 97: 125–141.

40.

Csató

. Quantifying incentive (in)compatibility: A case study from sports. Eur J Oper Res 2022; 302: 717–726.

41.

Stronka

. Demonstration of the collusion risk mitigation effect of random tie-breaking and dynamic scheduling. Sports Econ Rev 2024; 5: 100025.

42.

Appleton

. May the best man win? J R Stat Soc Ser D Stat 1995; 44: 529–538.

43.

FIFA. Regulations of the 2018 FIFA World Cup Russia

^{T M}

, 2018. Available at: https://digitalhub.fifa.com/m/74fabee6a36ad3b2/original/ejmfg94ac7hypl9zmsys-pdf.pdf.

44.

UEFA. UEFA Nations League 2020/21 – league phase draw procedure, 2020. Available at: https://www.uefa.com/MultimediaFiles/Download/competitions/General/02/63/57/88/2635788_DOWNLOAD.pdf.

45.

UEFA. UEFA Nations League 2022/23 – league phase draw procedure, 2021. Available at: https://editorial.uefa.com/resources/026f-13c241515097-67a9c87ed1b2-1000/unl_2022-23_league_phase_draw_procedure_en.pdf.

46.

IHF. Draw procedure for Germany/Denmark 2019. Available at: https://archive.ihf.info/en-us/mediacentre/announcements/announcementdetails.aspx?ID=262 (accessed 22 June 2018).

47.

IHF. Egypt 2021 draw details confirmed. Available at: https://www.ihf.info/media-center/news/egypt-2021-draw-details-confirmed (accessed 25 August 2020).

48.

von Neumann

. Various techniques used in connection with random digits. In: Householder AS, Forsythe GE and Germond HH (eds) Monte Carlo Method. National Bureau of Standards Applied Mathematics Series, volume 12, chapter 13. Washington, DC: US Government Printing Office, 1951, pp.36–38.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.02 MB

0.03 MB