I present a parsimonious family of scoring methods for ranking teams in incomplete tournaments. It is especially useful if teams play different number of games. I show that this family of scoring methods satisfies common sense theoretical properties studied by the literature on paired comparisons. I analyze it in terms of efficiency, defined as how close a scoring method comes to capturing what the teams’ win percentages would have been, in a complete tournament. Finally, using data on betting odds, I calibrate the family of scoring methods to match, as closely as possible, the actual rankings that were used to determine the teams that would go on to compete for the championship of the NCAA division 1 football tournament between 2011 and 2023. I find that the rankings used by the NCAA were generally efficient and I quantify the biases present in each year’s ranking.
AusloosM. (2024). Hierarchy selection: New team ranking indicators for cyclist multistage races. European Journal of Operational Research, 314(2), 807–816.
4.
BerkerY. (2014). Tie-breaking in round-robin soccer tournaments and its influence on the autonomy of relative rankings: UEFA vs. FIFA regulations. European Sport Management Quarterly, 14(2), 194–210.
5.
Bozó kiS.CsatóL.TemesiJ. (2016). An application of incomplete pairwise comparison matrices for ranking top tennis players. European Journal of Operational Research, 248(1), 211–218.
6.
ČaklovićL.KurdijaA. S. (2017). A universal voting system based on the potential method. European Journal of Operational Research, 259(2), 677–688.
7.
CanB. (2014). Weighted distances between preferences. Journal of Mathematical Economics, 51, 109–115.
8.
ChebotarevP. (1989). Generalization of the row sum method for incomplete paired comparisons. Automation and Remote Control, 50(8), 1103–1113.
9.
ChebotarevP. (1994). Aggregation of preferences by the generalized row sum method. Mathematical Social Sciences, 27, 293–320.
10.
ChebotarevP.ShamisE. (1997). “Constructing an objective function for aggregating incomplete preferences” In A. Tangian & J. Gruber (Eds.), Constructing Scalar-Valued Objective Functions, volume 453 of Lecture Notes in Economics and Mathematical Systems (pp. 100–124). Springer, Berlin Heidelberg.
11.
ChebotarevP.ShamisE. (1998). Characterizations of scoring methods for preference aggregation. Annals of Operations Research, 80, 299–332.
12.
ChebotarevP.ShamisE. (1999). Preference fusion when the number of alternatives exceeds two: Indirect scoring procedures. Journal of the Franklin Institute, 336(2), 205–226.
CsatóL. (2013). Ranking by pairwise comparisons for Swiss-system tournaments. Central European Journal of Operations Research, 21, 783–803.
16.
CsatóL. (2017). On the ranking of a Swiss system chess team tournament. Annals of Operations Research, 254(1-2), 17–36.
17.
CsatóL. (2019a). An impossibility theorem for paired comparisons. Central European Joural of Operations Research, 27(2), 497–514.
18.
CsatóL. (2019b). Some impossibilities of ranking in generalized tournaments. International Game Theory Review, 21(1), 1940002.
19.
CsatóL. (2020). The incentive (in)compatibility of group-based qualification systems. International Journal of General Systems, 49(4), 374–399.
20.
CsatóL. (2021a). Coronavirus and sports leagues: Obtaining a fair ranking when the season cannot resume. IMA Journal of Management Mathematics, 32(4), 547–560.
21.
CsatóL. (2021b). Tournament Design: How Operations Research Can Improve Sports Rules. Palgrave Pivots in Sports Economics. Palgrave McMillan, Cham, Switzerland.
22.
CsatóL. (2022). UEFA against the champions? An evaluation of the recent reform of the Champions League qualification. Journal of Sports Economics, 23(8), 991–1016.
23.
CsatóL. (2023). How to avoid uncompetitive games? The importance of tie-breaking rules. European Journal of Operational Research, 307(3), 1260–1269.
24.
CsatóL. (2024). Club coefficients in the UEFA Champions League: Time for shift to an ELO-based formula. International Journal of Performance Analysis in Sports, 24(2), 119–134.
25.
CsatóL.TóthC. (2020). University rankings from the revealed preferences of the applicants. European Journal of Operational Research, 286(1), 309–320.
26.
DabadghaoS.VaziriB. (2022). The predictive power of popular sports ranking methods in the NFL, NBA and NHL. Operational Research, 22(3), 2767–2783.
27.
DevlinS.TreloarT. (2018). A network diffusion ranking family that includes the methods of Markov, Massey and Colley. Journal of Quantitative Analysis in Sports, 14(3), 91–101.
28.
DevriesereK.CsatóL.GoosensD. (2024). Tournament design: A review from an operational research perspective. Manuscript. DOI:https://doi.org/10.48550/arXiv.2404.05034
29.
González-DíazJ.HendrickxR.LohmannE. (2014). Paired comparisons analysis: An axiomatic approach to ranking methods. Social Choice and Welfare, 42, 139–169.
30.
GorgiP.KoopmanS.LitR. (2023). Estimation of final standings in football competitions with premature ending: The case of Covid-19. AStA Advances in Statistical Analysis, 107(1-2), 233–250.
31.
GyimesiA. (2024). Competitive balance in the post-2024 Champions League and the European Super League: A simulation study. Journal of Sports Economics, in press. https://doi.org/10.1177/15270025241249362
32.
HenrietD. (1985). The copeland choice function: An axiomatic characterization. Social Choice and Welfare, 2(1), 49–63.
Palacios-HuertaI.VolijO (2004). The measurement of intellectual influence. Econometrica, 72(3), 963–977.
43.
PetróczyD. (2021). An alternative quality of life ranking on the basis of remittances. Socio-Economic Planning Sciences, 78, 101042.
44.
RubinsteinA. (1980). Ranking the participants in a tournament. SIAM Journal on Applied Mathematics, 38(1), 108–111.
45.
SauerP.CsehÁ.LenznerP. (2024). Improving ranking quality and fairness in Swiss-system chess tournaments. Journal of Quantitative Analysis in Sports, 20(2), 127–146.
46.
ScarfP.ParmaR.McHaleI. (2019). On outcome uncertainty and scoring rates in sport: The case of international Rugby Union. European Journal of Operational Research, 273(2), 721–730.
47.
ScarfP.YusofM.BilbaoM. (2009). A numerical study of designs for sporting contests. European Journal of Operational Research, 198(1), 190–198.
48.
SlutzkiG.VolijO. (2006). Scoring of web-pages and tournaments-axiomatizations. Social Choice and Welfare, 26(1), 75–92.
SternH. (1991). On the probability of winning a football game. The American Statistician, 45(3), 179–183.
51.
SziklaiB.BiróP.CsatóL. (2022). The efficacy of tournament designs. Computers and Operations Research, 144, 105821.
52.
VanEetveldeH.HvattumM.LeyC. (2023). The probabilistic final standing calculator: A fair stochastic tool to handle abruptly stopped football seasons. AStA Advances in Statistical Analysis, 107(1-2), 251–269.
53.
VaziriB.DabadghaoS.YihY.MorinT. (2018). Properties of Sports Ranking Methods. Journal of the Operational Research Society, 69(5), 776–787.
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
