The 20 th Top Chess Engine Championship,TCEC20

The TCEC Season 20 Championship started on 30^th November, 2020, shortly after TCEC19 ended (Haworth and Hernandez, 2020; TCEC, 2020a/b). The league structure was extended to six tiers to accommodate 39 engines, five of which were new to TCEC. These were Halogen, Koivisto, MrBob, Seer and Stash: a hearty welcome to their respective authors.

Four engines were either promoted or demoted throughout. After Stockfish adopted NNUE successfully for TCEC19, it was going to be interesting to see what other engines included the NNUE code library (Cong, 2020) deemed freely accessible. TCEC discarded the TCEC ‘win rule’ adjudication so decisive games were played to mate or to 6-man finales. TCEC also made the first two tiebreakers head-to-head results and Aloril’s r-mobility (Haworth, 2021).1

¹

In drawn chess games, the r-mobility rule of Aloril’s Mobility Chess is that the side wins which, since the last zeroing of the ply-count, most constrains the opponent first, preferably with a checking move.

The platform for the ‘Shannon AB’ engines was as for TCEC19, courtesy of ‘noobpwnftw’, the major sponsor, four Intel (2016) Xeon E5-4669v4 processors: Linux, 88 cores, 176 process threads and 128GiB of RAM with the sub-7-man Syzygy ‘EGT’ endgame tables in their own 1TB RAM. The TCEC GPU server was a 2.5GHz Intel Xeon Platinum 8163 CPU providing 32 threads, 48GiB of RAM and four Nvidia (2019) V100 GPUs. It is not clear how many CPU threads each NN-engine used. The ‘EGT’ platform was less than on the CPU side: 500 GB of SSD fronted by a 128GiB RAM buffer.

Fig. 1.

The logos for the engines originally in the TCEC20 Qualification League and in Leagues 4, 3, 2 and 1.

Table 1

The TCEC20 engines (CPW, 2020)

1. The Qualification League: 7 engines, 1 DRR, 42 games @ 30′+5″/m

With four promotion places on offer, there were plenty of incentives for the engine-authors if not for the engines. Koivisto spurned the opportunity with crashes in games 1, 11, 12 and 20 but now, engines are not discounted after three crashes. The kibitzing Stockfish-variety Crystal with its 7-man EGTs, clearly sniffed a win in game 4.3/21 and saw it vaporised by A0lite twice before the 50-move draw. Half-time standings were Seer +4, Halogen and MrBob +2, Cheese 0 and A0lite −1.

Koivisto crashed again in games 33, 34 and 41 and was looking good in this last against MrBob. It set a somewhat unwanted TCEC record with these 7 crashes and would still have been controversially promoted had not A0lite beaten Cheese in the final game. Koivisto promisingly won all the games it did not forfeit. Knockout punch, glass chin? Next time, it should be better. Seer, Halogen and MrBob promoted as expected. Wool (2020) touched on games 2, 5, 6, 18, 21, 25, 27, 28, 37 and 41. Without the TCEC win rule, many games did end in mate, see Table 9.

Table 2

The TCEC20 Qualification League cross-table

2. League 4: 10 engines, 1 DRR, 2 RRs, 90 games @ 30′+5″/m

First, an apology to Assaf Wool who contacted us during this event. Sadly, overtrained by two other commentators ‘GMTheChessPuzzler’ and ‘Kingscrusher’, we mistakenly took his actual name for a pseudonym. We could have been more successful in looking for his home page.

‘Cato’, and here the inverted commas are well placed, provided 2-move openings for this league. There were some upsets in the opening rounds. Monolith lost to A0lite in the opening game, and again to Fabchess in game 3.2/12. In its turn and in g4.4/19, Monolith beat Weiss, the clear leader at half-time. At that point, promotee Seer (+4) was second and Tucano (+2) third.

In the return games of the second half, A0lite – to its detriment, not using EGTs – missed a 7-man win in game 14.1/66 with 51.Ra6 when one of R(c/g/h)7/Ra6 was required. There were more upsets: MrBob beat Weiss in game 12.3/58 and in the next game, Halogen beat Tucano. Seer lost to ChessFighter in g17.3/83. The three leaders were however not to be overtaken and duly promoted. Particular congratulations to newcomer Seer’s connections. Bagatur could not land a blow on the top seven, was not helped by the one technical default and was duly relegated.

Table 3

The TCEC20 League 4 cross-table

3. League 3: 10 engines, 1 DRR, 2 RRs, 90 games @ 30′+5″/m

Cato’s next book provided 4-move openings here. As there was some churn at the top of the standings, principally because Weiss’ star faded, the surprise results at the time are not quite the surprise results one sees in retrospect. But as the event has happened, we take the latter perspective with the full benefit of hindsight. Weiss seemed the sole giant-slayer in the first half. It beat Combusken in game 6.2/27 and beat Seer from the Black side in g7.5/35. Seer did not have use of the provided EGTs and let go of the latter game with 60.Nxb6+, Fig. 2a. After nine rounds, the leading standings were Minic +4, Nemorino +3, Weiss, Combusken and Tucano +2.

In the second half, a rare gem came into view: g15.5/75, Marvin-Pirarucu could have ended with 67...f1=B#, Fig. 2b. Sadly, this underpromotion was not necessary and the more prosaic 67...f1=Q# was played instead. No doubt, other interesting moments escape those not in the chat room. Weiss lost to Topple in g13.3/63 but beat Tucano in g17.1/81. Shortly after and more significantly, Combusken joined the victorious underdogs, beating Minic in g17.3/83. There were no other upsets. Nemorino and Minic moved clear to promote, unbeaten Nemorino given the leading place only on the basis of Aloril’s r-mobility score.2

²

Principally because of Nemorino’s 66.Rh8+ giving it goal G_1.0 against Minic, restricting it to just one legal move in position 7R/8/1r5k/6pP/6K1/6P1/8/8 b, a subtle ‘two point swing’ of a sort.

The splendid newcomer Seer earned its luck, also moving up as Demolito did not take up its position in League 2. Marvin made a strong claim for the first relegation place and was not opposed. Pirarucu took the penultimate place and goes down unless reprieved.

Table 4

The TCEC20 League 3 cross-table

Fig. 2.

L3 (a) g35 Seer–Weiss 60.Nxb6??, (b) g75 Marvin–Pirarucu 67...f1=Q(/B)#; L2 (c) g17 Seer–Defenchess 67...Ke7 force, (d) g24 Defenchess–Winter RP-bp win, (e) g49 Nemorino-Minic 76...Kg7 force.

4. League 2: 10 engines, 1 DRR, 2 RRs, 90 games @ 30′+5″/m

Eduardo Saucedo again cut loose with his even more aggressive openings which came in a variety of lengths. In game 24, the ‘6-man adjudication’ rule pre-empted Defenchess which, informed by the EGTs, was about to demonstrate a tortuous 55-move RP-bp mate (Müller and Konoval, 2019, esp. §2c3), Fig. 2d. Pedone sustained the two most surprising first half losses, first to Arasan in g23 and then in the last game g45 to Winter. The contest remained close at both ends of the table.

In the second half, a private relegation battle developed between Wasp, Minic and Seer while the other promotee, Nemorino, headed the rest of the field in close order. It was the last to lose its unbeaten record, Arasan victorious in g52. Fritz beat Pedone in g46 but Pedone completed the double over Vajolet2 in g55. Nemorino also took Vajolet2 ‘to love’, a remarkable nulling at the top of the table. In game 69, Minic beat Pedone, the chief donor to the lower half of the table.

Seer just escaped relegation on r-mobility grounds. This was because it restricted Defenchess to one move with 96.Qh8+in game 17, Fig. 2c, while Nemorino restricted Minic to one move with 76.Qh4+ in game 49, Fig. 2e. And so a splendid run of three Seer promotions ended with a ‘hold’. Minic and Wasp are therefore destined for League 3 in TCEC21. Defenchess itself benefited from the r-mobility metric and promoted in a less than intuitive way which needs some explaining.

Game 50 between Defenchess and Arasan was terminated by the 50-move draw rule but the rules (TCEC, 2020c) required that in ‘EGT positions’, that rule should be ignored. Unfortunately for Vajolet2, Arasan saw the 50-move finishing line approaching and played 115...Ke6 which loses if the plycount is ignored. Defenchess was therefore credited with goal G0.0 in the r-mobility context and this won it the tiebreak. Hopefully, Vajolet2 will feature again in TCEC21.

Table 5

The TCEC20 League 2 cross-table

5. League 1: 8 engines, 2 DRRs, 4 RRs, 112 games @ 45′+5″/m

Here, we moved into the more leisurely tempo of 45′ per side. The distinguished academic Bastiaan Braams also brought some very biased openings to the table. So, from the off, it was clear that the engines’ prospects would not be clear until White’s advantage had been handed out to both sides in turn. In the first round robin, every engine lost and, apart from Xiphos, lost at least twice. Ethereal conceded to Igel and Rubichess in games 6 and 24 but still led the field by half a point. rofChade lost to Xiphos but won the return game with colours reversed. Wool (2020) himself reserved judgement until colours were fully reversed but then touched on the mini-matches starting with games 1-3, 6-7, 16-18, 22, 25 and 27. Add 28 for the return games.

At the half-way point, the first reasonable opportunity to sample the ranking with any confidence, the leading standings were Ethereal/rofChade +2 and Igel +1. Defenchess seemed untimely thrust upon the stage with −4 but here the similarity to Richard III ended. It held its own in the second half.

In the third round robin, Igel was the mover and shaker. Ethereal lost to Igel again, this time in g62. In its turn, rofChade lost to Igel in g83. However, Igel’s prospects had been set back by a loss to Xiphos in g78. In none of these mini-matches were there reprisals with colours reversed.

In the last quarter, Ethereal, having moved clear at the end of the third round robin, continued undefeated and duly promoted. Wool visited the mini-matches starting with games 57-58, 62, 64, 66-67, 71, 73-74, 77-78, 81, 83 and 84. rofChade promoted to the Premier Division for the first time and ahead of Igel on the r-mobility tiebreak.3

³

Both rofChade and Igel took G2.0 hits, rofChade in g86 with Nemorino’s 46.Rd5+, and Igel in g3 with RubiChess’ 60 Qe6+. So it was rofChade’s achievement of G3.0 against Igel itself in g27 with 62...Qg2+ that decided matters. It had taken the 8^th tie-breaker to separate these two engines.

We shall certainly see more of the rapidly improving Igel. Defenchess seemed headed for demotion from the start. The other promotee, Nemorino, also returned to Division 2 but finished up one tier to the good in net terms, presumably thanks to its newly-acquired NNUE capability.

Black not surprisingly had a torrid time of it with only three wins, games 30, 84 and 88. In the latter two cases, the weaker engine was simply beaten both times. Table 6 shows that some 14 of the 56 mini-matches ended 2-0 for White after two wins, adding those openings to the ‘possibly questionable’ list. While ‘57% wins’ was not a TCEC record, did the new openings chef spice up the sauce too much for our entertainment or are we happy to assess the games in their matched pairs?

Table 6

The TCEC20 League 1 cross-table

6. The Premier Division: 8 engines, 4 DRRs, 8 RRs, 224 games @ 60′+7″/m

The Premier Division arrived like a Boxing Day present, relatively up-tempo this time with an initial allowance of 60′ rather than 90′. This made time for eight round-robins rather than six, reducing the luck factor. Cato, the second author here, returned to serve up the openings. Only the six engines of Fig. 3 remain to be introduced even if they are all longtime Premier engines and familiar to the regular followers of TCEC. ‘KomodoDragon’ was a rebrand with new logo and it promised to breath more fire after stalking its prey more strategically with added NNUE. ScorpioNN was also rebadged. The neural network now pervades the whole of the Premier Division and half the field, including promotees Ethereal and rofChade, have taken on the exciting NNUE technology.

Stockfish started strong with +4 after key wins in games 6 and 20 against leading contenders Leela and KomodoDragon: in the first round-robin, it was the only unbeaten engine. Stoofvlees made a good early showing on +2. Leela was behind KomodoDragon on +1 with two losses and rofChade felt the most pain on −4 after a rough initiation.

Fig. 3.

The logos for the engines originally in the Premier Division.

The Dragon took out The Fish in the last vestiges of return game 34’s time, forcing P=Q after a very temporary rook sac. Even more surprising was that rofChade beat Leela from the apparently innocuous opening of game 42: Black looked lost after only six played moves. In games 48 and 49, Leela had two consecutive Whites and scored significant wins against Stockfish and AllieStein. Game 54 was Ethereal’s first win. At the end of the first quarter, the standings were Stockfish +6, Leela +4, Stoofvlees and KomodoDragon +2, AllieStein +1, ScorpioNN −4, Ethereal −5, rofChade −6. We were seeing better than expected performance from Stoofvlees and KomodoDragon. The three donors, as T.S.Eliot would say, were clearly going to have a hard time of it.

The third round robin saw few major upsets with the field seemingly dividing into three parts. Stockfish and Leela consolidated their leads while KomodoDragon continued to impress in third. AllieStein and Stoofvlees bossed the midfield as in TCEC19. The last three engines were destined to see which would escape relegation to Division 1. At halfway, Stockfish (+12) and Leela (+9) seemed clear of KomodoDragon (+4) which had eased ahead of AllieStein (+2). ScorpioNN and Ethereal were hoping to avoid the drop. The selected openings at this point were very impressive in terms of yielding decisive mini-matches: 39 mini-matches went 1-0/½ while only 8 went 1-0/1-0.

The fifth round robin was quieter. Komodo lost to Stoofvlees, g71, but Leela lost g76 to Stockfish by move 15. Undeterred, Leela held AllieStein in the next game, the longest draw of the event so far. After the return games, Stockfish (+16) and Leela (+15) were clear favourites for the Superfinal with Komodo a very creditable third on +8. ScorpioNN and Ethereal were both looking at relegation which, in turn, was not even socially distant from rofChade. Wool (2020) picked out games 28 and 76 as highlights amongst the 39 games he had covered so far.4

⁴

From the first double round-robin, Wool noted games 1/29, 6/34, 7, 8, 11/39, 15/43, 16, 18, 20/48, 27/55, 28; 30, 31, 38, 41, 42, 45, 49, 52 and 53. From the second DRR, he noted games 62, 66, 71/99, 76, 81, 84, 85, 95, 97, 105, 106, 110.

The seventh and eigth round robins were a bit of a formality except that Leela drew level with Stockfish on points and in fact took the top spot on the basis of the r-mobility tiebreaker. Stoofvlees helped, winning its seventh encounters with Komodo and with Stockfish, games 46.3/187 and 48.2/190. In g52.1/205, AllieStein progressed towards a difficult QP-rpp win but failed to beat the 50-move count. Given a reasonable 100 moves, it would probably have hit on the right plan and captured the f7 pawn. Games 202 and 207 confirmed Leela and Stockfish as Superfinalists, still with four rounds to go. Game 216 saw Ethereal demoted with rofChade as expected: a pity to see both promotees return so rapidly to League 1 but it will take a very good engine to prevent them both returning.

Table 7

The TCEC20 Premier League cross-table

The main winner in Elo terms was KomodoDragon, clearly refreshed with added NNUE, especially after its increasing flirtation with demotion in previous TCEC seasons. The Dragon is back! It is early days for NNUE and Komodo is clearly the leading contender to challenge the two established leaders. Wool (2020) focused on game 52.3/207 in which Leela beat Komodo and his continued coverage is very welcome.5

⁵

Wool (2020) also focused on games 174, 178, 183, 190-2, 194, 196-7, 205 and 218.

Table 7 shows that few games and only three mini-matches were won by an engine that finished lower in the standings. The choice of openings hit the sweet spot for bias, differentiating between engines: no less than 81 of the 112 mini-matches were decisive while only 14 resulted in two 1-0 wins. Black won only three games and no mini-matches: it seems hard to give Black a significant edge in the openings without giving the appearance of artificiality.

7. The Superfinal: 2 engines, 100 games @ 120′+10″/m

The stage was perfectly set. Stockfish and Leela, the two remaining gladiators, walked on in their first 2021 garb and looking forward to a close fight at classic tempo. Pre-match sentiment about the outcome was fairly even with Stockfish a slight favourite on the basis of 15 Elo points.

Jeroen Noomen (2021) ordered his fifty chosen openings with increasing bias which may or may not have been a good idea in that it threatened to starve the early games of oxygen. It also rather conditioned the expectations of the audience by telegraphing what was likely to happen. Jeroen reined back the bias from TCEC18. After the engines exchanged wins in games 5-6, the audience became increasingly concerned as the long sequence of draws stretched out across 26 games, see Fig. 4.

While Stockfish got the first mini-match win with games 33-34, the deadlock continued until its next win came in games 49-50. Leela started to notch its own wins in the second half, specifically in games 51-52 and 55-56. We were all square again, and the engines exchanged wins so that things were still level after 70 games even if Stockfish had never been headed.

As opening biases increased from Jeroen Noomen’s (2021) and mostly from the engines’ perspectives, see Fig. 5, Stockfish began to capitalise on opening advantages which evaded Leela. The close Superfinal we had expected receded from view: Stockfish was in its own personal Goldilocks region and won in games 71, 77, 79, 85, 87 and 89 without reply. Only when the match was decided did Leela come back in games 92, 96 and 100. At least, it got the last word and was closer than in TCEC19. In Elo terms, this Superfinal was as expected: Stockfish gained one Elo point but, due to rounding, Leela did not lose one point. These two engines represent computer chess’ state of the art on this platform and it’s getting increasingly hard to distinguish between them. One wonders how they would now fare against Deep Mind’s AlphaZero of 2019 (Sadler and Regan, 2019).

There were 22 wins in this Superfinal, an allocation the audience should have been happy with. Unfortunately, they were highly concentrated in the back end of this event. It was like visiting your favourite restaurant in good company only to endure an indifferent starter and main course before being blown away by the chef’s signature dessert. Matthew Sadler (2021c), ‘GMTheChessPuzzler’ (2021) and Wool (2020)6

⁶

Wool (2020) is as good a starting place as any, nobly touching on all games with useful introductory notes.

undoubtedly find quality play in the ‘sumo’ contest of thrust and counterthrust when there is no winner but we lesser mortals need an expert kibitzer to help us. It is difficult to spot an idea when it is snuffed out on the next move. The ICGA has never persuaded such a kibitzer to come forward in computer form even though a specific prize was offered. Maybe the TCEC-trailed DecodeChess (Decodea, 2021) or Nibbler (‘fohristiwhirl’, 2021) will provide some support in TCEC21.

‘Kingscrusher’ (2021a/2021b) commentated on games 66 and 91. GM Matthew Sadler (2021a/2021b/2021c) brings the whole Superfinal to life in the way that only he can, this time mentioning no less than 29 games. The rich context which he provides with his own reactions, past play from GMs and engines, and contemporaneous ‘side games’ adds huge value to the event. Matthew particularly picks out games 52 and 59, the latter being nominated as ‘game of the Sufi so far’ at the time. At the next level, games 29, 33, 47, 77, 96 and 97 get the big focus, and games 9, 39, 40 and 40 are only slightly less in the spotlight.7

⁷

And in addition, Matthew mentions games 2-5, 7, 11, 17-20, 22, 28, 34, 37, 43, 49 and 60. A feast!

Fig. 4.

The TCEC20 Superfinal: Stockfish’s net score.

Fig. 5.

The TCEC20 Superfinal: (a) the apparent |opening bias| of each game, and (b) probability[win given |bias| limit].

What can we add here? The evaluation graphs for the games that decided pairs of games for Stockfish are featured in Fig. 6. We note that the two engines’ assessments of position varied dramatically in games 59, 77, 85 and 97. Leela’s decisive wins, Fig. 7, do not feature such distanced perspectives. We also pick out in Fig. 8 five positions that stood out for us. In game 53, Jeroen put four queens on the board in the mandated opening. Game 75 was hailed as an interesting draw and game 86 was the longest draw. Game 77 stirred up the chat room as did the ‘sacrifice’ 42. Rg6 in game 99.

Table 8

The TCEC20 Superfinal: score detail

Fig. 6.

Evaluations for Stockfish wins in decisive mini-matches.

Fig. 7.

Evaluations for Leela Chess Zero wins in decisive mini-matches.

Fig. 8.

(a) game 49 p79w (Sadler, 2021a); (b) g53 p14b, the ‘4 Qs’ opening; (c) game 77 p26w; (d) game 97, p42w, evals wide apart; (e) game 99, p42b after Rg6!

8. In conclusion

Tables 9 and 10 give the usual statistics across the tournament. The pgn files and other data are available (Haworth and Hernandez, 2021) with 7-man points marked (de Man et al., 2018; Lomonosov, 2013) and the endings played out, less of an addition now that the previous ‘TCEC win’ adjudication has been dropped. It would be good to see the 6-man ‘endgame table wins’ played out by TCEC as well. The engines could do this in no time flat if they wanted to, even if we would need the GUI to slow down the tempo of moves played to less than one per second.

TCEC Season 1 started in December 2010 just one decade ago: it’s a good time to look back and take stock. Across these twenty tournaments, TCEC has clearly been a key force for change, providing a platform for several dimensions of improvement including the exploitation of parallelism, of crowd-sourced engine evolution, of neural networks and most recently of NNUE. There is however a danger of NNUE leading to a loss of engine identity if a number of NNUE-engines differ only in their training data. Should we be giving prizes shortly for non-NNUE engines as well as ‘open’ prizes?

The Grand Champion of TCEC Seasons 1, 2 and 4 also won Season 10 but does not now compete. The top step has been shared by Komodo (TCEC Seasons 5, 7-8), Leela Chess Zero (Seasons 15 and 17) and the dominant Stockfish (Seasons 6, 9, 11-14, 16, 18-20) which perhaps has the largest team behind it. It is no coincidence that these are the top three engines in TCEC20. Behind all this evolution, a coterie of engine authors and testers have been encouraged by the interest of us onlookers and by the very generous provision of hardware platforms and time by TCEC’s main sponsors.

Table 9

Generic statistics for each phase of TCEC20: results, terminations, average game-length, mini-matches

Table 10

The shortest and longest 1-0, drawn and 0-1 games in each phase of TCEC20