Making a Discovery in Chess

Introduction

We have added a new puzzle theme called Discovered Check (see also in practice section) and I would like to use the opportunity to layout theory around this chess topic. This includes more details on how puzzles are generated at Lichess to give everyone a better understanding of how puzzle themes are created.

A Chess Piece Can See

A chess move is sometimes described with a phrase like: "I move the light squared Bishop to B5 (Bb5) and now the bishop sees the king on e8, and therefore, the king is being checked (Bb5+)." And I always found this metaphor interesting, because, if a light squared bishop sees only the pieces on white squares, and the dark squared bishop sees only the pieces on black squares, -the two bishops see a completely different world, even though, they are standing on the same board. But apart from this philosophical aspect, there is also a more practical application, which is:

We can move one piece, to reveal another piece to the piece behind the moved piece, which was previously blocked by the moved piece. I know this sounds confusing, but it becomes very clear, if you consider the 4 move sequence below (taken from the Discovered Check sample puzzle #W5zMY).

Here, the rook on a6, is behind the pawn on d6, when viewed from the black king on h6, in the left most image. The rook on a6 does not see the king on h6. This changes with the next move: pawn to d7. Now the rook on a6 can see the king h6 and the king is in check.

And even though, the black knight on e5 could capture the pawn on d7, -it cannot, -because black must respond to the check. So, the king moves from h6 to h7 (Kh7) out of sight of the rook on a6, and the pawn on d7, beliefs in re-incarnation, and promotes to a queen on d8.

This small example gives us a small glimpse into the power of a discovery. The above example sequence shows one variation of a Discovered Check. A Discovered Check requires a piece that can see along a line of the board (rook, queen, or bishop) and any blocking piece (along with the opponents king, of course). This type of check can be particularly venomous, because:

1. it can often be advantageous (like promoting a pawn through a defended file),
2. is surprising, because the opponent may not have noticed the true venom, yet, or
3. the discovered check allows us to capture a hanging piece, while the opponent is dealing with the check.

A common threat (see CCT/CCA in my last post), that everyone should know is, therefore, to setup a piece like a bishop, queen, or rook behind a blocking piece opposite from the king or queen, to reveal this in the right moment.

Discovered Checks come in many variations and colors. Here is a small non-exhaustive list of interesting samples to get your imagination started:

1. Counter Check: #QiklW, #tdLWe
2. Smothered Mate: #Gdz89
3. The King initiates a Discovered Check: #WQO6f, #vKsSB, #tdLWe
4. Promoting a pawn with a Discovered Check: #Pm5T4
5. Double Check (see details below): #g9MKZ, #u4BgB
6. Promoting a pawn: #9HgCD
7. Capturing the queen in the middle game: #0ZfO2

A Discovered Check is one part of a group of motifs that are all very similar. This group includes:

1. Discovered Checks,
2. Discovered Attacks,
3. Double Checks, and
4. Double Attacks.

Let's look at the patterns in 2-3 next to understand the larger set of chess patterns, we're dealing with here.

Discovered Attack

A Discovered Attack occurs when a player moves one piece, revealing an attack by another piece that was previously blocked. This tactic can create two threats at once, making it difficult for the opponent to defend against both simultaneously. It is of course also dangerous, when it creates only one threat that is large enough to decide the game, or on the small side, to gain a tempo.

A Discovered Attacks is a Discovered Check if the revealed piece checks the opponent's king. Therefore, Discovered Checks are a subset of Discovered Attacks. But the revealed piece may also see another opponents (hanging) piece, in which case, the motif is just a Discovered Attack, since we're just attacking, but not checking through the discovery. This becomes clearer, if we review samples form the puzzle theme:

1. #H0lKh: The rook reveals the queen on h4 against the bishop on e7, via Rxg5+, Kh1, Rg1+
2. #LNN69: A classic Attraction example, where the black queen is tricked into taking on d4, -the white bishop moves to h7, checking the king and also revealing the white queen on d1 against the black queen on d4
3. #ztHz8: In this example we are exchanging a knight for a pawn and a bishop, via Nxe4 (which threatens the queen on f6) and reveals the rook on c1 against the bishop on c6

Double Check

A Double Check is a special case of a Discovered Attack and Discovered Check, in the way that not only the revealing piece is checking the king, but the moved piece that was previously blocking is also checking the king. This often leads to a direct check mate on the spot or decides the game in 1 or 2 more mating moves:

1. #EvKJQ: Here Rf6 is immediate check mate (Rf6#), even though, the f6 is defended by the pawn on g7. But the king is checked by the rook on f6 and the bishop on c5. This means that the king has to move in order to resolve the situation. But there is no free square to move to.
2. #gZDZb: A classic example of mating the king on the back rank using the Opera mate as final mating position.
3. #lfzeI: Another example using a Double Check to end the game in a Smothered mate position.

Double Attack

It is very likely that you already know the Definition of a Double Attack, if you followed the previously shown Discovered Check, Discovered Attack, and Double Check. But here it is for completeness. A Double Attack occurs when a piece is attacked by 2 different pieces in one single move. A similar situation is a fork, but there is only one piece forking/attacking 2 other pieces. A double Attack has 2 pieces attacking 1 piece. This is almost the same idea as the Double Check, except, it's not the king being attacked here, but any other piece on the board:

1. #lWqLE: The first move queen to b4 (Qb4) threatens the pawn on b2 (checkmate!) and the hanging rook on e1.
2. #OvcgX: The knight move to b4 (Nb4) not only forks the queen and bishop, but also attacks the bishop twice (note: the bishop was not attacked previously). Making its capture very likely, because the attacked queen is its only defender.
3. #jRj90: The knight move to c4 (Nc4) reveals the bishop on d4 seeing the pawn on g7, which is also attacked by the queen on g3 (threatening checkmate!), and the knight on c4 also attacks the queen on a5.

Lichess Puzzle Pipeline

I have received some feedback indicating that many lichess users are confused as to how and where we are getting these puzzle themes from. Some even think I would actually sit down and design them myself :-) And while I would like to be that clever, and have this amount of time in my live, I have to admit that a lot of the actual work is already done when I join the party. I have summarized the current workflow in the graphic below:

We can see that the current process is divided into 3 stages:

1. Game Play
2. Online Puzzle Extraction, and
3. Offline Puzzle Tagging

Let's detail these stages below.

Game Play and Online Puzzle Extraction

About 6 million games are played on lichess on an average day. The sequences from these games are stored in the lichess database and are then used to produce standard puzzles (mate in 1, 2,3, Attraction, Deflection etc.) on a daily base. The module that generates these puzzles is the lichess-puzzler and every available motif/theme can be found here: PuzzleTheme.scala. Additionally, every puzzle that has been generated can be changed, if enough users vote for it to be tagged differently.

The lichess database contains about 7.3 billion games at the time of this writing. But only the games, which:

1. have already been analyzed by Stockfish (via Request Computer Analysis),
2. plus additional filters for rating, time control etc)

are used to generate puzzles. This means about 5% of all games are used to generate a puzzle on a daily base.

The additional filters have changed (and will change) over time to control the number of puzzles generated, and their quality. The additional filters are based on 4 groups of scores for a game:

1. The rating of the player being lower than 1500, higher than 1500, 1600, or 1750 (each case is scored differently),
2. The time control being <= 60 seconds, > 60 seconds, >= 180 seconds, or >= 480 seconds
3. giving a score to the number of moves a game has,
4. scoring whether the player has a title, or not.

Final step, discard games that are not scoring high enough.

So, in a nutshell: Make sure you have a title, play above an ELO of 1750, and play longer time controls with many moves, and analyze your games to increase the likelihood of contributing to puzzles. Or do nothing of the above, if you do not want to contribute here. Either way, there is a dedicated lichess page, where you can enter your login handle and check out your puzzles contribution.

We are required to analyze 'only' 5% of 7.3 billion games, which amounts to about 365 million games. And the usual conversion rate requires 5-6 games to generate 1 puzzle. The lead developer of the lichess project Thibault has spend the last ~5 years using ~6 servers processing this amount of data in order to produce the currently available 5.6 million puzzles. He calls this a mining operation that is at times obscene and difficult to justify, because hundreds of years of CPU time have gone into it... but we can be glad this work has been done, because I find puzzles so intuitive to understand, and a lot more applied and teaching, than any chess theory book can ever be :-) And while 3 million puzzles being solved every day, -versus 6 million games being played, -shows that puzzles are not the most important thing in the chess world, they are being used a fair deal, as far as we can see.

Offline Puzzle Tagging

Getting back to the above workflow, we now know that generating puzzles is all, but a piece of cake. And I join the party, once that cake is available. I look at the 5.6 million puzzles that have been generated previously and process them with Python (mainly using Python-Chess, Pandas DataFrame in Google's CoLab environment) to identify the puzzles that should belong to a particular theme. I write 1 script for each theme and they all look very similar (Dirkster99/ChessMatingPatterns). The parser script typically looks at each available puzzle using Python-Chess and determines, if the conditions/requirements for a given tactic or pattern (e.g.: Discovered Check) are satisfied. The puzzle is tagged in the lichess puzzles database, if yes, otherwise, the next puzzle is considered.

But what does offline tagging really mean? Well, it means that we use a process that is no longer tightly coupled with the generation of new data (games being played). But instead we are using scripts that are run, once or twice a year, to identify the new puzzles that should belong to a theme. And tagging refers to the same mechanism that users have to tag a puzzle using the lichess puzzle UI. This means, the third and last puzzle identification step, does not create new puzzles, but only identifies those that should belong to a defined group of puzzles (a theme).
The above VEN diagram tries to express this idea with the largest white bubble referring to the 5.6 million puzzles. Inside that bubble we have Mate in 1, 2, 3 , 4 puzzles that may overlap with the Discovered Check bubble, if the given mate in 1, 2, 3, 4 puzzle is in fact done using a Discovered Check.

You can obviously do the same thing using other tools. For instance using a different cloud environment, such as, AWS, Azure, or your local computer will do. The runtime of each puzzle tagger differs widely depending on the complexity of the theme, but it usually runs anywhere between 20 minutes and 3 hours.

Now that we know that some themes are tagged offline, we should also understand that not every puzzle is always tagged correctly. But users are always welcome to add a missing puzzle tag using the Puzzle UI, to contribute here, as well.

And we can also make 2 obvious training observations about puzzles and themes in lichess. A training with a random mix from all 5.6 million puzzles is more difficult than a training with a random selection from Discovered Checks, because there is a lot more in terms of variations in the former compared to the later. And similar things can be observed about Mate in 1, 2, 3, 4, 5 puzzles versus Discovered Checks or other themes like Discovered Attack etc. The point is that you can select the themes that you have most trouble with, train with it until you improve (like the coming soon Opera mate that requires rook and bishop coordination), and go back to training with 5.6 million puzzles to enjoy a more complete understanding than before.

Conclusion

I hope you found this blog post interesting with regard to the chess content and/or the technical content. I am still fascinated by the fact that we can use data to navigate such difficult concepts like Discovered Attacks and come up with something so intuitive like puzzles. I am grateful for the opportunities given to contribute here and want to express my gratitude towards everyone contributing to lichess.

References

1. chessfox.com Double Attack
2. Parsing Chess Data to Discover Mating Patterns
3. the new Lichess.org puzzles out for a test drive! YouTube video by ChessCoach Andras from December/2020
4. Lichess Puzzles: The Fastest Way to Improve Your Chess | Puzzle Storm | Puzzle Streak | Puzzle Racer YouTube video by Sri Lanka Chess Lessons