When Should You Resign?

When Should You Resign?

At which point is the game probably lost beyond repair? Should you resign after you lose a pawn? A piece? Does it depend on whether the game is a slow or a fast game, or on your opponent's rating?

Grandmasters often resign when they cumulate a relatively small disadvantage. On the other hand, it's frequent to see players online drag the defeat until mate (or even until one of the players run out of time). In this article, we investigate the following questions:

1. What are the odds of winning a game once you've reached a certain disadvantage? (as measured by computer evaluation)

2. Do the odds of winning after reaching a given disadvantage vary by time control?

3. Do the odds of winning after reaching a given disadvantage vary by rating?

The Data: 146k+ Online Games Evaluated by Stockfish

To determine the odds of coming back from behind and winning a game, we've used the database of games played on lichess in May 2019 which had the Stockfish evaluation at each move. We sampled 200,000 games at random, and we only kept games that matched these criteria:

• Normal termination (i.e., we removed games tagged with "abandoned game" and "rules infraction")

• Decisive games (draws accounted for only a small proportion of games, about 2.5% of all games)

• Players were similarly rated (rated within 100 rating points of each other)

• Players were rated 800-2300 (since there were too few games outside of these ratings, particularly above 2300)

In total, we used 146,840 online games that had a Stockfish evaluation at every move.

We determined the worst disadvantage reached during each game by looking at the Stockfish evaluation after the opponent's move, to make sure to exclude mistakes that were not capitalized on.

For example, here are two different games in which White came back from behind:

Should these White players have resigned? What were the odds of winning after White reached a disadvantage of -2 in the first game? And in the second game, what were the odds of winning after White reached a disadvantage of -6?

Odds of Winning as a Function of Computer Evaluation

The graph below shows the odds of winning (in %) as the worst disadvantage reached during the game gets larger and larger as we move from left to right on the x axis (for example, White is trailing behind by 1 pawn, 2 pawns, a piece, etc.):

As shown in the graph, and as expected, the worst White is doing according to Stockfish, the less likely White is to come back and win the game. When the game starts, White has a small advantage and wins slightly over 50% of the time. However, as Black gains an advantage, White's chances of ultimately winning the game decreases.

If White is down less than a pawn (i.e., the Stockfish evaluation is between -0.00 and -1.00), White still has almost a 45% chance of coming back and winning the game. However, being down more than a pawn decreases that chance to less than 35%; and White wins less than 25% of games after being down a piece.

Somewhat surprisingly, there is still a non-negligible chance for White to come back and win even after being down a Queen or more (about 12.5% or 1 in 8 games), or even facing checkmate (about 7% or 1 in 14 games). This is partly (but not entirely) due to flagging: Black loses because he or she has run out of time.

Effect of Time Control

Do the odds of winning after trailing behind depend on time control? Yes.

Below we show the same graph (% chance of winning a game after trailing behind by a certain amount on the x axis) separately for Bullet, Blitz, Rapid, and Classical games:

As show above, there is a clear effect of time control: White is much more likely to come back and win the game after trailing behind if the game is fast-paced (a bullet game) than any other category. In fact, this is entirely due to flagging: When we exclude games in which the game ended after one player ran out of time, the four lines overlap almost exactly.

Effect of Rating

Do the odds of winning after trailing behind depend on rating? Also yes.

Below we show the same graph (% chance of winning a game after trailing behind by a certain amount on the x axis) separately for opponent ratings ranging from 800s to 2200s (for example, 2170 is placed in the 2100s category):

Here we see a very large effect of rating: White is much less likely to come back from behind when the game is between stronger opponents. Conversely, White is much more likely to pull a swindle and win after trailing behind when facing a weaker opponent. For example, the topmost line shows that when Black is rated in the 800s, White has about a 25% of winning the game after being down a Queen! Conversely, the yellow and green lines show that when White is facing an opponent rated above 1500, White can only win about 10% of games after being down a Queen.

what to make of this?

Some of these results are somewhat surprising. On the one hand, as expected, White is much more likely to lose the game after being down 2 pawns or more, even when including games in which White tried to flag Black and win on time. On the other hand, there is still a surprisingly large percentage of games (almost 1 in 8!) in which White wins the game after being down by as much as a whole Queen.

Nonetheless, it seems clear that much of the action happens between 0 and 3 pawns, particularly for players rated above 1500. For this reason, we think it would be wise (and sportsmanlike) to resign after losing a piece or more.

One interesting use of these graphs is prediction: It would be possible to use these graphs to predict the outcome of an online game based on the computer evaluation (for example, as a fun metric to include during a live stream). Who knows, maybe someday we'll be allowed to bet on a winner as the game is being played?

Also, it would be interesting to look at the relationship between the habit of resigning early and improvement. With the data presented here, it is unclear how to evaluate the impact of resigning habits on improvement; in particular, how should we assess improvement? In any case, resigning early can save time and frustration when you start trailing behind during a game, as we've seen that you are more likely to lose than to win regardless of the time control and your opponent's rating. If you've reached a plateau and your rating has stagnated, changing your resigning habits might be an idea to try out.

How to cite this article

Data & Code