OpenChess

Dear wise chess programmers, I have an interesting riddle for you.

So I've written this fairly primitive chess engine that does a very basic evaluation. One of its evaluations is a mobility calculation, which is basically just the number of available moves in any position. It added this number to the results of the other evaluations, in order to bias toward that sort of position. It evaluates between 40k and 200k nodes per second depending on the complexity of the position.

Now I came to realize that it might be a better to choose a mobility score based on the number of moves currently available in this position less the number of moves available by the opponent after a null move. This would acknowledge when the other side has better play and reduce the score accordingly.

It was quick to implement but reduced the performance of the engine down to around 45k nodes per second in the worse case. But the scores were much more accurate.

Now I played the engine against itself and what do you know? The engine's ELO dropped like 95 points. WTF? I would have expected that a more accurate evaluation function would have produced better results, even if it were slower.

Has anyone else seen behavior like this? Am I just wrong in my assumptions? Is it better to have an incorrect but fast evaluation function?

Welcome to the world of computer chess programming.

a chess engine is nothing but a bunch of compromises. You can do things accurately, or you can do them quickly. rarely can you do both. Sometimes accuracy improves Elo, if you ignore the computational costs (IE you test with fixed-depth or fixed-node-count rather than a time limit). But in general, for each gain in accuracy, there is a loss in speed, and you have to REALLY be careful as losing speed hurts, as you have seen. The art of chess programming is primarily focused on trying to balance speed and accuracy to get the best of both by finding the "sweet spot."

It takes LOTS of testing.

Respect to the great Bob Hyatt Thank you for both myself and the legions of wannabe chess programmers that you so graciously donate your time to help.

This is chess programming. Chess programming is actually very simple if you knew whether an algorithm would work without spending hours on testing. It takes more than simply speed vs accuracy, it's much harder than that. You need to devise a way to achieve good speed while searching for nodes that are relevant. You need to reduce your search while accurately making sure it's not something it's relevant to the position.