Experience version 3 : investigation

[Robert57]

I understand, thanks for the explanation.

[sarona]

. . . I bet none of them train their engines with the 500 games/opening they advise lol. At best, they fill their experience files by playing on chess servers with default settings (=training mode).

Well. . . I do not know what said authors do, but I focused on training specific openings where the total amount of games played exceeded 500. You even recommended that in 2021. In fact, your chart in the link below even references 2000 games - a practice which I support. With good hardware, this is not difficult to achieve. viewtopic.php?f=3&t=3317#p25326

The individual experience files were never merged into one. That was deliberately done. I followed your method and refined it a bit for my group's specific goals. I never used the servers to gather experience data. The games for a section of our database; yes. That (and selfplay) is more for collecting certain positions for our project. But playing? Never. That served no useful purpose to me.

We do not know what the goals of the others are regarding experience data (maybe, you do). I know M.Z often partakes in server competitions. Perhaps he is doing that to test HypnoS source modifications? That would seem logical.

And we know what is in those exp files because I used several of the tools you provided - starting in 2021 and continuing into 2023. https://www.open-chess.org/viewtopic.ph ... 314#p25315 In fact we have even refined the vb Experience Tools source that is publicly available on a Github page (not mine) to better suit our specific needs.

If there are trainers who do not know what is in their experience files, then they are lazy. All of the resources are there - thanks to you, Chris.

[sarona]

To me, 500 games is totally inadequate. Your own posts show that.



I archived everything in order to have a future reference guide. One of our team did the same when you had the private learning section at Outskirts - not to get any exp files (they were ignored), but to archive the methodology used. It's closing was a major loss for engine learning advancement. I understand the reasons for its closing, but it was unfortunate.

On a well deserved vacation and going for a short evening hike in the mountains at beautiful Banff Provincial Park with my wife. Stay out of trouble Mr. deeds

[deeds]

- 4 bytes to store the count value (ex : [00 00 00 01] = 1, it means the "e2e4" move was played 1 time)

v2 : With 4 bytes, from [00 00 00 00] to [FF FF FF FF] there are 4'294'967'295 available values.

v3 : We only need 2 bytes (-50%) because the learning needs at least 500 games/opening so the first learned moves can be played at least 500 times.

v2 : Until now, we use the counter of visits to evaluate the moves "popularity". In the same position, if several moves have a similar evaluation, their "popularity" can favor them when calculating their quality.

But when we reinforce experience data or analyze games, these visit counters increase which artificially increases the moves "popularity" even if the engine does not play a whole game. And I'm not even talking about when we merge multiple files...

v3 : Instead of evaluating the moves "popularity", this visit counter could represent :

- a winrate. It would only be updated when the outcome of the game takes shape. From -100% to +100% with a margin of +/- 1%, it only takes 1 byte (0-255) but we would often have moves with an identical winrate. With 2 bytes (0-65535), we lower the margin to +/- 0.01%, very little chance of having 2 moves with a similar winrate.

- a ponder's move counter. It would only be updated when the opponent plays the ponder's move. By playing enough games, it should be possible to establish which values of the ponder's move counter represent the most effective move. As most of the moves can be played more than 256 times, especially in training, we need 2 bytes.

- etc.

[deeds]

The old v2 experience file format contains :

- 8 bytes to store the position's key (ex : [B4 D3 0C D1 5A 43 43 2D] = [rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w KQkq - 0 1] for the start position)

V2 : As theses position's keys are attached to their engines, others engines may don't find them. Without a giant database, we only have the bruteforce search to retrieve the original FEN strings.

V3 : maybe we should use an another conversion (FEN string to 64 bits key) in order to easily retrieve the original FEN...

[deeds]

v3 : We only need 3 bytes (-25%) because generally at chess :
- the normal scores go from "-100'000" cp (-1000.00) to "+100'000" cp (+1000.00) so we need a range of 200'001 values
- the mate scores go from "mate -1000" to "mate +1000" so we need an additional range of 2'000 values
From [00 00 00] to [04 00 00] there are already 262'144 available values.
Until [FF FF FF], there are 16'777'215 available values so we even have room to handle tablebases scores.

Normal scores (score cp xxx) and mate scores (M +/- xxx) :

With each group separated by +/- 20k available values, we even have a error margin.

About how to use the 16M available values, it's still under investigation...

[deeds]

expToV3
To convert "eman-like" EXP files from "SugaR Experience version 2" to "Experience version 3.0".

https://github.com/chris13300/expTov3

[Amos 4ever]

expToV3
To convert "eman-like" EXP files from "SugaR Experience version 2" to "Experience version 3.0".

https://github.com/chris13300/expTov3

Thanks for this tool.
Clarification please, suppose I have a v2 exp file after converting it to v3 and still using the file playing games, will I have new data stored with v2 again so can I safely reuse the tool to update it again to v3?

[Anton101]

I had the same doubt.

Many thanks for this tool.

Regards.

A.Ces.

[deeds]

Clarification please, suppose I have a v2 exp file after converting it to v3 and still using the file playing games, will I have new data stored with v2 again so can I safely reuse the tool to update it again to v3?

Huh !? When Eman 7.00 first loads a v1 experience file, it converts it to a v2 experience file and updates it with new v2 data.

Be serious, once a chess engine or chess system adopts a file format for its experience data, there's no point in jumping back and forth between versions.

Given all the investigations presented above and the possible gains, do you really think I'm going to stop at the version 3.0 which saves only 33% ? loool