Experience version 3 : investigation

[deeds]

The old v2 experience file format contains :

- 8 bytes to store the position's key (ex : [B4 D3 0C D1 5A 43 43 2D] = [rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w KQkq - 0 1] for the start position)



- 4 bytes to store the move + promoted piece (ex : [00 00 03 1C] = [0 000 001(2) 100(e) 011(4) 100(e)] for the "e2e4" move)



- 4 bytes to store the move score (ex : [00 00 00 46] = 33 centipawns, it means "+0.33/4")



- 4 bytes to store the score depth (ex : [00 00 00 04] = 4, it means "+0.33/4")



- 4 bytes to store the count value (ex : [00 00 00 01] = 1, it means the "e2e4" move was played 1 time)

[deeds]

- 8 bytes to store the position's key (ex : [B4 D3 0C D1 5A 43 43 2D] = [rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w KQkq - 0 1] for the start position)

v2 : If a position contains several moves, the 8 bytes of its key are duplicated as many times as there are moves.
Example with the starting position which contains its 20 legal moves :
v2 : 20 * 24 bytes [8 (key) + 4 (move) + 4 (score) + 4 (depth) + 4 (count)] = 480 bytes

v3: We could only store the 8 bytes of the position key once and use a 9th byte (=0 to 255) to indicate the number of moves to load.
Example with the starting position which contains its 20 legal moves :
v3: 8 (key) + 1 (nb moves) + 20 * 16 [4 (move) + 4 (score) + 4 (depth) + 4 (count)] = 329 bytes (-31%)

[deeds]

- 4 bytes to store the move + promoted piece (ex : [00 00 03 1C] = [0 000 001(2) 100(e) 011(4) 100(e)] for the "e2e4" move)

v2: All moves consume 32 bits or 4 bytes

v3: All possible moves can fit in 15 bits or 2 bytes (-50%):
- 12 bits are sufficient from A(000) 1(000) to H(111) 8(111)
- 3 bits are enough for special moves, no promotion (000) or promotion to queen (001), to rook (010), to bishop (011), to knight (100), short castling (101), long castling (110)

[deeds]

- 4 bytes to store the move score (ex : [00 00 00 46] = 33 centipawns, it means "+0.33/4")

v2 : With 4 bytes, from [00 00 00 00] to [FF FF FF FF] there are 4'294'967'295 available values.

v3 : We only need 3 bytes (-25%) because generally at chess :
- the normal scores go from "-100'000" cp (-1000.00) to "+100'000" cp (+1000.00) so we need a range of 200'001 values
- the mate scores go from "mate -1000" to "mate +1000" so we need an additional range of 2'000 values
From [00 00 00] to [04 00 00] there are already 262'144 available values.
Until [FF FF FF], there are 16'777'215 available values so we even have room to handle tablebases scores.

[deeds]

- 4 bytes to store the score depth (ex : [00 00 00 04] = 4, it means "+0.33/4")

v2 : With 4 bytes, from [00 00 00 00] to [FF FF FF FF] there are 4'294'967'295 available values.

v3 : We only need 1 byte (-75%) because so far most engines are capping at D245.

[deeds]

- 4 bytes to store the count value (ex : [00 00 00 01] = 1, it means the "e2e4" move was played 1 time)

v2 : With 4 bytes, from [00 00 00 00] to [FF FF FF FF] there are 4'294'967'295 available values.

v3 : We only need 2 bytes (-50%) because the learning needs at least 500 games/opening so the first learned moves can be played at least 500 times.

[deeds]

v3 : We only need 2 bytes (-50%) because the learning needs at least 500 games/opening so the first learned moves can be played at least 500 times.

v3 : We only need 1 byte (-75%) because if the move is played 255 times, it's even sufficient to obtain its effectiveness

[deeds]

In sum,

v2 : we have 8 bytes/position + 16 bytes/moves
ex : 1 move/position = 24 bytes, 2 moves/position = 48 bytes, 4 moves/position = 96 bytes, etc.

v3 : we have 9 bytes/position + 7 bytes/moves
ex : 1 move/position = 16 bytes (-33%), 2 moves/position = 23 bytes (-52%), 4 moves/position = 37 bytes (-61%), etc.

[Robert57]

@deeds what is meant by experience v3?
no code has been released with further learning update by the author of SugaR, am I getting confused?
If so, can you explain better, thanks

[deeds]

@deeds what is meant by experience v3?

SugaR Experience version 1 contains :
- position's key
- move
- score
- depth
- count
- next score
- next 10th score

SugaR Experience version 2 contains :
- position's key
- move
- score
- depth
- count

Etc.

no code has been released with further learning update by the author of SugaR, am I getting confused?

None of the authors of SugaR, Eman, BrainLearn is the author of the learning code. I bet none of them train their engines with the 500 games/opening they advise lol. At best, they fill their experience files by playing on chess servers with default settings (=training mode).

With D.C.S. i already can read/write all their experience file formats but as i'm planning to add a learning feature to any non-learning engines, i'm already working on the experience version 3.