TRAINING MONITOR

[deeds]

Private graphical tool to monitor engines training in real time, draw learning curves of openings, obtain engines and games statistics, etc.



It just needs to select the PGN file to monitor then after each game it updates.
It draws different curves, manually or automatically (carousel mode).



It shows engines/games statistics, estimated end of training, etc.

[rulleeeee]

Where can I download it?

Because I don't see it on github...

[deeds]

Hi Rulleeeee, it's normal because i already said :

Private graphical tool...

But if you get a good score at our learning quiz (look at my website) i'll send you a copy (link by PM at infinitychess)

[rulleeeee]

Ah, I didn't read "private"

[deeds]

When we train the engines on an opening, they will replay it hundreds/thousands of times.
Each time, they test different moves in order to learn the most effective ones.
The "First Novelty" curve represents the position where a new move was first tested.

Example with the "1.g4" opening :



For its first game, so without experience data, Eman evaluated "7.Nf3" as its bestmove and got a draw.
For its second game, since it didn't win with "7.Nf3", Eman tried "7.Qc2" as its bestmove and it lost.

It's what we call "training mode" !

[deeds]

If we use a database's tool like SCID, it looks like these 2 moves are very different :



But if we look at the experience file, they are very close :



In training mode (default settings), learning engines always seek to acquire new experience data, so they often try new moves, so they don't play the most efficient moves, so this mode isn't suitable for tourneys.

In tourney mode (experience book true or d.c.s tool or others), learning engines would have chosen "7.Nf3" until the engine's trainer updates the experience data (analyzes at greater depths, trains more games, etc.).

[deeds]

This is what First Novelty's curves look like when engines learn drawish openings :

Eman experience vs Eman only (LS1)

Here, Eman Experience trained with a sparring-partner having the same playing style and without learning feature (Eman only = read-only true, no exp file)


Eman experience vs BrainLearn (LS2)

Here, Eman Experience trained with a sparring-partner having a different playing style and which can learn


Eman experience vs JudaS / HypnoS (LS4)

Here, Eman Experience trained with 2 sparring-partners having different playing styles and which can learn


Eman experience vs JudaS / BrainLearn / HypnoS / Stockfish (LS7)

Here, Eman Experience trained with 4 sparring-partners having diffrent playing styles and which can learn except Stockfish


Eman experience vs BrainLearn / HypnoS / ShashChess / Aurora (LS10)

Here, Eman Experience trained with 4 sparring-partners having diffrent playing styles and which can learn
Here, the average time/move is longer (e.g. 5.1 sec/move) because the TC was 2m+2s (instead of 1m+1s for the 7 first learning sessions)


Common points :
- the average First Novelty is very close to the average moves/game so with only 2000 games, engines can almost learn 100% of some drawish openings
- tons of duplicated games even with different sparring-partners (=different playing styles)

[deeds]

This is what First Novelty's curves look like when engines try to learn short openings (1 to 15 plies) :

1.g4 (LS2)

32th / 148 moves/games = opening only learned at about 22%


1.f3 (LS4)

27th / 140 moves/games = opening only learned at about 19%


1.e4 h5 (LS3)

27th / 160 moves/games = opening only learned at about 17%


1.g4 d5 2.h3 (LS3)

25th / 160 moves/games = opening only learned at about 15%


1.g4 d5 2.e3 (LS3)

21th / 155 moves/games = opening only learned at about 14%


Common points :
- the First Novelty is very low so with only 2000 games, engines only learn about 25% of short openings (average based on 54 non-drawish openings)
- very few of duplicated games even with only 1 or 2 different sparring-partners (=different playing styles)

Tip: If your engine doesn't play better after training it on short openings, try to train it on "key" positions

[deeds]

This is what First Novelty's curves look like when engines try to learn medium openings (16 to 31 plies) :

d41 h2h4 (ls04)

64th / 174 moves/games = opening only learned at about 37%


e99 c4c5 (ls04)

58th / 134 moves/games = opening only learned at about 43%


ls06 - e89 h2h3 (ls06)

57th / 137 moves/games = opening only learned at about 42%


c11 c5c4 (ls07)

54th / 141 moves/games = opening only learned at about 38%


b90 f8e7 (ls09)

66th / 147 moves/games = opening only learned at about 45%


Common points :
- the First Novelty is > 50th but remember that the openings already contain until 31 plies
- with only 2000 games, engines only learn about 41% of medium openings (average based on 112 non-drawish openings)
- some duplicated games but it is attached to the openings and the playing style of sparring-partners

Tip : training 500 games is not always enough to get a positive elo's gain, try to train at least 1000 games (2000 games are recommended)