Train your engine !

[deeds]

E21 Nimzo-Indian, 3 Knights, Euwe Variant :

[Event "Eman's learning"]
[Site "deeds"]
[Date "2021.05.24"]
[Round "0073-0074"]
[White "E21"]
[Black "E21"]
[Result "*"]
[ECO "E21"]
[PlyCount "20"]

1. d4 Nf6 2. c4 e6 3. Nc3 Bb4 4. Nf3 c5 5. d5 Ne4 6. Qd3 Qf6 7. Bd2 Nxd2
8. Nxd2 O-O 9. Nde4 Qg6 10. O-O-O e5 *

Code: Select all

   # PLAYER                             :  RATING  ERROR  POINTS  PLAYED   (%)    W    D    L  D(%)  OppAvg  OppN
   1 Eman experience                    :     154     18  1179.5    2000  59.0  904  551  545  27.6       0     1
   2 brainlearn 13.1 nn-6762d36ad265    :       0   ----   820.5    2000  41.0  545  551  904  27.6     154     1

White advantage = 367.55 +/- 8.81
Draw rate (equal opponents) = 50.00 % +/- 0.00

brainlearn 13.1 nn-6762d36ad265 with 545/2000 wins (27,25%, D30/245, 1 422 ms) :

1. d4 @ 100,00%

Eman experience with 904/2000 wins (45,20%, D34/245, 1 568 ms) :

1. d4 @ 100,00%

Averages :
275 sec/game
205 plies/game

[deeds]

B89 Sicilian, Velimirovic :

[Event "Eman's learning"]
[Site "deeds"]
[Date "2021.05.24"]
[Round "0077-0078"]
[White "B89"]
[Black "B89"]
[Result "*"]
[ECO "B89"]
[PlyCount "20"]

1. e4 c5 2. Nf3 d6 3. d4 cxd4 4. Nxd4 Nf6 5. Nc3 Nc6 6. Bc4 e6 7. Be3 Be7
8. Qe2 O-O 9. O-O-O a6 10. Rhg1 Qe8 *

Code: Select all

   # PLAYER                             :  RATING  ERROR  POINTS  PLAYED   (%)    W    D    L  D(%)  OppAvg  OppN
   1 Eman experience                    :      54     18  1053.5    2000  52.7  856  395  749  19.8       0     1
   2 brainlearn 13.1 nn-6762d36ad265    :       0   ----   946.5    2000  47.3  749  395  856  19.8      54     1

White advantage = 394.09 +/- 9.30
Draw rate (equal opponents) = 50.00 % +/- 0.00

brainlearn 13.1 nn-6762d36ad265 with 749/2000 wins (37,45%, D28/245, 1 469 ms) :

1. e4 @ 100,00%

Eman experience with 856/2000 wins (42,80%, D33/245, 1 680 ms) :

1. e4 @ 100,00%

Averages :
242 sec/game
174 plies/game

[deeds]

B18 Caro-Kann, Classical :

[Event "Eman's learning"]
[Site "deeds"]
[Date "2021.05.24"]
[Round "0119-0120"]
[White "B18"]
[Black "B18"]
[Result "*"]
[ECO "B18"]
[PlyCount "20"]

1. e4 c6 2. d4 d5 3. Nc3 dxe4 4. Nxe4 Bf5 5. Ng3 Bg6 6. Nf3 Nd7 7. Bc4 e6
8. Qe2 Qc7 9. O-O O-O-O 10. c3 Ngf6 *

Code: Select all

   # PLAYER                             :  RATING  ERROR  POINTS  PLAYED   (%)    W     D    L  D(%)  OppAvg  OppN
   1 Eman experience                    :       8     11  1022.0    2000  51.1  174  1696  130  84.8       0     1
   2 brainlearn 13.1 nn-6762d36ad265    :       0   ----   978.0    2000  48.9  130  1696  174  84.8       8     1

White advantage = 51.57 +/- 5.53
Draw rate (equal opponents) = 50.00 % +/- 0.00

brainlearn 13.1 nn-6762d36ad265 with 130/2000 wins (6,50%, D27/245, 1 459 ms) :

1. e4 @ 100,00%

Eman experience with 174/2000 wins (8,70%, D29/245, 1 753 ms) :

1. e4 @ 100,00%

Averages :
198 sec/game
144 plies/game

[deeds]

Here is a summary of a few openings where [Eman experience] tried to learn something against [Eman only] :

[Eman only] didn't used an experience file and [Eman experience] started with an empty experience file.
Settings : both engines used 8 threads (e3-1275v2), a hash of 512 MB, no contempt, no syzygy, no book, no livebook, strongest net handled by the engines, cutechess-cli (concurrency 1) ran 2000 games with reversed colors at a time control of 60 sec + 1 sec.

Code: Select all

   # PLAYER             :  RATING  ERROR   POINTS  PLAYED   (%)      W      D      L  D(%)  OppAvg  OppN
   1 Eman experience    :      59      2  27788.0   48000  57.9  11690  32196   4114  67.1       0     1
   2 Eman only          :       0   ----  20212.0   48000  42.1   4114  32196  11690  67.1      59     1

White advantage = 87.60 +/- 1.17
Draw rate (equal opponents) = 50.00 % +/- 0.00

Conclusion :
After 48k games (24 openings * 2000 games), [Eman experience] learned around 59 elo +/-2 against [Eman only], not so bad !

As [brainlearn 13.1 nn-6762d36ad265] is even stronger than [Eman experience] because brainlearn's engine/net are better updated than the eman 7.40 ones, I think [Eman experience] is learning stronger moves so the elo difference will be a lot reduced with the new openings.

[deeds]

B16 Caro-Kann, Bronstein-Larsen :

[Event "Eman's learning"]
[Site "deeds"]
[Date "2021.05.24"]
[Round "0137-0138"]
[White "B16"]
[Black "B16"]
[Result "*"]
[ECO "B16"]
[PlyCount "20"]

1. e4 c6 2. d4 d5 3. Nd2 dxe4 4. Nxe4 Nf6 5. Nxf6+ gxf6 6. Nf3 Bf5 7. Bd3
Bg4 8. Be2 Qc7 9. O-O Nd7 10. c4 O-O-O *

Code: Select all

   # PLAYER                             :  RATING  ERROR  POINTS  PLAYED   (%)    W     D    L  D(%)  OppAvg  OppN
   1 Eman experience                    :      10     12  1022.0    2000  51.1  461  1122  417  56.1       0     1
   2 brainlearn 13.1 nn-6762d36ad265    :       0   ----   978.0    2000  48.9  417  1122  461  56.1      10     1

White advantage = 164.33 +/- 6.07
Draw rate (equal opponents) = 50.00 % +/- 0.00

brainlearn 13.1 nn-6762d36ad265 with 417/2000 wins (20,85%, D27/245, 1 463 ms) :

1. e4 @ 100,00%

Eman experience with 461/2000 wins (23,05%, D31/245, 1 719 ms) :

1. e4 @ 100,00%

Averages :
222 sec/game
161 plies/game

[deeds]

For the next openings, I will replace :

- the brainlearn's default "nn-6762d36ad265" net by the new "nn-cdf1785602d6" net which will be called [brainlearn 13.1 nn-cdf1785602d6]

- the old eman's external "nn-46832cfbead3" net by the new "nn-cdf1785602d6" net and also the old engine "eman 7.40" by the new release "eman 7.80" which will be called [eman experience]

[deeds]

B06 Modern :

[Event "Eman's learning"]
[Site "deeds"]
[Date "2021.05.24"]
[Round "0149-0150"]
[White "B06"]
[Black "B06"]
[Result "*"]
[ECO "B06"]
[PlyCount "20"]

1. e4 g6 2. d4 Bg7 3. Nc3 d6 4. Be2 Nc6 5. Be3 e5 6. dxe5 Nxe5 7. f4 Nc6
8. Qd2 Nf6 9. h3 O-O 10. O-O-O Re8 *

Code: Select all

   # PLAYER                             :  RATING  ERROR  POINTS  PLAYED   (%)    W     D    L  D(%)  OppAvg  OppN
   1 brainlearn 13.1 nn-cdf1785602d6    :       0   ----  1083.5    2000  54.2  396  1375  229  68.8     -32     1
   2 Eman experience                    :     -32     11   916.5    2000  45.8  229  1375  396  68.8       0     1

White advantage = 112.30 +/- 5.77
Draw rate (equal opponents) = 50.00 % +/- 0.00

brainlearn 13.1 nn-cdf1785602d6 with 396/2000 wins (19,80%, D26/245, 1 459 ms) :

1. e4 @ 100,00%

Eman experience with 229/2000 wins (11,45%, D36/245, 1 778 ms) :

1. e4 @ 100,00%

Averages :
170 sec/game
133 plies/game

The first thousand games were played with Experience MultiPV = true (default).
The second thousand games were played without it (=Experience MultiPV = false).
But the damage was done.

[deeds]

B08 Pirc, Classical :

[Event "Eman's learning"]
[Site "deeds"]
[Date "2021.05.24"]
[Round "0171-0172"]
[White "B08"]
[Black "B08"]
[Result "*"]
[ECO "B08"]
[PlyCount "20"]

1. e4 d6 2. d4 Nf6 3. Nc3 g6 4. Nf3 Bg7 5. Bg5 O-O 6. Qd2 c6 7. Bd3 Nbd7
8. O-O-O e5 9. dxe5 dxe5 10. h4 h5 *

Code: Select all

   # PLAYER                             :  RATING  ERROR  POINTS  PLAYED   (%)    W     D    L  D(%)  OppAvg  OppN
   1 brainlearn 13.1 nn-cdf1785602d6    :       0   ----  1029.0    2000  51.5  221  1616  163  80.8     -11     1
   2 Eman experience                    :     -11     11   971.0    2000  48.5  163  1616  221  80.8       0     1

White advantage = 66.75 +/- 5.61
Draw rate (equal opponents) = 50.00 % +/- 0.00

brainlearn 13.1 nn-cdf1785602d6 with 221/2000 wins (11,05%, D27/245, 1 440 ms) :

1. e4 @ 100,00%

Eman experience with 163/2000 wins (8,15%, D34/245, 1 763 ms) :

1. e4 @ 100,00%

Averages :
211 sec/game
153 plies/game

It seems [Eman experience] used too much time in the first moves and be a lot stressed in endgame phase.
For the next openings, i will try with Automatic Time Management = false.

[deeds]

After a few tries, I discovered that the following settings allowed better learning :

Automatic Time Management = false
what must be understood is that after a few dozen games, the first moves which are the most effective are known and the learning will take place above all in the middle of the game. In addition, if the engine lacks time at the end of the game, it will distort learning.

Experience MultiPV = false
when this value is true (default), unnecessary data (= like weaker moves with very shallow depths) is written to the experience file.

Experience Book = true
without this function, the engine does not always choose the most effective moves from the experience file => it will lose more often => the effectiveness of the learned moves will be distorted.

Experience Book Min Depth = 40
D27 by default is too little ! When the engine chooses a move from the experience file, you have to be sure that it is better than the one it would have obtained by wasting time analyzing the position. With a tc 60s+1s, the average depth of moves is about D30 with an average analysis time of 1.7 sec/ply (at 7 threads@3.7GHz).

Experience Book Eval Importance = 0
In order to understand how this option worked, I tested with a value of 10 => a disaster ! In the starting position, while the experience file contained the 20 legal moves analyzed in depth D60, the engine chose 1. g4 and so on !?
If we leave the default value at 5, according to the data of the experience file, the engine still loses time to analyze the position to fall back on the same move as that of the experience file.
With a value of 0, the motor always chooses the most efficient move of the experience file.

Experience Book Max Moves = 100
A value of 100 only in learning mode because the engine will play hundreds/thousands of games so it will learn much more than 16 moves (default).

[deeds]

B63 Sicilian, Richter-Rauzer, Poderbrad variation :

[Event "Eman's learning"]
[Site "deeds"]
[Date "2021.05.24"]
[Round "0179-0180"]
[White "B63"]
[Black "B63"]
[Result "*"]
[ECO "B63"]
[PlyCount "20"]

1. e4 c5 2. Nf3 Nc6 3. d4 cxd4 4. Nxd4 Nf6 5. Nc3 d6 6. Bg5 Qb6 7. Nb3 e6
8. Qd2 Be7 9. O-O-O O-O 10. Be2 a6 *

Code: Select all

   # PLAYER                             :  RATING  ERROR  POINTS  PLAYED   (%)    W     D    L  D(%)  OppAvg  OppN
   1 brainlearn 13.1 nn-cdf1785602d6    :       0   ----  1012.5    2001  50.6  164  1697  140  84.8      -4     1
   2 Eman experience                    :      -4     11   988.5    2001  49.4  140  1697  164  84.8       0     1

White advantage = 52.95 +/- 5.55
Draw rate (equal opponents) = 50.00 % +/- 0.00

brainlearn 13.1 nn-cdf1785602d6 with 164/2001 wins (8,20%, D27/245, 1 461 ms) :

1. e4 @ 100,00%

Eman experience with 140/2001 wins (7,00%, D50/245, 1 392 ms) :

1. e4 @ 100,00%

Averages :
155 sec/game
129 plies/game

During the first 1000 games, I tested different settings, hence the variations of the red curve (elo).
During the next 1000 games, I used the settings mentioned above and you can clearly see the difference : eman goes up the slope !