Designing an analysis friendly Stockfish?

[Jeremy Bernstein]

Hot off the presses, Gran2h! I am including a 32-bit build this time, since I'm pretty impressed with this one in my own testing. I haven't seen a single time loss in a few hours of (Arena) testing, and I'm getting very good performance. This is almost Gran2g, but a pretty embarrassing bug involving a misplaced parenthesis has been fixed (which has been around since the dawn of the GTB series, and should only have affected tablebase access, albeit both at the root and not). While Gran2g wasn't really rocking my world, I'm feeling very good about this one. I'm running some 30/0,3 games overnight (against Gran2i (Gran2f + paren fix) and Gran2j (Gran2e + paren fix), Gran2e and JA), but the first 150 games are finished and Gran2h is leading the pack.

So here you are, enjoy! Please test it with OnlyProbeAtRoot DISABLED (default), so I can be sure that the time loss issue is history.

[Uly]

Thanks Jeremy, do I need to upgrade if I don't use TBs nor play games?

[Jeremy Bernstein]

Thanks Jeremy, do I need to upgrade if I don't use TBs nor play games?

No. But the bits are fresher, so you might want to update anyway.

[Ancalagon]

My mistake, I only just saw I had opened the wrong download stockfish_201_pamc.7z package yesterday and did not notice there were no sources anymore included in the Gran2g.7z I had downloaded. Also not in the Gran2h.7z. I know it is a bit of a bother, but without the license and the updated sources the admin might complain about point 5 of the Forum Rules... With a link to the forum members only, I'm not sure this not still is a 'distribution' in the GPL sense. It is a bit of a sensitive issue at the moment, it now even made the pages of 'Der Spiegel' I believe. It would better to include them in the downloads as there were also bugs in the older versions?

Regards, Eelco

[LucenaTheLucid]

Hot off the presses, Gran2h! I am including a 32-bit build this time, since I'm pretty impressed with this one in my own testing. I haven't seen a single time loss in a few hours of (Arena) testing, and I'm getting very good performance. This is almost Gran2g, but a pretty embarrassing bug involving a misplaced parenthesis has been fixed (which has been around since the dawn of the GTB series, and should only have affected tablebase access, albeit both at the root and not). While Gran2g wasn't really rocking my world, I'm feeling very good about this one. I'm running some 30/0,3 games overnight (against Gran2i (Gran2f + paren fix) and Gran2j (Gran2e + paren fix), Gran2e and JA), but the first 150 games are finished and Gran2h is leading the pack.

So here you are, enjoy! Please test it with OnlyProbeAtRoot DISABLED (default), so I can be sure that the time loss issue is history.

Adding it to my Round Robin:

Code: Select all

    Program                          Elo    +   -   Games   Score   Av.Op.  Draws

  1 Houdini 1.5a                   : 2923   27  26   739    82.2 %   2657   20.7 %
  2 Deep Rybka 4                   : 2810   21  21   803    70.7 %   2657   29.6 %
  3 Stockfish 2.0.1                : 2780   20  20   802    66.6 %   2659   34.5 %
  4 Critter 0.90                   : 2770   21  20   803    65.3 %   2660   31.0 %
  5 Stockfish 2.0.1 Lucena 1       : 2764   20  20   803    64.4 %   2661   31.9 %
  6 Naum 4.2                       : 2686   20  20   802    52.9 %   2667   32.9 %
  7 Spike 1.4                      : 2672   20  20   803    50.5 %   2668   31.5 %
  8 Komodo 1.3                     : 2636   20  20   803    45.0 %   2671   29.1 %
  9 Spark 1.0                      : 2618   21  21   803    42.3 %   2672   26.7 %
 10 Gull 1.1                       : 2610   21  21   803    41.2 %   2672   28.1 %
 11 Stockfish 2.0.1 PA             : 2598  259 409     2    25.0 %   2789   50.0 %
 12 Thinker 5.4D Inert             : 2584   21  21   801    37.3 %   2674   27.8 %
 13 TogaII1.4 5c                   : 2534   24  24   672    33.0 %   2657   23.7 %
 14 Protector 1.4.0                : 2517   22  23   802    28.1 %   2680   23.6 %
 15 Zappa Mexico II                : 2446   25  25   801    20.2 %   2685   20.7 %

[Jeremy Bernstein]

My mistake, I only just saw I had opened the wrong download stockfish_201_pamc.7z package yesterday and did not notice there were no sources anymore included in the Gran2g.7z I had downloaded. Also not in the Gran2h.7z. I know it is a bit of a bother, but without the license and the updated sources the admin might complain about point 5 of the Forum Rules... With a link to the forum members only, I'm not sure this not still is a 'distribution' in the GPL sense. It is a bit of a sensitive issue at the moment, it now even made the pages of 'Der Spiegel' I believe. It would better to include them in the downloads as there were also bugs in the older versions?

Regards, Eelco

https://github.com/jeremybernstein/Stockfish_PA_GTB

For ease, the sources are on github, and I've posted this link several times. The sources to Gran2h are not yet pushed, but will be as soon as I'm sure it isn't catastrophically busted or anything. This entire series is for testing and certainly doesn't rise to the level of a distribution (yet), and I don't think there's an issue with posting the binaries locally.

Jeremy

[BB+]

and I don't think there's an issue with posting the binaries locally.

http://www.gnu.org/licenses/gpl.html
From GPLv3:

To “convey” a work means any kind of propagation that enables other parties to make or receive copies.

Which leaves "propagate" to be defined:

To “propagate” a work means to do anything with it that, without permission, would make you directly or secondarily liable for infringement under applicable copyright law, except executing it on a computer or modifying a private copy.

OTOH, my recollections is that you don't have to include the sources, but only make them available on request (cf. Section 6d, maybe?). E.g. I think Jim Ablett has some older things (like Gerbil 0.2) on his site w/o source (I found the source to this elsewhere I think).

[Jeremy Bernstein]

and I don't think there's an issue with posting the binaries locally.

http://www.gnu.org/licenses/gpl.html
From GPLv3:

To “convey” a work means any kind of propagation that enables other parties to make or receive copies.

Which leaves "propagate" to be defined:

To “propagate” a work means to do anything with it that, without permission, would make you directly or secondarily liable for infringement under applicable copyright law, except executing it on a computer or modifying a private copy.

OTOH, my recollections is that you don't have to include the sources, but only make them available on request (cf. Section 6d, maybe?). E.g. I think Jim Ablett has some older things (like Gerbil 0.2) on his site w/o source (I found the source to this elsewhere I think).

Fair enough. To avoid any ambiguity, here is a complete distribution of Gran2k (which is the same as Gran2h, but skipping over the retired i and j to avoid additional ambiguity). Mac and Windows binaries are included, as well as the complete source (which has been updated on github in the meantime, as well).

http://www.open-chess.org/stockfish_pa_gtb_gran2k.7z

[Jeremy Bernstein]

Some results (note that Gran2k is exactly the same as Gran2h):

.  Engine              Score                                                    Gr                                                 Gr                                                 St                                                 Gr                                                 Gr    S-B
1: Gran2h_x64          105,0/200 ·················································· ===11==11=1=10==0=0=10000=====1=0============1=1== =0=01===========1=====00===0=1=====00==1010==1=1== =====1==0==010====01=0=1=11====1==1=0===1011====01 1100==1==01101=1===10==1====1=====0=1=0==101====1=  10362,
2: Gran2e_x64          100,5/200 ===00==00=0=01==1=1=01111=====0=1============0=0== ·················································· =======001=1====0=1010===1==1==01======0==10===1=0 =====1==0==0==1=1=0=10=======1====11=======11=1=== ==1001==10=1100===10==00=1=011=0=0=0====1=1=0=0===  10035,
3: Stockfish-201-64-ja 99,5/200  =1=10===========0=====11===1=0=====11==0101==0=0== =======110=0====1=0101===0==0==10======1==01===0=1 ·················································· 0====0=0=====1=0=11==0=10==0=11===0010====10==1010 =1==00==11=1=0==1=1=====0==0=0=01===0===10=1=0==1=  9969,0
4: Gran2i_x64          99,0/200  =====0==1==101====10=1=0=00====0==0=1===0100====10 =====0==1==1==0=0=1=01=======0====00=======00=0=== 1====1=1=====0=1=00==1=01==1=00===1101====01==0101 ·················································· =110=11=01=1===1==010===1========1====0===1=0===01  9902,7
5: Gran2j_x64          96,0/200  0011==0==10010=0===01==0====0=====1=0=1==010====0= ==0110==01=0011===01==11=0=100=1=1=1====0=0=1=1=== =0==11==00=0=1==0=0=====1==1=1=10===1===01=0=1==0= =001=00=10=0===0==101===0========0====1===0=1===10 ··················································  9688,2

500 games played / Tournament is finished
Name of the tournament: TCTests3
Level: Blitz 0:30/0,3
Hardware: Intel(R) Xeon(R) CPU  W3520 @ 2.67GHz 2659 MHz with 1,0 GB Memory
Operating system: Windows 7 Professional Professional (Build 7600) 64 bit

PGNs attached, not a single time loss for Gran2h/k.

EDIT: pgnscanner ELO calc, using 2919 (current IPON ELO) as starting value:

pgnscanner> elo

> please note that the start elo is : 2919 (see pgnscanner.ini to change it)
> filling repository
> scanning game 529
> end of file
> 542 games scanned
> process time : 1.8s
> clustering...
> adding cluster 6
> calculating ratings for cluster 6
> 14 iteration(s), 0 correction(s), var = 0.000000, stab. factor = 10 {10}

made by PgnScanner 0.80
starting elo : 2919
maximum iterations : 200
minimal number of games : 10
cluster : 6

1.Gran2h_x64 :                  elo=2934, x=2916, sc=105/200
2.Gran2e_x64 :                  elo=2921, x=2919, sc=100/200
3.Stockfish-201-64-ja :         elo=2918, x=2920, sc=99/200
4.Gran2i_x64 :                  elo=2917, x=2920, sc=99/200
5.Gran2j_x64 :                  elo=2909, x=2922, sc=96/200

[keoki010]

Some results (note that Gran2k is exactly the same as Gran2h):

.  Engine              Score                                                    Gr                                                 Gr                                                 St                                                 Gr                                                 Gr    S-B
1: Gran2h_x64          105,0/200 ·················································· ===11==11=1=10==0=0=10000=====1=0============1=1== =0=01===========1=====00===0=1=====00==1010==1=1== =====1==0==010====01=0=1=11====1==1=0===1011====01 1100==1==01101=1===10==1====1=====0=1=0==101====1=  10362,
2: Gran2e_x64          100,5/200 ===00==00=0=01==1=1=01111=====0=1============0=0== ·················································· =======001=1====0=1010===1==1==01======0==10===1=0 =====1==0==0==1=1=0=10=======1====11=======11=1=== ==1001==10=1100===10==00=1=011=0=0=0====1=1=0=0===  10035,
3: Stockfish-201-64-ja 99,5/200  =1=10===========0=====11===1=0=====11==0101==0=0== =======110=0====1=0101===0==0==10======1==01===0=1 ·················································· 0====0=0=====1=0=11==0=10==0=11===0010====10==1010 =1==00==11=1=0==1=1=====0==0=0=01===0===10=1=0==1=  9969,0
4: Gran2i_x64          99,0/200  =====0==1==101====10=1=0=00====0==0=1===0100====10 =====0==1==1==0=0=1=01=======0====00=======00=0=== 1====1=1=====0=1=00==1=01==1=00===1101====01==0101 ·················································· =110=11=01=1===1==010===1========1====0===1=0===01  9902,7
5: Gran2j_x64          96,0/200  0011==0==10010=0===01==0====0=====1=0=1==010====0= ==0110==01=0011===01==11=0=100=1=1=1====0=0=1=1=== =0==11==00=0=1==0=0=====1==1=1=10===1===01=0=1==0= =001=00=10=0===0==101===0========0====1===0=1===10 ··················································  9688,2

500 games played / Tournament is finished
Name of the tournament: TCTests3
Level: Blitz 0:30/0,3
Hardware: Intel(R) Xeon(R) CPU  W3520 @ 2.67GHz 2659 MHz with 1,0 GB Memory
Operating system: Windows 7 Professional Professional (Build 7600) 64 bit

PGNs attached, not a single time loss for Gran2h/k.

EDIT: pgnscanner ELO calc, using 2919 (current IPON ELO) as starting value:

pgnscanner> elo

> please note that the start elo is : 2919 (see pgnscanner.ini to change it)
> filling repository
> scanning game 529
> end of file
> 542 games scanned
> process time : 1.8s
> clustering...
> adding cluster 6
> calculating ratings for cluster 6
> 14 iteration(s), 0 correction(s), var = 0.000000, stab. factor = 10 {10}

made by PgnScanner 0.80
starting elo : 2919
maximum iterations : 200
minimal number of games : 10
cluster : 6

1.Gran2h_x64 :                  elo=2934, x=2916, sc=105/200
2.Gran2e_x64 :                  elo=2921, x=2919, sc=100/200
3.Stockfish-201-64-ja :         elo=2918, x=2920, sc=99/200
4.Gran2i_x64 :                  elo=2917, x=2920, sc=99/200
5.Gran2j_x64 :                  elo=2909, x=2922, sc=96/200

Here's last nights run. No time losses. I'll start a new tour tonight and include 2h but not 2d.

-----------------------------Rating.dat:-----------------------------

3/4/2011 9:05:42 AM :

Program Elo + - Games Score Av.Op. Draws

1 Stockfish_201_PA_GTB_Gran2g_x64 : 2410 35 34 93 52.2 % 2395 76.3 %
2 Stockfish_201_PA_GTB_Gran2f_x64 : 2397 40 40 93 49.5 % 2401 68.8 %
3 Stockfish_201_PA_GTB_Gran2d_x64 : 2393 38 38 94 48.4 % 2404 71.3 %


----------------------------Programs.dat:----------------------------
Individual statistics:

1 Stockfish_201_PA_GTB_Gran2g_x64: 2410 93 (+ 13,= 71,- 9), 52.2 %

Stockfish_201_PA_GTB_Gran2d_x64: 47 (+ 5,= 37,- 5), 50.0 %
Stockfish_201_PA_GTB_Gran2f_x64: 46 (+ 8,= 34,- 4), 54.3 %

2 Stockfish_201_PA_GTB_Gran2f_x64: 2397 93 (+ 14,= 64,- 15), 49.5 %

Stockfish_201_PA_GTB_Gran2d_x64: 47 (+ 10,= 30,- 7), 53.2 %
Stockfish_201_PA_GTB_Gran2g_x64: 46 (+ 4,= 34,- 8), 45.7 %

3 Stockfish_201_PA_GTB_Gran2d_x64: 2393 94 (+ 12,= 67,- 15), 48.4 %

Stockfish_201_PA_GTB_Gran2f_x64: 47 (+ 7,= 30,- 10), 46.8 %
Stockfish_201_PA_GTB_Gran2g_x64: 47 (+ 5,= 37,- 5), 50.0 %


----------------------------General.dat:-----------------------------

Games : 140 (finished)

White Wins : 16 (11.4 %)
Black Wins : 23 (16.4 %)
Draws : 101 (72.1 %)
Unfinished : 0

White Perf. : 47.5 %
Black Perf. : 52.5 %

ECO A = 19 Games (13.6 %)
ECO B = 38 Games (27.1 %)
ECO C = 49 Games (35.0 %)
ECO D = 20 Games (14.3 %)
ECO E = 14 Games (10.0 %)


----------------------------Cluster.dat:-----------------------------
Cluster No. 1:

Stockfish_201_PA_GTB_Gran2d_x64 (2)
Stockfish_201_PA_GTB_Gran2f_x64 (2)
Stockfish_201_PA_GTB_Gran2g_x64 (2)

3 programs, 140 games

itoffset = 0.001752