Designing an analysis friendly Stockfish?

LucenaTheLucid · Post by **LucenaTheLucid** » Sat Mar 05, 2011 2:59 am

3/4/2011 7:59:00 PM :

    Program                          Elo    +   -   Games   Score   Av.Op.  Draws

  1 Houdini 1.5a                   : 2913   25  25   821    81.7 %   2653   21.2 %
  2 Deep Rybka 4                   : 2803   21  20   844    69.7 %   2658   29.9 %
  3 Stockfish 2.0.1                : 2775   19  19   843    65.9 %   2661   35.7 %
  4 Critter 0.90                   : 2764   20  20   844    64.4 %   2661   31.6 %
  5 Stockfish 2.0.1 Lucena 1       : 2760   20  20   803    64.4 %   2657   31.9 %
  6 Stockfish 2.0.1 PA             : 2760   25  25   535    64.5 %   2656   33.3 %
  7 Naum 4.2                       : 2683   19  19   843    52.3 %   2667   32.7 %
  8 Spike 1.4                      : 2667   19  19   844    49.6 %   2669   31.5 %
  9 Komodo 1.3                     : 2629   20  20   844    43.9 %   2672   28.8 %
 10 Spark 1.0                      : 2616   20  20   844    42.0 %   2672   27.6 %
 11 Gull 1.1                       : 2608   20  20   844    40.9 %   2672   28.7 %
 12 Thinker 5.4D Inert             : 2580   20  20   842    36.7 %   2675   28.0 %
 13 TogaII1.4 5c                   : 2536   23  23   754    31.2 %   2673   23.2 %
 14 Protector 1.4.0                : 2513   22  22   843    27.6 %   2680   23.5 %
 15 Zappa Mexico II                : 2443   24  25   842    19.9 %   2685   20.1 %

keoki010 · Post by **keoki010** » Sat Mar 05, 2011 3:48 pm

Here is a partial run. 4/4 TC Ponder=off Sleeping threads=on 1024 hash i7980@3.2
Pgn attached at the bottom. Some time losses to all programs. But only a few. OK a bit of a problem on the pgn it's too big and I'll have to edit it later today.

-----------------------------Rating.dat:-----------------------------

3/5/2011 8:39:50 AM :

Program Elo + - Games Score Av.Op. Draws

1 Stockfish-201-64-ja : 2424 82 81 33 54.5 % 2392 54.5 %
2 Critter_0.90_64bit_SSE4 : 2423 90 89 34 54.4 % 2392 44.1 %
3 Stockfish_201_PA_GTB_Gran2g_x64 : 2385 91 92 34 47.1 % 2405 41.2 %
4 Stockfish_201_PA_GTB_Gran2h_x64 : 2368 94 95 33 43.9 % 2411 39.4 %

----------------------------Programs.dat:----------------------------
Individual statistics:

1 Stockfish-201-64-ja : 2424 33 (+ 9,= 18,- 6), 54.5 %

Critter_0.90_64bit_SSE4 : 11 (+ 1,= 7,- 3), 40.9 %
Stockfish_201_PA_GTB_Gran2g_x64: 11 (+ 5,= 5,- 1), 68.2 %
Stockfish_201_PA_GTB_Gran2h_x64: 11 (+ 3,= 6,- 2), 54.5 %

2 Critter_0.90_64bit_SSE4 : 2423 34 (+ 11,= 15,- 8), 54.4 %

Stockfish_201_PA_GTB_Gran2g_x64: 12 (+ 3,= 5,- 4), 45.8 %
Stockfish-201-64-ja : 11 (+ 3,= 7,- 1), 59.1 %
Stockfish_201_PA_GTB_Gran2h_x64: 11 (+ 5,= 3,- 3), 59.1 %

3 Stockfish_201_PA_GTB_Gran2g_x64: 2385 34 (+ 9,= 14,- 11), 47.1 %

Critter_0.90_64bit_SSE4 : 12 (+ 4,= 5,- 3), 54.2 %
Stockfish-201-64-ja : 11 (+ 1,= 5,- 5), 31.8 %
Stockfish_201_PA_GTB_Gran2h_x64: 11 (+ 4,= 4,- 3), 54.5 %

4 Stockfish_201_PA_GTB_Gran2h_x64: 2368 33 (+ 8,= 13,- 12), 43.9 %

Critter_0.90_64bit_SSE4 : 11 (+ 3,= 3,- 5), 40.9 %
Stockfish_201_PA_GTB_Gran2g_x64: 11 (+ 3,= 4,- 4), 45.5 %
Stockfish-201-64-ja : 11 (+ 2,= 6,- 3), 45.5 %

----------------------------General.dat:-----------------------------

Games : 67 (finished)

White Wins : 26 (38.8 %)
Black Wins : 11 (16.4 %)
Draws : 30 (44.8 %)
Unfinished : 0

White Perf. : 61.2 %
Black Perf. : 38.8 %

ECO A = 10 Games (14.9 %)
ECO B = 14 Games (20.9 %)
ECO C = 30 Games (44.8 %)
ECO D = 8 Games (11.9 %)
ECO E = 5 Games ( 7.5 %)

----------------------------Cluster.dat:-----------------------------
Cluster No. 1:

Critter_0.90_64bit_SSE4 (3)
Stockfish_201_PA_GTB_Gran2g_x64 (3)
Stockfish-201-64-ja (3)
Stockfish_201_PA_GTB_Gran2h_x64 (3)

4 programs, 67 games

itoffset = -0.015423

keoki010 · Post by **keoki010** » Sun Mar 06, 2011 4:10 pm

another 60 games only one time forfeit by 2h. I think it's just because my TB's are on a Sata disc.
h and g are pretty much even.

-----------------------------Rating.dat:-----------------------------

3/6/2011 9:03:28 AM :

Program Elo + - Games Score Av.Op. Draws

1 Critter_0.90_64bit_SSE4 : 2416 62 62 67 53.0 % 2395 46.3 %
2 Stockfish-201-64-ja : 2408 56 56 67 51.5 % 2397 55.2 %
3 Stockfish_201_PA_GTB_Gran2g_x64 : 2400 56 56 67 50.0 % 2400 55.2 %
4 Stockfish_201_PA_GTB_Gran2h_x64 : 2377 60 60 67 45.5 % 2408 49.3 %

----------------------------Programs.dat:----------------------------
Individual statistics:

1 Critter_0.90_64bit_SSE4 : 2416 67 (+ 20,= 31,- 16), 53.0 %

Stockfish_201_PA_GTB_Gran2g_x64: 23 (+ 3,= 13,- 7), 41.3 %
Stockfish-201-64-ja : 22 (+ 8,= 11,- 3), 61.4 %
Stockfish_201_PA_GTB_Gran2h_x64: 22 (+ 9,= 7,- 6), 56.8 %

2 Stockfish-201-64-ja : 2408 67 (+ 16,= 37,- 14), 51.5 %

Critter_0.90_64bit_SSE4 : 22 (+ 3,= 11,- 8), 38.6 %
Stockfish_201_PA_GTB_Gran2g_x64: 22 (+ 7,= 12,- 3), 59.1 %
Stockfish_201_PA_GTB_Gran2h_x64: 23 (+ 6,= 14,- 3), 56.5 %

3 Stockfish_201_PA_GTB_Gran2g_x64: 2400 67 (+ 15,= 37,- 15), 50.0 %

Critter_0.90_64bit_SSE4 : 23 (+ 7,= 13,- 3), 58.7 %
Stockfish-201-64-ja : 22 (+ 3,= 12,- 7), 40.9 %
Stockfish_201_PA_GTB_Gran2h_x64: 22 (+ 5,= 12,- 5), 50.0 %

4 Stockfish_201_PA_GTB_Gran2h_x64: 2377 67 (+ 14,= 33,- 20), 45.5 %

Critter_0.90_64bit_SSE4 : 22 (+ 6,= 7,- 9), 43.2 %
Stockfish_201_PA_GTB_Gran2g_x64: 22 (+ 5,= 12,- 5), 50.0 %
Stockfish-201-64-ja : 23 (+ 3,= 14,- 6), 43.5 %

----------------------------General.dat:-----------------------------

Games : 134 (finished)

White Wins : 37 (27.6 %)
Black Wins : 28 (20.9 %)
Draws : 69 (51.5 %)
Unfinished : 0

White Perf. : 53.4 %
Black Perf. : 46.6 %

ECO A = 20 Games (14.9 %)
ECO B = 32 Games (23.9 %)
ECO C = 52 Games (38.8 %)
ECO D = 18 Games (13.4 %)
ECO E = 12 Games ( 9.0 %)

----------------------------Cluster.dat:-----------------------------
Cluster No. 1:

Critter_0.90_64bit_SSE4 (3)
Stockfish_201_PA_GTB_Gran2g_x64 (3)
Stockfish-201-64-ja (3)
Stockfish_201_PA_GTB_Gran2h_x64 (3)

4 programs, 134 games

itoffset = -0.013949

keoki010 · Post by **keoki010** » Mon Mar 07, 2011 12:46 am

Ok, I think that 2h is the best of the lot. It seems to analyse without bouncing all over the place and the only fault I can see is that in the opening it will over evaluate the position. Once it gets into the middlegame it usually gets back to about the evals of the best engines. I am going to go to 5/8 because there is definitely something wrong with the TC even with the JA build in 3/3 and 4/4. It runs out of time and just is moving in emergency. This means that sometimes it will start with a positive value for the color and then move so fast that it ends up in a draw or a time loss.

If someone has better TC values I can try please post them.\

keoki010

Jeremy Bernstein · Post by **Jeremy Bernstein** » Tue Mar 08, 2011 7:56 am

keoki010 wrote:Ok, I think that 2h is the best of the lot. It seems to analyse without bouncing all over the place and the only fault I can see is that in the opening it will over evaluate the position. Once it gets into the middlegame it usually gets back to about the evals of the best engines. I am going to go to 5/8 because there is definitely something wrong with the TC even with the JA build in 3/3 and 4/4. It runs out of time and just is moving in emergency. This means that sometimes it will start with a positive value for the color and then move so fast that it ends up in a draw or a time loss.

If someone has better TC values I can try please post them.\

keoki010

I've been doing a 4/4 tournament (4 minutes, 4 second increment) with JA, Gran2h/k, Gran2i and Gran2j. I only have 180 games or so, but I don't think there is any reason to believe that any of these builds is stronger than the other at LTC.

I haven't checked the games yet for TC problems. My principle concern, though, was that our changes could negatively impact overall engine performance. This doesn't seem to be the case.

: Bildschirmfoto 2011-03-08 um 07.52.05.png (10.7 KiB) Viewed 3308 times

jb

keoki010 · Post by **keoki010** » Thu Mar 10, 2011 5:35 pm

Jeremy Bernstein wrote:
keoki010 wrote:Ok, I think that 2h is the best of the lot. It seems to analyse without bouncing all over the place and the only fault I can see is that in the opening it will over evaluate the position. Once it gets into the middlegame it usually gets back to about the evals of the best engines. I am going to go to 5/8 because there is definitely something wrong with the TC even with the JA build in 3/3 and 4/4. It runs out of time and just is moving in emergency. This means that sometimes it will start with a positive value for the color and then move so fast that it ends up in a draw or a time loss.

If someone has better TC values I can try please post them.\

keoki010
I've been doing a 4/4 tournament (4 minutes, 4 second increment) with JA, Gran2h/k, Gran2i and Gran2j. I only have 180 games or so, but I don't think there is any reason to believe that any of these builds is stronger than the other at LTC.

I haven't checked the games yet for TC problems. My principle concern, though, was that our changes could negatively impact overall engine performance. This doesn't seem to be the case.

Bildschirmfoto 2011-03-08 um 07.52.05.png
jb

Jeremy I'll try and pick out some of the games with time problems. From what I've seen though the problems are the same for the GTB variants and the 201 JA. It looks like towards the end of some games they all have to start making emergency moves when they are down to seconds. This seems to happen fairly often. Look at the endgames and you will see only a 1 instead of xs where x is a number of seconds.

keoki010 · Post by **keoki010** » Thu Mar 10, 2011 8:07 pm

Jeremy, here are a few games with the time problems I mentioned.

LucenaTheLucid · Post by **LucenaTheLucid** » Fri Mar 11, 2011 2:12 am

Code: Select all

Program                          Elo    +   -   Games   Score   Av.Op.  Draws

  1 Houdini 1.5a                   : 2918   22  22  1026    82.1 %   2654   20.7 %
  2 Deep Rybka 4.1                 : 2813   26  26   532    72.0 %   2649   29.7 %
  3 Deep Rybka 4                   : 2808   19  19   971    70.3 %   2658   29.2 %
  4 Stockfish 2.0.1                : 2776   17  17  1025    65.8 %   2662   35.6 %
  5 Stockfish 2.0.1 PA             : 2773   19  19   905    66.2 %   2656   32.2 %
  6 Critter 0.90                   : 2772   18  18  1026    65.3 %   2662   31.6 %
  7 Stockfish 2.0.1 Lucena 1       : 2767   20  20   860    65.0 %   2659   31.9 %
  8 Naum 4.2                       : 2688   18  17  1026    53.0 %   2668   32.5 %
  9 Gull 1.2                       : 2675   20  20   848    51.3 %   2666   28.3 %
 10 Spike 1.4                      : 2670   18  18  1026    50.1 %   2669   31.6 %
 11 Komodo 1.3                     : 2631   18  18  1027    44.3 %   2671   28.8 %
 12 Spark 1.0                      : 2619   18  18  1025    42.5 %   2672   27.7 %
 13 Gull 1.1                       : 2613   20  20   861    40.5 %   2680   28.8 %
 14 Thinker 5.4D Inert             : 2589   18  19  1024    38.0 %   2674   27.4 %
 15 TogaII1.4 5c                   : 2545   20  20   951    32.1 %   2675   23.3 %
 16 Protector 1.4.0                : 2523   20  20  1026    29.1 %   2678   23.7 %
 17 Hannibal 1.0a                  : 2521   22  22   850    29.1 %   2676   23.9 %
 18 Loop 2007                      : 2514   51  52   160    30.6 %   2656   20.0 %
 19 Zappa Mexico II                : 2445   22  22  1025    20.3 %   2682   19.8 %
 20 Jonny 4.00                     : 2444   56  58   160    22.2 %   2662   16.9 %

keoki010 · Post by **keoki010** » Fri Mar 11, 2011 4:31 pm

LucenaTheLucid wrote:

Code: Select all

Program                          Elo    +   -   Games   Score   Av.Op.  Draws

  1 Houdini 1.5a                   : 2918   22  22  1026    82.1 %   2654   20.7 %
  2 Deep Rybka 4.1                 : 2813   26  26   532    72.0 %   2649   29.7 %
  3 Deep Rybka 4                   : 2808   19  19   971    70.3 %   2658   29.2 %
  4 Stockfish 2.0.1                : 2776   17  17  1025    65.8 %   2662   35.6 %
  5 Stockfish 2.0.1 PA             : 2773   19  19   905    66.2 %   2656   32.2 %
  6 Critter 0.90                   : 2772   18  18  1026    65.3 %   2662   31.6 %
  7 Stockfish 2.0.1 Lucena 1       : 2767   20  20   860    65.0 %   2659   31.9 %
  8 Naum 4.2                       : 2688   18  17  1026    53.0 %   2668   32.5 %
  9 Gull 1.2                       : 2675   20  20   848    51.3 %   2666   28.3 %
 10 Spike 1.4                      : 2670   18  18  1026    50.1 %   2669   31.6 %
 11 Komodo 1.3                     : 2631   18  18  1027    44.3 %   2671   28.8 %
 12 Spark 1.0                      : 2619   18  18  1025    42.5 %   2672   27.7 %
 13 Gull 1.1                       : 2613   20  20   861    40.5 %   2680   28.8 %
 14 Thinker 5.4D Inert             : 2589   18  19  1024    38.0 %   2674   27.4 %
 15 TogaII1.4 5c                   : 2545   20  20   951    32.1 %   2675   23.3 %
 16 Protector 1.4.0                : 2523   20  20  1026    29.1 %   2678   23.7 %
 17 Hannibal 1.0a                  : 2521   22  22   850    29.1 %   2676   23.9 %
 18 Loop 2007                      : 2514   51  52   160    30.6 %   2656   20.0 %
 19 Zappa Mexico II                : 2445   22  22  1025    20.3 %   2682   19.8 %
 20 Jonny 4.00                     : 2444   56  58   160    22.2 %   2662   16.9 %

What are your time controls? I'm current running 5"+8' increment round robin with your top 5 engines. I get slightly different results. Will post when I get 200 games.

keoki010 · Post by **keoki010** » Sat Mar 12, 2011 5:03 pm

Here are some partial results. TC=5/8 Ponder=off hash=1024 I changed the TC on R4.1 and on Gran2h. I will post them when I finish the tour. Very few time losses.
-----------------------------Rating.dat:-----------------------------

3/12/2011 10:00:04 AM :

Program Elo + - Games Score Av.Op. Draws

1 Houdini_15a_x64 : 2498 74 73 68 66.9 % 2375 27.9 %
2 Rybka4.1 : 2451 64 63 67 59.0 % 2388 43.3 %
3 Stockfish-201-64-ja : 2409 62 62 67 51.5 % 2399 46.3 %
4 Stockfish_Gran2h_x64 : 2327 70 71 67 37.3 % 2417 32.8 %
5 Rybka4 : 2314 67 68 67 35.1 % 2421 37.3 %

----------------------------Programs.dat:----------------------------
Individual statistics:

1 Houdini_15a_x64 : 2498 68 (+ 36,= 19,- 13), 66.9 %

Stockfish_Gran2h_x64 : 17 (+ 11,= 4,- 2), 76.5 %
Rybka4.1 : 17 (+ 4,= 6,- 7), 41.2 %
Rybka4 : 17 (+ 11,= 5,- 1), 79.4 %
Stockfish-201-64-ja : 17 (+ 10,= 4,- 3), 70.6 %

2 Rybka4.1 : 2451 67 (+ 25,= 29,- 13), 59.0 %

Houdini_15a_x64 : 17 (+ 7,= 6,- 4), 58.8 %
Stockfish_Gran2h_x64 : 16 (+ 10,= 4,- 2), 75.0 %
Rybka4 : 17 (+ 4,= 10,- 3), 52.9 %
Stockfish-201-64-ja : 17 (+ 4,= 9,- 4), 50.0 %

3 Stockfish-201-64-ja : 2409 67 (+ 19,= 31,- 17), 51.5 %

Houdini_15a_x64 : 17 (+ 3,= 4,- 10), 29.4 %
Stockfish_Gran2h_x64 : 17 (+ 6,= 11,- 0), 67.6 %
Rybka4.1 : 17 (+ 4,= 9,- 4), 50.0 %
Rybka4 : 16 (+ 6,= 7,- 3), 59.4 %

4 Stockfish_Gran2h_x64 : 2327 67 (+ 14,= 22,- 31), 37.3 %

Houdini_15a_x64 : 17 (+ 2,= 4,- 11), 23.5 %
Rybka4.1 : 16 (+ 2,= 4,- 10), 25.0 %
Rybka4 : 17 (+ 10,= 3,- 4), 67.6 %
Stockfish-201-64-ja : 17 (+ 0,= 11,- 6), 32.4 %

5 Rybka4 : 2314 67 (+ 11,= 25,- 31), 35.1 %

Houdini_15a_x64 : 17 (+ 1,= 5,- 11), 20.6 %
Stockfish_Gran2h_x64 : 17 (+ 4,= 3,- 10), 32.4 %
Rybka4.1 : 17 (+ 3,= 10,- 4), 47.1 %
Stockfish-201-64-ja : 16 (+ 3,= 7,- 6), 40.6 %

----------------------------General.dat:-----------------------------

Games : 168 (finished)

White Wins : 67 (39.9 %)
Black Wins : 38 (22.6 %)
Draws : 63 (37.5 %)
Unfinished : 0

White Perf. : 58.6 %
Black Perf. : 41.4 %

ECO A = 25 Games (14.9 %)
ECO B = 39 Games (23.2 %)
ECO C = 66 Games (39.3 %)
ECO D = 19 Games (11.3 %)
ECO E = 19 Games (11.3 %)

----------------------------Cluster.dat:-----------------------------
Cluster No. 1:

Houdini_15a_x64 (4)
Stockfish_Gran2h_x64 (4)
Rybka4.1 (4)
Rybka4 (4)
Stockfish-201-64-ja (4)

5 programs, 168 games

itoffset = 0.068830

OpenChess

OpenChess

Designing an analysis friendly Stockfish?

Re: Designing an analysis friendly Stockfish?

Re: Designing an analysis friendly Stockfish?

Re: Designing an analysis friendly Stockfish?

Re: Designing an analysis friendly Stockfish?

Re: Designing an analysis friendly Stockfish?

Re: Designing an analysis friendly Stockfish?

Re: Designing an analysis friendly Stockfish?

Re: Designing an analysis friendly Stockfish?

Re: Designing an analysis friendly Stockfish?

Re: Designing an analysis friendly Stockfish?