Page 35 of 37
Re: Designing an analysis friendly Stockfish?
Posted: Sat Mar 05, 2011 2:59 am
by LucenaTheLucid
Code: Select all
3/4/2011 7:59:00 PM :
Program Elo + - Games Score Av.Op. Draws
1 Houdini 1.5a : 2913 25 25 821 81.7 % 2653 21.2 %
2 Deep Rybka 4 : 2803 21 20 844 69.7 % 2658 29.9 %
3 Stockfish 2.0.1 : 2775 19 19 843 65.9 % 2661 35.7 %
4 Critter 0.90 : 2764 20 20 844 64.4 % 2661 31.6 %
5 Stockfish 2.0.1 Lucena 1 : 2760 20 20 803 64.4 % 2657 31.9 %
6 Stockfish 2.0.1 PA : 2760 25 25 535 64.5 % 2656 33.3 %
7 Naum 4.2 : 2683 19 19 843 52.3 % 2667 32.7 %
8 Spike 1.4 : 2667 19 19 844 49.6 % 2669 31.5 %
9 Komodo 1.3 : 2629 20 20 844 43.9 % 2672 28.8 %
10 Spark 1.0 : 2616 20 20 844 42.0 % 2672 27.6 %
11 Gull 1.1 : 2608 20 20 844 40.9 % 2672 28.7 %
12 Thinker 5.4D Inert : 2580 20 20 842 36.7 % 2675 28.0 %
13 TogaII1.4 5c : 2536 23 23 754 31.2 % 2673 23.2 %
14 Protector 1.4.0 : 2513 22 22 843 27.6 % 2680 23.5 %
15 Zappa Mexico II : 2443 24 25 842 19.9 % 2685 20.1 %
Re: Designing an analysis friendly Stockfish?
Posted: Sat Mar 05, 2011 3:48 pm
by keoki010
Here is a partial run. 4/4 TC Ponder=off Sleeping threads=on 1024 hash i7980@3.2
Pgn attached at the bottom. Some time losses to all programs. But only a few. OK a bit of a problem on the pgn it's too big and I'll have to edit it later today.
-----------------------------Rating.dat:-----------------------------
3/5/2011 8:39:50 AM :
Program Elo + - Games Score Av.Op. Draws
1 Stockfish-201-64-ja : 2424 82 81 33 54.5 % 2392 54.5 %
2 Critter_0.90_64bit_SSE4 : 2423 90 89 34 54.4 % 2392 44.1 %
3 Stockfish_201_PA_GTB_Gran2g_x64 : 2385 91 92 34 47.1 % 2405 41.2 %
4 Stockfish_201_PA_GTB_Gran2h_x64 : 2368 94 95 33 43.9 % 2411 39.4 %
----------------------------Programs.dat:----------------------------
Individual statistics:
1 Stockfish-201-64-ja : 2424 33 (+ 9,= 18,- 6), 54.5 %
Critter_0.90_64bit_SSE4 : 11 (+ 1,= 7,- 3), 40.9 %
Stockfish_201_PA_GTB_Gran2g_x64: 11 (+ 5,= 5,- 1), 68.2 %
Stockfish_201_PA_GTB_Gran2h_x64: 11 (+ 3,= 6,- 2), 54.5 %
2 Critter_0.90_64bit_SSE4 : 2423 34 (+ 11,= 15,- 8), 54.4 %
Stockfish_201_PA_GTB_Gran2g_x64: 12 (+ 3,= 5,- 4), 45.8 %
Stockfish-201-64-ja : 11 (+ 3,= 7,- 1), 59.1 %
Stockfish_201_PA_GTB_Gran2h_x64: 11 (+ 5,= 3,- 3), 59.1 %
3 Stockfish_201_PA_GTB_Gran2g_x64: 2385 34 (+ 9,= 14,- 11), 47.1 %
Critter_0.90_64bit_SSE4 : 12 (+ 4,= 5,- 3), 54.2 %
Stockfish-201-64-ja : 11 (+ 1,= 5,- 5), 31.8 %
Stockfish_201_PA_GTB_Gran2h_x64: 11 (+ 4,= 4,- 3), 54.5 %
4 Stockfish_201_PA_GTB_Gran2h_x64: 2368 33 (+ 8,= 13,- 12), 43.9 %
Critter_0.90_64bit_SSE4 : 11 (+ 3,= 3,- 5), 40.9 %
Stockfish_201_PA_GTB_Gran2g_x64: 11 (+ 3,= 4,- 4), 45.5 %
Stockfish-201-64-ja : 11 (+ 2,= 6,- 3), 45.5 %
----------------------------General.dat:-----------------------------
Games : 67 (finished)
White Wins : 26 (38.8 %)
Black Wins : 11 (16.4 %)
Draws : 30 (44.8 %)
Unfinished : 0
White Perf. : 61.2 %
Black Perf. : 38.8 %
ECO A = 10 Games (14.9 %)
ECO B = 14 Games (20.9 %)
ECO C = 30 Games (44.8 %)
ECO D = 8 Games (11.9 %)
ECO E = 5 Games ( 7.5 %)
----------------------------Cluster.dat:-----------------------------
Cluster No. 1:
Critter_0.90_64bit_SSE4 (3)
Stockfish_201_PA_GTB_Gran2g_x64 (3)
Stockfish-201-64-ja (3)
Stockfish_201_PA_GTB_Gran2h_x64 (3)
4 programs, 67 games
itoffset = -0.015423
Re: Designing an analysis friendly Stockfish?
Posted: Sun Mar 06, 2011 4:10 pm
by keoki010
another 60 games only one time forfeit by 2h. I think it's just because my TB's are on a Sata disc.
h and g are pretty much even.
-----------------------------Rating.dat:-----------------------------
3/6/2011 9:03:28 AM :
Program Elo + - Games Score Av.Op. Draws
1 Critter_0.90_64bit_SSE4 : 2416 62 62 67 53.0 % 2395 46.3 %
2 Stockfish-201-64-ja : 2408 56 56 67 51.5 % 2397 55.2 %
3 Stockfish_201_PA_GTB_Gran2g_x64 : 2400 56 56 67 50.0 % 2400 55.2 %
4 Stockfish_201_PA_GTB_Gran2h_x64 : 2377 60 60 67 45.5 % 2408 49.3 %
----------------------------Programs.dat:----------------------------
Individual statistics:
1 Critter_0.90_64bit_SSE4 : 2416 67 (+ 20,= 31,- 16), 53.0 %
Stockfish_201_PA_GTB_Gran2g_x64: 23 (+ 3,= 13,- 7), 41.3 %
Stockfish-201-64-ja : 22 (+ 8,= 11,- 3), 61.4 %
Stockfish_201_PA_GTB_Gran2h_x64: 22 (+ 9,= 7,- 6), 56.8 %
2 Stockfish-201-64-ja : 2408 67 (+ 16,= 37,- 14), 51.5 %
Critter_0.90_64bit_SSE4 : 22 (+ 3,= 11,- 8), 38.6 %
Stockfish_201_PA_GTB_Gran2g_x64: 22 (+ 7,= 12,- 3), 59.1 %
Stockfish_201_PA_GTB_Gran2h_x64: 23 (+ 6,= 14,- 3), 56.5 %
3 Stockfish_201_PA_GTB_Gran2g_x64: 2400 67 (+ 15,= 37,- 15), 50.0 %
Critter_0.90_64bit_SSE4 : 23 (+ 7,= 13,- 3), 58.7 %
Stockfish-201-64-ja : 22 (+ 3,= 12,- 7), 40.9 %
Stockfish_201_PA_GTB_Gran2h_x64: 22 (+ 5,= 12,- 5), 50.0 %
4 Stockfish_201_PA_GTB_Gran2h_x64: 2377 67 (+ 14,= 33,- 20), 45.5 %
Critter_0.90_64bit_SSE4 : 22 (+ 6,= 7,- 9), 43.2 %
Stockfish_201_PA_GTB_Gran2g_x64: 22 (+ 5,= 12,- 5), 50.0 %
Stockfish-201-64-ja : 23 (+ 3,= 14,- 6), 43.5 %
----------------------------General.dat:-----------------------------
Games : 134 (finished)
White Wins : 37 (27.6 %)
Black Wins : 28 (20.9 %)
Draws : 69 (51.5 %)
Unfinished : 0
White Perf. : 53.4 %
Black Perf. : 46.6 %
ECO A = 20 Games (14.9 %)
ECO B = 32 Games (23.9 %)
ECO C = 52 Games (38.8 %)
ECO D = 18 Games (13.4 %)
ECO E = 12 Games ( 9.0 %)
----------------------------Cluster.dat:-----------------------------
Cluster No. 1:
Critter_0.90_64bit_SSE4 (3)
Stockfish_201_PA_GTB_Gran2g_x64 (3)
Stockfish-201-64-ja (3)
Stockfish_201_PA_GTB_Gran2h_x64 (3)
4 programs, 134 games
itoffset = -0.013949
Re: Designing an analysis friendly Stockfish?
Posted: Mon Mar 07, 2011 12:46 am
by keoki010
Ok, I think that 2h is the best of the lot. It seems to analyse without bouncing all over the place and the only fault I can see is that in the opening it will over evaluate the position. Once it gets into the middlegame it usually gets back to about the evals of the best engines. I am going to go to 5/8 because there is definitely something wrong with the TC even with the JA build in 3/3 and 4/4. It runs out of time and just is moving in emergency. This means that sometimes it will start with a positive value for the color and then move so fast that it ends up in a draw or a time loss.
If someone has better TC values I can try please post them.\
keoki010
Re: Designing an analysis friendly Stockfish?
Posted: Tue Mar 08, 2011 7:56 am
by Jeremy Bernstein
keoki010 wrote:Ok, I think that 2h is the best of the lot. It seems to analyse without bouncing all over the place and the only fault I can see is that in the opening it will over evaluate the position. Once it gets into the middlegame it usually gets back to about the evals of the best engines. I am going to go to 5/8 because there is definitely something wrong with the TC even with the JA build in 3/3 and 4/4. It runs out of time and just is moving in emergency. This means that sometimes it will start with a positive value for the color and then move so fast that it ends up in a draw or a time loss.
If someone has better TC values I can try please post them.\
keoki010
I've been doing a 4/4 tournament (4 minutes, 4 second increment) with JA, Gran2h/k, Gran2i and Gran2j. I only have 180 games or so, but I don't think there is any reason to believe that any of these builds is stronger than the other at LTC.
I haven't checked the games yet for TC problems. My principle concern, though, was that our changes could negatively impact overall engine performance. This doesn't seem to be the case.
- Bildschirmfoto 2011-03-08 um 07.52.05.png (10.7 KiB) Viewed 3588 times
jb
Re: Designing an analysis friendly Stockfish?
Posted: Thu Mar 10, 2011 5:35 pm
by keoki010
Jeremy Bernstein wrote:keoki010 wrote:Ok, I think that 2h is the best of the lot. It seems to analyse without bouncing all over the place and the only fault I can see is that in the opening it will over evaluate the position. Once it gets into the middlegame it usually gets back to about the evals of the best engines. I am going to go to 5/8 because there is definitely something wrong with the TC even with the JA build in 3/3 and 4/4. It runs out of time and just is moving in emergency. This means that sometimes it will start with a positive value for the color and then move so fast that it ends up in a draw or a time loss.
If someone has better TC values I can try please post them.\
keoki010
I've been doing a 4/4 tournament (4 minutes, 4 second increment) with JA, Gran2h/k, Gran2i and Gran2j. I only have 180 games or so, but I don't think there is any reason to believe that any of these builds is stronger than the other at LTC.
I haven't checked the games yet for TC problems. My principle concern, though, was that our changes could negatively impact overall engine performance. This doesn't seem to be the case.
Bildschirmfoto 2011-03-08 um 07.52.05.png
jb
Jeremy I'll try and pick out some of the games with time problems. From what I've seen though the problems are the same for the GTB variants and the 201 JA. It looks like towards the end of some games they all have to start making emergency moves when they are down to seconds. This seems to happen fairly often. Look at the endgames and you will see only a 1 instead of xs where x is a number of seconds.
Re: Designing an analysis friendly Stockfish?
Posted: Thu Mar 10, 2011 8:07 pm
by keoki010
Jeremy, here are a few games with the time problems I mentioned.
Re: Designing an analysis friendly Stockfish?
Posted: Fri Mar 11, 2011 2:12 am
by LucenaTheLucid
Code: Select all
Program Elo + - Games Score Av.Op. Draws
1 Houdini 1.5a : 2918 22 22 1026 82.1 % 2654 20.7 %
2 Deep Rybka 4.1 : 2813 26 26 532 72.0 % 2649 29.7 %
3 Deep Rybka 4 : 2808 19 19 971 70.3 % 2658 29.2 %
4 Stockfish 2.0.1 : 2776 17 17 1025 65.8 % 2662 35.6 %
5 Stockfish 2.0.1 PA : 2773 19 19 905 66.2 % 2656 32.2 %
6 Critter 0.90 : 2772 18 18 1026 65.3 % 2662 31.6 %
7 Stockfish 2.0.1 Lucena 1 : 2767 20 20 860 65.0 % 2659 31.9 %
8 Naum 4.2 : 2688 18 17 1026 53.0 % 2668 32.5 %
9 Gull 1.2 : 2675 20 20 848 51.3 % 2666 28.3 %
10 Spike 1.4 : 2670 18 18 1026 50.1 % 2669 31.6 %
11 Komodo 1.3 : 2631 18 18 1027 44.3 % 2671 28.8 %
12 Spark 1.0 : 2619 18 18 1025 42.5 % 2672 27.7 %
13 Gull 1.1 : 2613 20 20 861 40.5 % 2680 28.8 %
14 Thinker 5.4D Inert : 2589 18 19 1024 38.0 % 2674 27.4 %
15 TogaII1.4 5c : 2545 20 20 951 32.1 % 2675 23.3 %
16 Protector 1.4.0 : 2523 20 20 1026 29.1 % 2678 23.7 %
17 Hannibal 1.0a : 2521 22 22 850 29.1 % 2676 23.9 %
18 Loop 2007 : 2514 51 52 160 30.6 % 2656 20.0 %
19 Zappa Mexico II : 2445 22 22 1025 20.3 % 2682 19.8 %
20 Jonny 4.00 : 2444 56 58 160 22.2 % 2662 16.9 %
Re: Designing an analysis friendly Stockfish?
Posted: Fri Mar 11, 2011 4:31 pm
by keoki010
LucenaTheLucid wrote:Code: Select all
Program Elo + - Games Score Av.Op. Draws
1 Houdini 1.5a : 2918 22 22 1026 82.1 % 2654 20.7 %
2 Deep Rybka 4.1 : 2813 26 26 532 72.0 % 2649 29.7 %
3 Deep Rybka 4 : 2808 19 19 971 70.3 % 2658 29.2 %
4 Stockfish 2.0.1 : 2776 17 17 1025 65.8 % 2662 35.6 %
5 Stockfish 2.0.1 PA : 2773 19 19 905 66.2 % 2656 32.2 %
6 Critter 0.90 : 2772 18 18 1026 65.3 % 2662 31.6 %
7 Stockfish 2.0.1 Lucena 1 : 2767 20 20 860 65.0 % 2659 31.9 %
8 Naum 4.2 : 2688 18 17 1026 53.0 % 2668 32.5 %
9 Gull 1.2 : 2675 20 20 848 51.3 % 2666 28.3 %
10 Spike 1.4 : 2670 18 18 1026 50.1 % 2669 31.6 %
11 Komodo 1.3 : 2631 18 18 1027 44.3 % 2671 28.8 %
12 Spark 1.0 : 2619 18 18 1025 42.5 % 2672 27.7 %
13 Gull 1.1 : 2613 20 20 861 40.5 % 2680 28.8 %
14 Thinker 5.4D Inert : 2589 18 19 1024 38.0 % 2674 27.4 %
15 TogaII1.4 5c : 2545 20 20 951 32.1 % 2675 23.3 %
16 Protector 1.4.0 : 2523 20 20 1026 29.1 % 2678 23.7 %
17 Hannibal 1.0a : 2521 22 22 850 29.1 % 2676 23.9 %
18 Loop 2007 : 2514 51 52 160 30.6 % 2656 20.0 %
19 Zappa Mexico II : 2445 22 22 1025 20.3 % 2682 19.8 %
20 Jonny 4.00 : 2444 56 58 160 22.2 % 2662 16.9 %
What are your time controls? I'm current running 5"+8' increment round robin with your top 5 engines. I get slightly different results. Will post when I get 200 games.
Re: Designing an analysis friendly Stockfish?
Posted: Sat Mar 12, 2011 5:03 pm
by keoki010
Here are some partial results. TC=5/8 Ponder=off hash=1024 I changed the TC on R4.1 and on Gran2h. I will post them when I finish the tour. Very few time losses.
-----------------------------Rating.dat:-----------------------------
3/12/2011 10:00:04 AM :
Program Elo + - Games Score Av.Op. Draws
1 Houdini_15a_x64 : 2498 74 73 68 66.9 % 2375 27.9 %
2 Rybka4.1 : 2451 64 63 67 59.0 % 2388 43.3 %
3 Stockfish-201-64-ja : 2409 62 62 67 51.5 % 2399 46.3 %
4 Stockfish_Gran2h_x64 : 2327 70 71 67 37.3 % 2417 32.8 %
5 Rybka4 : 2314 67 68 67 35.1 % 2421 37.3 %
----------------------------Programs.dat:----------------------------
Individual statistics:
1 Houdini_15a_x64 : 2498 68 (+ 36,= 19,- 13), 66.9 %
Stockfish_Gran2h_x64 : 17 (+ 11,= 4,- 2), 76.5 %
Rybka4.1 : 17 (+ 4,= 6,- 7), 41.2 %
Rybka4 : 17 (+ 11,= 5,- 1), 79.4 %
Stockfish-201-64-ja : 17 (+ 10,= 4,- 3), 70.6 %
2 Rybka4.1 : 2451 67 (+ 25,= 29,- 13), 59.0 %
Houdini_15a_x64 : 17 (+ 7,= 6,- 4), 58.8 %
Stockfish_Gran2h_x64 : 16 (+ 10,= 4,- 2), 75.0 %
Rybka4 : 17 (+ 4,= 10,- 3), 52.9 %
Stockfish-201-64-ja : 17 (+ 4,= 9,- 4), 50.0 %
3 Stockfish-201-64-ja : 2409 67 (+ 19,= 31,- 17), 51.5 %
Houdini_15a_x64 : 17 (+ 3,= 4,- 10), 29.4 %
Stockfish_Gran2h_x64 : 17 (+ 6,= 11,- 0), 67.6 %
Rybka4.1 : 17 (+ 4,= 9,- 4), 50.0 %
Rybka4 : 16 (+ 6,= 7,- 3), 59.4 %
4 Stockfish_Gran2h_x64 : 2327 67 (+ 14,= 22,- 31), 37.3 %
Houdini_15a_x64 : 17 (+ 2,= 4,- 11), 23.5 %
Rybka4.1 : 16 (+ 2,= 4,- 10), 25.0 %
Rybka4 : 17 (+ 10,= 3,- 4), 67.6 %
Stockfish-201-64-ja : 17 (+ 0,= 11,- 6), 32.4 %
5 Rybka4 : 2314 67 (+ 11,= 25,- 31), 35.1 %
Houdini_15a_x64 : 17 (+ 1,= 5,- 11), 20.6 %
Stockfish_Gran2h_x64 : 17 (+ 4,= 3,- 10), 32.4 %
Rybka4.1 : 17 (+ 3,= 10,- 4), 47.1 %
Stockfish-201-64-ja : 16 (+ 3,= 7,- 6), 40.6 %
----------------------------General.dat:-----------------------------
Games : 168 (finished)
White Wins : 67 (39.9 %)
Black Wins : 38 (22.6 %)
Draws : 63 (37.5 %)
Unfinished : 0
White Perf. : 58.6 %
Black Perf. : 41.4 %
ECO A = 25 Games (14.9 %)
ECO B = 39 Games (23.2 %)
ECO C = 66 Games (39.3 %)
ECO D = 19 Games (11.3 %)
ECO E = 19 Games (11.3 %)
----------------------------Cluster.dat:-----------------------------
Cluster No. 1:
Houdini_15a_x64 (4)
Stockfish_Gran2h_x64 (4)
Rybka4.1 (4)
Rybka4 (4)
Stockfish-201-64-ja (4)
5 programs, 168 games
itoffset = 0.068830