Page 6 of 7
Re: value of LMR and null-move
Posted: Thu Jul 15, 2010 9:45 am
by Rebel
Thanks for the efforts and figures Bob. Very interesting.
'
Ed
Re: value of LMR and null-move
Posted: Thu Jul 15, 2010 5:11 pm
by hyatt
Morning update, +63 at present for LMR over non-LMR. Going slowly, but going:
Code: Select all
Rank Name Elo + - games score oppo. draws
1 Stockfish 1.8 64bit 2826 13 13 1991 80% 2586 32%
2 Stockfish 1.8a 64bit 2763 12 12 1933 69% 2610 35%
3 Toga2 2572 23 23 609 19% 2795 28%
4 Glaurung 2.2 2543 24 24 631 16% 2795 27%
5 Fruit 2.1 2469 27 27 628 11% 2796 18%
6 Glaurung 1.1 SMP 2427 27 27 746 9% 2795 14%
This poking along is a pain, but we can't power up all the cluster hardware due to A/C issues. We have seen the computer room spike way past 135F and it does not take very long, when the A/C fails.
Re: value of LMR and null-move
Posted: Thu Jul 15, 2010 10:08 pm
by hyatt
Big pain in the a**. This is slow.
now +64 after almost 2400 games for each...
Code: Select all
Rank Name Elo + - games score oppo. draws
1 Stockfish 1.8 64bit 2827 12 12 2387 80% 2590 33%
2 Stockfish 1.8a 64bit 2763 11 11 2355 68% 2612 35%
3 Toga2 2574 21 21 764 20% 2795 28%
4 Glaurung 2.2 2545 21 21 791 16% 2795 27%
5 Fruit 2.1 2463 25 25 782 11% 2796 17%
6 Glaurung 1.1 SMP 2429 26 26 803 9% 2795 14%
Re: value of LMR and null-move
Posted: Fri Jul 16, 2010 5:35 am
by hyatt
Going to be out of town tomorrow and Saturday. If I have internet access, I will post final results, otherwise here is the current status:
Code: Select all
Rank Name Elo + - games score oppo. draws
1 Stockfish 1.8 64bit 2826 11 11 2785 79% 2590 33%
2 Stockfish 1.8a 64bit 2766 10 10 2780 69% 2610 35%
3 Toga2 2579 19 19 912 20% 2796 29%
4 Glaurung 2.2 2538 20 20 919 16% 2796 26%
5 Fruit 2.1 2465 23 23 922 11% 2796 17%
6 Glaurung 1.1 SMP 2426 24 24 946 9% 2796 14%
Re: value of LMR and null-move
Posted: Sat Jul 17, 2010 8:17 am
by tom
So the big question is why LMR only gives 65 Elo for SF, but is worth closer to 200 elo to crafty..
Any theories on this?
- better lmr implementation in crafty?
- other fwd pruning in SF means lmr benefit is reduced?
- some unexpected, emergent behaviour, e.g. unexpectedly good interaction between crafty lmr and crafty's eval?
It would be interesting to know the "bounds" for lmr. Could a really good implementation give 300 elo? I guess I really bad implementation could lose 300 elo, easily
Cheers, all.
Regards,
Tom
Re: value of LMR and null-move
Posted: Sat Jul 17, 2010 9:15 am
by Hood
tom wrote:So the big question is why LMR only gives 65 Elo for SF, but is worth closer to 200 elo to crafty..
Any theories on this?
- better lmr implementation in crafty?
- other fwd pruning in SF means lmr benefit is reduced?
- some unexpected, emergent behaviour, e.g. unexpectedly good interaction between crafty lmr and crafty's eval?
It would be interesting to know the "bounds" for lmr. Could a really good implementation give 300 elo? I guess I really bad implementation could lose 300 elo, easily
Cheers, all.
Regards,
Tom
Hi,
the green question suggestion
Shall be all the programms sensible in the same way to the part(element) of their functionality ?
I suppose that not. It shall depend on what role is playing that element in the whole construction.
The programm is the engineering construction and its strength could come from differrent parts.
If the constructions are similar the sensitivity to the element shall be similar if the designs are differrent then not.
Rgds
Hood
Re: value of LMR and null-move
Posted: Sat Jul 17, 2010 9:33 am
by tom
tom wrote:So the big question is why LMR only gives 65 Elo for SF, but is worth closer to 200 elo to crafty..
Any theories on this?
- better lmr implementation in crafty?
- other fwd pruning in SF means lmr benefit is reduced?
- some unexpected, emergent behaviour, e.g. unexpectedly good interaction between crafty lmr and crafty's eval?
It would be interesting to know the "bounds" for lmr. Could a really good implementation give 300 elo? I guess I really bad implementation could lose 300 elo, easily
Cheers, all.
Regards,
Tom
I'll try and run a test on my prog, so we get another view - value of LMR in a much weaker engine! It would be handy to have a cluster, you know..
Re: value of LMR and null-move
Posted: Sat Jul 17, 2010 1:43 pm
by hyatt
tom wrote:So the big question is why LMR only gives 65 Elo for SF, but is worth closer to 200 elo to crafty..
Any theories on this?
- better lmr implementation in crafty?
- other fwd pruning in SF means lmr benefit is reduced?
- some unexpected, emergent behaviour, e.g. unexpectedly good interaction between crafty lmr and crafty's eval?
It would be interesting to know the "bounds" for lmr. Could a really good implementation give 300 elo? I guess I really bad implementation could lose 300 elo, easily
Cheers, all.
Regards,
Tom
I am not sure, at the present. I am going to try to go back and figure out which version I tested last year when I got +80, and re-run the test to sanity-check things. But in any case, current version is significantly more aggressive in LMR, using 2 almost everywhere. And the pruning stuff is more aggressive as well. How they "add up" and influence each other is unknown.
Re: value of LMR and null-move
Posted: Wed Jul 21, 2010 8:49 pm
by tom
hyatt wrote:tom wrote:So the big question is why LMR only gives 65 Elo for SF, but is worth closer to 200 elo to crafty..
Any theories on this?
- better lmr implementation in crafty?
- other fwd pruning in SF means lmr benefit is reduced?
- some unexpected, emergent behaviour, e.g. unexpectedly good interaction between crafty lmr and crafty's eval?
It would be interesting to know the "bounds" for lmr. Could a really good implementation give 300 elo? I guess I really bad implementation could lose 300 elo, easily
Cheers, all.
Regards,
Tom
I am not sure, at the present. I am going to try to go back and figure out which version I tested last year when I got +80, and re-run the test to sanity-check things. But in any case, current version is significantly more aggressive in LMR, using 2 almost everywhere. And the pruning stuff is more aggressive as well. How they "add up" and influence each other is unknown.
Well, a few test games (60 moves in 1 min) suggest that I'm getting somewhere around 50 Elo from my LMR implementation. Room for improvement, then? Wonder what I've missed..
Code: Select all
Rank Name Elo + - games score oppo. draws
1 Rybka 2.2 32 bit 241 34 32 320 86% -81 14%
2 Ruffian 1.0.5 44 28 27 320 57% -15 18%
3 fran015x -115 27 28 320 33% 39 22%
4 fran015x_nolmr -170 28 29 320 24% 57 19%
Re: value of LMR and null-move
Posted: Thu Jul 22, 2010 10:12 am
by Uly
I don't think it will measure correctly with engines so far apart in strength.