OpenChess

kranium wrote:i.e. If your testing techniques are superior, and perhaps empirical....then why is StockFish not as strong than the rest of the field?

It seems to me that Stockfish is stronger than most of the rest of the field.

Rank Name Elo + - games score oppo. draws
1 Houdini 1.5 x64 3000 18 18 1408 80% 2763 28%
2 Rybka 4 x64 Exp. 42 2969 20 19 1240 80% 2719 25%
- Rybka 4.1 x64 Exp. 79TD v.1 2966 19 19 1200 79% 2738 26% NEW
- Rybka 4.1 x64 2965 26 25 630 77% 2753 30% NEW + 26
- Rybka 4 x64 Exp. 61 2961 21 20 1000 78% 2743 27%
- Houdini 1.03a x64 2949 22 21 920 79% 2719 29%
3 IvanHoe B47cB x64 2944 24 24 670 74% 2767 36% NEW + 11
- Rybka 4 x64 2939 17 17 1520 80% 2700 29%
- IvanHoe B49jA x64 2933 19 18 1160 76% 2739 34%
- IvanHoe B52aC x64 2923 21 21 880 77% 2728 34%
4 Stockfish 1.8.0 JA x64 2906 18 18 1200 75% 2715 33%
- Stockfish 1.9.1 JA x64 2906 18 17 1280 73% 2729 33%
5 Critter 1.01 x64 2906 23 23 669 69% 2768 37% NEW + 34]

It seems clear that Stockfish has benefited enormously from the release of RobboLito, IvanHoe, etc...
(this has in fact been shown)...
not because of 'empirical' testing methods or 'new' ideas from the developers.

Don't forget the release of several major embarassing 'bug' versions, and next day fixes,
...wonder why these problems weren't discovered during the incredibly thorough SF 'empirical' testing procedures?

I suggest we get back to topic:
"Fire's null_new_depth"

Uly wrote: It seems to me that Stockfish is stronger than most of the rest of the field.

?
Not according to the 'open' (and unbiased) ranking lists I see

http://www.talkchess.com/forum/viewtopic.php?t=37502

kranium wrote:Rank Name Elo + - games score oppo. draws
1 Houdini 1.5 x64 3000 18 18 1408 80% 2763 28%
2 Rybka 4 x64 Exp. 42 2969 20 19 1240 80% 2719 25%
- Rybka 4.1 x64 Exp. 79TD v.1 2966 19 19 1200 79% 2738 26% NEW
- Rybka 4.1 x64 2965 26 25 630 77% 2753 30% NEW + 26
- Rybka 4 x64 Exp. 61 2961 21 20 1000 78% 2743 27%
- Houdini 1.03a x64 2949 22 21 920 79% 2719 29%
3 IvanHoe B47cB x64 2944 24 24 670 74% 2767 36% NEW + 11
- Rybka 4 x64 2939 17 17 1520 80% 2700 29%
- IvanHoe B49jA x64 2933 19 18 1160 76% 2739 34%
- IvanHoe B52aC x64 2923 21 21 880 77% 2728 34%
4 Stockfish 1.8.0 JA x64 2906 18 18 1200 75% 2715 33%
- Stockfish 1.9.1 JA x64 2906 18 17 1280 73% 2729 33%
5 Critter 1.01 x64 2906 23 23 669 69% 2768 37% NEW + 34]

It seems clear that Stockfish has benefited enormously from the release of RobboLito, IvanHoe, etc...
(this has in fact been shown)...
not because of 'empirical' testing methods or 'new' ideas from the developers.

Don't forget the release of several major embarassing 'bug' versions, and next day fixes,
...wonder why these problems weren't discovered during the incredibly thorough SF 'empirical' testing procedures?

I suggest we get back to topic:
"Fire's null_new_depth"

I see two potential engines in that list. Rybka and Stockfish. The rest are derivatives. And we now know that Rybka itself is a derivative as well. Houdini? pure clone with a few small changes. Robo/ip*? Reverse-engineered clones. Ivanhoe? Clone of a clone. It would appear to me that stockfish is #2 on that list, and probably #1 overall if you only count _original_ engines...

mcostalba wrote:

kranium wrote: i.e. If your testing techniques are superior, and perhaps empirical....then why is StockFish not as strong than the rest of the field?

Because we are not able to come up with winning ideas: we test a lot, but for the most part candidate changes result in no ELO change or even in a weaker engine.

Regarding the rest of the field, apart from Houdini, we think we are almost already there...

There might still be room to earn some ELO points by only fixing bugs. For example in evaluate.c, evaluate_passed_pawns(..), line 831

ebonus -= Value(square_distance(pos.king_square(Us), blockSq + pawn_push(Us)) * rr);

if blockSq is already on rank 8, blockSq + pawn_push(Us) is on rank 9, outside of board. Makes it sense to measure king distance to a field outside the board? Probably not much.

In evaluate_unstoppable_pawns(...) within the first while loop you do not check for enemy pieces blocking a square in front of a passed pawn. In that case you should "give up" on that pawn. There might be more to fix or make better in that function. Passed pawn detection is very ELO critical. So it probably would pay off to invest some serious time there.

hyatt wrote: Houdini? pure clone with a few small changes. Robo/ip*? Reverse-engineered clones...

adding SMP and increasing strength by 100 ELO above Robbo is 'a few small changes'...and the engine is not a derivative... it's a pure clone?
?
That's ridiculous...

hyatt wrote: Ivanhoe? Clone of a clone.

Sorry Bob, you've gone too far...
?
This is absolute nonsense and you know it.
Suddenly, anything that doesn't have a 'Crafty-like pedigree' is simply cast as a 'clone'...?

The facts:
The development of IvanHoe has been ongoing for 2 years now...
the program has grown from a basic skeleton engine to a full-featured analysis tool...
they have added feature after feature... including many technical innovations.

the development includes (but not limited to):

Windows and Linux support
SMP support,
a complete bitbase solution that surpasses Nalimov EGTB,
Montecarlo analysis,
sophisticated hashing options,
compiling tracing,
Magic Bitboards,
ZugZwang detection
Large pages
Eval and Material explanation modes,
etc., etc.
the list goes on and on...

but according to you it's a 'clone'

Stop it Bob...it's utterly ridiculous.

You know damn well that these engines have been extensively developed, calling them pure 'clones' is baloney.

I like Crafty...it's a great program, and I like you, you did a fantastic job...and deserve much respect for your 30 years.
But, haven't you received enough recognition for it?
Now you're on the warpath to eradicate any program you don't like?
I like you, am gratified that Rybka has been uncovered for what it is and discredited, but...must you now mount a hit-squad campaign to stamp out and discredit all other chess programming competition?
Keep it up, and if enough ignorant people simply take you at your word, there will be only 'Crafty' left...the one and only 'original' #1.

I simply don't understand why you re-iterate Crafty's utter originality, it's blessings, and extensive history over and over, ad-nauseum...
I could find dozens of examples posted just in the last month...I suggest you 'give it a break'.
(no worries: you get the nobel-prize...from the Wizard of Oz).

The real world is not a Univ of Alabama programming classroom, and Talkchess should not IMHO serve as a lecture hall for your academic ideals...
'many' and probably 'most' don't care if the program is a 'derivative' or not.
Besides even if a program is based on something else, a 'derivative' is a real program deserving of recognition too!

It's bad enough that as founding member and moderator, you have managed to saturate the Talkchess lemmings with your overbearing and puritanical academic ideals...
(i.e. 'the only valid program is a 100% original Hemmingway-like creation, ala 'Crafty')...here's where you sing it's praises for the 10000th time...

What on earth is happening to you? It's scary...
You've been posting utter nonsense on Talkchess as well..not even bothering to check your facts.
It's as if you believe that just because your post it, everyone will simply (or is required to) accept it as true...

Please stop making a fool of yourself, maybe a little humility and tolerance?...
last time I checked, the world is not your private classroom!

Norm

What has happened to me? Nothing except for developing an immense feeling of disgust with these so-called "top programs". Clones. Derivatives. Doesn't matter. Either is just as bad (and unacceptable) as the other. From the list presented, Stockfish appears to be the _only_ legitimate program that was listed...

"These engines have been extensively ______" should have "copied" in the blank, not "developed". Sorry.

kranium wrote:adding SMP and increasing strength by 100 ELO above Robbo

A don't really understand this firm belief that Houdini is a derivative of Robbo. Why not Ivanhoe's? I can't see any reason for RH to write the SMP code from scratch while there's quite a good solution already implemented...

hyatt wrote:Stockfish appears to be the _only_ legitimate program that was listed...

You suggest Critter to be a clone? Looks unlikely to me.

P.S. I thought it's quite clear for everyone that Rybka 4 has almost nothing common with Rybka 1.0 and Fruit 2.1. So, IMHO, there are no reasons to call it "clone".

I did not notice critter's name in that list. I only saw a bunch of Rybka/Robo/ip/houdini/etc derivative/clone names. If I overlooked it, it was unintentional. As far as how Houdini's parallel search works, how do you _know_ he developed his own? Ever seen his source? That's the trouble with this entire kit and kaboodle. I suppose we will reach a point where computer chess implodes on itself and interest drops to zero, at least from the developmental side. It doesn't appear to be that far off at the present rate of expansion.

Chris Conkie has provided the best examples of program output showing _identical_ output for some version of Houdini compared to some version of Robo*. Same depths, same PVs, same fail highs, fail lows, etc.... And we are not talking about forced mate positions where one might expect everyone to agree. The fact that an unknown author pops up out of the woodwork, with a program ranked #1, after Robo came out and was ranked near the top as well, is just a bit too much for coincidence. And then the output, particularly for some oddball positions, was revealing since only the robo* family seemed to show the same analysis.

As far as Rybka 4, there is no evidence showing it has almost no fruit/rybka 1 code in it. How much remains is a valid question. Whether any remains or not is the more important issue, as none is allowable...

hyatt wrote:Chris Conkie has provided the best examples of program output showing _identical_ output for some version of Houdini compared to some version of Robo*. Same depths, same PVs, same fail highs, fail lows, etc.... And we are not talking about forced mate positions where one might expect everyone to agree. The fact that an unknown author pops up out of the woodwork, with a program ranked #1, after Robo came out and was ranked near the top as well, is just a bit too much for coincidence. And then the output, particularly for some oddball positions, was revealing since only the robo* family seemed to show the same analysis.

I see two possibilities here:
1) Houdini is derived from Robbo. Hence the SMP code is written from scratch.
2) Houdini is derived from Ivanhoe. This seems more logical to me.
However if the output of Houdini and Robbo is identical in some peculiar positions and different from the output of Ivanhoe then the first option is likely to be the case...

P.S. The fact that Houdini is a derivative became clear for me when I examined it with a disassembler:). The same algorithms of IID, single reply exclusion. Almost the same prunings (same structure, slightly increased margins). The same search functions which have the same structure, move generation phases...

hyatt wrote:As far as Rybka 4, there is no evidence showing it has almost no fruit/rybka 1 code in it. How much remains is a valid question. Whether any remains or not is the more important issue, as none is allowable...

Well, the similarities between Rybka 1.0 beta and Fruit 2.1 are concentrated mostly in the evalution function. And it's quite clear from the BB's report on the Rybka3/Robbo issue that in Rybka 3 the evaluation was rewritten completely. So it seems quite unlikely that any traces of the Fruit code are still there...