Feb 12 version: Rybka 1.0 Beta / Fruit 2.1 document

[BB+]

Here is another way of looking at the evaluation feature comparison (YMMV):

Exactly the same usage as Fruit 2.1:

Rybka 1.0 Beta  12/24
Crafty 19.0      2/24
Faile 1.4        2/24
Pepito 1.59      1/24
Phalanx XXII     1/24

Substantially the same usage:

Rybka 1.0 Beta  15/24
Crafty 19.0      4/24
Pepito 1.59      3/24
Phalanx XXII     2/24
Faile 1.4        2/24

More-likely-than-not usage similarity, given the feature is re-used:

Rybka 1.0 Beta  16/24
Crafty 19.0     12/24
Phalanx XXII     6/24
Faile 1.4        5/24
Pepito 1.59      4/24

I'm not exactly sure about some things in this last table. For instance, is additionally considering tripled pawns "more likely than not", given that doubled pawns are considered -- I gave this 0.7. Similarly with draw recognisers, where I would need a better knowledge of which types are commonly used.

As in the text, the "inverse" relationship must be considered too -- Rybka 1.0 Beta has only 3 features not in Fruit 2.1 by my count, while Crafty 19.0 has more than 10 such features.

EDIT: I guess for doubled pawns you can look for whether an engine penalises both pawns, or just one of them (with a score twice as large), which would then also separate the tripled pawn case.

[Jeremy Bernstein]

As in the text, the "inverse" relationship must be considered too -- Rybka 1.0 Beta has only 3 features not in Fruit 2.1 by my count, while Crafty 19.0 has more than 10 such features.

I think that this is the most telling metric. If Fruit has 24 evaluation features, Rybka shares 20 of them and adds 3 new ones (sorry, I'm too sleepy to get the real numbers -- I think that's nearly correct, though), that's +3=20-4. Which is pretty f'ing similar. Of further interest, of course, would be whether those 3 eval features were "new" features of Crafty in a Crafty/Fruit comparison. Do the 3 new features come, rather than from Fruit, from Crafty (I only ask this because of the demonstrated use of Crafty code in Rybka 1.6.1 -- obviously, you won't go fishing in random engines for the "original")?

Once the overall similarity is known, we can hone in on differences in implementation, if any.

jb

[BB+]

Do the 3 new features come, rather than from Fruit, from Crafty (I only ask this because of the demonstrated use of Crafty code in Rybka 1.6.1 -- obviously, you won't go fishing in random engines for the "original")?

Of the 3 "added" features (and others might count these differently), the material imbalance table seems sufficiently Rajlich-specific (even if others also had this) due to his promotion of it in CCC posts, the tempo bonus seems to be a coin-flip as to whether a random engine has it (Zach had even forgotten about it as being somewhat irrelevant), and lazy eval can hardly be said to be specific to any engine (one could argue this isn't really a "feature" in the sense of the others, though I included it because it affects eval). So there's no real sign of an additional Crafty/Rybka overlap from this [nothing like the EvaluateWinner() function from Rybka 1.6.1/Crafty 19 re-appearing for instance].

[BB+]

If Fruit has 24 evaluation features, Rybka shares 20 of them and adds 3 new ones (sorry, I'm too sleepy to get the real numbers -- I think that's nearly correct, though),

As I say, it all depends on how you count these things. You could just enumerate "features" in a broad sense, and then I think Rybka has everything in Fruit except draw recognition, while Fruit has everything in Rybka except lazy eval, material imbalances, and a tempo bonus (Toga included the latter, among other things). Given that there are some broad features that are common to almost every engine (doubled pawns for instance), I tried to go further and demand that the feature conditions also be the same.

I guess the best way to present this material would be to enumerate a large number of features (essentially the union of those in the engines being compared), and then for each give a description of whether/how each engine implements it, and then derive the scores from a rationale therein. This would be a nontrivial amount of work, and I'm not sure it really adds much to any quasi-scientific capacity herein.

[benstoker]

Before we make a decision, let's look at another case with similar evaluation features, i.e. R3H and Komodo. Both have evaluation functions designed by Larry Kaufman.

Komodo? Now Komodo is getting sucked into the vortex of BB+ Chess Engine Metaphysics.

There had been a number of comments that I should (say) investigate all engines from tournament X, for completeness. While this seems a bit outlandish to me, in this particular case Alan Sassler mentioned a specific pair, and the work was not too difficult, so I found it reasonable to comment.

Contingency planning: what happens if ICGA does NOTHING?

Seems to me that then the ICGA does nothing... not sure I can say more than that? [And do I really need a contingency plan for this?]

Just imagining the howls of execration IF the ICGA does nothing . . . that's all ...

[BB+]

For reference, the White pawn values in Rybka 1.0 Beta are loaded into the 256 bytes starting at location 0x64bdf0 in the 64-bit version,

Should be 0x64b4f0. Sorry for any confusion.

[BB+]

Regarding http://www.open-chess.org/viewtopic.php ... =80#p10927

Here is my "probabilisitic" analysis (YMMV): consider a post-Fruit 2.1 world in which ideas are free to take, but code is not. What [is] the probability (or an upper bound for it) that two random programs would:
*) Have the same 20+ evaluation features: [...] I can't put the odds of this match at more than 1 in 10000.

A firmer quantitative analysis has now been performed for evaluation feature overlap, and from the "control group" of engines chosen (none of which are post-Fruit 2.1 currently), crude statistics imply less than 1 in a trillion (12 zeros) for the chance event of the Fruit 2.1 and Rybka 1.0 Beta evaluation feature overlap.

R1x64.eval.txt
Updated version of Rybka 1.0 Beta main eval function (64-bit) with comments

I snarfed up the explanation of unstoppable pawns. As ZW had originally noted, Rybka 1.0 Beta does indeed have both the "square" and "shepherding" conditions. Here is the relevant snippet with the comments corrected:

Code: Select all

0x40244d: test   %r11,%r11                      IF black has no pieces
0x402450: jne    0x40248a
0x402452: test   %rcx,%r12                       and the pawn is not blocked 
0x402455: jne    0x402478
0x402457: mov    0x26c1b3(%rip),%eax # 0x66e610   (get wtm)
0x40245d: lea    (%rax,%r10,2),%rcx               ("square" depends on wtm)
0x402461: lea    -0x2468(%rip),%rax # 0x400000
0x402468: test   %r13,0x262280(%rax,%rcx,8)      and bK is not in the "square"
0x402470: je     0x402478
0x402472: add    $0x6400,%edi                     add 25600 in the endgame
0x402478: test   %r15,0x262a80(%rax,%r10,8)      or if wK shepherds
0x402480: je     0x4024d1
0x402482: add    $0x6400,%edi                     add 25600 in the endgame
0x402488: jmp    0x4024d1

[hyatt]

For the record, Crafty has had that "shepherding idea" for ages. Both likely got the idea from there (fruit and then rybka from fruit).

[BB+]

I haven't been keeping up with a lot of this. Are you sure that Fabien is writing a complaint to the FSF?

From what I hear, Fabien is now in contact with an FSF License Compliance Engineer.