Questions for BB about Rybka PST = Fruit PST

[BB+]

BB wasn't measuring "overlap" he was measuring chess programming skill. If anybody cares to put the program ELO's against BB's "scores" he will find high correlation.

Note the lack of any doubt in his conclusion (though it was later rebutted by Adam Hair, who evidently did care), as could have been engendered by a word like "possibly".

Here are some a few other "certainties" from CW in that thread:

our side would appeal to the arbiter/court that given the obvious ELO correlation, that the BB results are both questionable and unreliable on the basis that we can't be sure what is being measured and the document "eval function - BB" should be omitted from consideration, or junked to put it more rudely.

Once the "obvious ELO correlation" is demoted to a lesser certainty standard, I think little is left but the rudeness.

When BB presented that paper, somebody should have said "er, but your results map to ELO, do they not? go away, do your homework again and come back with something better that isn't going to make us look stupid". [...] Time to get grovelling, apologise to Vas and get ready with the donation to a charity of his choice.

Meanwhile, the scientific way, and I assume you are seeking after truth, with the BB eval function paper would be to go away and redo it using programs of comparable ELO from 2005-6, ie top programs, then you could eliminate the obvious "what exactly are we measuring" ELO problem from the process. Until then the paper is not acceptable to any reasonable person due to ambiguity in what is being measured.

That there could be a "what are we measuring" problem with ELO was also "obvious" to the Panel (even with its "unreasonable" members like Ken Thompson), but they managed to form a consensus that EVAL_COMP was indeed measuring what it said. Again there seems to be little comprehension of (and gross speculation regarding) the machinations of the Panel.

[wgarvin]

Wow. Just to be clear, I was only speculating about the material table, not making accusations. My idea has already been refuted, so there's no need for Ed to get all indignant about it. (When I feel like accusing someone of something, I will be very clear about it, I assure you).

I am not too familiar with early Rybka history. But since that 2003 stuff predates Fruit 2.1 we can be pretty sure that it wasnt the source, and it also makes sense that if Vas had already implemented the material table in his engine 1.6.1, it would not be hard to stitch it into whatever other codebase became Rybka 1.0 Beta. I don't know if that was Fruit in its entirety, or a completely new engine by Vas (except for the Fruit-derived eval). I haven't seen much evidence concerning the origins of anything in Rybka 1.0 Beta except for the eval. Just that its considerably different from Rybka 1.6.1, and there were a few comments about search structure etc. being similar to Fruit.

We've also seen evidence of some direct copying of Crafty code in earlier Rybka 1.6.1, although I think some of Bob's comments about the extent of that copying are, how shall I put it, not completely substantiated by the evidence collected. Yes, there was evidence of some direct copying, but claims that "practically everything" in it was from Crafty are too broad to agree with based on the evidence I saw.

That is not the case with the eval of Rybka 1.0 Beta, though. I think it was shown very thoroughly that all of the eval, including PSTs, features, pawn stuff, king safety, etc. is very similar to Fruit 2.1. Yes there are minor differences and enhancements, and I suppose the weights were tuned. But anybody claiming there's no derivation there should be laughed out of the room. There's a big grey unknown area between "completely original" and "definitely a clone", and there's no clear consensus in the chess programming community exactly how to draw a line through there and say what should be allowed and what shouldn't. Fortunately, Rybka 1.0 Beta does not fall anywhere near the fuzzy area: its very clearly a derivative, and if the ICGA allowed that they would have to allow almost anything.

[mballicora]

Miguel came up with a different way of making his point, good enough that I will copy it here:

Just to make this more clear: I wrote that Fruit's PSTs could be derived from (and indeed are derived from): a specific algorithmic process, an array of weights (like -3 -1 0 1), and however many tunable parameters. What Miguel did (IMO) was point out that the array of weights could instead be embedded in the algorithmic process (via Taxicab considerations, and the like). I don't know whether it is worthwhile to debate which method has more/less "information content", as the process itself must be included in any such comparison. The most confusing thing I found about his presentation is that he doesn't specify the various linear relations between his variables, in the manner that both Rybka and Fruit have, and this can tend to an over-parametrisation of the system, particularly when comparing to other engines.

As I mentioned before, the alternative approach does not have more parameters. You need to count -3,-1,0,+1 as parameters. As you imply, I embedded that knowledge in the extra four parameters I used, ending up with the same number. In fact, with the knight tables, I used less parameters than Zach's code.

The point is not whether VR Rajlich used, Fruit code, my code, or Bob's code, which all of them produce the same numbers. The point is that the fact that there are real alternative ways to do this alerts us about something else. There is an underlying common knowledge that can be hardly "copyrightable". Namely, the secret is in the -3,-1,0,+1 vector.

That is
[0,1,2,3] + [-1,0,0,0] + [-2,-2,-2,-2] = [-3,-1,0,+1]
Blunt centralization ramp + edge penalty + normalization => final vector

There are many ways to end up with the allegedly magic [-3,-1,0,+1], which can ultimately be decomposed in rudimentary, non-unique, non-copyrightable chess knowledge. It is not a surprise that we find this in Crafty and Stockfish!

When I saw [-3,-1,0,+1] I immediately got the pattern and I figured the chess knowledge involved. That is why it was not so difficult to find a reasonable code alternative. Anybody who sees that pattern may or may not mentally decompose it, but I figure that if I show it with code, the people would get what I mean. I never expected to find an identical B table in Crafty, but I certainly expected to find it somewhere. Looks like Stockfish used it too (at least in the N tables and Q).

As you mentioned in a previous post, of course, all ways to come to the same numbers are related. But you also have to take into account that even Bishop, Queen, and Kind endgame tables from Fruit, can be derived from only one table, the Fruit Bishop table! How unique is this table? not much, we can find it in Crafty.

If Vas "copy" anything from the PSTs, it was something like this vector = [-3,-1,0,+1]. I cannot see how this could be an issue of copying and that is the point I am trying to make, not only that there are some alternatives. Other things such as the concept of diagonal bonus and backrank penalties should not be considered a problem (or trapped pieces in A8 etc.).

Miguel

I don't think this changes the conclusion about the PST evidence (which, as I say, never was even considered by the Panel). Namely, whether one views it an as array plus a process, or just an algorithmic process, what is being done is an encoding of some specific rendition of general chess knowledge (such as "centralisation"). There are relevant 11 PSTs, of which 8 of them have noticeable similarity in their renditions of this.

[wgarvin]

Miguel, I respect what you're trying to do, to show that the PST similarity may not be so significant. If it were the only similarity, all this debate about whether or not the PSTs could end up being structurally similar if they were independently developed from first principles, would be more interesting. I think Mark W. and Zach both acknowledged in their reports that the PSTs by themselves were not strong evidence of copying (but I dont have them handy right now to go back and check).

But the PST similarity is only one element of the eval that ended up being extremely similar between Rybka 1.0 Beta and Fruit 2.1. And if you consider the totality of that evidence, and apply the principle of Occam's Razor, the obvious conclusion is that Vas simply copied them from Fruit along with everything else, scaled the weights to finer values, and tuned them or adjusted them.

I considered the evidence of the evaluation features much more damning than the PSTs, which are only one piece of the puzzle here (though it does fit with the rest).

[Rebel]

Finally, I find any suggestion of "lack of time" (5.5 months) to be at best unsupported [either by evidence or argument], and most likely just plain wrong, as I argued above. Sorry if most of points here have already been stated -- it's just easier to state them again in a direct response, rather than point elsewhere. Also it now seems that I might have used "laugh test" to apply to Ed's suggestion that 5.5 months was not enough, while I meant it to apply to my own's attempts to form a reasonable alternative narrative to the Rybka development storyline. This said, I think it is incumbent upon those who suggest that 5.5 months is not often time to explain why they might specifically posit this, and not just leave it as a "possible". As a counterpoint to this, and something the Panel did discuss, it was generally accepted that, even if Rybka 1.0 Beta was Fruit-based, there was (more than) enough time for Rajlich to have re-written his engine between Dec 2005 and Jun 2006 (the first ICGA event he entered) so as to make it Fruit-free. Another possible counterpoint is to actually look at later versions in the Rybka 1 series, and note how many search changes there are, and from this determine a guess at a rate of possible development.

It's one thing to say you can do all that in just 5.5 months than actually doing it. Following your profile you are not chess programmer or the document must be incomplete. Are you?

I mean this, it's one thing to understand existing code, writing your own bug-free stuff is something entirely different. So it makes me wonder how you can judge the 5½ month issue on its merit.

[Rebel]

About the material table in Rybka 1.0 Beta. Its possible that was also originally from Fruit.
[...]
Anyway, my point is just that counting the "material table" as an original part of Rybka that Vas added during those 5.5 months might be incorrect

Simply outrageous. The man introduces something entirely new and even that must be stolen from Fruit. [...]

I would say that your "must" does not equal his "possibly" as a choice of word. [And at any rate, the idea came from an article of LK, while WG's later prose suggested more of a Fruit-link via data structures]. Furthermore, he did at least give a few anecdotal points as additional adumbration regarding his suggestion, plus the exterior reason he was voicing it [not as a direct accusation per se but rather that one might be more hesitant before counting it as a time-consuming part of the 5.5 months -- a point which indeed seems to be valid, as it was pre-existent].

Perhaps you could suggest a re-phrasing of Wylie's thought in a manner you would prefer, or are you against the idea of Wylie voicing it publicly in the first place? Something like: "Maybe we should look further at this Rybka material table, as there's some issues with Fruit there that I'm not completely happy with [...]" -- but then, should we expect everyone to speak so cautiously? [And as above, I think you took (perhaps wrenched) his comment/wording out of its proper context, namely of how/whether to account the material table toward the 5.5 month timeline].

Suggestive. Predefined. Tunnel vision. Unproven. Speculation.

Which of those 5 words would correspond to Chris's conclusion regarding EVAL_COMP?

BB wasn't measuring "overlap" he was measuring chess programming skill. If anybody cares to put the program ELO's against BB's "scores" he will find high correlation.

Note the lack of any doubt in his conclusion (though it was later rebutted by Adam Hair, who evidently did care), as could have been engendered by a word like "possibly". [I do, though, appreciate his suggestion that TSCP and GnuChess should be included]. If Chris was just being a gadfly devil's advocate, I don't see why Wylie's comment can't be taken the same way?

I am speechless.

I would tend not to be so theatrical. Do think Wylie's verbiage is any more transgressive than Chris's in (for instance) his "suggestive, predefined, tunnel-vision, unproven, and speculative" comments about Hyatt and the Panel?

It all points to Hyatt loudly dominating the conclusion phase of the "investigation" and forcing his line on the others through his usual techniques. The exaggerations and lies continue on Rybka forum, one suspects much to the squirming dislike of Hyatt's onetime collaborators.

Basically, guys, he is singlehandedly destroying your case for his own wild motives.

I will address this in a new thread.

[Rebel]

Wow. Just to be clear, I was only speculating about the material table, not making accusations. My idea has already been refuted, so there's no need for Ed to get all indignant about it. (When I feel like accusing someone of something, I will be very clear about it, I assure you).

But Wylie, nice to meet you BTW, it is in the final document too. And that's the problem.

The material tables in Rybka were one of the more interesting features introduced. Their implementation was a new way to evaluate material imbalances. The indexing and evaluations in the table seem to be unique, but there are some very interesting similarities in the information stored in the table with Fruit.

[Rebel]

I think Mark W. and Zach both acknowledged in their reports that the PSTs by themselves were not strong evidence of copying (but I dont have them handy right now to go back and check).

Sorry, but the documents label the PST as a major issue.

2 Outline of the evidence
There are various major points of evidence between Fruit 2.1 and Rybka 1.0
Beta, and a number of minor and/or more circumstantial ones.
The major points of evidence include:
. the use of exactly the same evaluation features;
. the identical ordering of operations at the root node in the search;
. the same type of PST-scheme, re-using the identical File/Rank/Line weights.

7 Conclusion
This document has highlighted a number of places where Rybka 1.0 Beta can be said to have over-stretched an \originality" barrier with respect to Fruit 2.1. These include a borrowing of arrays in PST, a peculiar match in data structures for hashing, a wholesale re-use of evaluation features, and a repetition of the same ordering of operations in root search.

[wgarvin]

I guess I am missing something here. You are suggesting that the PST contents were independently arrived at, despite the fact that they can be generated using the PST initialization code from Fruit with only tweaking of the constants, and that the PSTs of other engines from that period can not, and that Vas reproduced every feature from Fruit's evaluation in his own program, in the same order as they appeared in Fruit?

Give me a break. The most probable explanation is that he copied it.

By the way, the ICGA made a ruling on all of this already. You're fighting the last war. All of the evidence presented to the panel was available to Vas. He was invited, multiple times, to come and view it, and make whatever statements he wanted about it. He was sent a copy of the reports by Zach and Mark W. He chose to make no effort to defend himself. He didn't raise any argument about how he had independently created his own PST from first principles, and how even though he had been through Fruit forwards and backwards and "took many things", the PST initialization code from Fruit was not one of them.

There's no point in second-guessing now, the investigation is over and Vas is banned and I don't think that's going to change. But since you guys obviously care about these issues, the next time the panel is given a case to investigate, you should participate. I'm sure your input would be welcome, and if you are on the panel then your arguments will get considered while they still have a chance to affect the outcome.

[Rebel]

It's far more likely I will never enter an IGCA panel that first refuses to enter the only critical voice (how convenient) and then punish out of proportion without even considering their own role in the drama, not even have the politeness of an answer when you address it.