On Dalke

Code, algorithms, languages, construction...
hyatt
Posts: 1242
Joined: Thu Jun 10, 2010 2:13 am
Real Name: Bob Hyatt (Robert M. Hyatt)
Location: University of Alabama at Birmingham
Contact:

Re: On Dalke

Post by hyatt » Tue Feb 21, 2012 11:51 pm

User923005 wrote:
hyatt wrote:
User923005 wrote:I should also mention that what was demonstrate in the documentation that I read was a few tiny patches of algorithmic similarity (which were nonetheless different because of bitboard verses 0x88).
The Rybka binary is 2.5 MB for even the 1.0 beta with binaries as large as 8 MB for later versions of Rybka. Obviously, to claim a 1 to 1 binary match in these cases is utterly ludicrous.
Of course, due to similar algorithms, small patches of code have very similar structure. But the vast majority of the code is extremely different.

As I have said, Vas could indeed have done something wrong. But the arguments put forward have been entirely unconvincing. And those convicting Vas have turned themselves into wrongdoers in the actions that they performed (e.g. the illegal reproduction and display of reverse engineered data to the public).
no one is claiming a 1-to-1 binary match, unless you look at rybka 1.6.1 and crafty 19.x. For Rybka 1.0 beta vs Fruit, we are talking about semantic matches instead. Of course Rybka would be larger. It has the big rotated bitboard arrays. Fruit does not. From your comment, I presume that assembly language / compilers / etc are not that familiar to you...
I have been programming since 1976.
If you are not claiming an exact match, and if (indeed) most of the code is different, then you are claiming that Vas does the same thing as everyone else, including you.
Your semantic matches are nothing but algorithm matches, and tiny little patches at that. You should be ashamed to call yourself a computer scientist.

Funny stuff. Since 1976 but you don't understand semantic equivalence, or anything else in the discussion? Whoop-de-doo... We claimed that in the case of Rybka 1.6.1 there was an EXACT match. We claimed in Fruit that there was far too much in "non-exact matches." There can't be many exact matches since one uses bitboards, and one uses mailbox. But if one has the experience to understand what it would take to convert from mailbox to bitboard, one can certainly compare the two with great accuracy. Just because you can't, doesn't mean others can't...

syzygy
Posts: 148
Joined: Sun Oct 16, 2011 4:21 pm

Re: On Dalke

Post by syzygy » Wed Feb 22, 2012 12:12 am

User923005 wrote:You do realize that algorithms are not protected? So the blueprint for the sailboat is free for anyone to use, it is only the individually constructed sailboat that is protected.
Reasonable analogy, but wrong conclusion. A blueprint for a sailboat will certainly be protected by copyright, as will be the sailboat built on the basis of the blueprint. The blueprint for a program performing an algorithm is the source code implementing the algorithm, which is also protected by copyright (in so far as there are still choices to be made), as will be the object code built on the basis of the source code.
So Joe can make a carbon fiber pumpkinseed boat using the plan and Fred can make one out of ferro-cement and Larry can make one out of laminated wood.
Only with permission of the blueprint's author.

Anyway, I fail to see why so much attention is paid to copyright issues. The right to participate with a program in a computer chess tournaments has little to do with owning the copyright on the program. If I hire someone to write a chess program from scratch and have the copyright transferred to me, this would not entitle me in any way to enter a computer chess tournament pretending that I am the program's author.

So principles of copyright are of very little importance here. It is much more a question of good sportsmanship and academic integrity.
However, to publicly publish the results of the reverse engineering is almost always illegal. Though I am not a lawyer, I guess that the publishing of the information obtained by reverse engineering of Rybka in this case was also illegal.
Of course it is not. (But I am wondering... according to you a conclusion that large parts of Rybka are based on Fruit cannot be made public if it was arrived at through reverse engineering?)
If others have taken the ideas from Fruit to make their programs stronger, and even taken the ideas from Rybka to make their programs stronger, what exactly is it that Vas has done that is different from what every one else is doing?
As I understand it, the "others" essentially took high-level ideas discussed in internet fora and spent a lot of blood, sweat and tears to mold these ideas into functioning code added to their home-made programs. As I understand it, what Vas is being accussed of is that he took Fruit's source code and either loaded it directly into his editor to work further on it, or had it in a window next to his editor to essentially copy/convert it line by line.

I am planning a rewrite of my own engine. I will probably write large parts from scratch. Other parts I will most likely "copy" by shutting off most of my brain and more or less mechanically converting it line by line into the structure of my new engine. I would not consider an engine to be my "own" if I used let's say crafty source code for that (even if I did it with full permission).

User923005
Posts: 616
Joined: Thu May 19, 2011 1:35 am

Re: On Dalke

Post by User923005 » Wed Feb 22, 2012 2:03 am

First let me apologize. I should not have used the term "lies" because that requires intent.
Second, your Rybka 1.6.1 allegations may be true, because I have not studied them carefully and I do not have the Rybka binary. However, it has no bearing on the ICGA because Rybka 1.6.1 never participated in any ICGA events.
Third, I have investigated reverse engineering law and it is only strictly illegal to publish the results of reverse engineering in Europe. In the US, from what I have read, it would only have been illegal if trade secrets were published.

User923005
Posts: 616
Joined: Thu May 19, 2011 1:35 am

Re: On Dalke

Post by User923005 » Wed Feb 22, 2012 2:12 am

syzygy wrote:
User923005 wrote:You do realize that algorithms are not protected? So the blueprint for the sailboat is free for anyone to use, it is only the individually constructed sailboat that is protected.
Reasonable analogy, but wrong conclusion. A blueprint for a sailboat will certainly be protected by copyright, as will be the sailboat built on the basis of the blueprint. The blueprint for a program performing an algorithm is the source code implementing the algorithm, which is also protected by copyright (in so far as there are still choices to be made), as will be the object code built on the basis of the source code.
Do you know what it means that an algorithm is not protected by a copyright? It means that I can use exactly the same algorithm as long as I did not simply copy your original one. So, for instance, if I study an algorithm I can write my own version of it. Even though it is extremely similar to the original algorithm, it is not any sort of a violation.
So Joe can make a carbon fiber pumpkinseed boat using the plan and Fred can make one out of ferro-cement and Larry can make one out of laminated wood.
Only with permission of the blueprint's author.

Anyway, I fail to see why so much attention is paid to copyright issues. The right to participate with a program in a computer chess tournaments has little to do with owning the copyright on the program. If I hire someone to write a chess program from scratch and have the copyright transferred to me, this would not entitle me in any way to enter a computer chess tournament pretending that I am the program's author.

So principles of copyright are of very little importance here. It is much more a question of good sportsmanship and academic integrity.
However, to publicly publish the results of the reverse engineering is almost always illegal. Though I am not a lawyer, I guess that the publishing of the information obtained by reverse engineering of Rybka in this case was also illegal.
Of course it is not. (But I am wondering... according to you a conclusion that large parts of Rybka are based on Fruit cannot be made public if it was arrived at through reverse engineering?)
If others have taken the ideas from Fruit to make their programs stronger, and even taken the ideas from Rybka to make their programs stronger, what exactly is it that Vas has done that is different from what every one else is doing?
As I understand it, the "others" essentially took high-level ideas discussed in internet fora and spent a lot of blood, sweat and tears to mold these ideas into functioning code added to their home-made programs. As I understand it, what Vas is being accussed of is that he took Fruit's source code and either loaded it directly into his editor to work further on it, or had it in a window next to his editor to essentially copy/convert it line by line.
Yes, these are the accusations. However, any intelligent examination of the evidence will reveal that this standard has not been met. There were a few tiny patches of assembly that were shown to be similar. Even those tiny patches of assembly were necessarily different than the original programs. It takes magical clairvoyance to determine that the bitboard routines of Rybka have been obtained via cut and paste and not by understanding the algorithms and typing in his own version.

I am planning a rewrite of my own engine. I will probably write large parts from scratch. Other parts I will most likely "copy" by shutting off most of my brain and more or less mechanically converting it line by line into the structure of my new engine. I would not consider an engine to be my "own" if I used let's say crafty source code for that (even if I did it with full permission).
If you were to use alpha-beta in your program is it original? In some very real sense, the answer is 'No." If you were to use LMR in your chess program is it original? Again, the answer is no. What I have seen of the evidence clearly shows that Vas has done this sort of copying -- reuse of Fruit ideas is as plain as the nose on my face. This is not the same thing as proving code copying (which IS claimed in the documents). Every chess author whose program is among the top 50 has reused ideas written by others recently. That does not mean that they have done something wrong.

Chess programming has become a cesspool of petty, mean-spirited, selfish jerks. Or maybe it was always that way and I simply failed to recognize it.

User923005
Posts: 616
Joined: Thu May 19, 2011 1:35 am

Re: On Dalke

Post by User923005 » Wed Feb 22, 2012 2:20 am

syzygy wrote:
User923005 wrote:
However, to publicly publish the results of the reverse engineering is almost always illegal. Though I am not a lawyer, I guess that the publishing of the information obtained by reverse engineering of Rybka in this case was also illegal.
Of course it is not. (But I am wondering... according to you a conclusion that large parts of Rybka are based on Fruit cannot be made public if it was arrived at through reverse engineering?)
I have tempered my view on reverse engineering. I think that it is probable that pronouncing the finding would be legal even in Europe (if the reverse engineering were legal) but not display of the code.

I have been referring to this sort of thing:

"The European Software Directive also limits follow-on uses that can be made of information
obtained in the course of decompilation. See European Software Directive, supra, Art. 6(2). One cannot,
for example, publish information learned during reverse engineering. This puts at risk authors of books
such as ANDREW SCHULMAN, DAVID MAXEY, & MATT PIETREK, UNDOCUMENTED WINDOWS: A
PROGRAMMERS’ GUIDE TO RESERVED MICROSOFT WINDOWS API FUNCTIONS (1992). Under Article 6(2),
European decompilers are at risk if they try to recoup their reverse engineering expenses by licensing the
information it learned in the course of its reverse engineering efforts. The Official Commentary to the
Software Directive asserts that Article 6(2)(b) “prevents the publication or trafficking in information by
those who have decompiled existing programs, since it would be inequitable to impose conditions on the
decompiler but allows others access to the information which he had then made public.” Czarnota & Hart,
supra, at 81. The European software directive, in essence, converts copyright into a trade secrecy law as to
internal elements of programs."

I suspect that if you do a web search for salient words from that quote, you will find several interesting references.

User923005
Posts: 616
Joined: Thu May 19, 2011 1:35 am

Re: On Dalke

Post by User923005 » Wed Feb 22, 2012 2:35 am

From:
http://www.mccormacklegal.com/blog/copy ... d-elements
We have this:

"Algorithms

An algorithm in the most general sense is an uncopyrightable idea. As used below, the term “algorithm” refers to a “finite sequence of instructions, each of which has a clear meaning and can be performed with a finite amount of effort in a finite length of time.”(52) For example, an algorithm for Christmas shopping could be: (1) find appropriate gift, (2) purchase gift, (3) wrap gift, and (4) present gift to recipient. “When viewed from the highest level, algorithms represent the very essence of abstract ideas; as such, algorithms are ineligible for copyright or patent protection. . . . [and] always remain in the public domain.”(53)"

User923005
Posts: 616
Joined: Thu May 19, 2011 1:35 am

Re: On Dalke

Post by User923005 » Wed Feb 22, 2012 2:55 am

I also wish to make clear that I have never claimed Vas was innocent.

I make the following claim:
The intent to prove that Vas has copied code is not satisfying to me. It has not been convincingly demonstrated that Vas has not simply reused ideas in the same way that all other chess program authors reuse ideas. Note that this claim of mine is not evidence of innocence, only that to me the evidence presented does not sufficiently back the claims of guilt that were pronounced.

I further state that Chess programmers do not know what an algorithm is. (No, really, they don't or they would not think that "semantic equivalence" (IOW doing the same thing) is an illegal act.)

I admit that some of the evidence uncovered makes me very uncomfortable about the possibility that Vas has done something wrong. However, if some sort of official body is to go so far as to emit information to destroy someone's reputation, they should have actual proof. As I see it, there is a volume of zero when it comes to real, formal proof of wrongdoing.

It is also interesting that the great majority of the evidence presented has nothing to do with any program that ever participated in an ICGA event and it is therefore almost totally irrelevant. After all, the specific charges stem from competition in an ICGA event.

syzygy
Posts: 148
Joined: Sun Oct 16, 2011 4:21 pm

Re: On Dalke

Post by syzygy » Wed Feb 22, 2012 3:28 am

User923005 wrote:Do you know what it means that an algorithm is not protected by a copyright?
I certainly do understand that mere ideas are not copyrightable and that only the expression is. Maybe we have a misunderstanding: for me the "blueprint" of a sailboat are the drawings based on which the sailboat is constructed.
It means that I can use exactly the same algorithm as long as I did not simply copy your original one. So, for instance, if I study an algorithm I can write my own version of it. Even though it is extremely similar to the original algorithm, it is not any sort of a violation.
I don't think anybody is disagreeing here.

However, when you go from a highly non-trivial algorithm to the actual source code, there are many choices to be made. Those make the resulting source code copyrightable. I you do a conversion of let's say C into Java based on a line-by-line understanding, you will copy enough of those choices to infringe the copyright. If you study the C source code long enough to actually fully grasp the algorithm, and implement that algorithm in your own programming style, you will probably be fine from the point of view of copyright.
Yes, these are the accusations. However, any intelligent examination of the evidence will reveal that this standard has not been met. There were a few tiny patches of assembly that were shown to be similar. Even those tiny patches of assembly were necessarily different than the original programs. It takes magical clairvoyance to determine that the bitboard routines of Rybka have been obtained via cut and paste and not by understanding the algorithms and typing in his own version.
I haven't studied the evidence myself, so I can't say too much about this. What I do know is that Vas didn't really attempt to cooperate where there seems to have been a case to answer. There are also these other things such as obfuscating the node count.

If, as you seem to say, it is more or less mathematically impossible to show guilt based on the reverse engineering of the Rybka executable, it seems to me that Vas had an obligation to cooperate (or bear the consequences). There is nothing justified in being able to hide behind obscurity. (It might be different in a real criminal trial, but this is a civil matter.)
If you were to use alpha-beta in your program is it original? In some very real sense, the answer is 'No."
Why come back to this when there is absolutely no disagreement that everybody is free to reuse high-level ideas? If I respond by "if it rains you might get wet", do I win the argument?
Chess programming has become a cesspool of petty, mean-spirited, selfish jerks. Or maybe it was always that way and I simply failed to recognize it.
Well, among the *programmers* (as opposed to the religious fanatics that seem to be crowding some of the fora), there seem to be a great many that are willing to discuss and share ideas in the open. Btw, I don't think Vas was ever one of them? (Feel free to correct me on this, since I might really be wrong here.)

syzygy
Posts: 148
Joined: Sun Oct 16, 2011 4:21 pm

Re: On Dalke

Post by syzygy » Wed Feb 22, 2012 3:39 am

User923005 wrote:I think that it is probable that pronouncing the finding would be legal even in Europe (if the reverse engineering were legal) but not display of the code.
Display of the reverse engineered code would indeed infringe copyright, but then there are fair use (like) exceptions. In so far as it serves the legitimate purpose of backing up the finding, it is most likely legal.

User923005
Posts: 616
Joined: Thu May 19, 2011 1:35 am

Re: On Dalke

Post by User923005 » Wed Feb 22, 2012 3:48 am

syzygy wrote:
User923005 wrote:Do you know what it means that an algorithm is not protected by a copyright?
I certainly do understand that mere ideas are not copyrightable and that only the expression is. Maybe we have a misunderstanding: for me the "blueprint" of a sailboat are the drawings based on which the sailboat is constructed.
It means that I can use exactly the same algorithm as long as I did not simply copy your original one. So, for instance, if I study an algorithm I can write my own version of it. Even though it is extremely similar to the original algorithm, it is not any sort of a violation.
I don't think anybody is disagreeing here.

However, when you go from a highly non-trivial algorithm to the actual source code, there are many choices to be made. Those make the resulting source code copyrightable. I you do a conversion of let's say C into Java based on a line-by-line understanding, you will copy enough of those choices to infringe the copyright. If you study the C source code long enough to actually fully grasp the algorithm, and implement that algorithm in your own programming style, you will probably be fine from the point of view of copyright.
Yes, but the code is not identical. It is only highly similar. And those highly similar patches are tiny.
If, for instance, you studies a simple string reversal algorithm and wrote your own version the assembly language might look extremely similar even though it really is your own implementation {see below}
Yes, these are the accusations. However, any intelligent examination of the evidence will reveal that this standard has not been met. There were a few tiny patches of assembly that were shown to be similar. Even those tiny patches of assembly were necessarily different than the original programs. It takes magical clairvoyance to determine that the bitboard routines of Rybka have been obtained via cut and paste and not by understanding the algorithms and typing in his own version.
I haven't studied the evidence myself, so I can't say too much about this. What I do know is that Vas didn't really attempt to cooperate where there seems to have been a case to answer. There are also these other things such as obfuscating the node count.

If, as you seem to say, it is more or less mathematically impossible to show guilt based on the reverse engineering of the Rybka executable, it seems to me that Vas had an obligation to cooperate (or bear the consequences). There is nothing justified in being able to hide behind obscurity. (It might be different in a real criminal trial, but this is a civil matter.)
If you were to use alpha-beta in your program is it original? In some very real sense, the answer is 'No."
Why come back to this when there is absolutely no disagreement that everybody is free to reuse high-level ideas? If I respond by "if it rains you might get wet", do I win the argument?
We are in agreement, then that algorithms are fair game?
Chess programming has become a cesspool of petty, mean-spirited, selfish jerks. Or maybe it was always that way and I simply failed to recognize it.
Well, among the *programmers* (as opposed to the religious fanatics that seem to be crowding some of the fora), there seem to be a great many that are willing to discuss and share ideas in the open. Btw, I don't think Vas was ever one of them? (Feel free to correct me on this, since I might really be wrong here.)

It is possible for algorithms that look very different but accomplish the same purpose to produce remarkably similar assembly language. If you looked at the assembly routines below, you might very well conclude that we have copying going on (there are two pairs that are highly similar to each other {E,5}, {B,9}). But if you look at the C code that generated the assembly language, you can see that each routine is different from the others (they are actually taken from different author’s attempts to reverse a string in an old news:comp.lang.c posting). It is also true that different compilers or different optimization settings can generate different assembly from the same source code. In other words, to look at two patches of assembly language generated from a higher order language and assume that the high order language has been directly copied is not verifiable. It is possible, of course, to see that the same *algorithm* has been used. See below.

To say that high level language has been directly copied by examination of assembly language *requires* a great deal of poetic license. You can clearly say that the same *algorithms* have been used, but that is neither here nor there.

From the following old CCC thread:
http://talkchess.com/forum/viewtopic.ph ... 18&t=35128

We have this:

Code: Select all

/*
* Various methods to reverse a string, cobbled from ancient clc posts.
*/
#include <string.h>
#include <stdlib.h>
#include <stdio.h>

char *reverse9 (char *string)
{
    char *start, *end;
    for (end = start = string; *end && *(end + 1); ++end);
    while (end > start)
    {
        char temp;
        temp = *start;
        *start = *end;
        *end = temp;
        ++start;
        --end;
    }
    return string;
}


char *reverseE (char *sp)
{
    char *cp, *ep;
    char ch;
    if (!*sp)
        return (sp); /* need this so we can do 1st ep-- below */
    for (ep = sp; *ep; ep++);
    for (ep--, cp = sp; cp < ep; cp++, ep--)
    {
        ch = *cp;
        *cp = *ep;
        *ep = ch;
    }
    return (sp);
}

char *reverseB (char *s)
{
    char t, *e, *ps;
    ps = s;
    e = s;
    while (*e) /* go to end of string */
        e++;
    *e--;
    while (e > s)
    {
        t = *s;
        *s = *e;
        *e = t;
        e--;
        s++;
    }
    s = ps;
    return s;
}
char *reverse5 (char *string)
{
    char *backward = string;
    char *forward = string;
    char temp;
    /* * Set 'backward' to the null byte * terminating the string: */
    while (*backward)
        backward++;
    /* * Move 'forward' towards end of string * and 'backward' towards
    * start of string * until 'forward' and 'backward' meet, * swapping
    * characters as we go: */
    while (forward < backward)
    {
        --backward;
        temp = *backward;
        *backward = *forward;
        *forward = temp;
        ++forward;
    }
    return string;
}



reverseE PROC ; COMDAT 
                cmp BYTE PTR [rcx], 0 
                mov r8, rcx 
                je SHORT $LN1@reverseE 
                mov rdx, rcx 
                npad 5 
$LL6@reverseE: 
                inc rdx 
                cmp BYTE PTR [rdx], 0 
                jne SHORT $LL6@reverseE 
                dec rdx 
                mov r9, rcx 
                cmp rcx, rdx 
                jae SHORT $LN1@reverseE 
                npad 13 
$LL3@reverseE: 
                movzx eax, BYTE PTR [rdx] 
                movzx ecx, BYTE PTR [r9] 
                dec rdx 
                mov BYTE PTR [r9], al 
                mov BYTE PTR [rdx+1], cl 
                inc r9 
                cmp r9, rdx 
                jb SHORT $LL3@reverseE 
$LN1@reverseE: 
                mov rax, r8 
                ret 0 
                reverseE ENDP 
_TEXT ENDS 

reverseB PROC ; COMDAT 
                cmp BYTE PTR [rcx], 0 
                mov r9, rcx 
                mov rax, rcx 
                mov r8, rcx 
                je SHORT $LN3@reverseB 
                npad 2 
$LL4@reverseB: 
                inc r8 
                cmp BYTE PTR [r8], 0 
                jne SHORT $LL4@reverseB 
$LN3@reverseB: 
                dec r8 
                cmp r8, rcx 
                jbe SHORT $LN11@reverseB 
$LL2@reverseB: 
                movzx ecx, BYTE PTR [r8] 
                movzx edx, BYTE PTR [r9] 
                dec r8 
                mov BYTE PTR [r9], cl 
                mov BYTE PTR [r8+1], dl 
                inc r9 
                cmp r8, r9 
                ja SHORT $LL2@reverseB 
$LN11@reverseB: 
                fatret 0 
                reverseB ENDP 
_TEXT ENDS 

reverse5 PROC ; COMDAT 
                cmp BYTE PTR [rcx], 0 
                mov r8, rcx 
                mov rdx, rcx 
                mov r9, rcx 
                je SHORT $LN12@reverse5 
                npad 2 
$LL4@reverse5: 
                inc rdx 
                cmp BYTE PTR [rdx], 0 
                jne SHORT $LL4@reverse5 
                cmp rcx, rdx 
                jae SHORT $LN12@reverse5 
                npad 3 
$LL2@reverse5: 
                movzx eax, BYTE PTR [r9] 
                movzx ecx, BYTE PTR [rdx-1] 
                dec rdx 
                mov BYTE PTR [rdx], al 
                mov BYTE PTR [r9], cl 
                inc r9 
                cmp r9, rdx 
                jb SHORT $LL2@reverse5 
$LN12@reverse5: 
                mov rax, r8 
                ret 0 
                reverse5 ENDP 
_TEXT ENDS 

reverse9 PROC ; COMDAT 
                cmp BYTE PTR [rcx], 0 
                mov r8, rcx 
                mov r9, rcx 
                mov rdx, rcx 
                je SHORT $LN13@reverse9 
                npad 2 
$LL5@reverse9: 
                cmp BYTE PTR [rdx+1], 0 
                lea rax, QWORD PTR [rdx+1] 
                je SHORT $LN9@reverse9 
                mov rdx, rax 
                jmp SHORT $LL5@reverse9 
$LN9@reverse9: 
                cmp rdx, rcx 
                jbe SHORT $LN13@reverse9 
                npad 12 
$LL2@reverse9: 
                movzx eax, BYTE PTR [rdx] 
                movzx ecx, BYTE PTR [r9] 
                dec rdx 
                mov BYTE PTR [r9], al 
                mov BYTE PTR [rdx+1], cl 
                inc r9 
                cmp rdx, r9 
                ja SHORT $LL2@reverse9 
$LN13@reverse9: 
                mov rax, r8 
                ret 0 
                reverse9 ENDP 
_TEXT ENDS

Post Reply