Loop 2007 / Fruit 2.1

[Michel Van den Bergh]

I don't quite know the history but didn't Fritz Reul get a PhD for writing Loop?

[hyatt]

I don't quite know the history but didn't Fritz Reul get a PhD for writing Loop?

Yes, but I don't think the purpose of his dissertation was to write a chess program. I think it had to do with some research into algorithms, which would mean it has nothing to do with any outcome of an investigation into reuse of code...

[Damir Desevac]

Hi Bob,

That's a quite an erroneous statement in my opinion, because at some time both Loop/List were commercially available, which shows the fact that there was a whole lot more than just research into algorithms. He could have published his work as an open source, but he did not, which shows that he clearly had a whole lot more in mind than just ''research into algorithms''. Would not you agree with that ?

[Michel Van den Bergh]

That's a quite an erroneous statement in my opinion, because at some time both Loop/List were commercially available, which shows the fact that there was a whole lot more than just research into algorithms. He could have published his work as an open source, but he did not, which shows that he clearly had a whole lot more in mind than just ''research into algorithms''. Would not you agree with that ?

What he "had in mind" is not important I think. He was working on a PhD problem, presumably given to him by his advisor(s).
A valid question is I believe: would Loop not being an original work invalidate his PhD? I had a cursory look at the thesis and he clearly implies that the strength of Loop is the result of the developed algorithms. If it turns out that Loop is simply a Fruit clone then this should at least be regarded as severely misleading the examiners.

[hyatt]

Hi Bob,

That's a quite an erroneous statement in my opinion, because at some time both Loop/List were commercially available, which shows the fact that there was a whole lot more than just research into algorithms. He could have published his work as an open source, but he did not, which shows that he clearly had a whole lot more in mind than just ''research into algorithms''. Would not you agree with that ?

I was addressing the question relative to his Ph.D. dissertation. If his Ph.D. research (I do not recall and have not looked) was about search techniques, there would be no requirement that the rest of the program be original. And if it is discovered to not be original, it would have no bearing on his dissertation. If his dissertation was about developing a complete chess engine, then there could well be a big problem...

[hyatt]

That's a quite an erroneous statement in my opinion, because at some time both Loop/List were commercially available, which shows the fact that there was a whole lot more than just research into algorithms. He could have published his work as an open source, but he did not, which shows that he clearly had a whole lot more in mind than just ''research into algorithms''. Would not you agree with that ?

What he "had in mind" is not important I think. He was working on a PhD problem, presumably given to him by his advisor(s).
A valid question is I believe: would Loop not being an original work invalidate his PhD? I had a cursory look at the thesis and he clearly implies that the strength of Loop is the result of the developed algorithms. If it turns out that Loop is simply a Fruit clone then this should at least be regarded as severely misleading the examiners.

Certainly, although to date the allegation is relative to the evaluation only. If the search is different, and his Ph.D. claims the improvements are related only to search, there's no problem I can think of so long as he mentions the origins of the rest of the program somewhere to avoid any issue related to plagiarism.

[BB+]

Yes, but I don't think the purpose of his dissertation was to write a chess program.

What he "had in mind" is not important I think. He was working on a PhD problem, presumably given to him by his advisor(s).
A valid question is I believe: would Loop not being an original work invalidate his PhD? I had a cursory look at the thesis and he clearly implies that the strength of Loop is the result of the developed algorithms. If it turns out that Loop is simply a Fruit clone then this should at least be regarded as severely misleading the examiners.

I was asked the same by David Levy (I have not spoken directly to his advisor, Jaap van den Herik). Here was my response:

Regarding the thesis of Fritz Reul, I think it is a stretch to think there is any academic dishonesty that would disqualify his thesis on the whole.

He had three main Research questions:

1. To what extent can we develop non-bitboard computer-chess architectures, which are competitive in speed, simplicity, and ease of implementation?

2. To what extent is it possible to use hash functions and magic multiplications in order to examine bitboards in computer chess?

3. How can develop an alpha-beta approach in order to implement pruning conditions in the domain of static exchange evaluation?

and a Problem Statement:

1. How can we develop new computer-chess architectures in such a way that computer-chess engines combine the requirement on knowledge expressiveness with a maximum of efficiency?

Only #1 (with the "competitive" word) much relates to the Loop results. And the specific of the evaluation function is somewhat tangential here. I might think that using the Fruit evaluation function in this guise could even be argued to be "fair use" for an academic project, though once he starts "competing" with it, the situation is more dicey.

The question of "competitive in speed" here in #1 is also touched upon in his Conclusion on the Problem Statement, where nodes per seconds-cores is used as a metric, as opposed to any tournament result.

He does mention the Loop results again for #2 (page 97, 111 of the PDF), and I actually think he is likely wrong to say that Loop did so well due to "the use of highly sophiscated perfect hash functions and magic multipliers for the computation of compound bit-patterns (bitboards) via perfect hashing". One could argue that the Fruit evaluation was more beneficial that any speed-ups he gained from internal representations. At any rate, there is no evidence and/or quantification regarding this in his work.

Also, academics don't care that much about results (though usually the writer of a thesis is sure to mention them). E.g. a thesis on talkbots would be judged more upon whether the methods were novel, and well-explained with proper relation to previous work, as opposed as to how well it fared in some competition.

Overall, my opinion is that he should have mentioned the fact that the Loop evaluation function was from Fruit, but this is in some sense "de minimis" with respect to the entirety and focus of the thesis. Jaap may have a different opinion.

[Adam Hair]

I have been curious about the connection between List and Aristarch. The two engines showed some similarities in move selection. But I decided that this was due to an exchange of ideas between Fritz Reul and Stephan Zipproth. After all, List 5.12 is hosted at http://www.zipproth.com (or was hosted. The site is now down). That was until I found this:

Key:

1) Aristarch 4.4.131 (time: 910 ms scale: 1.0)
2) Crafty 18.15 (time: 910 ms scale: 1.0)
3) List 5.12 (time: 538 ms scale: 1.0)

1 2 3
1. ----- 61.36 60.12
2. 61.36 ----- 56.17
3. 60.12 56.17 -----

I believe the 5 standard deviations is an adequate point to start questioning the origins of an engine. After all, I am certain 1000 is an upper limit for unique engines (Unique as in different authors and no shared code). That would represent 499,500 pairs. 5 st dev represents an event that happens once in 3,488,556 times. So, we would not necessarily expect a 5 sigma event to happen in the population of unique engines. For my newer data, 5 st dev = ~59%. So, the Crafty/Aristarch and Aristarch/List relations look a little suspicious. And I believe if I could acquire the move selection data for List 4.61, its similarity with Crafty 18.15 would be at the same level as Aristarch 4.4.131. Also, an earlier Crafty 18.xx may produce higher numbers.

To finish this, here are some additional comparisons for Loop:

Key:

1) Fruit 2.1 (time: 290 ms scale: 1.0)
2) Fruit 2.2.1 (time: 217 ms scale: 1.0)
3) Loop 10.32f (time: 178 ms scale: 1.0)
4) Loop 2007 (Loop 13.6) (time: 178 ms scale: 1.0)
5) Rybka 1.0 Beta (time: 171 ms scale: 1.0)
6) TogaII 1.0 (time: 327 ms scale: 1.0)

1 2 3 4 5 6
1. ----- 71.78 70.67 70.81 55.75 72.43
2. 71.78 ----- 84.13 84.73 56.15 67.61
3. 70.67 84.13 ----- 98.56 56.48 67.92
4. 70.81 84.73 98.56 ----- 56.62 67.88
5. 55.75 56.15 56.48 56.62 ----- 55.33
6. 72.43 67.61 67.92 67.88 55.33 -----

70-71% is approximately 9 st dev for my data
84% is approximately 13 st dev.

[JcMaTe]

the loopchess website is down

[Prima]

Actually Loop site has been down since it was determined Fruit codes to be in Loop, and will be subject to investigation. The author has also coincidentally vanished.

But not to worry. Money/monetary is a funny thing. It has the tendency to draw people from fox holes. There's suggestion that the author of Loop still collects royal payments through a "different" e-mail account, given in one of CCC (Talk Chess) threads. Let's see him continue pretending to be disconnected from the world, when contacted through that royal payment receivable e-mail address.