On a 5.5 month timeline

[BB+]

Have thought about this line of reasoning as well. Goes a bit far don't you think?

Imagine all the work involved, first getting familiar with the Fruit source code understanding the in's and out's before even dare to change a thing would take how much time? Then the supposed Mail-box to Bit-board conversion and all the testing involved, how much time? Then all the supposed obfuscations (100=3200) among them the PST's (import the calculated Fruit numbers into Rybka) on top of that adding an extra hiding element, the 3399 obfuscation. During the process adding 100 elo points. Loads of testing of course. All of that in 5½ months? I am counting from the release of Fruit 2.1 sources (17.06.2005) and the release of Rybka 1.0 beta (4.12.2005) The 5½ month being the best case scenario because nobody knows if Vas downloaded the Fruit sources on 17.6.2005 and nobody knows when Vas had his supposed eureka moment, "yes, this is good stuff, I am going to use Fruit as the base for my own engine".

I still think this argument could use some additional bolstering, but I will rebut what I think it is the main contention.

Firstly, as "100 elo" was bolded (in green) in the original, I should first point out that the number might be closer to 75 Elo (see this evidence, where the number is 68), though I'll stick with 100 for the sake of argument. Further, Fabien gained around 50-60 Elo from mid-June to mid-August via tuning. With this in mind, here is an enumeration of gaining 100 Elo in 5.5 months -- I've tried to overestimate every time constraint:

Looking at Fruit, quitting job, etc., 1.5 months
Bitboard conversion, 20-25 Elo from general speed improvement, 2-4 weeks 
                        (including removing/improving various Fruit development/debugging code]
Material table 20-25 Elo, 1-2 days to plug in pre-existing work
Search ideas 20-25 Elo, 1-2 weeks for the "beyond AEL" pruning, also might have had the idea previously
                                  maybe another 1-2 weeks to get rid of Fruit history pruning, reshape its search, etc.
Tuning eval 50 Elo, 2 months [note that various other development can be done during this time, if it is sufficiently automated]
Bugs introduced, -20 Elo 

If nothing else, these are ballpark estimates to the cited Elo gain, and is comfortably within 5.5 months, even with somewhat gratuitous padding. I might note in passing that Fabien rather fervently (and thus, quite out of character) said that he didn't think the fact that Rybka 1.0 Beta gained so much Elo over Fruit 2.1 to be much of a big deal, perhaps in comparison to his own experience with Fruit.

I will deal with the specifics (file-by-file) of "bitboard conversion" in a later post below (if my "time-wasting" budget for the day does not over-run by then).

It's one thing to say you can do all that in just 5.5 months than actually doing it. Following your profile you are not chess programmer or the document must be incomplete. Are you?

I dabbled in writing a computer chess program when I was in Bristol, in 2007. It was (surprise!) a bitboard program with a lot of Fruit influence. It took me less than 5.5 months (working part-time) to get something decent working, though I can't say I was necessarily that interested in tooling around to gain Elo [it did take me more than a week to chase down a hashing bug at one point -- I had to instrument I/O for all accesses, trace them, etc.].

I might in reply ask about your commensurate qualifications in estimating the time necessary to take an existing codebase (not necessarily chess or open-source) and modify according it to one's needs?

I mean this, it's one thing to understand existing code, writing your own bug-free stuff is something entirely different. So it makes me wonder how you can judge the 5½ month issue on its merit.

I don't quite follow this statement. I think the contention is that Rajlich didn't write his "own" bug-free(?!) stuff for at least some of Rybka 1.0 Beta parts, but rather borrowed/modified them from Fruit.

[hyatt]

I mentioned this elsewhere, but if you look at Crafty 22.1 and 22.1, there is a 6 month interval between them, and +120 Elo as I reported from millions of games in cluster testing. So nothing seems impossible with regard to 6 months worth of effort.

[BB+]

I will deal with the specifics (file-by-file) of "bitboard conversion" in a later post below (if my "time-wasting" budget for the day does not over-run by then).

Since my above post was mostly done at the request of Ed, and in the other thread he has now essentially concluded that the only relevant issue for Rybka/Fruit is ordering of evaluation terms, I've chosen not to bother with adding the discussion proposed in this quotation.

[Rebel]

So ok, if you think that all this was doable within 5½ months then let's consider the time line of events we know and the options available.

Time-line of events,

18. Alexander Schmidt: What was the point that you decided to go the commercial route, and was it clear from the beginning to go this way? I think it would be much easier for the programmer of such a strong engine to talk to an existing company like Chessbase or Lokasoft with their big marketing advantage. Did you at least talk to them? Wasn't their offer interesting for you?

Vasik Rajlich: Already when I started in March 2003, I had some plans to go into the field full-time. This is probably a very unusual way to start working on a chess engine, but that is how it was for me. I don't remember when exactly I made the final decision, probably sometime in early 2004, long before Rybka was among the very top engines.

CCT-6
January 31-February 1 2004
Rybka 1.6.1

Fruit release June 17 2005

Rybka release December 4 2005

Plenty of time for Vas to take a freeware source code, do a major rewrite till it was to his satisfaction. Then after the Fruit release took ideas and implemented them (own code) into Rybka (own code).

Plenty of time even to write his own engine from scratch using all the material available on the net how to do things.

Interesting is also the self confidence in Vas wordings, some would call it arrogance, not one byte written (2003) but already knowing where the path would lead him. And he proved it. But maybe that's typical for MIT guys.

[hyatt]

What, EXACTLY, is your point here? 100 Elo in 5.5 months? Been there. 22.1 to 22.2 crafty versions.

So perhaps you can explain exactly what your point is in the above post???

[BB+]

CCT-6
January 31-February 1 2004
Rybka 1.6.1

This is incorrect nomenclature. The Rybka 1.6.1 version was from Nov 2004. The Rybka 1.3-1.5 versions were from Mar/Apr 2004. The fact that the Panel didn't have the exact version of Rybka for CCT6 participation was raised in the Rybka Forum as a nugatory aspect to any evidence therein -- VR did not respond to the question as to whether the above versions were suitable surrogates. As the Rybka 1.6 version was already entered by Rajlich in an "author-based tourney", the Panel never discussed anything specific about CCT6, and Peter Skinner made the decision therein himself.

Plenty of time for Vas to take a freeware source code, do a major rewrite till it was to his satisfaction. Then after the Fruit release took ideas and implemented them (own code) into Rybka (own code).

Plenty of time even to write his own engine from scratch using all the material available on the net how to do things.

As for your argument on the whole, I admit that it is remotely "possible", and indeed slightly more likely than a "conspiracy theory", but if this were the case, I'd expect VR to say something along these lines. Contrarily, he has said (to Zach) that "everything" in Rybka development since March 2003 has been "incremental"-- one can see a similar sense in VR's (public) request to Nalimov in Dec 2005: A little over a year ago, you gave me permission to use your EGTB access code in my engine Rybka, provided that the engine remained amateur. Now, Rybka is going commercial -- he seems to talk of "Rybka" as essentially the same underlying engine as with his prior request in 2004, for if not, he should need to explicate the situation more.

Furthermore, on the evidentiary level [which I realise is not of much interest to some persons by this point] there is the "direct copying" of Fruit 2.1 iterative driver and search control code (Appendix A and end of 6.3.2 of RYBKA_FRUIT -- one might note the former is quite different [e.g., no bad_1 and bad_2] in Fruit 1.0), whereas I would expect that one's "own" engine would have its own version of these, and not follow the idiosyncratic pattern of Fruit 2.1. Or do I not [as you've implied a few times] have a good grasp of computer chess engine development?

Interesting is also the self confidence in Vas wordings, some would call it arrogance, not one byte written (2003) but already knowing where the path would lead him. And he proved it. But maybe that's typical for MIT guys.

I know various persons at MIT, and I'd say that this kind of talk is only about as typical there as it is in other places -- it has more to do with one's "worldview" (or something like this) than anything to do with technical training.

Incidentally, here are two additional examples of VR's visionary sense:

Subject: Re: Unmasking the Secrets of Rybka and Fruit
From: Vasik Rajlich
Message Number: 470751
Date: December 16, 2005 at 03:42:44

In addition, there is vast room for further improvement. If I could get a team of let's say four
smart people to work for four years full time (and this of course won't happen), the engine
could be improved by probably 500 rating points.

Subject: Re: Predictions 2006. Post them now and i will repost predictions end 2006
From: Vasik Rajlich
Message Number: 471686
Date: December 19, 2005 at 07:24:30

Somebody - and there several suspects - will raise the bar by >= 150 ELO points in 2006.
All the ingredients are there for this ..

In both cases, he seems to have over-estimated slightly, but was certainly closer than what I would call the "mainstream" view.

In any event, you completely changed the subject from your original "5.5 months is too short" line, and I don't see much reason to continue a game of "Whac-A-Mole" against whatever new idea [with flimsy support, at best] you might invent. As Wikipedia says/quotes when discussing evidential burden:

The reason for imposing an evidential burden is to ensure the prosecution does not have to disprove all imaginable defences, only those properly supported by sufficient evidence. [Quoting Lord Morris]:
As human behaviour may manifest itself in infinite varieties of circumstances it is perilous to generalize, but it is not every facile mouthing of some easy phrase of excuse that can amount to an explanation. It is for a judge to decide whether there is evidence fit to be left to the jury which could be the basis of some suggested verdict...

An alternative rendering of the same concept is from Thurber: A pinch of probably is worth a pound of perhaps.

[BB+]

Another [possible] example of working in this type of timeframe (slightly tangential as it is Shogi/Chess -- I will let the development experts say more) is this:

Hi Tord,

Yes, I produced a very agressive Shogi engine in about three months. I think it was built on the back of the routines in CSTal3 (not the bitboard ones!) in 1998/99 or so.

What has happened was that OS had licenced Jeff Rollason's Shogi to its Japanese distributor, the revenues were large and important (PC and Play Station). There was strong Shogi competition and Jeff was struggling a bit at one stage. OS had very little control over what he did since nobody understood, and there was always the threat of losing the Japanese deal to competitors. So I attacked the Shogi problem for two reasons, one to put a rocket up Jeff's backside and the other to try and understand the basis of Shogi and Shogi engines in order to manage the project better.

The result was immensley agressive, the underlying routines of movgen, move, unmove are fairly simple to change, so really all the work for a chess programmer is evaluation. I just went for massive king attack and used standard piece values for the different Shogi pieces.

The Japanese distributor came over and watched it play a couple fo games and liked it. Jeff had got everything back together again by then, so I didn't do any more with mine and the source probably sits around somewhere.

Any, the moral is that Shogi is not very difficult for a chess programmer who already has a working chess program.

Also, in reading this, I'm not sure whether the consideration that VR might have hired someone in the Jun-Dec 2005 period has been raised [though I agree it is unlikely].