Houdini 1.02

[Sentinel]

surprised no one has mentioned houdini is now being 'officially' tested by ingo bauer -- http://www.inwoba.de/index.html

currently in the #2 spot trailing rybka by 11 points, which is roughly what i'd expect. more games to go.

It's only -3 elo atm, so I think the current number of games Ingo plays will be well inside error margin and we won't know which one is the strongest.
Unless he decides just for R4 and Houdini to play like 10k games...

[Eduard Nemeth]

In my opinion, the endgame databases (EGTB's) makes +10 to +20 ELO more. In the case Houdini 1.02 almost twice ~ +20 to +40 ELO!

Reason: Houdini 1.02 can not access to these EGTB databases, Shredder 12 and Rybka 4 (as opponents!) they can! A test without EGTB would therefore be better for such a list. If we deduct the EGTB's accumulated points of the opponents, Houdini 1.02 would be in my opinion, even a better as pure ENGINE than Rybka 4!

ED.

[Robert Houdart]

Hi Ed,

A test without EGTB would therefore be better for such a list.

The use of EGTB is part of the strength of an engine, so it's probably fair that EGTB are used for the test.
How did you estimate the +10 to +20 Elo strength improvement from EGTB?

Robert

[yanquis1972]

i agree that EGTBs should be used; as many as the tester has on hand. really, with a file client 3-4-5 man TBs are not that big a pain to download. i currently have them on a thumb drive. but i do see where the other POV comes from & i do think all engines should be able to play extremely well in the endgames without tablebases.

robert, is there any chance you will have bitbase or tablebase support in the near future? i agree that this would be another 10 elo, which alone would probably pull houdini even with the current rybka 4. (i know everyone talks about error margins, but i think houdini is 5-15 elo weaker than R4 -- i expect ingo's test will agree with this...though there is the issue of rybka's default time control settings).


update on the testing, it is now trailing R4 by 1 point after 798 games, & has drawn dead even with R4 head to head. pretty decent rivalry, & i really hope robert is serious about his work.

[Robert Houdart]

It may surprise you that I'm not really driven by a rivalry or by any particular eagerness to "pull even" with Rybka.

Surely at some point EGTB support will be added to Houdini, but not in the near future (i.e. not the coming 3 months).
FYI, the next endgame related activity will be to try to improve Houdini's knowledge of pawn endings, I hope that you find that "serious" enough .

Robert

[Harvey Williamson]

It may surprise you that I'm not really driven by a rivalry or by any particular eagerness to "pull even" with Rybka.

Surely at some point EGTB support will be added to Houdini, but not in the near future (i.e. not the coming 3 months).
FYI, the next endgame related activity will be to try to improve Houdini's knowledge of pawn endings, I hope that you find that "serious" enough .

Robert

That is of course if you can get permission to use EGTB?

[Eduard Nemeth]

Hi Ed,
How did you estimate the +10 to +20 Elo strength improvement from EGTB?
Robert

Hi Robert!

Since few years, has been shown statistically, that an engine with EGTB's approximately +10 to +20 points plays stronger.

ED.

By the way: In the latest interview (on ChessBase.de) V. Rajlich said, that for Rybka 4, using EGTB is very good.

[yanquis1972]

i would also wonder if it would be possible to incorporate robbobases. imo these give a similar boost in performance. since it's apparently hard to get permission for nalimov tablebases, gaviota tablebases may be an option as well...i am not familiar with these as they haven't really caught on yet, but it's something to look into.

although you may not personally be interested in an arms race, i find it fascinating & fun that the two top uci engines are so close in strength, & i wish you the best in developing your engine. as you well know the only other engine currently close in strength to rybka that is truly & surely being actively developed is stockfish (we will see shortly if the new ivanhoes are worth anything), and, well, as much as i respect its authors, i just don't like the engine...it's evals are simply too weird for me to really understand yet. houdini btw, almost goes too far in the opposite direction (overly pessimistic) but i find this easier to mentally compensate for & less 'wild' than stockfish's output. looking forward to mpv, hope that's just around the corner.

[yanquis1972]

just another note re the testing, i see ingo only uses 3-4 pieces, not 5 -- i believe these only take up a few megabytes and can be very easily had by anyone with an in internet connection.

Ponder : ON
CPU : 3 GHz C2Q
Operating system : XP-64
Cores : Engines with 2T used 2 cores, all other 1 core
Hash : 256 MB
Table-/Shredder-/Bitbases : 4 pieces
Cache for Tbs : 64 MB
Time control : 5 min/game + 3 sec/move (mean game length ~ 16 Minutes)
Openings : 50 different opening positions (no books)
Positions : Changing colors (100 games engine vs engine)
Learning : OFF

Enignes are tested in 64 bit if available.

[LetoAtreides82]

Does he use those settings for all his testings? I've been following his Houdini 1.02 tournament and it seems everytime I refresh the page several games have finished, the page updates every three minutes and his settings indicate 5 minute plus 3 sec increments so I don't see how it's possible for so many games to finish in just three minutes, unless he has multiple machines running?