SedatChess

[Sedat Canbaz]

UPDATE

1st of all, here are the conditions:
Ryzen7 4800H, CuteChess, 1 Core, Ponder OFF, 1m+1s, 128 MB Hash, 4-MEN
As book, Unique v50 opening suite is used for all and played as twice!

Code: Select all

1st Test Results: +17 Elo (in favor for SF-PB)

1  SF-PB 060623  +12/-7/=81 52.50%   52.5/100
2  Eman 8.91     +7/-12/=81 47.50%   47.5/100

Note: Eman via Rulleeeee.exp / Experience MultiPV ON
----------------------------------------------------
2nd Test Results: +17 Elo (in favour for SF-PB)
                   
1  SF-PB 060623  +16/-11/=73 52.50%   52.5/100
2  Eman 8.91     +11/-16/=73 47.50%   47.5/100

Note: No Pre Learned Exp / Experience MultiPV ON
----------------------------------------------------

3rd Test Results: +14 Elo (in favor for Eman)
                                   
1   Eman 8.91     +14/-10/=76 52.00%   52.0/100
2   SF-PB 060623  +10/-14/=76 48.00%   48.0/100

Note: Pre Learned Exp of 2nd test/Exp. MultiPV OFF 
----------------------------------------------------
4rd Test Results: +7 Elo (in favor for Eman)
                                                            
1 Eman 8.91      +14/-12/=74 51.00%   51.0/100
2 SF-PB 060623   +12/-14/=74 49.00%   49.0/100

Note: Pre Learned Exp of 2nd+3rd test/Exp. MultiPV ON
----------------------------------------------------

As final words,
Sure for a better conclusion we need more games...
But anyhow it's clear that Eman's learning is quite
Effective even via concurrent (parallel matches..)
So this is wrong who claims that Eman is buggy or not
Able to learn even via parallel/concurrent matches!

On other hand,
Experience MultiPV ON is performed as best (than all)
As 2nd best performance, we noticed comes in 4rd test!
So..once more we noticed that Eman's learning is valid
Even under SCCT conditions (even via parallel matches)

What I can say more about Eman 8.91 engine:
Via pre learned exp file ( sure learned on same openings):
In next re-run: Eman is performing much better for sure!

That means also that...
There is a possibility to see Exp Champs or higher ranks at:
https://sites.google.com/site/computers ... t-ben-cs-1

Hope helps...

Note: very soon all games will be uploaded...

Greetings

[Sedat Canbaz]

As I promised,

GAMES:
https://mega.nz/file/20h0xaZC#osOID5DqV ... 1TsPkdsApU

[Sedat Canbaz]

EDIT:
Experience MultiPV OFF is performed as best (than all)

[IbaiBuR]

UPDATE

1st of all, here are the conditions:
Ryzen7 4800H, CuteChess, 1 Core, Ponder OFF, 1m+1s, 128 MB Hash, 4-MEN
As book, Unique v50 opening suite is used for all and played as twice!

Code: Select all

1st Test Results: +17 Elo (in favor for SF-PB)

1  SF-PB 060623  +12/-7/=81 52.50%   52.5/100
2  Eman 8.91     +7/-12/=81 47.50%   47.5/100

Note: Eman via Rulleeeee.exp / Experience MultiPV ON
----------------------------------------------------
2nd Test Results: +17 Elo (in favour for SF-PB)
                   
1  SF-PB 060623  +16/-11/=73 52.50%   52.5/100
2  Eman 8.91     +11/-16/=73 47.50%   47.5/100

Note: No Pre Learned Exp / Experience MultiPV ON
----------------------------------------------------

3rd Test Results: +14 Elo (in favor for Eman)
                                   
1   Eman 8.91     +14/-10/=76 52.00%   52.0/100
2   SF-PB 060623  +10/-14/=76 48.00%   48.0/100

Note: Pre Learned Exp of 2nd test/Exp. MultiPV OFF 
----------------------------------------------------
4rd Test Results: +7 Elo (in favor for Eman)
                                                            
1 Eman 8.91      +14/-12/=74 51.00%   51.0/100
2 SF-PB 060623   +12/-14/=74 49.00%   49.0/100

Note: Pre Learned Exp of 2nd+3rd test/Exp. MultiPV ON
----------------------------------------------------

As final words,
Sure for a better conclusion we need more games...
But anyhow it's clear that Eman's learning is quite
Effective even via concurrent (parallel matches..)
So this is wrong who claims that Eman is buggy or not
Able to learn even via parallel/concurrent matches!

On other hand,
Experience MultiPV ON is performed as best (than all)
As 2nd best performance, we noticed comes in 4rd test!
So..once more we noticed that Eman's learning is valid
Even under SCCT conditions (even via parallel matches)

What I can say more about Eman 8.91 engine:
Via pre learned exp file ( sure learned on same openings):
In next re-run: Eman is performing much better for sure!

That means also that...
There is a possibility to see Exp Champs or higher ranks at:
https://sites.google.com/site/computers ... t-ben-cs-1

Hope helps...

Note: very soon all games will be uploaded...

Greetings

Hello, just some friendly comments about this, with the aim to help, please dont misunderstand the intention:

If you think that Eman "learned" something after a few 100 game matches, with parallel concurrent matches, and different openings you are quite wrong, believe me...

Eman usually needs 500 games or more per single opening line to learn something and this is something that even the Eman's author said, it is not something I am inventing.

Eman may have saved some of the moves played in the experience file, but probably not all of them because of the parallel matches!

Anyways, thanks a lot for the test and the games. I hope you see my comment in a friendly, constructive way, because this is they way it is intended

Regards,

Ibai

[Sedat Canbaz]

Hello, just some friendly comments about this, with the aim to help, please dont misunderstand the intention:

If you think that Eman "learned" something after a few 100 game matches, with parallel concurrent matches, and different openings you are quite wrong, believe me...

Eman usually needs 500 games or more per single opening line to learn something and this is something that even the Eman's author said, it is not something I am inventing.

Eman may have saved some of the moves played in the experience file, but probably not all of them because of the parallel matches!

Anyways, thanks a lot for the test and the games. I hope you see my comment in a friendly, constructive way, because this is they way it is intended

Regards,

Ibai

Well,
I never said that 400 games is enough...

But what we noticed,
1 concurrent ( parallel) does not make any sense, even weaker!
Sure I mean if Eman is not trained vs/in same opening positions...

What we noticed also that,
SCCT is valid...

The proof is proof...

That's why since 1997,
I prefer to relay on results than comments...

On other hand, that's true that,
Many more trainings are needed.. sure with same openings, then
Very likely not only Eman or I mean any other Top engine, which is
Capable to learn properly.. will be performed better...!!

In other words, this is nothing new...
This happens in every area...with learning...
Including books too... if I start to create a very strong book,
I will run and test many games I mean in various openings..
So my new created book will be performed much better... !)
It all depends on the techniques...on the conditions etc.

But this is also true that,
A very strong exp may be Champ / overcome some strong books as well...
But before, a very serious testings required... sure that's all from me )
In other words, the turn is now in the hands of Exp authors.. !)

Greetings

[Sedat Canbaz]

And for anyone missed,

More testings, but via 64 parallel matches

And for those who say 400 games is a small number..

I suggest to check to this tour..where 2200 games (per player)
https://sites.google.com/site/computers ... t-nn-cs-32

And the leading players are engines with learning capacity..!!

If STILL not enough proof...just let me know please
I may share more...

Hope helps...


Greetings

[IbaiBuR]

And for those who say 400 games is a small number..

I suggest to check to this tour..where 2200 games (per player)
https://sites.google.com/site/computers ... t-nn-cs-32

Just checked, thats a little bit better, tho, still quite far from the 500 games per opening recommendation, but yeah, definetely better, thanks for sharing

Btw, I was thinking a bit... Do you have / recommend any small testsuite for testing engines (not more than 50 opening lines)?

I may train a modded version of Stockfish + learning on it and I'll report the results of training and testing against other engines (big number, as you said)

This will take a big amount of time tho...

Greetings and thanks in advance

[Sedat Canbaz]

And for those who say 400 games is a small number..

I suggest to check to this tour..where 2200 games (per player)
https://sites.google.com/site/computers ... t-nn-cs-32

Just checked, thats a little bit better, tho, still quite far from the 500 games per opening recommendation, but yeah, definetely better, thanks for sharing

Btw, I was thinking a bit... Do you have / recommend any small testsuite for testing engines (not more than 50 opening lines)?

I may train a modded version of Stockfish + learning on it and I'll report the results of training and testing against other engines (big number, as you said)

This will take a big amount of time tho...

Greetings and thanks in advance

You are welcome )

Now I feel much better..I mean all in friendly way )

Agreed...500+ games (per opening) would be much better!
But in my guess, even several games (per same opening) can be
Ranked at higher places..than the ones which never tested..

Why I say like this (if check latest current test-400 games),
We see that even via small number of learned games
Eman's performance is increased...and I strongly believe that
Pre-learned Eman. exp file is helped a lot...

Btw, BIG thanks to you, for your idea: Experience MultiPV OFF
And it seems that Eman 8.91 is stronger under these settings...

On other hand,
Nowadays I have no much free time for chess but later,
Maybe in September, I may test RapTora 2.2 as without learning:
https://sites.google.com/site/computers ... t-nn-cs-33

Then we will have chances to compare, e.g
RapTora 2.2 NON-Learning will be Champ or not..
And frankly in my estimation: no much chances!

About your book question,
You can try to test the full version of Unique - 300 openings:
https://sites.google.com/site/computers ... test-suite

Greetings )

[Sedat Canbaz]

And let me say a few more words about this tour:
https://sites.google.com/site/computers ... t-nn-cs-33

As far as I remember,
Cutechess's GUI concurrent game option was: 64

Yes.. in this tour: Eman is not crashed...but it seems
Eman's performance is suffered but if Concurrent was via 32
Then very likely Eman would be ranked at higher placers..

What I can tell you more,
In direct matches (if 2 players via concurrent 64): Eman crashes+GUI terminates..
But this is also true that if via many players and let's say in round robin.. even via
Concurrent 64, no any crash is recorded..but this is normal, because other engines
Are as active in same time (in game-play...)

For anyone missed, I'd like to point out that e.g in direct matches, even
If only 2 players via concurrent 64: other Top engines are not crashing..
So it is clear that Eman is not stable via concurrent 64

As conclusion, after resuming BOOK/EXP NNUE tour:
In my next gauntlet testings, I will set concurrent 32
Sure I mean in case of Eman exp entries...

Greetings

[IbaiBuR]

You are welcome )

Now I feel much better..I mean all in friendly way )

Agreed...500+ games (per opening) would be much better!
But in my guess, even several games (per same opening) can be
Ranked at higher places..than the ones which never tested..

Why I say like this (if check latest current test-400 games),
We see that even via small number of learned games
Eman's performance is increased...and I strongly believe that
Pre-learned Eman. exp file is helped a lot...

Btw, BIG thanks to you, for your idea: Experience MultiPV OFF
And it seems that Eman 8.91 is stronger under these settings...

On other hand,
Nowadays I have no much free time for chess but later,
Maybe in September, I may test RapTora 2.2 as without learning:
https://sites.google.com/site/computers ... t-nn-cs-33

Then we will have chances to compare, e.g
RapTora 2.2 NON-Learning will be Champ or not..
And frankly in my estimation: no much chances!

About your book question,
You can try to test the full version of Unique - 300 openings:
https://sites.google.com/site/computers ... test-suite

Greetings )

In friendly way much better for sure... I dont like arguments...

Well yeah several games (not much but some as you say) can help but normally are not enough, but for sure I believe they can help...

Regarding the Experience MultiPV OFF, you are welcome, this option is better turned off because of a simple but logical reason, if it is set to true, the engine saves a lot of useless, shallow depth entries, that are not even played in the game. The consequence of this is, useless experience data that makes the experience file bigger and slows down the engine in future games / matches. Thanks for testing it!

I hope that after this people stop doing tests with Experience MultiPV ON, because people mostly believe you, and you showed that ON is worse...

Thanks for the test-suite, now its time to train Stockfish + learning and Eman...

Regards,

Ibai