OpenChess

WOW I just cannot understand it,

the vesrion that I compiled from the source wins the tournament (1 0), no books, no exp files.... 4 rounds
it lost only one game with ShashChess 33.2
All twelve engines are downloaded AVX2 bin files, only the Tactical is compiled. Strange is that LittleBeats, Brainlearn, Raptora, Shash, Raid and BlueMarlin have BENCH tests almost DOUBLE than Tactical..... but they still lose on Tactical with default values.

    Engine                     Score   TA   St   Li   SF   EM   Bl   Ra   Sh   Ra   Li   SF   Bl    S-B
01: TACTICAL 220823_a          28,5/44 ···· ==== =1=1 1=== ==== ==1= ==== 01=1 =1=1 1=1= =1=1 1=1=  604,00
02: Stockfish-dev              24,5/44 ==== ···· ==0= ==== ==== ==1= 0=== 01=1 1=== 1=1= =1=1 01==  525,50
03: LittleBeast2               24,0/44 =0=0 ==1= ···· 10== ==== =1=1 =1=1 1=1= 1==0 0100 11=0 1=0=  518,00
04: SF_23061113_avx2           24,0/44 0=== ==== 01== ···· ==== ==10 1=== 1=== 10== ===1 ==1= ===1  515,00
05: EMAN94                     23,0/44 ==== ==== ==== ==== ···· 1=== ==== ==== ==== ===1 =01= ====  502,75
06: Brainlearn 25.2            22,0/44 ==0= ==0= =0=0 ==01 0=== ···· 1=0= 0101 ==1= ==1= =1== 1=1=  468,75
07: RapTora 2.2-avx2           21,5/44 ==== 1=== =0=0 0=== ==== 0=1= ···· ==1= ==== ==== ==== ====  471,75
08: ShashChess33.2-x86-64-avx2 21,0/44 10=0 10=0 0=0= 0=== ==== 1010 ==0= ···· =1== ==== 1==1 ==1=  450,00
09: Raid v2.73_avx2            19,5/44 =0=0 0=== 0==1 01== ==== ==0= ==== =0== ···· 0101 1010 0101  426,75
10: Little Beast 3 sl-avx2     19,5/44 0=0= 0=0= 1011 ===0 ===0 ==0= ==== ==== 1010 ···· 0101 1010  425,50
11: SF-PB-160823_AVX2          18,5/44 =0=0 =0=0 00=1 ==0= =10= =0== ==== 0==0 0101 1010 ···· 1101  400,00
12: Bluemarlin 15.7            18,0/44 0=0= 10== 0=1= ===0 ==== 0=0= ==== ==0= 1010 0101 0010 ····  400,50

Hi!

It's a great result, in any case if you help me with the pgn, what opening book or pgn you used and interface will be very helpful, I will send you the tool to measure the bench.

And thank you for the valuable information about the compilation, the next update of TACTICAL and Cool Iris, will be done with the default commands for you as users.

Best regards.

PD. Give me about three hours to send you the tool, I'm not at home.

wolfie74 wrote: Thu Aug 24, 2023 2:24 pm WOW I just cannot understand it,

the vesrion that I compiled from the source wins the tournament (1 0), no books, no exp files.... 4 rounds
it lost only one game with ShashChess 33.2
All twelve engines are downloaded AVX2 bin files, only the Tactical is compiled. Strange is that LittleBeats, Brainlearn, Raptora, Shash, Raid and BlueMarlin have BENCH tests almost DOUBLE than Tactical..... but they still lose on Tactical with default values.

    Engine                     Score   TA   St   Li   SF   EM   Bl   Ra   Sh   Ra   Li   SF   Bl    S-B
01: TACTICAL 220823_a          28,5/44 ···· ==== =1=1 1=== ==== ==1= ==== 01=1 =1=1 1=1= =1=1 1=1=  604,00
02: Stockfish-dev              24,5/44 ==== ···· ==0= ==== ==== ==1= 0=== 01=1 1=== 1=1= =1=1 01==  525,50
03: LittleBeast2               24,0/44 =0=0 ==1= ···· 10== ==== =1=1 =1=1 1=1= 1==0 0100 11=0 1=0=  518,00
04: SF_23061113_avx2           24,0/44 0=== ==== 01== ···· ==== ==10 1=== 1=== 10== ===1 ==1= ===1  515,00
05: EMAN94                     23,0/44 ==== ==== ==== ==== ···· 1=== ==== ==== ==== ===1 =01= ====  502,75
06: Brainlearn 25.2            22,0/44 ==0= ==0= =0=0 ==01 0=== ···· 1=0= 0101 ==1= ==1= =1== 1=1=  468,75
07: RapTora 2.2-avx2           21,5/44 ==== 1=== =0=0 0=== ==== 0=1= ···· ==1= ==== ==== ==== ====  471,75
08: ShashChess33.2-x86-64-avx2 21,0/44 10=0 10=0 0=0= 0=== ==== 1010 ==0= ···· =1== ==== 1==1 ==1=  450,00
09: Raid v2.73_avx2            19,5/44 =0=0 0=== 0==1 01== ==== ==0= ==== =0== ···· 0101 1010 0101  426,75
10: Little Beast 3 sl-avx2     19,5/44 0=0= 0=0= 1011 ===0 ===0 ==0= ==== ==== 1010 ···· 0101 1010  425,50
11: SF-PB-160823_AVX2          18,5/44 =0=0 =0=0 00=1 ==0= =10= =0== ==== 0==0 0101 1010 ···· 1101  400,00
12: Bluemarlin 15.7            18,0/44 0=0= 10== 0=1= ===0 ==== 0=0= ==== ==0= 1010 0101 0010 ····  400,50

Great work, thanks.

Here you have the tool: https://pixeldrain.com/u/dWyKXMXk

Regards.

Homayoun wrote: Thu Aug 24, 2023 6:27 pm

wolfie74 wrote: Thu Aug 24, 2023 2:24 pm WOW I just cannot understand it,

the vesrion that I compiled from the source wins the tournament (1 0), no books, no exp files.... 4 rounds
it lost only one game with ShashChess 33.2
All twelve engines are downloaded AVX2 bin files, only the Tactical is compiled. Strange is that LittleBeats, Brainlearn, Raptora, Shash, Raid and BlueMarlin have BENCH tests almost DOUBLE than Tactical..... but they still lose on Tactical with default values.

    Engine                     Score   TA   St   Li   SF   EM   Bl   Ra   Sh   Ra   Li   SF   Bl    S-B
01: TACTICAL 220823_a          28,5/44 ···· ==== =1=1 1=== ==== ==1= ==== 01=1 =1=1 1=1= =1=1 1=1=  604,00
02: Stockfish-dev              24,5/44 ==== ···· ==0= ==== ==== ==1= 0=== 01=1 1=== 1=1= =1=1 01==  525,50
03: LittleBeast2               24,0/44 =0=0 ==1= ···· 10== ==== =1=1 =1=1 1=1= 1==0 0100 11=0 1=0=  518,00
04: SF_23061113_avx2           24,0/44 0=== ==== 01== ···· ==== ==10 1=== 1=== 10== ===1 ==1= ===1  515,00
05: EMAN94                     23,0/44 ==== ==== ==== ==== ···· 1=== ==== ==== ==== ===1 =01= ====  502,75
06: Brainlearn 25.2            22,0/44 ==0= ==0= =0=0 ==01 0=== ···· 1=0= 0101 ==1= ==1= =1== 1=1=  468,75
07: RapTora 2.2-avx2           21,5/44 ==== 1=== =0=0 0=== ==== 0=1= ···· ==1= ==== ==== ==== ====  471,75
08: ShashChess33.2-x86-64-avx2 21,0/44 10=0 10=0 0=0= 0=== ==== 1010 ==0= ···· =1== ==== 1==1 ==1=  450,00
09: Raid v2.73_avx2            19,5/44 =0=0 0=== 0==1 01== ==== ==0= ==== =0== ···· 0101 1010 0101  426,75
10: Little Beast 3 sl-avx2     19,5/44 0=0= 0=0= 1011 ===0 ===0 ==0= ==== ==== 1010 ···· 0101 1010  425,50
11: SF-PB-160823_AVX2          18,5/44 =0=0 =0=0 00=1 ==0= =10= =0== ==== 0==0 0101 1010 ···· 1101  400,00
12: Bluemarlin 15.7            18,0/44 0=0= 10== 0=1= ===0 ==== 0=0= ==== ==0= 1010 0101 0010 ····  400,50

Great work, thanks.

+1

The more i go deep in the chess engines the more puzzled I am. I used the build tester tool to measure the strength of the engines, and
Tactical although won the previous tournament with flying colours, is (according to the test tool) weakest engine.
11 tested engines, sorted by strenght:

LittleBeast 2	- 	1 402 697
RapTora 2.2 	-	1 363 903
BrainLearn 25.1	-	1 251 501
ShashChess 33.2	-	1 242 585
Raid v 2.5	-	1 216 725

and the rest 6 engines showed approx. 50% of this strength

SF-PLB_240823	-	733 715
SF-PB-220811	-	733 090
EMAN94		-	729 494
Raid v2.73	-	728 320
StockFish-Dev	-	722 516
TAC220823	-	708 010

I am enclosing the PGN from the 220 games https://www.transfernow.net/dl/20230826maOW4Pw3

and screenshot of the tool test result https://www.transfernow.net/dl/20230826GVFjBCA9

(both links will be valid only 7 days - I did not know where else to upload the files)

AMD Ryzen 5 2600 Six-Core Processor            
SafeMode: No 
Running In VM: No 
HyperThreading Enabled: Yes
CPU Warmup: Yes
Command Line: bench
Tests per Build: 21
ANOVA: Passed

                Engine# (NPS)                     Speedup     Sp     Conf. 95%    S.S.
2  (1 216 725,7   ) ---> 1  (729 494,3     ) ---> 66,790%  73 526,5     Yes       No 
1  (729 494,3     ) ---> 3  (708 010,3     ) --->  3,034%  48 619,1     No        No 
1  (729 494,3     ) ---> 4  (728 320,0     ) --->  0,161%  50 631,4     No        No 
1  (729 494,3     ) ---> 5  (722 516,8     ) --->  0,966%  58 044,1     No        No 
6  (1 363 903,4   ) ---> 1  (729 494,3     ) ---> 86,966%  68 721,5     Yes       No 
7  (1 242 585,8   ) ---> 1  (729 494,3     ) ---> 70,335%  75 392,2     Yes       No 
8  (1 251 501,9   ) ---> 1  (729 494,3     ) ---> 71,557%  79 474,2     Yes       No 
9  (733 715,2     ) ---> 1  (729 494,3     ) --->  0,579%  54 177,9     No        No 
10 (1 402 697,0   ) ---> 1  (729 494,3     ) ---> 92,283%  60 168,0     Yes       No 
11 (733 090,9     ) ---> 1  (729 494,3     ) --->  0,493%  56 625,8     No        No 

I would have one question though,

would it be possible to add live book to Tactical ?

something like BrainLearn so that in UCI would be:

Live Book=true
Live Book URL=http://www.chessdb.cn/cdb.php

Many thanks and regards

Hi!

You confuse speed (NPS) with strength.

On the other hand, will it really be necessary to add a "live book"? A good training experience file is enough.

You can train two engines (One for your OTB games and the second experience file for vs engine games). Since I don't have proof on hand of how many users use the "live book" tool, I'm not going to add it.

Regards.

wolfie74 wrote: Thu Aug 24, 2023 2:24 pm Strange is that LittleBeats, Brainlearn, Raptora, Shash, Raid and BlueMarlin have BENCH tests almost DOUBLE than Tactical..... but they still lose on Tactical with default values.

Bench does not really matter. Sometimes a tiny change can change the bench a lot. Sometimes it is almost the same even if there are big changes. Latest Stockfish and its derivatives has usually lower bench because of reduced classical evaluation.

I was under impression that NPS and strenght are connected. i.e. higher NPS = higher strenght.
NPS can be measured by BENCH, but how can you measure the strenght? only by tournaments?