Test : who is right

[Hamfer]

Hello,
First, sorry for bad englsih.

I'm faced to strange behaviour when testing my experimental engine.
I lunch the same test (same conditions) under cuteches-cli and Arena and the result was totally different.
With cutechess-cli, it loses => 70%
With arena, it wins => 90%.

This the cutechess-cli command :
%CHEMIN%\tools\cutechess-cli-0-7-2\cutechess-cli.exe ^
-engine name=DEV cmd=DEV.exe dir="D:\arena_3.0\Engines\DEV" ^
-engine name=ENG cmd=ENG.exe dir="D:\arena_3.0\Engines\ENG" ^
-each proto=uci tc=10+0.5 ^
-openings file=%CHEMIN%\pgnsets\Test.epd format=epd ^
-pgnout %CHEMIN%\PGN_RES\%PGN_DATABASE% ^
-games 2 ^
-rounds 7536 ^
-repeat ^
-recover ^
-concurrency 3

Help

[hyatt]

Hello,
First, sorry for bad englsih.

I'm faced to strange behaviour when testing my experimental engine.
I lunch the same test (same conditions) under cuteches-cli and Arena and the result was totally different.
With cutechess-cli, it loses => 70%
With arena, it wins => 90%.

This the cutechess-cli command :
%CHEMIN%\tools\cutechess-cli-0-7-2\cutechess-cli.exe ^
-engine name=DEV cmd=DEV.exe dir="D:\arena_3.0\Engines\DEV" ^
-engine name=ENG cmd=ENG.exe dir="D:\arena_3.0\Engines\ENG" ^
-each proto=uci tc=10+0.5 ^
-openings file=%CHEMIN%\pgnsets\Test.epd format=epd ^
-pgnout %CHEMIN%\PGN_RES\%PGN_DATABASE% ^
-games 2 ^
-rounds 7536 ^
-repeat ^
-recover ^
-concurrency 3

Help

Have you looked at the games to see what is different? I'd suspect a lot of time losses. Some programs seem to not behave well. For instance, if you use Gaviota, it dumps a BUNCH of unnecessary stuff to the GUI. Using xboard, this can cause the opponent to lose on time when Gaviota makes a move, then dumps many hundreds of lines of garbage output to xboard. While xboard is processing all that input, the opponent can lose on time when time is short.

You might also have a GUI with bugs that can cause similar bias. You have to look at the games to see why it is losing...

[Hamfer]

Thank you for the response.

Yes, there is some games that terminate with "White's connection stalls".
But there is another problem, tc is 10+0.5 and the game starts with 1. Bf3 {+0.20/11 0.69s} => how the time jumped from 10s to 69s.

[Event "?"]
[Site "?"]
[Date "2015.11.24"]
[Round "1"]
[White "DEV"]
[Black "ENG"]
[Result "0-1"]
[FEN "r4rk1/1q2bppp/p1bppn2/1p6/3BPP2/P1N3Q1/1PP1B1PP/4RR1K w - -"]
[PlyCount "182"]
[SetUp "1"]
[TimeControl "10+0.5"]

1. Bf3 {+0.20/11 0.69s} Rad8 {-0.09/11 1.1s} 2. e5 {+0.16/11 0.68s}
dxe5 {-0.09/9 0.37s} 3. fxe5 {+0.15/13 0.66s} Nd5 {0.00/11 0.67s}
4. Ne4 {+0.37/11 0.67s} Qc7 {-0.13/11 0.46s} 5. c3 {+0.26/10 0.66s}
... {-M2/20 0.10s, White's connection stalls} 0-1