Page 2 of 5
Re: IPON Rating's List Bash Script
Posted: Wed Sep 07, 2011 7:55 pm
by Uly
kingliveson wrote:I saw the thread and have to agree with Uly...publishing the games wont ruin his list.
Whoa! Amazing how we had the same idea in very close times, with the difference that I did nothing and you fully implemented it

(it would have taken me more than two weeks to do so and my results wouldn't have been as good as yours so I appreciate the effort!)
Now I'm going to re-post this at Rybka Forum

Re: IPON Rating's List Bash Script
Posted: Wed Sep 07, 2011 8:50 pm
by Jeremy Bernstein
Re: IPON Rating's List Bash Script
Posted: Wed Sep 07, 2011 9:27 pm
by Uly
Oh well, Prima went on a posting spree burying the crosspost, I deleted it and may attempt to repost it later.
Re: IPON Rating's List Bash Script
Posted: Wed Sep 07, 2011 10:05 pm
by Prima
Jeremy Bernstein wrote:
LOL, Uly.
jb
Now my response to other Threads, in Rybka forum, is blamed for truncating a Topic, as if I had the power to. Or as if THAT was my intention. What next? Blame me for Vas' breach of GPL and breaking the ICGA rule #2 repetitively?
It never ends

.....
Re: IPON Rating's List Bash Script
Posted: Thu Sep 08, 2011 3:10 pm
by kingliveson
uh...what did I miss?
By the way, the weight used for Stockfish is incorrect. It should have been 36 instead of 46, and 40 seems more appropriate.
Re: IPON Rating's List Bash Script
Posted: Fri Sep 09, 2011 1:09 am
by Uly
Prima wrote:Now my response to other Threads, in Rybka forum, is blamed for truncating a Topic
No. It was a coincidence, but my cross-post was badly timed (as can be seen when a newly created thread rushes to be buried in the second page, regardless of who is making a posting spree).
I just wanted to explain why I deleted the thread.
Re: IPON Rating's List Bash Script
Posted: Fri Sep 09, 2011 7:05 am
by kingliveson
Important correction to getting draws -- new source attached.
Removed:
Code: Select all
# difference weight table scale
{
black_difference_table[1]=45 # 0-8
black_difference_table[2]=35 # 9-16
black_difference_table[3]=25 # 17-24
black_difference_table[4]=15 # 25-32
black_difference_table[5]=4 # > 32
white_difference_table[1]=10 # > 32
white_difference_table[2]=8 # 25-32
white_difference_table[3]=6 # 17-24
white_difference_table[4]=4 # 9-16
white_difference_table[5]=2 # 0-8
}
# difference weight table scale
{
black_difference_table[1]=45 # 0-8
black_difference_table[2]=35 # 9-16
black_difference_table[3]=25 # 17-24
black_difference_table[4]=15 # 25-32
black_difference_table[5]=4 # > 32
white_difference_table[1]=10 # > 32
white_difference_table[2]=8 # 25-32
white_difference_table[3]=6 # 17-24
white_difference_table[4]=4 # 9-16
white_difference_table[5]=2 # 0-8
}
# Weight look-up table scale
Weight_Lookup_Scale()
{
if [ $lookup_table_bit -eq 0 ]; then
if (( $engine_weight_difference <= 8 )); then
black_table_weight=${black_difference_table[1]}
elif (( $engine_weight_difference >= 9 && $engine_weight_difference <= 16 )); then
black_table_weight=${black_difference_table[2]}
elif (( $engine_weight_difference >= 17 && $engine_weight_difference <= 24 )); then
black_table_weight=${black_difference_table[3]}
elif (( $engine_weight_difference >= 25 && $engine_weight_difference <= 32 )); then
black_table_weight=${black_difference_table[4]}
elif (( $engine_weight_difference > 32 )); then
black_table_weight=${black_difference_table[5]}
fi
white_table_weight=0
else
if (( $engine_weight_difference <= 8 )); then
white_table_weight=${white_difference_table[5]}
elif (( $engine_weight_difference >= 9 && $engine_weight_difference <= 16 )); then
white_table_weight=${white_difference_table[4]}
elif (( $engine_weight_difference >= 17 && $engine_weight_difference <= 24 )); then
white_table_weight=${white_difference_table[3]}
elif (( $engine_weight_difference >= 25 && $engine_weight_difference <= 32 )); then
white_table_weight=${white_difference_table[2]}
elif (( $engine_weight_difference > 32 )); then
white_table_weight=${white_difference_table[1]}
fi
black_table_weight=0
fi
}
Replaced with:
Code: Select all
# Draw formula -- decrease to increase win
Draw_Weight_Scale()
{
if [ $scale_side_switch -eq 0 ]; then
black_scale_weight=`expr $weaker_engine_weight*.425 | bc`
else
white_scale_weight=`expr $weaker_engine_weight*.185 | bc`
fi
}
The new approach is more efficient; rather than hard-coding constants on how often draws occur, the formula uses the engines' defined weights. The current ratio is tuned to IPON's list previously posted.
Re: IPON Rating's List Bash Script
Posted: Fri Sep 09, 2011 4:57 pm
by kingliveson
So what was added this time? User interaction, Gauntlet and Round-Robin tournaments. It is pretty easy to use. Execute it and just follow the prompt -- source and database attached.
-Run a Gauntlet tournament after selecting an engine, then number of rounds. Round Robin tournament will allow all engines to play against each another. A game database will be generated
-Add/remove engine requires 3 changes:
MAXENGINECOUNT=23
chess_engine[23]="Engine Name"
engine_weight[23]=100
-To increase/decrease wins, edit values in Draw_Weight_Scale()
Code: Select all
Chess Engine Rating List Simulator
Enter 1 for Gauntlet or 2 for Round-Robin: 1
List Engines? Enter y or n: n
Enter engine number: 1
Enter number of rounds: 100
Generating game database.......................Complete!
Code: Select all
Chess Engine Rating List Simulator
Enter 1 for Gauntlet or 2 for Round-Robin: 2
List Engines? Enter y or n: n
Enter number of rounds: 300
Generating game database......................Complete!
Simulated Tournament:
Program Elo + - Games Score Av.Op. Draws
1 Houdini 2.0 STD : 2649 9 9 6300 81.7 % 2390 19.8 %
2 Critter 1.2 : 2604 8 8 6300 77.2 % 2392 22.8 %
3 Komodo64 3 SSE42 : 2595 8 8 6300 76.2 % 2392 23.6 %
4 Deep Rybka 4.1 SSE42 : 2589 8 8 6300 75.5 % 2393 23.8 %
5 Stockfish 2.1.1 JA : 2584 8 8 6300 75.0 % 2393 24.8 %
6 Naum 4.2 : 2484 7 7 6300 62.2 % 2398 29.4 %
7 Deep Shredder 12 : 2484 7 7 6300 62.1 % 2398 29.4 %
8 Spike 1.4 32b : 2448 7 7 6300 57.0 % 2399 31.3 %
9 Deep Sjeng c't 2010 32b : 2443 7 7 6300 56.1 % 2400 31.3 %
10 Deep Junior 12.5 : 2426 7 7 6300 53.6 % 2401 31.4 %
11 Gull 1.2 : 2394 7 7 6300 48.8 % 2402 31.2 %
12 HIARCS 13.2 MP 32b : 2387 7 7 6300 47.8 % 2402 32.1 %
13 Hannibal 1.1 : 2374 7 7 6300 45.9 % 2403 32.1 %
14 Protector 1.4.0 x64 : 2366 7 7 6300 44.6 % 2403 31.0 %
15 spark-1.0 SSE42 : 2362 7 7 6300 44.1 % 2404 30.3 %
16 Deep Onno 1-2-70 : 2328 7 7 6300 39.1 % 2405 30.2 %
17 Zappa Mexico II : 2316 7 7 6300 37.3 % 2406 28.6 %
18 Loop 13.6/2007 : 2303 8 8 6300 35.5 % 2406 27.1 %
19 Strelka 2.0 B : 2236 8 8 6300 27.0 % 2410 22.6 %
20 Crafty 23.3 JA : 2177 9 9 6300 20.5 % 2412 19.0 %
21 Umko 1.2 SSE42 : 2177 9 9 6300 20.5 % 2412 19.7 %
22 Jonny 4.00 32b : 2075 11 11 6300 12.2 % 2417 12.0 %
At last, your own chess engine rating's list! Just pick weights from lists such as CEGT, CCRL, SWCR, SSDF. When people ask for games, you can always provide database with results useful for generating a stats table such as above.
Re: IPON Rating's List Bash Script
Posted: Sun Sep 11, 2011 12:37 am
by kingliveson
A new feature has been added; it now supports choosing which engines participate in a tournament (Gauntlet or Round-Robin) -- Just like regular GUIs. Source and game database are attached.
Usage/Description:
Code: Select all
This program (bash script) is used for simulating and predicting outcome for chess engine matches and tournaments.
It generates a game (CRLS.pgn) database at the conclusion of each run.
The results should be compared to rating lists such CEGT, CCRL, SWCR, SSDF, etc.
How to run:
sh CRLS or ./CRLS
# Windows users need to research openSUSE (Linux) LiveCD, or cygwin and notepad++
# There is also http://win-bash.sourceforge.net/
Prompt/Usage:
"Enter 1 for Gauntlet or 2 for Round-Robin: "
# Gauntlet allows an engine to compete against a group of engines
# Round-Robin allows all engines in a tournament to play each other
"List Engines? Enter y or n: "
# List all available chess programs
"Enter engine number (1-25), l (list engines), f (finish), or q (quit): "
# Choose up to 16 programs. Enter each, one at a time
# Enter l to see available programs
# Enter f after selecting the program(s)
# Enter p to see already selected programs
# Enter q to quit the program
"Enter number of round(s) (1-999999): "
# Enter the number of round(s)/game(s) to play
Gauntlet:
"Enter engine number (1-25): "
# Enter engine number from available engines' list
"Play against all engines? Enter y or n: "
# If yes, selected engine plays all available programs
"Enter engine number (1-25), l (list engines), f (finish), or q (quit): "
# Choose program(s) to play against selected engine.
Round-Robin
"List Engines? Enter y or n: "
# List all available chess programs
"Play against all engines? Enter y or n: "
# If yes, all available programs play round-robin style tournament
"Enter engine number (1-25), l (list engines), f (finish), or q (quit): "
# Select the programs to compete in the tournament
Source Editing
Change Engine Weight
# For example, Critter 1.2:
engine_weight[21]=445 # 40
Add/Remove engine requires 3 changes:
# For example: adding a new program
MAXENGINECOUNT=26
chess_engine[26]="Engine Name"
engine_weight[26]=100
To increase/decrease wins, edit values in Draw_Weight_Scale()
Here are some stats from a simulated tournament of all "installed" engines playing each other:
Code: Select all
Games : 150000 (finished)
White Wins : 58442 (39.0 %)
Black Wins : 44737 (29.8 %)
Draws : 46821 (31.2 %)
Unfinished : 0
White Perf. : 54.6 %
Black Perf. : 45.4 %
9/10/2011 7:00:20 PM :
Program Elo + - Games Score Av.Op. Draws
1 Houdini 2.0 STD : 3023 6 6 12000 79.0 % 2793 25.4 %
2 Houdini 1.5a : 3012 6 6 12000 77.9 % 2793 26.3 %
3 Critter 1.2 : 2972 6 6 12000 73.5 % 2795 29.3 %
4 Komodo64 3 SSE42 : 2972 6 6 12000 73.5 % 2795 29.9 %
5 Deep Rybka 4.1 SSE42 : 2969 6 5 12000 73.1 % 2795 30.3 %
6 Stockfish 2.1.1 JA : 2960 5 5 12000 72.1 % 2795 30.8 %
7 IvanHoe 9.47b : 2957 5 5 12000 71.7 % 2795 31.2 %
8 Deep Shredder 12 : 2869 5 5 12000 60.0 % 2799 35.3 %
9 Naum 4.2 : 2860 5 5 12000 58.6 % 2799 35.1 %
10 Spike 1.4 32b : 2828 5 5 12000 53.9 % 2801 36.4 %
11 Deep Sjeng c't 2010 32b : 2826 5 5 12000 53.6 % 2801 36.9 %
12 Deep Fritz 12 32b : 2810 5 5 12000 51.3 % 2802 36.8 %
13 Deep Junior 12.5 : 2806 5 5 12000 50.6 % 2802 36.3 %
14 Gull 1.2 : 2785 5 5 12000 47.5 % 2803 36.8 %
15 HIARCS 13.2 MP 32b : 2770 5 5 12000 45.3 % 2803 37.3 %
16 Hannibal 1.1 : 2757 5 5 12000 43.4 % 2804 35.6 %
17 Protector 1.4.0 x64 : 2748 5 5 12000 42.0 % 2804 35.7 %
18 spark-1.0 SSE42 : 2747 5 5 12000 41.8 % 2804 36.0 %
19 Deep Onno 1-2-70 : 2711 5 5 12000 36.7 % 2806 33.6 %
20 Zappa Mexico II : 2701 5 5 12000 35.3 % 2806 31.8 %
21 Loop 13.6/2007 : 2683 5 5 12000 32.8 % 2807 30.9 %
22 Strelka 2.0 B : 2611 6 6 12000 24.1 % 2810 24.9 %
23 Umko 1.2 SSE42 : 2566 6 6 12000 19.5 % 2812 21.1 %
24 Crafty 23.3 JA : 2559 6 6 12000 18.9 % 2812 21.0 %
25 Jonny 4.00 32b : 2499 7 7 12000 14.0 % 2815 15.7 %
IWB wrote:kingliveson wrote:
And by the way, what you published as results (which snippets I pasted), can easily be generated without ever playing a single game. I can produce you a script that does exactly that.
Franklin
In advance and accordiing to the other list? I would be really impressed!
BYe
Ingo
I am hoping that after this demonstration, Ingo would reconsider publishing actual games.
Re: IPON Rating's List Bash Script
Posted: Sun Sep 11, 2011 9:35 am
by ThinkingALot
kingliveson wrote:A new feature has been added; it now supports choosing which engines participate in a tournament (Gauntlet or Round-Robin) -- Just like regular GUIs. Source and game database are attached.
It's clear that your simulation is a fake. Gull is underrated

.