To kick off some technical discussions

[orgfert]

I'm still waiting for objective measurements. Looking for material down won games is subjective. Which is better, to be a queen down for 10 moves and then win, or to be a rook down for 24 moves and then win? Which is better, to be a queen down and almost win, but ultimately lose, or being a queen up and almost losing but ultimately win?

True science requires accurate measurement. Elo gave us an accurate measuring tool. Since there is no other measurement around, discarding the only objective one we have seems a bit rash.

Now if one wants to simply rank players on a "Tal-ish" scale (or a Karpov-ish scale for boring) there's nothing wrong with doing that. And if one wants to move from the boring side to the Tal side, without hurting overall performance, there's nothing wrong with doing that. However, the flashiest sword-fighter in the world still has the same problem against a gunfighter. He ends up dead, no matter how dashing his personality is.

Chess is a game for humans to enjoy. They enjoy playing and watching exciting games. Exciting games very often involve sacrifices. So, measuring programs on a scale which assesses excitability (with various fuzzy parameters to account for your above and other stuff) seems eminently senseful to me. And quite easy to do.

Maybe you are just worried Crafty is not a very exciting opponent, after all, as you keep telling us, it is optimised for boring old win/loss/draw/ELO performance?

btw, the swordfighter lives again to fight another day (nobody is killed in chess) but he does get the newspaper coverage, the articles and almost certainly the girls, not to mention the wine. The boring gunfighter goes home to loneliness and his Wii .....

The flat earth is also more romantic. But people who cling to it are called stupid. Science banishes darkness. You may think chess is only a game so we can cling to darkness there in safety. But this is harboring a viral infection in our brain that can only end in religion. Be careful of protecting treasured feelings at the expense of science. A joy like that must be dropped like a bad habit.

[Chris Whittington]

I'm still waiting for objective measurements. Looking for material down won games is subjective. Which is better, to be a queen down for 10 moves and then win, or to be a rook down for 24 moves and then win? Which is better, to be a queen down and almost win, but ultimately lose, or being a queen up and almost losing but ultimately win?

True science requires accurate measurement. Elo gave us an accurate measuring tool. Since there is no other measurement around, discarding the only objective one we have seems a bit rash.

Now if one wants to simply rank players on a "Tal-ish" scale (or a Karpov-ish scale for boring) there's nothing wrong with doing that. And if one wants to move from the boring side to the Tal side, without hurting overall performance, there's nothing wrong with doing that. However, the flashiest sword-fighter in the world still has the same problem against a gunfighter. He ends up dead, no matter how dashing his personality is.

Chess is a game for humans to enjoy. They enjoy playing and watching exciting games. Exciting games very often involve sacrifices. So, measuring programs on a scale which assesses excitability (with various fuzzy parameters to account for your above and other stuff) seems eminently senseful to me. And quite easy to do.

Maybe you are just worried Crafty is not a very exciting opponent, after all, as you keep telling us, it is optimised for boring old win/loss/draw/ELO performance?

btw, the swordfighter lives again to fight another day (nobody is killed in chess) but he does get the newspaper coverage, the articles and almost certainly the girls, not to mention the wine. The boring gunfighter goes home to loneliness and his Wii .....

The flat earth is also more romantic. But people who cling to it are called stupid. Science banishes darkness. You may think chess is only a game so we can cling to darkness there in safety. But this is harboring a viral infection in our brain that can only end in religion. Be careful of protecting treasured feelings at the expense of science. A joy like that must be dropped like a bad habit.

There are more assertive conjectures in the above than I care to argue about, but, for starters, your initial assumption that computer chess as it currently is performed is science is staggeringly unsound. Computer chess may have once held out some sort of hope for usefulness or even as a hope as a platform on which science can be done, but the science is long gone, the development goal of ELO, ELo and more ELO is a) going nowhere and b) utterly obsessive and the whole process is about as unscientific as you can get.

Gimme fun every time instead.

[Uly]

One important point is that to measure strength you have to play thousands and thousands of games, but to see the engine's playing style you just need to play a few of them, and watch them.

Say, play 16 games and see how the engines play, with just 16 games it was very evident that the style of Rybka 2.3.2a was extremely dull, though of course more games against different opponents were played, but she kept playing like that, so 16 games against each opponent seem to be enough for a decision.

But the decision would be subjective, I recall having discussions on the past where I perceived some style as less exciting, while the other person liked the style more (this was a Toga's style vs. Fruit style IIRC).

But style isn't just about sacrifices, otherwise my Pro Deo Xtreme personality would reach the highest "playing style" even if it loses against everyone. No, that doesn't suffice, style also needs to show that the sacrifices are sound (so that engines that go for positions in where sacrifices work are rewarded), and, engines that go for positions where the material imbalance is huge provide a game more enjoyable to watch, say, 3 sacrifices that leave to a position with several pawns against a queen is much better than a game with 5 sacrifices where in the end the other side gets eventually the material back and it's draw.

As for strength-style ratio (or whatever) I suggest playing with time handicap or in extreme cases depth handicap. Then you can play Rybka 4 at 1 minute for the whole game against another engine with one hour on the clock, or in any case Rybka 4 at depth -2 (the real depth "1") against another engine with 2 hours on the clock, that way you can get to see the engine's styles without worrying about strength differences (though how are we going to judge the style is critical).

[orgfert]

I'm still waiting for objective measurements. Looking for material down won games is subjective. Which is better, to be a queen down for 10 moves and then win, or to be a rook down for 24 moves and then win? Which is better, to be a queen down and almost win, but ultimately lose, or being a queen up and almost losing but ultimately win?

True science requires accurate measurement. Elo gave us an accurate measuring tool. Since there is no other measurement around, discarding the only objective one we have seems a bit rash.

Now if one wants to simply rank players on a "Tal-ish" scale (or a Karpov-ish scale for boring) there's nothing wrong with doing that. And if one wants to move from the boring side to the Tal side, without hurting overall performance, there's nothing wrong with doing that. However, the flashiest sword-fighter in the world still has the same problem against a gunfighter. He ends up dead, no matter how dashing his personality is.

Chess is a game for humans to enjoy. They enjoy playing and watching exciting games. Exciting games very often involve sacrifices. So, measuring programs on a scale which assesses excitability (with various fuzzy parameters to account for your above and other stuff) seems eminently senseful to me. And quite easy to do.

Maybe you are just worried Crafty is not a very exciting opponent, after all, as you keep telling us, it is optimised for boring old win/loss/draw/ELO performance?

btw, the swordfighter lives again to fight another day (nobody is killed in chess) but he does get the newspaper coverage, the articles and almost certainly the girls, not to mention the wine. The boring gunfighter goes home to loneliness and his Wii .....

The flat earth is also more romantic. But people who cling to it are called stupid. Science banishes darkness. You may think chess is only a game so we can cling to darkness there in safety. But this is harboring a viral infection in our brain that can only end in religion. Be careful of protecting treasured feelings at the expense of science. A joy like that must be dropped like a bad habit.

There are more assertive conjectures in the above than I care to argue about, but, for starters, your initial assumption that computer chess as it currently is performed is science is staggeringly unsound. Computer chess may have once held out some sort of hope for usefulness or even as a hope as a platform on which science can be done, but the science is long gone, the development goal of ELO, ELo and more ELO is a) going nowhere and b) utterly obsessive and the whole process is about as unscientific as you can get.

Gimme fun every time instead.

I hope you will not take offense if I observe that for a statement about science this seems very subjective. To say the science is gone and that Elo is going nowhere is backed up by nothing but the air it takes to speak. What you are really saying is that the "art" is gone and that the science of game tree search is not lending itself to subjective ideals of this supposed art. I would contend the art was a side-effect of recovering from or punishing less than optimal play. It was an illusion of beauty that could only arise from errors.

[Chris Whittington]

One important point is that to measure strength you have to play thousands and thousands of games, but to see the engine's playing style you just need to play a few of them, and watch them.

Say, play 16 games and see how the engines play, with just 16 games it was very evident that the style of Rybka 2.3.2a was extremely dull, though of course more games against different opponents were played, but she kept playing like that, so 16 games against each opponent seem to be enough for a decision.

But the decision would be subjective, I recall having discussions on the past where I perceived some style as less exciting, while the other person liked the style more (this was a Toga's style vs. Fruit style IIRC).

But style isn't just about sacrifices, otherwise my Pro Deo Xtreme personality would reach the highest "playing style" even if it loses against everyone. No, that doesn't suffice, style also needs to show that the sacrifices are sound (so that engines that go for positions in where sacrifices work are rewarded), and, engines that go for positions where the material imbalance is huge provide a game more enjoyable to watch, say, 3 sacrifices that leave to a position with several pawns against a queen is much better than a game with 5 sacrifices where in the end the other side gets eventually the material back and it's draw.

As for strength-style ratio (or whatever) I suggest playing with time handicap or in extreme cases depth handicap. Then you can play Rybka 4 at 1 minute for the whole game against another engine with one hour on the clock, or in any case Rybka 4 at depth -2 (the real depth "1") against another engine with 2 hours on the clock, that way you can get to see the engine's styles without worrying about strength differences (though how are we going to judge the style is critical).

of course the 'style' thing is not only tremendously subjective but also subject to lies and propaganda especially for real or semi-commercial programs - Hiarcs plays human style propaganda nonsense being an obvious case. Genius was said by the 'expert reviewers' once to be interesting and attacking, then they changed their tune, it was dull and boring. You should read my old rgcc? post on search gap for naive perceived style, it should be around somewhere.

btw, there is no such thing as a 'sound sacrifice', the essential element is that we don't know the answer to the question: "is this attack worth my rook?" because we have no scientific (that stupid word again) or accurate way to measure the material value of an attack over the board. If we can measure it and the answer is positive, it is no sacrifice

[Chris Whittington]

I'm still waiting for objective measurements. Looking for material down won games is subjective. Which is better, to be a queen down for 10 moves and then win, or to be a rook down for 24 moves and then win? Which is better, to be a queen down and almost win, but ultimately lose, or being a queen up and almost losing but ultimately win?

True science requires accurate measurement. Elo gave us an accurate measuring tool. Since there is no other measurement around, discarding the only objective one we have seems a bit rash.

Now if one wants to simply rank players on a "Tal-ish" scale (or a Karpov-ish scale for boring) there's nothing wrong with doing that. And if one wants to move from the boring side to the Tal side, without hurting overall performance, there's nothing wrong with doing that. However, the flashiest sword-fighter in the world still has the same problem against a gunfighter. He ends up dead, no matter how dashing his personality is.

Chess is a game for humans to enjoy. They enjoy playing and watching exciting games. Exciting games very often involve sacrifices. So, measuring programs on a scale which assesses excitability (with various fuzzy parameters to account for your above and other stuff) seems eminently senseful to me. And quite easy to do.

Maybe you are just worried Crafty is not a very exciting opponent, after all, as you keep telling us, it is optimised for boring old win/loss/draw/ELO performance?

btw, the swordfighter lives again to fight another day (nobody is killed in chess) but he does get the newspaper coverage, the articles and almost certainly the girls, not to mention the wine. The boring gunfighter goes home to loneliness and his Wii .....

The flat earth is also more romantic. But people who cling to it are called stupid. Science banishes darkness. You may think chess is only a game so we can cling to darkness there in safety. But this is harboring a viral infection in our brain that can only end in religion. Be careful of protecting treasured feelings at the expense of science. A joy like that must be dropped like a bad habit.

There are more assertive conjectures in the above than I care to argue about, but, for starters, your initial assumption that computer chess as it currently is performed is science is staggeringly unsound. Computer chess may have once held out some sort of hope for usefulness or even as a hope as a platform on which science can be done, but the science is long gone, the development goal of ELO, ELo and more ELO is a) going nowhere and b) utterly obsessive and the whole process is about as unscientific as you can get.

Gimme fun every time instead.

I hope you will not take offense if I observe that for a statement about science this seems very subjective. To say the science is gone and that Elo is going nowhere is backed up by nothing but the air it takes to speak. What you are really saying is that the "art" is gone and that the science of game tree search is not lending itself to subjective ideals of this supposed art. I would contend the art was a side-effect of recovering from or punishing less than optimal play. It was an illusion of beauty that could only arise from errors.

The science has disappeared partly in that people who call themselves 'scientists' are no longer interested in the subject and partly because the field practitioners are engaged in a sort-of obsessive hobby rather than a scientific discipline. Despite the pretense over many years, the pretense of the academic journal, the pseudo mathematics and statistics, this never was a science, it was a con-trick designed to get some money through bullshit. That money has now dried up.

Science re rating lists, for example.

1. The data is improperly collected without controls

2. There is political-personal interference in what is and what isn't allowed as data (banned programs)

3. The games played come from a play pool that is restricted to like-playing entities and therefore veers off to nowhere or at best a cul-de-sac

etc.


ELO going nowhere, means: what is the point of obsessively working on programs and then having others obsessively test them, publishing allegedly statistical scientific enumerated lists of how these programs (crippled chess entities with super lookahead) perform against each other in an endless quest for more ELO points.

What are they going to do with all these ELO points? What does it matter? Isn't it just an obsessive behaviour, pointless, filling in the void?

[orgfert]

I'm still waiting for objective measurements. Looking for material down won games is subjective. Which is better, to be a queen down for 10 moves and then win, or to be a rook down for 24 moves and then win? Which is better, to be a queen down and almost win, but ultimately lose, or being a queen up and almost losing but ultimately win?

True science requires accurate measurement. Elo gave us an accurate measuring tool. Since there is no other measurement around, discarding the only objective one we have seems a bit rash.

Now if one wants to simply rank players on a "Tal-ish" scale (or a Karpov-ish scale for boring) there's nothing wrong with doing that. And if one wants to move from the boring side to the Tal side, without hurting overall performance, there's nothing wrong with doing that. However, the flashiest sword-fighter in the world still has the same problem against a gunfighter. He ends up dead, no matter how dashing his personality is.

Chess is a game for humans to enjoy. They enjoy playing and watching exciting games. Exciting games very often involve sacrifices. So, measuring programs on a scale which assesses excitability (with various fuzzy parameters to account for your above and other stuff) seems eminently senseful to me. And quite easy to do.

Maybe you are just worried Crafty is not a very exciting opponent, after all, as you keep telling us, it is optimised for boring old win/loss/draw/ELO performance?

btw, the swordfighter lives again to fight another day (nobody is killed in chess) but he does get the newspaper coverage, the articles and almost certainly the girls, not to mention the wine. The boring gunfighter goes home to loneliness and his Wii .....

The flat earth is also more romantic. But people who cling to it are called stupid. Science banishes darkness. You may think chess is only a game so we can cling to darkness there in safety. But this is harboring a viral infection in our brain that can only end in religion. Be careful of protecting treasured feelings at the expense of science. A joy like that must be dropped like a bad habit.

There are more assertive conjectures in the above than I care to argue about, but, for starters, your initial assumption that computer chess as it currently is performed is science is staggeringly unsound. Computer chess may have once held out some sort of hope for usefulness or even as a hope as a platform on which science can be done, but the science is long gone, the development goal of ELO, ELo and more ELO is a) going nowhere and b) utterly obsessive and the whole process is about as unscientific as you can get.

Gimme fun every time instead.

I hope you will not take offense if I observe that for a statement about science this seems very subjective. To say the science is gone and that Elo is going nowhere is backed up by nothing but the air it takes to speak. What you are really saying is that the "art" is gone and that the science of game tree search is not lending itself to subjective ideals of this supposed art. I would contend the art was a side-effect of recovering from or punishing less than optimal play. It was an illusion of beauty that could only arise from errors.

The science has disappeared partly in that people who call themselves 'scientists' are no longer interested in the subject and partly because the field practitioners are engaged in a sort-of obsessive hobby rather than a scientific discipline. Despite the pretense over many years, the pretense of the academic journal, the pseudo mathematics and statistics, this never was a science, it was a con-trick designed to get some money through bullshit. That money has now dried up.

Science re rating lists, for example.

1. The data is improperly collected without controls

2. There is political-personal interference in what is and what isn't allowed as data (banned programs)

3. The games played come from a play pool that is restricted to like-playing entities and therefore veers off to nowhere or at best a cul-de-sac

etc.


ELO going nowhere, means: what is the point of obsessively working on programs and then having others obsessively test them, publishing allegedly statistical scientific enumerated lists of how these programs (crippled chess entities with super lookahead) perform against each other in an endless quest for more ELO points.

What are they going to do with all these ELO points? What does it matter? Isn't it just an obsessive behaviour, pointless, filling in the void?

It is not pointless in actual fact. The reason is because the search is always going deeper. Search eventually corrects errors because search eventually discovers the end of the game.

You criticize programs playing each other, but it's like Kasparov's comment on why he engaged in post-game analysis with Karpov, and man he detests. His answer? "Who else can I discuss these things with?" Who else was able to play at Kasparov's level? Only Karpov at the time.

The same is now true of the computer. No human is in the same league with the best programs anymore. Therefore, how is your statement being objective in saying no science has happened or is happening?

[Chris Whittington]

I'm still waiting for objective measurements. Looking for material down won games is subjective. Which is better, to be a queen down for 10 moves and then win, or to be a rook down for 24 moves and then win? Which is better, to be a queen down and almost win, but ultimately lose, or being a queen up and almost losing but ultimately win?

True science requires accurate measurement. Elo gave us an accurate measuring tool. Since there is no other measurement around, discarding the only objective one we have seems a bit rash.

Now if one wants to simply rank players on a "Tal-ish" scale (or a Karpov-ish scale for boring) there's nothing wrong with doing that. And if one wants to move from the boring side to the Tal side, without hurting overall performance, there's nothing wrong with doing that. However, the flashiest sword-fighter in the world still has the same problem against a gunfighter. He ends up dead, no matter how dashing his personality is.

Chess is a game for humans to enjoy. They enjoy playing and watching exciting games. Exciting games very often involve sacrifices. So, measuring programs on a scale which assesses excitability (with various fuzzy parameters to account for your above and other stuff) seems eminently senseful to me. And quite easy to do.

Maybe you are just worried Crafty is not a very exciting opponent, after all, as you keep telling us, it is optimised for boring old win/loss/draw/ELO performance?

btw, the swordfighter lives again to fight another day (nobody is killed in chess) but he does get the newspaper coverage, the articles and almost certainly the girls, not to mention the wine. The boring gunfighter goes home to loneliness and his Wii .....

The flat earth is also more romantic. But people who cling to it are called stupid. Science banishes darkness. You may think chess is only a game so we can cling to darkness there in safety. But this is harboring a viral infection in our brain that can only end in religion. Be careful of protecting treasured feelings at the expense of science. A joy like that must be dropped like a bad habit.

There are more assertive conjectures in the above than I care to argue about, but, for starters, your initial assumption that computer chess as it currently is performed is science is staggeringly unsound. Computer chess may have once held out some sort of hope for usefulness or even as a hope as a platform on which science can be done, but the science is long gone, the development goal of ELO, ELo and more ELO is a) going nowhere and b) utterly obsessive and the whole process is about as unscientific as you can get.

Gimme fun every time instead.

I hope you will not take offense if I observe that for a statement about science this seems very subjective. To say the science is gone and that Elo is going nowhere is backed up by nothing but the air it takes to speak. What you are really saying is that the "art" is gone and that the science of game tree search is not lending itself to subjective ideals of this supposed art. I would contend the art was a side-effect of recovering from or punishing less than optimal play. It was an illusion of beauty that could only arise from errors.

The science has disappeared partly in that people who call themselves 'scientists' are no longer interested in the subject and partly because the field practitioners are engaged in a sort-of obsessive hobby rather than a scientific discipline. Despite the pretense over many years, the pretense of the academic journal, the pseudo mathematics and statistics, this never was a science, it was a con-trick designed to get some money through bullshit. That money has now dried up.

Science re rating lists, for example.

1. The data is improperly collected without controls

2. There is political-personal interference in what is and what isn't allowed as data (banned programs)

3. The games played come from a play pool that is restricted to like-playing entities and therefore veers off to nowhere or at best a cul-de-sac

etc.


ELO going nowhere, means: what is the point of obsessively working on programs and then having others obsessively test them, publishing allegedly statistical scientific enumerated lists of how these programs (crippled chess entities with super lookahead) perform against each other in an endless quest for more ELO points.

What are they going to do with all these ELO points? What does it matter? Isn't it just an obsessive behaviour, pointless, filling in the void?

It is not pointless in actual fact. The reason is because the search is always going deeper. Search eventually corrects errors because search eventually discovers the end of the game.

You criticize programs playing each other, but it's like Kasparov's comment on why he engaged in post-game analysis with Karpov, and man he detests. His answer? "Who else can I discuss these things with?" Who else was able to play at Kasparov's level? Only Karpov at the time.

The same is now true of the computer. No human is in the same league with the best programs anymore. Therefore, how is your statement being objective in saying no science has happened or is happening?

it is pointless because search will NEVER get to the end of the game. You can stop the development process now, next year or in ten years time. My contention is stop now because there's nothing to be gained anymore, the target is unattainable anyway and any targets that there were, were reached long ago - so why continue the incestuous fight other than for reasons of obsessiveness?

It is not science, it's an obsessive hobby.

[orgfert]

btw, there is no such thing as a 'sound sacrifice', the essential element is that we don't know the answer to the question: "is this attack worth my rook?" because we have no scientific (that stupid word again) or accurate way to measure the material value of an attack over the board. If we can measure it and the answer is positive, it is no sacrifice

A good point at last. The lament over loss of "art" is rooted in our ignorance of best play. I found a link to unusual openings that commented on the "vulture und woozle" variations as viable openings at club level. They work at lower levels of play for dynamic and interesting games to those at that level of play. But these would be jokes at top level human play. Computers shred these openings.

I know you might think that at the limit of a machines "knowledge" it should be able to speculate like a super-constellation or a CSTal of olden times. But at 23-ply search, can the human even appreciate or even notice what has happened, like a club player against a GM not understanding how he has been so royally screwed? It will look just as dull since we are become as patzers at a masters table. The super-constellation and CSTal could be appreciated becasue they were not out of our league. We will never be at that place again in top computer play. So ultimately this art is just a historical artifact with no basis in scientific reality. This must be the more objective of the assessments.

[orgfert]

it is pointless because search will NEVER get to the end of the game. You can stop the development process now, next year or in ten years time. My contention is stop now because there's nothing to be gained anymore, the target is unattainable anyway and any targets that there were, were reached long ago - so why continue the incestuous fight other than for reasons of obsessiveness?

It is not science, it's an obsessive hobby.

I meant at some point in the game, the computer search finds an objectively decisive advantage be it checkmate or anything that means the game is basically over with anything resembling good play. That is how search corrects errors in evaluation. The programmer is at times able to detect where the search is refuting the evaluation, the scientific check that balances evaluative guessing.