To kick off some technical discussions

[orgfert]

Bob,

I like reading your posts and I think what you've done over the years, with Crafty and other stuff is really excellent. But, there's always a but ....

Fernando does have some understanding of what I tried to do in the past and what he spoke of just now is part of that but it's as if what he said doesn't exist in your answer and possibly even in your mindset. All your posts that begin "In Crafty we do ...." or similar, I think oh god, here we go again, Crafty and Bob have to be doing or have done all that is possible in computer chess - it's as if you want to own all the computer chess space and deny others a look in for other ideas. What you are referring to in the above post "In Crafty ..." has nothing to do with the idea postulated by Fernando, although you make it seem that it does. It may be of little loss to you, but if I'm going to have a dialog with someone else in the field, I expect some of my ideas and thoughts to get across and be reflected back to me some way by the other person, in other words, I need to feel as part of the dialog that my space and your space overlap somehow. Not a feeling I get with result I tend to drop out of the conversation. Probably no loss to you

But there is yet another difference from your side that you do not see, maybe. It is that Bob shares all. Secretive commercial programmers do not. I never see you say, "In my program, we achieved our unique advantage by this or that means." I can understand this position, but it seems to explain why your space never overlaps with Bob's.

[Chris Whittington]

Bob,

I like reading your posts and I think what you've done over the years, with Crafty and other stuff is really excellent. But, there's always a but ....

Fernando does have some understanding of what I tried to do in the past and what he spoke of just now is part of that but it's as if what he said doesn't exist in your answer and possibly even in your mindset. All your posts that begin "In Crafty we do ...." or similar, I think oh god, here we go again, Crafty and Bob have to be doing or have done all that is possible in computer chess - it's as if you want to own all the computer chess space and deny others a look in for other ideas. What you are referring to in the above post "In Crafty ..." has nothing to do with the idea postulated by Fernando, although you make it seem that it does. It may be of little loss to you, but if I'm going to have a dialog with someone else in the field, I expect some of my ideas and thoughts to get across and be reflected back to me some way by the other person, in other words, I need to feel as part of the dialog that my space and your space overlap somehow. Not a feeling I get with result I tend to drop out of the conversation. Probably no loss to you

But there is yet another difference from your side that you do not see, maybe. It is that Bob shares all. Secretive commercial programmers do not. I never see you say, "In my program, we achieved our unique advantage by this or that means." I can understand this position, but it seems to explain why your space never overlaps with Bob's.

Utterly wrong in fact. You got the wrong commercial programmer - maybe many of them are silent, but, as a commercial programmer I was hardly secretive at all. My technical 'verbal' output onto chess forums was probably higher than any other programmer (Bob excepted, accepted) and my program pumped out huge amounts of data such that anyone who delved deep enough could work out exactly how it analysed positions and, with some work, how it conducted its search.

Part of Bob's problem may be that he never acquired a copy of CSTal (which says much for his honesty imo, most people had an illegal copy or so I am told) but even his never seeing it didn't stop of from saying it "was a load of crap". Lack of overlap does not come from my direction.

[Chris Whittington]

What a pleasure to see you here, Bob! Respect what Chris W told to you and what you answered, I cannot add nothing but just the banal assertion that when you have a massive tool to do something, better to drop the old stone hammer.
What I do fully take from Chris thinking, not expressed here but elsewhere and lot time ago, is his idea that there is something Ok in the engine pushing the adversary to uncharted waters, that is, choosing sometimes lines that are not the best in terms of conventional scores, but are the best in terms of risky sub lines coming from there. Against humans this is usually lethal. At that you can add a superb improvement in the amount of interesting games.
I wonder if you could program a Crafty tailored specifically to take humans.
My best
Fern

Older versions of crafty had a lot of anti-human type code in them. Avoid trading material, avoid locked pawn structures, avoid certain types of problematic opening positions (Stonewall as an example where a program usually mishandles things and falls into a kingside attack before it realizes what is going on). But over time, I have removed many of those crutches because as depth/speed has gone up, the ability to handle those with general knowledge has improved. Nowadays there is really nothing wrong with playing black against a strong human, and walking into a Stonewall setup. Programs should not fear endgames and therefore avoid trading, because they play endgames extremely accurately also.

In short, for a good while we needed "anti-human" strategies in a chess engine to avoid playing into positions where the computer's strength was nullified. Nowadays, it seems to be the humans that are trying to rely on anti-computer strategies, and the computers are getting better at dealing with them, without needing special-case code...

Yes there are a few small holes scattered around, but overall...

Bob,

I like reading your posts and I think what you've done over the years, with Crafty and other stuff is really excellent. But, there's always a but ....

Fernando does have some understanding of what I tried to do in the past and what he spoke of just now is part of that but it's as if what he said doesn't exist in your answer and possibly even in your mindset. All your posts that begin "In Crafty we do ...." or similar, I think oh god, here we go again, Crafty and Bob have to be doing or have done all that is possible in computer chess - it's as if you want to own all the computer chess space and deny others a look in for other ideas. What you are referring to in the above post "In Crafty ..." has nothing to do with the idea postulated by Fernando, although you make it seem that it does. It may be of little loss to you, but if I'm going to have a dialog with someone else in the field, I expect some of my ideas and thoughts to get across and be reflected back to me some way by the other person, in other words, I need to feel as part of the dialog that my space and your space overlap somehow. Not a feeling I get with result I tend to drop out of the conversation. Probably no loss to you

I understood your old comments of "steer into the fog". But I've never found a workable way to do that, directly. I have found ideas to complicate the position (when it is appropriate) by piling on pieces when the opponent's king is unsafe, or maintaining tension rather than relieving it by trading pawns or pushing to lock them. But I've not found anything that works as you described. Not that it can't be done, but that I have not thought of a way to make it happen. I am not trying to dictate what others do, by any means. I do often mention what I have tried and whether or not it worked, and I often mention things I plan on trying in the future. And I have also quite often taken an idea I have not tried, code it up, and find that it works. These are usually mentioned in the comments in main.c to give credit to the person that came up with the idea.

In the above, Fernando explicitly mentioned "in Crafty". And I explained what we had done previously (although as I explained, much of that has been removed as the search depths have grown so much). If you have ideas to suggest, fire away. I can't ever remember dismissing an idea I have not tried, outright. I sometimes might well decide "way too hard to code efficiently" or whatever.

well, the only way to "steer into the fog" is by including a king attack evaluation which is able to recognise certain powerful attacking situations and give them positional bonuses of more than a minor piece (ie make it interesting to sac a knight or bishop), some terms should be as high or higher than a rook. That's what the fog is. I don't think any program ever tried it, although I'm a little out of date now.
You should not claim btw that addition of a great mass of positional terms summing to more than a bishop/knight is the same thing - it's specifically recognition of wild/overwhelming attack situation.

[hyatt]

We've experimented, particularly using 2nd order and 3rd order interactions. Say an exponential king shelter term, plus terms to measure distance from pieces to king, which can be 2nd order by themselves (more pieces accelerates this term more). And then a 3rd order term that takes the "safety", the "attacker tropism" (BTW this varies by piece for us at present, knights need to be close, but sliders need open paths to the king) and then lets those two terms interact to produce the final 3rd order score. We've got stuff that can go to +5 given the right conditions, but we've had to tune this carefully as cluster testing generally highlights each failure by refuting the attacks and leaving us material down. Possibly better against humans, but the challenge today is to beat the best players, which are no longer human at all.

[Chris Whittington]

We've experimented, particularly using 2nd order and 3rd order interactions. Say an exponential king shelter term, plus terms to measure distance from pieces to king, which can be 2nd order by themselves (more pieces accelerates this term more). And then a 3rd order term that takes the "safety", the "attacker tropism" (BTW this varies by piece for us at present, knights need to be close, but sliders need open paths to the king) and then lets those two terms interact to produce the final 3rd order score. We've got stuff that can go to +5 given the right conditions, but we've had to tune this carefully as cluster testing generally highlights each failure by refuting the attacks and leaving us material down. Possibly better against humans, but the challenge today is to beat the best players, which are no longer human at all.

how do you know the best players aren't human?

it seems some comparitively regular players can thrash computers quite easily, it also seems, looking at various comp-comp games, that computers have absolutely zero strategic sense.

Do computers actually play chess? I don't think so, they play some strange subset with many human factors removed, with deep tactical search and almost zero strategic sense and compete with each other in nonsensically organised 'rating lists' where nobody even bothers to look at the game moves.


As to your first second and third order evaluation. Sounds good, but obviously it couldn't fight
That's not time to give up imo

[Fernando]

Chris:
perhaps you should take a look at Junior 11.1
As far as I have seen in some games, it does go ahead if he sniff a chance of killing you no matter how much infantry he loses to defensive musketry. It has an style near Cstal, perhaps less daring at times and without the funny face I am afraid.. . It is a very entertaining kind of game. Hiarcs, also, has lot of bowels to do the same.
Respect what Bob say about the fact that to win now means to win a computer, I wonder if computers are those that goes to the Internet to purchase programs. I am a 2100 player, I do not need to play a boring 3400 elo program top be trashed. I can be beaten by a 2300-2400, most of the time. The point is: how much fun I get from that couple of hours on the board. It amazes me the fact that so many deceive themselves about this simple fact and still cater for geeks prone to play comps versus comps.
I am not of that freak league, thanks God


Fern.

[hyatt]

We've experimented, particularly using 2nd order and 3rd order interactions. Say an exponential king shelter term, plus terms to measure distance from pieces to king, which can be 2nd order by themselves (more pieces accelerates this term more). And then a 3rd order term that takes the "safety", the "attacker tropism" (BTW this varies by piece for us at present, knights need to be close, but sliders need open paths to the king) and then lets those two terms interact to produce the final 3rd order score. We've got stuff that can go to +5 given the right conditions, but we've had to tune this carefully as cluster testing generally highlights each failure by refuting the attacks and leaving us material down. Possibly better against humans, but the challenge today is to beat the best players, which are no longer human at all.

how do you know the best players aren't human?

it seems some comparitively regular players can thrash computers quite easily, it also seems, looking at various comp-comp games, that computers have absolutely zero strategic sense.

Do computers actually play chess? I don't think so, they play some strange subset with many human factors removed, with deep tactical search and almost zero strategic sense and compete with each other in nonsensically organised 'rating lists' where nobody even bothers to look at the game moves.


As to your first second and third order evaluation. Sounds good, but obviously it couldn't fight
That's not time to give up imo

I am reasonably convinced that silicon chess players are stronger than humans now. I won't argue about how they compare when playing, long-range-strategy vs insanely deep tactical searches, but the age-old artificial intelligence arguments don't appear to hold water any longer. In any game, there is the "what" and the "how". The "what" is about what the player does in the game, and computers are quite strong there. The "how" is less important. Motorcycles don't have "legs". It would be mechanically ugly, unreliable, and uncomfortable to ride such a thing. Yet clearly motorcycles are faster than a human. So the "what" (getting from point A to point B) favors the machine, regardless of "how" it accomplishes this.

I've always held out hope that somehow humans are superior to all this stuff. I begin to suspect that somewhere in the future, we are going to discover that we are also "binary" at the most basic level of our biology. The "how" is only interesting to some. Which is better, the human that unerringly finds his way thru a deep maze, vs the machine that just flies up and over the thing to get to the other side quicker? If the goal is an elegant solution, the human is better. If the goal is to just get there, the helicopter is impossible to beat. It begins to look like the game of chess is not nearly as "cerebral" as has been thought, for many, many years.

Fern:

You have yet a different goal. Not the "if you lose" but "how you lose". Which is a different issue. I agree it is more fun to play against an opponent that doesn't seem to be omnipotent, as that is demoralizing. And there's nothing wrong with wanting or writing a program to do that. Some want to race on water, some on asphalt, some in mud. There's room for all. Unfortunately, in autos, it is pretty easy (ignoring money) to go out and buy something that can break 200mph on asphalt, it is not easy at all to find something that will run up a mountainside at a similar speed, you have to build those yourself.

[Fernando]

Demoralizing perhaps is not the word. Boring is nearer the target. I could feel demoralized if I ever begin to write under my usual standard, not just losing chess. Neither is the case that I do not want to get the full point or I want a losing program. For that I can use some commercial stuff with variable levels of strength. What I look is a really human kind of engine capable of playing at 3000 elo if you want, but not just adding some minuscule positional advantages to crush you after 560 moves. Junior and Hiarc are near that, but CStal was even nearer. It's a pity that kind of programming, that certainly can be done as Chris did, it is not anymore the first target of programmer, but to beat Rybka or something of the sort.
I would be enchanted if you, with all the knowledge you have, does try to do that, a Human-Crafty tailored for geeks as me, strong as hell but human, prone to sacrifices, to confuse the game, etc.
why not, Bob?

Fern

[hyatt]

I am working on something related, already. Crafty has had a "skill" command for a year or two now, one that _was_ quite effective. You could vary skill from 1 to 100, where 100 is normal crafty, no random eval, and skill 1 is 99% random, 1% normal eval. And it worked. But something recent is causing a big problem, in that with a 100% random eval (just a pure random number between 1 and 100) the damned thing is playing at over +1700 Elo now, where it used to be below 800. Why? All I can surmise so far is the "Don Beal effect" if you read his paper on a random evaluation playing much better than expected. He discovered that this is another way to indirectly simulate mobility. I am back to trying to find a way to fix this as 1700+ is still far too strong for most players. I could artificially reduce the depth, but that still leaves a program that would be positionally very aware, and if I just do pure random eval and somehow defeat this "Beal effect" that still leaves a positional idiot but a tactical genius. So both need to be trimmed back so that you play what appears to be a weak player, not a strong player that just hangs a rook every now and then for no reason, which doesn't quite feel right...

[Fernando]

Bob:
It is curious what you say about that Beal effect. Many many years ago when I played mostly Champion Chess Challenger, a machine around 1600 elo I think, I realized -and even I wrote about that- that the machine played better when in random mode. In that case the machine chose between the 2 or 3 - I suppose- best moves found so far and so i reasoned that, as much by definition the machine was not a chess genius, so his absolutely "best" moves probably were not such a thing many timers and then the random function gave it the chance to play the really best move in the position. I presume the Beal argument is lot different, but I could not help to remember my little piece of analysis made from the side of ignorance.
Respect what you say of that function of Crafty, it is not what I am thinking about as you surely realize. It is not a question of weakening same good old Crafty, but of creating a new kind of it. Don't you think it could be a new, exciting avenue for you to try? As both of us are arund 60 nor more men, we both knoiw how much we need some extra kicks to get fun...


Fern