UCI Interfacing question

[epideath]

Hi everyone,
I am coding an interface to uci chess engines and I can't seem to find much information about the GO parameters wTime, bTime, wInc, and bInc. I can add these values and track them with my own code. But I was wondering if this is something that the engine handles. If so then how would you go about getting this information from the chess engine? Or am I correct in that my interface software will have to manage movestogo and time remaining?

TIA for any help.
Gary

[BB+]

Usually wtime/btime are things that the GUI keeps track of, and get sent to the engine when it sends the "go" command. The winc/binc/movestogo are similarly kept by the GUI, and they let the engine know what the operative time control (for instance, so it can decide how to allocate its time -- admittedly UCI is quite weak in this regard wrt "changing" time controls like 40/90 SD/30). The philosophy of UCI is that the engine is completely a "dummy", and needs to be told what to do, always. Specifically, for your post, the engine is not keeping track of time used/elapsed itself (either its own, or the opponent's), but relies on the GUI for this.

[lucasart]

Hi everyone,
I am coding an interface to uci chess engines and I can't seem to find much information about the GO parameters wTime, bTime, wInc, and bInc. I can add these values and track them with my own code. But I was wondering if this is something that the engine handles. If so then how would you go about getting this information from the chess engine? Or am I correct in that my interface software will have to manage movestogo and time remaining?

TIA for any help.
Gary

Yes, indeed, the whole philosophy of the UCI protocol is that:
* everything that can be done by the GUI should be done by the GUI

That includes things like
(i) opening book
(ii) clock, movestogo, and generally all notion of state

The point is that, state is the root of all evil. Imagine if the state is stored by both the GUI and the Engine. What if they disagree? How do they reconcile their state values? Imagine some nasty bugs, because the state maintained by your engine is wrong, and some bugs cannot be reproduced as a result...

All these idiotic made up problems, that typically arise in the Xboard protocol, do not even exist in the UCI world. That is why, the UCI protocol is always a much better choice than the Xboard one, especially for beginners who want to keep it simple.

So a proper UCI engine has no notion of state, and just receives everything from the GUI everytime. Of course there is still some state somewhere: eg. the hash table, that you typically flush upon receiving a "ucinewgame" command.

If in doubt, read the manual instead, and get the information from the horse's mouth:
http://download.shredderchess.com/div/uci.zip

One of the awckward consequences of that principle (eliminating state) is that an UCI interface will send to your engine a position command that looks like this:

Code: Select all

position startpos moves m(1) ... m(n)

WHen I first saw that, I thought it would be a ridiculous waste of time, but I was completely wrong for a couple of reasons:
1/ the real bottle neck is pipe I/O and task switching. parsing the string and playing the moves (assuming a reasonable implementation) is much faster than people usually imagine.
2/ there is actually zero overhead, because the GUI is compelled to send you an 'isready' and wait for 'readyok' after the position command. Only at this point the clock is started.

PS: The UCI protocol is case sensitive, so 'GO', 'wTime' and 'bTime' do not mean anything. I suppose these were typos.

[BB+]

2/ there is actually zero overhead, because the GUI is compelled to send you an 'isready' and wait for 'readyok' after the position command.

I don't see this in the protocol description. In fact, the example given violates this.

Code: Select all

// tell the engine to search infinite from the start position after 1.e4 e5
position startpos moves e2e4 e7e5
go infinite

On the 250th move (not difficult to reach in computer shuffle-wars), the overhead to parse from "startpos" should be about 1ms. This might only be an issue if you are testing a sudden-death time control.

PS: The UCI protocol is case sensitive, so 'GO', 'wTime' and 'bTime' do not mean anything. I suppose these were typos.

Again I don't see general case-sensitivity discussed in the linked description, only an indication that option names are not case-sensitive.

Incidentally, how many GUIs respond (as demanded) to "readyok" with "isready" at any time?

[lucasart]

2/ there is actually zero overhead, because the GUI is compelled to send you an 'isready' and wait for 'readyok' after the position command.

I don't see this in the protocol description. In fact, the example given violates this.

The protocol states that any command that may take some time to complete should be followed by isready, in order to synchronize GUI and Engine. Besides, it is self-obvious, so any GUI developper who has a half brain will do it that way. Not everything is explicitly stated in this document.

On the 250th move (not difficult to reach in computer shuffle-wars), the overhead to parse from "startpos" should be about 1ms. This might only be an issue if you are testing a sudden-death time control.

Again isready solves this problem. And even if the GUI is defective and doesn't do that, 1ms is really small. And you're taking an extreme edge case: 250 moves is 500 half moves:
* How often do you see games taking that long ?
* Also, a clever GUI could decide to send instead 'position fen ... moves ...' and start from the fen of the last position when a pawn move or capture occured, in order to reduce the worst case scenario overhead.

PS: The UCI protocol is case sensitive, so 'GO', 'wTime' and 'bTime' do not mean anything. I suppose these were typos.

Again I don't see general case-sensitivity discussed in the linked description, only an indication that option names are not case-sensitive.

Again it is self obvious. The very fact that the protocol insists that option names are not case sensitive implicitly means that everything is otherwise case sensitive. I don't know any Engine or GUI that uses, or even tolerates upper or mixed case UCI commands.

Incidentally, how many GUIs respond (as demanded) to "readyok" with "isready" at any time?

You're getting it the wrong way round: GUI sends isready, and Engine answers readyok (once it has completed whatever it was doing). All Engines should answer isready with readyok, otherwise they are defective, and I don't know any GUIs that will tolerate them.

[BB+]

* isready
        this is used to synchronize the engine with the GUI. When the GUI has sent a command or
        multiple commands that can take some time to complete,
        this command can be used to wait for the engine to be ready again or
        to ping the engine to find out if it is still alive.
        E.g. this should be sent after setting the path to the tablebases as this can take some time.

The standard here says that isready "can be used" (not "should", that only refers to the example of setting the tablebase path), and so this seems to be permissive rather than mandatory.

* isready
[...]
        This command is also required once before the engine is asked to do any search
        to wait for the engine to finish initializing.

Here it does say that "isready" is required, but my reading is that this refers to the first search the engine is asked to do after startup (the word "any" is unclear, but that's my guess), not every search, and the example given tends to agree with this.

Besides, it is self-obvious, so any GUI developper who has a half brain will do it that way. Not everything is explicitly stated in this document.

I disagree, a standard should be as descriptive as possible, with little left to interpretation. GUIs are written for different purposes, for instance if you want to test with 3 ply searches, worrying about back-and-forth overhead between engine/GUI might be a concern. Similarly on nontypical computing devices, with different interprocess communication characteristics (particularly context switching).

* How often do you see games taking that long ?

In self-play testing w/o TBs, I forget the statistics, but something like 5-10% of games last longer than 150 moves, 1-2% up to 200 moves, maybe 0.5% up to 250 moves (admittedly most of these are shuffling draws). Last year, TCEC [which is not self-play, so games tend to be shorter] had 16 games (of 358) over 300 ply, and 5 over 400.

Also, a clever GUI could decide to send instead 'position fen ... moves ...' and start from the fen of the last position when a pawn move or capture occured, in order to reduce the worst case scenario overhead.

Do you know of any such clever GUI? Some engines seem to have 1024 bytes as their input buffer size, so they crash at about 200 ply of a "moves" list. Anyway, one could certainly interpret the standard to say the GUI will give the move list from the start of the game.

* position [fen <fenstring> | startpos ]  moves <move1> .... <movei>
        set up the position described in fenstring on the internal board and
        play the moves on the internal chess board.
        if the game was played from the start position the string "startpos" will be sent [...]

[BB+]

Incidentally, how many GUIs respond (as demanded) to "readyok" with "isready" at any time?

You're getting it the wrong way round: GUI sends isready, and Engine answers readyok (once it has completed whatever it was doing). All Engines should answer isready with readyok, otherwise they are defective, and I don't know any GUIs that will tolerate them.

OK, I was parsing "It" in the readyok description grammatically to be the "readyok" command from engine->GUI, not the "isready" command from GUI->engine. You seem to take the opposite choice?

Engine to GUI:
* readyok
        This must be sent when the engine has received an "isready" command and has
        processed all input and is ready to accept new commands now.
        It is usually sent after a command that can take some time to be able to wait for the engine,
        but it can be used anytime, even when the engine is searching,
        and must always be answered with "isready".

So, who exactly is answering with "isready" here?! You say only the GUI sends isready, which would mean that this "answer" must be to the readyok from the engine, which is what I inquired about in the first place?

[lucasart]

* isready
        this is used to synchronize the engine with the GUI. When the GUI has sent a command or
        multiple commands that can take some time to complete,
        this command can be used to wait for the engine to be ready again or
        to ping the engine to find out if it is still alive.
        E.g. this should be sent after setting the path to the tablebases as this can take some time.

The standard here says that isready "can be used" (not "should", that only refers to the example of setting the tablebase path), and so this seems to be permissive rather than mandatory.

* isready
[...]
        This command is also required once before the engine is asked to do any search
        to wait for the engine to finish initializing.

Here it does say that "isready" is required, but my reading is that this refers to the first search the engine is asked to do after startup (the word "any" is unclear, but that's my guess), not every search, and the example given tends to agree with this.

Besides, it is self-obvious, so any GUI developper who has a half brain will do it that way. Not everything is explicitly stated in this document.

I disagree, a standard should be as descriptive as possible, with little left to interpretation. GUIs are written for different purposes, for instance if you want to test with 3 ply searches, worrying about back-and-forth overhead between engine/GUI might be a concern. Similarly on nontypical computing devices, with different interprocess communication characteristics (particularly context switching).

OK: In short, the standard should be more explicit. You're probably right.

Also, a clever GUI could decide to send instead 'position fen ... moves ...' and start from the fen of the last position when a pawn move or capture occured, in order to reduce the worst case scenario overhead.

Do you know of any such clever GUI? Some engines seem to have 1024 bytes as their input buffer size, so they crash at about 200 ply of a "moves" list. Anyway, one could certainly interpret the standard to say the GUI will give the move list from the start of the game.

* position [fen <fenstring> | startpos ]  moves <move1> .... <movei>
        set up the position described in fenstring on the internal board and
        play the moves on the internal chess board.
        if the game was played from the start position the string "startpos" will be sent [...]

[/quote]
The standard is just poorly written. The fact that fen and startpos are acceptable obviously overules this. Sending 'position fen ... moves ...' or 'position statpos moves ...' are both standard compliant. It's up to the GUI to decide. And yes, most GUIs do not bother to optimize and send 'position startpos moves ...' even when there are 500 half moves. We're talking sub millisecond here. Is it really worth the optimization ?

Regarding the 1024 byte limitation, that has nothing to do with the UCI protocol. It has all to do with the lazyness (or incompetence) of some amateur engine programmers. To solve this problem cleanly:

=> in C++

Code: Select all

getline(std::cin, somestring);

This apparently simple syntax hides a lot of complexity

=> in C
To avoid using a fixed size buffer like

Code: Select all

char buf[1024];
gets(buf);

I would recomment to write a tokenizer that uses getchar() and skips ' ', and '\n'. That way you call the tokenizer until the end of line is reached. And the token would have a fixed size buffer. So something like this:

Code: Select all

char token[8];
while (get_token(token)) {...}

And you can even do a get_token() function that is safe and prevents buffer overrun, in a similar fashion to strncpy() and the likes.

[bluefever]

UCI does seem simpler, but you can't use it play vs your engine in the console. Well, you could, I suppose, but typing out position startpos followed by the moves would get a bit tiresome!

imho, if you don't have "state" you don't have a chess game playing program, you have a "search for best move in a given position" program. The two are different.

regarding isready, I understood this to be the same as the ping in the Winboard protocol - a chance for the GUI to sync.

[H.G.Muller]

The point is that, state is the root of all evil. Imagine if the state is stored by both the GUI and the Engine. What if they disagree? How do they reconcile their state values? Imagine some nasty bugs, because the state maintained by your engine is wrong, and some bugs cannot be reproduced as a result...

All these idiotic made up problems, that typically arise in the Xboard protocol, do not even exist in the UCI world. That is why, the UCI protocol is always a much better choice than the Xboard one, especially for beginners who want to keep it simple.

Actually there is no practical difference between UCI and XBoard protocol, in this respect. UCI engines must keep track of the game state while they are parsing the position-move command move by move, and remember the state unil they receive the go command. If they mess up properly updating the state after encountering a move in their input, they end up in a different game state as the GUI thinks they are in, and you will get plenty of 'idiotic problems'. If engines can properly perform the moves fed to them on their internal board, there will never be any problems whatsoever. If they don't, they won't work no matter what protocol they use. In XBoard protocol this is usually much easier to debug, because there exists formal protocol for the engine to report the error condition (the Illegal Move command, pointing out the offending move), which are handled by the GUI, while in UCI you would have to improvise using home-grown debug output and log files of the GUI-engine communication to see them.

Beginners that want to keep it simple tend to prefer XBoard protocol, because they usually start by not using any formal protocol at all, and then converting their engine for running under a standard GUI only requires them to add parsing of the 'new' command, and printing 'move' in front of their output move. This is why nearly all engines use XBoard protocol.

As to UCI: 'isready' is of course never sent after the position-move command. The specs would allow it, but no GUI I know would do it. Setting up the position is not something 'expected to take time'. Neither would any GUI I know try to shorten the list of moves by sending a FEN in stead of startpos. That is only customary in the UCCI protocol.