How do I flush a pipe ?

[Matthias Gemuh]

From CCC:

One last trial to explain it.

Windows offers only WriteFile() to write into a pipe.
(Obselete write functions do exist that work like WriteFile())
WriteFile() has an internal buffer that makes WriteFile() non-blocking.
Windows flushes that buffer depending on how full it is or how long data has been standing in it. This efficiency decision effectively steals time from any engine waiting at the other end of the pipe.
The internal WriteFile() buffer can be forcefully flushed with FlushFileBuffers() to eliminate delays, but that is blocking

You can accomplish read()/write() effect using FILE_FLAG_NO_BUFFERING and FILE_FLAG_WRITE_THROUGH flags with WriteFile().

I know that those flags can be used in CreateFile() to obtain a handle that WriteFile() can use in a non-blocking manner to write unbuffered into a file on disk. But how can you use that handle to write into a pipe ?

[kingliveson]

From CCC:

One last trial to explain it.

Windows offers only WriteFile() to write into a pipe.
(Obselete write functions do exist that work like WriteFile())
WriteFile() has an internal buffer that makes WriteFile() non-blocking.
Windows flushes that buffer depending on how full it is or how long data has been standing in it. This efficiency decision effectively steals time from any engine waiting at the other end of the pipe.
The internal WriteFile() buffer can be forcefully flushed with FlushFileBuffers() to eliminate delays, but that is blocking

You can accomplish read()/write() effect using FILE_FLAG_NO_BUFFERING and FILE_FLAG_WRITE_THROUGH flags with WriteFile().

I know that those flags can be used in CreateFile() to obtain a handle that WriteFile() can use in a non-blocking manner to write unbuffered into a file on disk. But how can you use that handle to write into a pipe ?

Why can't you use the CreatePipe() handle and WriteFile()?

[Matthias Gemuh]

From CCC:

One last trial to explain it.

Windows offers only WriteFile() to write into a pipe.
(Obselete write functions do exist that work like WriteFile())
WriteFile() has an internal buffer that makes WriteFile() non-blocking.
Windows flushes that buffer depending on how full it is or how long data has been standing in it. This efficiency decision effectively steals time from any engine waiting at the other end of the pipe.
The internal WriteFile() buffer can be forcefully flushed with FlushFileBuffers() to eliminate delays, but that is blocking

You can accomplish read()/write() effect using FILE_FLAG_NO_BUFFERING and FILE_FLAG_WRITE_THROUGH flags with WriteFile().

I know that those flags can be used in CreateFile() to obtain a handle that WriteFile() can use in a non-blocking manner to write unbuffered into a file on disk. But how can you use that handle to write into a pipe ?

Why can't you use the CreatePipe() handle and WriteFile()?

I am actually using CreatePipe() and WriteFile(), but the handle from CreatePipe() does not seem to have the non-buffering attributes associate with FILE_FLAG_NO_BUFFERING and FILE_FLAG_WRITE_THROUGH.
Buffering seems to be occuring somewhere (but noticeable only under Win64).

[kingliveson]

From CCC:

One last trial to explain it.

Windows offers only WriteFile() to write into a pipe.
(Obselete write functions do exist that work like WriteFile())
WriteFile() has an internal buffer that makes WriteFile() non-blocking.
Windows flushes that buffer depending on how full it is or how long data has been standing in it. This efficiency decision effectively steals time from any engine waiting at the other end of the pipe.
The internal WriteFile() buffer can be forcefully flushed with FlushFileBuffers() to eliminate delays, but that is blocking

You can accomplish read()/write() effect using FILE_FLAG_NO_BUFFERING and FILE_FLAG_WRITE_THROUGH flags with WriteFile().

I know that those flags can be used in CreateFile() to obtain a handle that WriteFile() can use in a non-blocking manner to write unbuffered into a file on disk. But how can you use that handle to write into a pipe ?

Why can't you use the CreatePipe() handle and WriteFile()?

I am actually using CreatePipe() and WriteFile(), but the handle from CreatePipe() does not seem to have the non-buffering attributes associate with FILE_FLAG_NO_BUFFERING and FILE_FLAG_WRITE_THROUGH.
Buffering seems to be occuring somewhere (but noticeable only under Win64).

"Remarks

CreatePipe creates the pipe, assigning the specified pipe size to the storage buffer. CreatePipe also creates handles that the process uses to read from and write to the buffer in subsequent calls to the ReadFile and WriteFile functions.

To read from the pipe, a process uses the read handle in a call to the ReadFile function. ReadFile returns when one of the following is true: a write operation completes on the write end of the pipe, the number of bytes requested has been read, or an error occurs.

When a process uses WriteFile to write to an anonymous pipe, the write operation is not completed until all bytes are written. If the pipe buffer is full before all bytes are written, WriteFile does not return until another process or thread uses ReadFile to make more buffer space available."

Looks like we are back to fflush() again:

"(Follows my previous post)

With some experiments, I found the output is buffered at child side's STDOUT with every 4096 bytes. So, I add a "fflush(stdout)" after printf(), the output redirect to parent without problem then.

BreadPan 3/26/2009"

An alternative:
http://msdn.microsoft.com/en-us/library ... 71%29.aspx

[Matthias Gemuh]

...

When a process uses WriteFile to write to an anonymous pipe, the write operation is not completed until all bytes are written. If the pipe buffer is full before all bytes are written, WriteFile does not return until another process or thread uses ReadFile to make more buffer space available."[/i]

Looks like we are back to fflush() again:

"(Follows my previous post)

With some experiments, I found the output is buffered at child side's STDOUT with every 4096 bytes. So, I add a "fflush(stdout)" after printf(), the output redirect to parent without problem then.

BreadPan 3/26/2009"

An alternative:
http://msdn.microsoft.com/en-us/library ... 71%29.aspx

ChessGUI does not pump so much data into the pipe that WriteFile() fails to return immediately.
The problem seems to be that the data WriteFile() writes seems to hang around in some mysterious internal buffer before Windows forwards it into the pipe. That hanging around is the unwanted delay.
"Typically the WriteFile and WriteFileEx functions write data to an internal buffer that the operating system writes to a disk or communication pipe on a regular basis. "

I indeed thought of creating pipes with _pipe(), but refrained because I would also have to spawn() the child process.
I don't know why I prefer CreateProcess() to spawn().

CreatePipe()/CreateProcess() and _pipe()/spawn() go in pairs, right ?

I even see a bunch of issues if I spawn() chess engines.

Anyway, there is a way I can relax timing in ChessGUI and be perfectly strict only optionally.
Maybe I should simply do that.

Matthias.

[Monju Athanasius]

Hello Matthias,

Greetings, Sorry that I am not writing for the topic at stake. I am rather writing to comfort myself if you are Matthias Gemuh, origin from Babanki in Cameroon. My High school friend. If you are then I am indeed grateful to have traced you in to this forum. My name again is Monju Athanasius. I look forward to reading from you if you are the one I have been searching for, for many years. If you are not, then please, I am sorry for intruding.

Stay Blessed.
Monju Athanasius.