named pipe and Linux

[Thomas Bellman]

Cameron Simpson <cs at zip.com.au> wrote:

> On 07Apr2009 10:08, akineko <akineko at gmail.com> wrote:
>| I'm trying to use named pipes to fuse a Python program and a C
>| program.
>| One side creates pipes using os.mkfifo() and both sides use the same
>| named pipes (one side reads, another side writes). The read side uses
>| select.select() to wait for incoming messages and read the message
>| when select.select() says it is ready.
>| The length of the message is unknown to the read side.

> That's a serious flaw in the message protocol.

>| I cannot use file.read() because it will block waiting for an EOF.
>| I cannot use file.readline() because how many lines have arrived is
>| unknown.
>| So, I needed to use os.read() with the exact number of characters to
>| read.

> No!

> You should use os.read() with the maximum size of a message.
> It _should_ return with the number of bytes in the message, provided the
> C program writes messages with a single OS-level write() call.

No!

That's still broken.  You can't know if the writer has managed to
write one or several messages onto the pipe.  And if the messages
are large, you aren't guaranteed that the OS won't split up the
data into multiple segments that only become available to the
reader one or a few at a time.

You *need* to use a protocol where it is possible to determine
the message boundaries.  You can do that by:

- Using messages of a fixed size.

- Terminate each message with an octet sequence that cannot occur
  within a message.  For example, a linefeed without a backslash
  before it (and you would probably want a way to escape the
  backslash, in case you want to end a message with a backslash).

- Have small header of a fixed size at the start of each message,
  that includes the length of the message in octets.


-- 
Thomas Bellman,   Lysator Computer Club,   Linköping University,  Sweden
"Life IS pain, highness.  Anyone who tells   !  bellman @ lysator.liu.se
 differently is selling something."          !  Make Love -- Nicht Wahr!