named pipe and Linux

[Cameron Simpson]

On 07Apr2009 10:08, akineko <akineko at gmail.com> wrote:
| I'm trying to use named pipes to fuse a Python program and a C
| program.
| One side creates pipes using os.mkfifo() and both sides use the same
| named pipes (one side reads, another side writes). The read side uses
| select.select() to wait for incoming messages and read the message
| when select.select() says it is ready.
| The length of the message is unknown to the read side.

That's a serious flaw in the message protocol.

| I cannot use file.read() because it will block waiting for an EOF.
| I cannot use file.readline() because how many lines have arrived is
| unknown.
| So, I needed to use os.read() with the exact number of characters to
| read.

No!

You should use os.read() with the maximum size of a message.
It _should_ return with the number of bytes in the message, provided the
C program writes messages with a single OS-level write() call.

Forget all the fstat() stuff - it's racy.

Personally, I'd use a thread to just do continuous blocking os.read()s of
the pipe, and putting the resulting messages on a Queue for collection
by your main program. If you're the only consumer of a Queue it's safe
to poll it for emptiness or not, or to use a no-wait get().

All the above is untested, but absent a size in the protocol or other
ways of parsing message boundaries in data stream, you can only rely on
the C program writing messages with a single write() and collect using a
large os.read(), which should return with what is there.

Cheers,
-- 
Cameron Simpson <cs at zip.com.au> DoD#743
http://www.cskk.ezoshosting.com/cs/

Language... has created the word "loneliness" to express the pain of
being alone. And it has created the word "solitude" to express the glory
of being alone. - Paul Johannes Tillich