How to get number of bytes written to nonblocking FIFO when EAGAIN is raised?

[Aaron Staley]

On Jul 19, 8:15 pm, Adam Skutt <ask... at gmail.com> wrote:
> On Jul 19, 9:19 pm, Aaron Staley <usaa... at gmail.com> wrote:
>
> > However, if interpreter 1 overfills the FIFO, we get an error (EAGAIN)>>> f.write('a'*70000)
>
> > IOError: [Errno 11] Resource temporarily unavailable
>
> > However interpreter 2 still receives data>> len(f.read())
>
> > 65536
>
> > It looks like interpreter 1 pushed data until the FIFO was full and
> > then raised the IOError.  Interpreter 2 constantly received some, but
> > not all, of what interpreter 2 tried to send.
> > Unfortunately, the IOError seems to have no attribute indicating how
> > much data was successfully sent.  I've looked through the docs and
> > can't seem to figure out how; can anyone land some advice?
>
> You need to do as Roy Smith suggested and use the actual OS I/O calls
> os.read() and os.write().  os.write() returns the number of bytes
> actually written to the underlying descriptor.  Python file objects
> are akin to FILE structures in C: they perform buffering and other
> operations internally that makes them less than suitable for usage
> with asynchronous UNIX I/O. In particular, they buffer I/O internally
> before writing it to the descriptor, so there's no direct relationship
> between calling file.write() and how much data is written to the
> stream. In addition, file objects also simply raise the underlying OS
> error when it occurs.  The UNIX write(2) syscall assumes that you have
> been keeping track of how many bytes you've successfully written to
> the stream and does not track it for you.
>
> Adam

That's for the info; lower level I/O solved the issue.
That said, is such behavior with the file objects intended? It seems
they don't work correctly with an underlying non-blocking file
descriptor, but the documentation doesn't state such. In fact, it
suggests you can use non-blocking with this comment for file.read:

Also note that when in non-blocking mode, less data than was requested
may be returned, even if no size parameter was given.