[Python-Dev] Draft PEP to make file objects support non-blockingmode.

[Donovan Baarda]

G'day,

From: "Greg Ward" <gward at python.net>
> On 18 March 2005, Donovan Baarda said:
[...]
> > Currently the built in file type does not support non-blocking mode very
> > well.  Setting a file into non-blocking mode and reading or writing to
it
> > can only be done reliably by operating on the file.fileno() file
descriptor.
> > This requires using the fnctl and os module file descriptor manipulation
> > methods.
>
> Is having to use fcntl and os really so awful?  At least it requires
> the programmer to prove he knows what he's doing putting this file
> into non-blocking mode, and that he really wants to do it.  ;-)

It's not that bad I guess... but then I'm proposing a very minor change to
fix it.

The bit that annoys me is popen2() and select() give this false sense of
"File Object compatability", when in reality you can't use them reliably
with file objects.

It is also kind of disturbing that file.read() actually does work in
non-blocking mode, but file.write() doesn't. The source for file.read()
shows a fair bit of effort towards making it work for non-blocking mode...
why not do the same for file.write()?

> > Details
> > =======
> >
> > The documentation of file.read() warns; "Also note that when in
non-blocking
> > mode, less data than what was requested may be returned, even if no size
> > parameter was given".  An empty string is returned to indicate an EOF
> > condition.  It is possible that file.read() in non-blocking mode will
not
> > produce any data before EOF is reached.  Currently there is no
documented
> > way to identify the difference between reaching EOF and an empty
> > non-blocking read.
> >
> > The documented behaviour of file.write() in non-blocking mode is
undefined.
> > When writing to a file in non-blocking mode, it is possible that not all
of
> > the data gets written.  Currently there is no documented way of handling
or
> > indicating a partial write.
>
> That's more interesting and a better motivation for this PEP.

The other solution to this of course is to simply say "file.read() and
file.write() don't work in non-blocking mode", but that would be a step
backwards for the current file.read().

> > file.read([size]) Changes
> > --------------------------
> >
> > The read method's current behaviour needs to be documented, so its
actual
> > behaviour can be used to differentiate between an empty non-blocking
read,
> > and EOF.  This means recording that IOError(EAGAIN) is raised for an
empty
> > non-blocking read.
> >
> >
> > file.write(str) Changes
> > --------------------
> >
> > The write method needs to have a useful behaviour for partial
non-blocking
> > writes defined, implemented, and documented.  This includes returning
how
> > many bytes of "str" are successfully written, and raising
IOError(EAGAIN)
> > for an unsuccessful write (one that failed to write anything).
>
> Proposing semantic changes to file.read() and write() is bound to
> raise hackles.  One idea for soothing such objections: only make these
> changes active when setblocking(False) is in effect.  I.e., a
> setblocking(True) file (the default, right?) behaves as you described
> above, warts and all.  (So old code that uses fcntl() continues to
> "work" as before.)  But files that have had setblocking(False) called
> could gain these new semantics that you propose.

There is nothing in this proposal that would break or change the behaviour
of any existing code, unless it was relying on file.write() returning None.
or checking that file objects don't have a "setblocking" method.

Note that the change for file.read() is simply to document the current
behaviour... not to actually change it.

The change for file.write() is a little more dramatic, but I really can't
imagine anyone relying on file.write() returning None. A compromise would be
to have file.write() return None in blocking mode, and a count in
non-blocking mode... but I still can't believe people will rely on it
returning None :-) It would be more useful to always return a count, so that
methods using them could handle both modes easily.

Note that I did consider some more dramatic changes that would have made
them even easier to use. Things like raising an exception for EOF instead of
EAGAIN would actually make a lot of things easier to code... but it would be
too big a change.

----------------------------------------------------------------
Donovan Baarda                http://minkirri.apana.org.au/~abo/
----------------------------------------------------------------