non blocking read()

[Steve Holden]

Donn Cave wrote:
> In article <mailman.6995.1101939055.5135.python-list at python.org>,
>  Gustavo Córdova Avila <gustavo.cordova at q-voz.com> wrote:
> 
> 
>>David Bolen wrote:
>>
>>
>>>Jp Calderone <exarkun at divmod.com> writes:
>>>
>>>
>>>>   def nonBlockingReadAll(fileObj):
>>>>       bytes = []
>>>>       while True:
>>>>           b = fileObj.read(1024)
>>>>           bytes.append(b)
>>>>           if len(b) < 1024:
>>>>               break
>>>>       return ''.join(bytes)
>>>>
>>>
>>>Wouldn't this still block if the input just happened to end at a
>>>multiple of the read size (1024)?
>>>
>>>-- David
>>>
>>
>>No, it'll read up to 1024 bytes or as much as it can, and
>>then return an apropriatly sized string.
> 
> 
> Depends.  I don't believe the original post mentioned
> that the file is a pipe, socket or similar, but it's

It did actually specifically mention files.

> kind of implied by the use of select() also mentioned.
> It's also kind of implied by use of the term "block" -
> disk files don't block.
> 
Erm, you mean they operate at infinite speed? Wrong. There's a definite 
risk that a process will block when reading from disk, and the reason 
that (Unix) files can be the object of select() calls is to allow users 
to perform non-blocking reads and continue to process while waiting for 
disk data to arrive.

I agree the blocking period is *more predictable* than for network data, 
but "disk file access won't cause a process to block" is a misleading 
and incorrect, though frequent, assumption.

I agree that it's difficult to actually write asynchronous disk I/O in 
such a way as to improve performance, but it can be done. Except on 
platforms where select() can't be applied to disk files, of course. In 
practice most applications that need the scalability will use threading 
or multiple processes.

> If we are indeed talking about a pipe or something that
> really can block, and you call fileobject.read(1024),
> it will block until it gets 1024 bytes.
> 
>    Donn Cave, donn at u.washington.edu

Disk access can block. When access is made to a file object in 
non-blocking mode it will return whatever data there are immediately 
available in the buffers, up to the number of bytes requested. If the 
buffers are currently empty the process will generate an error (which in 
Python becomes an exception).

regards
  Steve
-- 
http://www.holdenweb.com
http://pydish.holdenweb.com
Holden Web LLC +1 800 494 3119