Can read() be non-blocking?

[Karthik Gurusamy]

On Nov 6, 2:54 pm, Thomas Christensen <thom... at thomaschristensen.org>
wrote:
> This issue has been raised a couple of times I am sure.  But I have yet
> to find a satisfying answer.
>
> I am reading from a subprocess and this subprocess sometimes hang, in
> which case a call to read() call will block indefinite, keeping me from
> killing it.
>
> The folloing sample code illustrates the problem:
>
>   proc = subprocess.Popen(['/usr/bin/foo', '/path/to/some/file'],
>                           stdout=subprocess.PIPE)
>   output = StringIO.StringIO()
>   while True:
>       r = select.select([proc.stdout.fileno()], [], [], 5)[0]
>       if r:
>           # NOTE: This will block since it reads until EOF
>           data = proc.stdout.read()
>           if not data:
>               break  # EOF from process has been reached
>           else:
>               output.write(data)
>       else:
>           os.kill(proc.pid, signal.SIGKILL)
>   proc.wait()
>
>   <Process the output...>
>
> As the NOTE: comment above suggests the call to read() will block here.
>
> I see two solutions:
>
> 1. Read one byte at a time, meaning call read(1).
> 2. Read non-blocking.
>
> I think reading one byte at a time is a waste of CPU, but I cannot find
> a way to read non-blocking.
>
> Is there a way to read non-blocking?  Or maybe event a better way in
> generel to handle this situation?

>From what I understand, you want a way to abort waiting on a blocking
read if the process is hung.
There are some challenges about how you decide if the process is hung
or just busy doing work without generating output for a while (or may
be the system is busy and the process didn't get enough CPU due to
other CPU hungry processes).
Assuming you have a valid way to figure this out, one option is to
have a timeout on the read.
If the timeout exceeds, you abort the read call. No, the read doesn't
provide a timeout, you can build one using alarm.

def alarm_handler(*args):
      """ This signal stuff may not work in non unix env """
      raise Exception("timeout")

signal.signal(signal.SIGALRM, alarm_handler)

try:
    signal.alarm(timeout)   # say timeout=60 for a max wait of 1
minute
    data = proc.stdout.read()
except Exception, e:
    if not str(e) == 'timeout':  # something else went wrong ..
        raise
     # got the timeout exception from alarm .. proc is hung; kill it

Karthik

>
> Thanks
>
>                 Thomas