non-blocking sockets

[James J. Besemer]

so very tired wrote:

> When trying to read from a socket, I don't want it hanging, so I set it to
> non-blocking by calling
>
> setblocking(0)
>
> but then when I try to read from it and there is no data in the socket, I
> get an error. I tried this in both windows and linux and in both I get an
> error and the program stops. It says there's an exception but it doesn't
> say the name of the exception so I can't even try to catch.
> Does anyone know how I can check if there's data coming from a socket
> connection without hanging or having the program barf on me?

I suspect you need to use "select" to query whether a given socket is ready
for reading or writing.  It has its own section in the library doc.  In
general it waits on multiple sockets and/or file descriptors and tells you
which ones are "ready".  (However, it does not work with anything EXCEPT
sockets on Windoz.)  Select also has a delay parameter so it will return after
an interval even if nothing is happening, so you program can pretend it's not
dead.

A newer alternative is "Polling Objects," in the section immediately following
"select".  I never used it but it appears marginally better than "select".

However, I personally find it MUCH easier to use blocking I/O in conjunction
with threads, rather than the older, historical techniques.  And since Python
threads are available in Windows and most Unix dialects (Linux, anyway),
there's little reason not to use them.  Furthermore, I find coding it out long
hand with threads to be lots easier to figuring out how to adapt library code,
such as "asyncore" or "SocketServer".  The result also is much clearer to
understand.

E.g., below is something I was working on this weekend (undefined variables
are left to the reader's imagination):

    import os, sys
    import socket, thread

    # first thread listens for incoming connects
    # and then creates a new thread to handle each one

    def Listener():
        s = socket.socket( socket.AF_INET, socket.SOCK_STREAM )
        s.bind( HOST, PORT )
        while 1:
            s.listen(3)
            conn, addr = s.accept()
            thread.start_new_thread( Reader, ( conn, addr ))

    # independent reader thread, processes incoming data as it arrives

    def Reader( conn, addr ):
        while 1:
            data = conn.recv( 1024 )    # blocking!
            if not data:
                break
            process( conn, data )
        conn.close()

    # main program launches threads and
    # loops, waiting to handle incoming SIGNALS

    def main():
        thread.start_new_thread( Listener, ())
        thread.start_new_thread( Reader, ())

        # signal handling code omitted

        while 1:
            time.sleep( 5 )

If your protocol is simple enough, 'process()' may result in sends on the
socket.  If the protocol is complex then you may need separate threads for
reading and for writing.

The one hitch is where you have different threads accessing common data then
you need to serialize access via LOCKS.  This is a problem you don't have with
a single, blocking socket, or with multiple sockets tested via "select".

    aLock = thread.allocate_lock()
    ...
    aLock.acquire()
    access_shared_resource()
    aLock.release()

Also, the Queue data structure is handy for queuing data between multiple
threads and if you poke around, there are other tools for concurrent
programming.

Lemme know if this helps.

Regards

--jb

--
James J. Besemer  503-280-0838 voice
http://cascade-sys.com  503-280-0375 fax
mailto:jb at cascade-sys.com