Problem with blocking portably on sockets and Queue?

[Tero Saarni]

"Alex Martelli" <aleax at aleax.it> wrote in message
news:9P9Ya.42349$cl3.1347211 at news2.tin.it...
> <posted & mailed>
>
> Tero Saarni wrote:
>
> > Hi,
> >
> > I have several threads communicating with each other using events
> > stored in Queues. Threads block on Queue.get() until somebody
> > publishes an event in thread's event queue.
>
> So far so wonderful -- excellent architecture.
>
>
> > I need to add support for sockets to the system. Thread needs to
> > unblock when:
> >
> >   - there is socket ready to be read, or
> >   - there is event waiting in the queue
> >
> > My first tought was to replace blocking on Queue.get() with blocking
> > on poll or select and dedicating file descriptors (created with
os.pipe())
> > as an semaphore. Event publisher would write something to the write
> > end of the pipe when it puts an event to the queue, effectively
> > unblocking the receiver.
> >
> > BUT I noticed that select.poll() is not available on Windows and
> > secondly Windows version of select.select() will accept only socket
> > descriptors.
> >
> > What options do I have that are still portable also to Windows
> > platform?
>
> I think you could devote a thread specifically to the task of handling
> sockets, only.  That special thread would not block on any queue but
> just on select; when its select shows that a socket is ready, you can
> have the thread itself do the reading and post the appropriate special
> message including the data read to the appropriate thread, or you could
> pass the ready-for-reading socket object to the appropriate thread, or
> the like.  This needs a bit more care if the set of sockets to be
> select'ed on changes with time; in this case, I think the socket-handler
> thread would have to use a timeout on its select so the set of sockets
> can be updated once in a while (either by another thread, in which
> case you need a lock to protect the "set of sockets" shared data, or
> by the socket-handling thread (SHT) itself in response to a queued
> message -- the SHT would read non-blockingly from that queue periodically
> when the select times out) -- or you could perhaps use a special
> identifiable socket for the SHT to receive from other threads requests
> to change the set of ('true') sockets to be selected on.
>
>
> Alex
>

Good idea, thanks! I will try dedicating a special thread for socket
handling, although breaking out of select() for modifying the set of
sockets gets a little tricky as you pointed out.

Momentarily I was even thinking that I could break out of select() by
sending a signal when I want to modify the set of sockets, but
as I was almost expecting, it appeared that also signal support is
quite limited on Windows platform.

I guess that fallback solution would be to learn Windows
sychronization primitives (wait functions etc) and write two separate
implementations of relevant parts of the system. With some work
that could probably be hidden behind matching interfaces and a
factory method choosing the implementation on the fly.

---
Tero