Help using Thread (or other method)

[Marko Rauhamaa]

"Brendan Simon (eTRIX)" <brendan.simon at etrix.com.au>:

> Is using `select` here redundant?

Yes, if you go the thread way, select() is usually redundant and just
gets in the way.

> Does readline block until it sees an end-of-line char? i.e. does it
> only wake up the thread when it has a _complete string_ or a timeout
> of 0.1 seconds?

You probably don't want to mix 0.1-second timeouts with readline but let
the readline thread do its job: call readline in an indefinite wait. You
can then have Queue.get(timeout=...) do the timeout if necessary.

> or will the process block other threads from running while it is
> gathering the chars until the newline?

It shouldn't, but I didn't try it out.

> My application mainloop (not the threads) needs to be high priority
> and always process as soon as it gets an interrupt. Is using select a
> good way of doing this? How can I ensure that no other threads are
> utilizing the CPU, etc? I'm worried about the GIL (I'm using CPython
> 2.7).

Here's what you can do:

 * Have your main, high-priority thread have an input Queue coming in.
   Have the peripheral threads put their notifications in that Queue.

 * Have each peripheral thread have an acknowledgement threading.Event
   object. Whenever a peripheral thread has put a notification in the
   Queue, have it wait for the main thread to set() the acknowledgement
   event before looking for more work. That prevents the peripheral
   threads from starving the CPU under any circumstances.

> Is using select only in the mainloop, with multiple file descriptors, a
> better way of doing things, so that I can process the file descriptor of
> interest first, before any others if set?
>
> Is using the multiprocessing module a better option?

I don't know enough of your exact problem to recommend anything. Both of
those are viable options. However, readline() and select() don't mix
well at all because readline() does buffered I/O and select() deals with
raw file descriptors.

If you *can* reduce all functionality to file descriptor operations, use
os.read() and os.write() only. Then, you can use select.select() or
select.epoll() and not bother with threads at all. However, then you'll
have to implement HTTP processing on your own.

(The same goes for 3.5's new async/await scheme: https://docs.python.or
g/3/whatsnew/3.5.html?highlight=asyncio#pep-492-coroutines-with-async-a
nd-await-syntax>.)

Given the nature of your questions, you might want to stick with
threads. Your problem will then be how to shut down those threads in the
end...


Marko