To Thread or not to Thread....?

[Antoine Pitrou]

On Wed, 1 Dec 2010 02:45:50 +0000
Jack Keegan <whatsjacksemail at gmail.com> wrote:

> Hi there,
> 
> I'm currently writing an application to control and take measurements during
> an experiments. This is to be done on an embedded computer running XPe so I
> am happy to have python available, although I am pretty new to it.
> The application basically runs as a state machine, which transitions through
> it's states based on inputs read in from a set of general purpose
> input/output (GPIO) lines. So when a certain line is pulled low/high, do
> something and move to another state. All good so far and since I get through
> main loop pretty quickly, I can just do a read of the GPIO lines on each
> pass through the loop and respond accordingly.

> However, in one of the states I have to start reading in, and storing frames
> from a camera. In another, I have to start reading accelerometer data from
> an I2C bus (which operates at 400kHz). I haven't implemented either yet but
> I would imagine that, in the case of the camera data, reading a frame would
> take a large amount of time as compared to other operations. Therefore, if I
> just tried to read one (or one set of) frames on each pass through the loop
> then I would hold up the rest of the application. Conversely, as the I2C bus
> will need to be read at such a high rate, I may not be able to get the
> required data rate I need even without the camera data. This naturally leads
> me to think I need to use threads.
> As I am no expert in either I2C, cameras, python or threading I thought I
> would chance asking for some advice on the subject. Do you think I need
> threads here or would I be better off using some other method. I was
> previously toying with the idea of using generators to create weightless
> threads (as detailed in
> http://www.ibm.com/developerworks/library/l-pythrd.html) for reading the
> GPIOs. Do you think this would work in this situation?

The main question IMO: the I2C bus operates at 400kHz, but how much
received data can it buffer? That will give you a hint as to how much
latency you can tolerate.

I don't think soft threads would work at all, since they wouldn't allow
overlapping between frame reading and other ops. If frame reading
releases the GIL (as any properly implemented IO primitive in Python
should), then at least you can try using OS threads.

Then, depending on the tolerable latency for I2C operation, you can try
to run it as an OS thread, or a separate process (if running as a
separate process, make sure it cannot block while sending IO to the
master process).

Regards

Antoine.