Threading questions

[Magnus Lycka]

Antal Rutz wrote:
> Hi, All!
> 
> I'm new to threading. I have some design questions:
> Task: I collect data and store them in an RDBMS (mysql or pgsql)
> 
> The question is how to do that with threading?
> The data-collecting piece of the code runs in a thread.
> 
> 1. Open the db, and each thread writes the result immediately.
>    (Sub-question: which is better: cursor object passed to the thread
>    or stored in a global var.)
> 2. Threads return data, which is written to DB after that.
> 3. Threads write to global variable, after 'join()' data is written.
>    (Can be used global (write-only) variables with threads at all?)
> 4. ?...

I'm not sure why you need threading at all here, but I would avoid
both globals and passing cursors between threads. I don't think all
DB-API implementations can handle threading properly.

I suspect it might be useful to do data collection in one thread,
DB communication in another, and to use a Queue.Queue object to
pass data from data collection to DB insertion thread.