multithreading - managing transactions and sequence of processing

[Jaco Smuts]

Hello there

question: I'm looking for information / patterns / recipe's for 
implementing a multi threaded program (multiple producers / consumers) 
that will manage transactions and maintain sequence across the different 
threads.

background:
I'm busy writing an adapter taking messages from ibm websphere mq using 
pymqi and inserting these messages into a database (after some 
processing, the messages come in as xml). If I comment out the database 
.execute bit of the code, i process hundreds of messages per second, the 
database slows the whole process down to around 60 - 100 messages per 
second. I'm hoping to speed this up some by having 2 threads reading the 
messages from mq, and another populating the database - at least then 
the database bit will never have to wait for me to fetch another 
message. (i'm thinking of later adding a second thread for the databse 
as well).

The challenge is that these messages need to be processed in exact 
sequence and transactionally. (ie. the thread that fetched a message 
will have to wait for confirmation that the message has successfully 
been inserted into the database before committing the transaction, in mq 
terms the message will only really be deleted then.)

I've started implementing a kind of state machine using dictionary's to 
facilitate this, but I'm now concerned that
  - there is a better way
  - i might get bitten with dictionary processing updating / setting of 
values not necessarily being thread safe.

I've looked at this recipe 
http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/302997 as a 
possible replacement for dictionary's but I'm not even sure if I'm using 
it correctly.

thank you
jaco