Read from database, write to another database, simultaneously

[Bjoern Schliessmann]

Sean Davis wrote:

> The author of one of the python database clients mentioned that
> using one thread to retrieve the data from the oracle database and
> another to insert the data into postgresql with something like a
> pipe between the two threads might make sense, keeping both IO
> streams busy.

IMHO he's wrong. Network interaction is quite slow compared with CPU
performance, so there's no gain (maybe even overhead due to thread
management and locking stuff). That's true even on multiprocessor
machines, not only because there's almost nothing to compute but
only IO traffic. CMIIW.

Using multiplexing, you'll get good results with simple code without
the danger of deadlocks. Have a look at asyncore (standard library)
or the Twisted framework -- personally, I prefer the latter.

Regards,


Björn

-- 
BOFH excuse #194:

We only support a 1200 bps connection.