Read from database, write to another database, simultaneously

Paul Boddie paul at boddie.org.uk
Fri Jan 12 05:42:06 EST 2007


Dennis Lee Bieber wrote:
>  The problem though, is that the original poster claimed the Oracle
> data was being emitted in multiple chunks, not as a single output -- and
> they want to avoid collecting the data in a temporary file...

I think he wanted to know whether concurrent reads from Oracle and
writes to PostgreSQL would be faster or more efficient. I'd argue that
the temporary file approach is likely to be faster and more efficient
(disk space usage for temporary files disregarded). Sure, if you can
have two concurrent processes with minimal contention, one reading from
Oracle, one writing to PostgreSQL, where the PostgreSQL one never lags
behind and thus completes shortly after the Oracle data has been
completely downloaded, then that might be the ideal solution, but I'm
inclined to believe that unless the stream of data from Oracle were
arriving really slowly, it wouldn't work out that way.

Anyway, that's what my experiences with PostgreSQL suggest so far.
Further commentary on the topic is, as noted, available in the manual.

Paul




More information about the Python-list mailing list