Parallel insert to postgresql with thread

Scott David Daniels Scott.Daniels at Acm.Org
Thu Oct 25 08:28:00 EDT 2007


Diez B. Roggisch wrote:
> Abandoned wrote:
> 
>> Hi..
>> I use the threading module for the fast operation. But ....
[in each thread]
>> def save(a,b,c):
>>             cursor.execute("INSERT INTO ...
>>             conn.commit()
>>             cursor.execute(...)
>> How can i insert data to postgresql the same moment ?...
> 
> DB modules aren't necessarily thread-safe. Most of the times, a connection
> (and of course their cursor) can't be shared between threads.
> 
> So open a connection for each thread.

Note that your DB server will have to "serialize" your inserts, so
unless there is some other reason for the threads, a single thread
through a single connection to the DB is the way to go.  Of course
it may be clever enough to behave "as if" they are serialized, but
mostly of your work parallelizing at your end simply creates new
work at the DB server end.

-Scott David Daniels
Scott.Daniels at Acm.Org



More information about the Python-list mailing list