Parallel insert to postgresql with thread

Diez B. Roggisch deets at nospam.web.de
Thu Oct 25 07:27:40 EDT 2007


Abandoned wrote:

> Hi..
> I use the threading module for the fast operation. But i have some
> problems..
> This is my code sample:
> =================
> conn =
> psycopg2.connect(user='postgres',password='postgres',database='postgres')
> cursor = conn.cursor()
> class paralel(Thread):
>     def __init__ (self, veriler, sayii):
>         Thread.__init__(self)
>     def run(self):
>          save(a, b, c)
> 
> def save(a,b,c):
>             cursor.execute("INSERT INTO keywords (keyword) VALUES
> ('%s')" % a)
>             conn.commit()
>             cursor.execute("SELECT
> CURRVAL('keywords_keyword_id_seq')")
>             idd=cursor.fetchall()
>             return idd[0][0]
> 
> def start(hiz):
>     datas=[........]
>     for a in datas:
>         current = paralel(a, sayii)
>         current.start()
> ==================
> And it gives me different errors to try parallel insert. My querys
> work in normal operation but in paralel don't work.
> How can i insert data to postgresql the same moment ?
> errors:
> no results to fetch
> cursor already closed

DB modules aren't necessarily thread-safe. Most of the times, a connection
(and of course their cursor) can't be shared between threads.

So open a connection for each thread.

Diez



More information about the Python-list mailing list