Tips on Speeding up Python Execution

[Abhijeet Mahagaonkar]

Dear Pythoners,
I have written a python application which authenticates a user, reads a
webpage and searches for pattern and builds a database ( In my case its a
dictinary with fixed set of keys).
Inputting the username and password for authentication and final display of
the results is done by GUI.
I was able to isolate that major chunk of run time is eaten up in opening a
webpages, reading from them and extracting text.
I wanted to know if there is a way to concurrently calling the functions.

here is my pseudo code:

database=[]
in a while loop:
   build the url by concatenating the query parametres
   temp=urlopen(url).read()
   dict['Key1']=get_key1_data(temp) ## passing the entire string obtained by
.read()
   dict['key2']=get_key2_data(temp)
   .
   .
   .
   dict['keyn']=get_keyn_data(temp)
   database=database+[dict]   ## building an array of dictionaries

My question here is can I have the functions get_key1_data to get_keyn_data
run in some concurrent way.??
I ask this because these functions are not dependent on one another. They
all have access to the same parsed url and independently pull data in order
to populate the final database

Appreciating your help in this one.

Warm Regards,
Abhijeet.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20110408/1a9205a3/attachment.html>