[Chicago] GIS Python and threads

[Robert Spelich]

I work with GIS data and write Python scripts to extract data from very
large datasets. I have a particular case where it is necessary to query on
the data and create a separate file object base on the queried data. For
reasons too difficult to explain, this has to be done over 800 times, once
for each row in a different dataset. The API that is provided by the GIS
application vendor is very good but the processing of these scripts is very
slow. I estimate 30s to a minute each, so the run takes hours. Each
execution of the script is independent of the other so I was wondering if I
should use threads. I read up on threads and was like, OK my head hurts now,
but if this will speed things up I will take a crack at it.



Is this a good case for threads? Also are they manageable, I don’t want 800
threads firing at once and crashing the machine.
Thank you.
-- 
-Bob
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/chicago/attachments/20100128/f700cfba/attachment.htm>