parallel programming in Python

[Werner Thie]

For such tasks my choice would be twisted combined with ampoule.
Let's you spread out work to whatever amount of processes you desire, 
maxing out whatever iron you're sitting on..

HTH, Werner

http://twistedmatrix.com/trac/
https://launchpad.net/ampoule

On 29.05.2012 16:43, Jabba Laci wrote:
> Hehe, I just asked this question a few days ago but I didn't become
> much cleverer:
>
> http://www.gossamer-threads.com/lists/python/python/985701
>
> Best,
>
> Laszlo
>
> On Thu, May 10, 2012 at 2:14 PM, Jabba Laci<jabba.laci at gmail.com>  wrote:
>> Hi,
>>
>> I would like to do some parallel programming with Python but I don't
>> know how to start. There are several ways to go but I don't know what
>> the differences are between them: threads, multiprocessing, gevent,
>> etc.
>>
>> I want to use a single machine with several cores. I want to solve
>> problems like this: iterate over a loop (with millions of steps) and
>> do some work at each step. The steps are independent, so here I would
>> like to process several steps in parallel. I want to store the results
>> in a global list (which should be "synchronised"). Typical use case:
>> crawl webpages, extract images and collect the images in a list.
>>
>> What's the best way?
>>
>> Thanks,
>>
>> Laszlo
-------------- next part --------------
A non-text attachment was scrubbed...
Name: werner.vcf
Type: text/x-vcard
Size: 297 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-list/attachments/20120529/f9f3e085/attachment.vcf>