[Python-Dev] Python parallel benchmark

[Jesse Noller]

Do you have the code posted someplace for this? I'd like to add it  
into the tests I am running

On May 15, 2008, at 11:56 AM, Tom Pinckney <thomaspinckney3 at gmail.com>  
wrote:

> All the discussion recently about pyprocessing got me interested in  
> actually benchmarking Python's multiprocessing performance to see if  
> reality matched my expectations around what would scale up and what  
> would not. I knew Python threads wouldn't be good for compute bound  
> problems, but I was curious to see how well they worked for i/o  
> bound problems. The short answer is that for i/o bound problems,  
> python threads worked just as well as using multiple operating  
> system processes.
>
> I wrote two simple benchmarks, one compute bound and the other i/o  
> bound. The compute bound one did a parallel matrix multiply and the  
> i/o bound one read random records from a remote MySQL database. I  
> ran each benchmark via python's thread module and via MPI (using  
> mpi4py and openmpi and Send()/Recv() for communication). Each test  
> was run multiple times and the numbers were consistent between test  
> runs. I ran the tests on a dual-core Macbook Pro running OS X 10.5  
> and the included python 2.5.1.
>
> 1) Python threads
>
> a) compute bound:
>
> 1 thread - 16 seconds
> 2 threads - 21 seconds
>
> b) i/o bound:
>
> 1 thread -- 13 seconds
> 4 threads -- 10 seconds
> 8 threads -- 5 seconds
> 12 threads - 4 seconds
>
> 2) MPI
>
> a) compute bound:
>
> 1 thread - 17 seconds
> 2 threads -- 11 seconds
>
> b) i/o bound
>
> 1 thread -- 13 seconds
> 4 threads -- 10 seconds
> 8 threads -- 6 seconds
> 12 threads -- 4 seconds
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/jnoller%40gmail.com