[IPython-dev] ZMQ Parallel IPython Performance preview

[MinRK]

I'll get on the new tests, I already have a bandwidth one written, so I'm
running it now.  As for Twisted's throughput performance, it's at least
partly our fault.  Since the receiving is in Python, every time we try to
send there are incoming results getting in the way.  If we wrote it such
that sending prevented the receipt of results, I'm sure the Twisted code
would be faster for large numbers of messages.  With ZMQ, though, we don't
have to be receiving in Python to get the results to the client process, so
they arrive in ZMQ and await simple memcpy/deserialization.

-MinRK

On Fri, Oct 22, 2010 at 09:27, Brian Granger <ellisonbg at gmail.com> wrote:

> Min,
>
> Also, can you get memory consumption numbers for the controller and queues.
>  I want to see how much worse Twisted is in that respect.
>
> Cheers,
>
> Brian
>
> On Thu, Oct 21, 2010 at 11:53 PM, MinRK <benjaminrk at gmail.com> wrote:
>
>> I have my first performance numbers for throughput with the new parallel
>> code riding on ZeroMQ, and results are fairly promising.  Roundtrip time for
>> ~512 tiny tasks submitted as fast as they can is ~100x faster than with
>> Twisted.
>>
>> As a throughput test, I submitted a flood of many very small tasks that
>> should take ~no time:
>> new-style:
>> def wait(t=0):
>>     import time
>>     time.sleep(t)
>> submit:
>> client.apply(wait, args=(t,))
>>
>> Twisted:
>> task = StringTask("import time; time.sleep(%f)"%t)
>> submit:
>> client.run(task)
>>
>> Flooding the queue with these tasks with t=0, and then waiting for the
>> results, I tracked two times:
>> Sent: the time from the first submit until the last submit returns
>> Roundtrip: the time from the first submit to getting the last result
>>
>> Plotting these times vs number of messages, we see some decent numbers:
>> * The pure ZMQ scheduler is fastest, 10-100 times faster than Twisted
>> roundtrip
>> * The Python scheduler is ~3x slower roundtrip than pure ZMQ, but no
>> penalty to the submission rate
>> * Twisted performance falls off very quickly as the number of tasks grows
>> * ZMQ performance is quite flat
>>
>> Legend:
>> zmq: the pure ZMQ Device is used for routing tasks
>> lru/weighted: the simplest/most complicated routing schemes respectively
>> in the Python ZMQ Scheduler (which supports dependencies)
>> twisted: the old IPython.kernel
>>
>> [image: roundtrip.png]
>> [image: sent.png]
>> Test system:
>> Core-i7 930, 4x2 cores (ht), 4-engine cluster all over tcp/loopback,
>> Ubuntu 10.04, Python 2.6.5
>>
>> -MinRK
>> http://github.com/minrk
>>
>
>
>
> --
> Brian E. Granger, Ph.D.
> Assistant Professor of Physics
> Cal Poly State University, San Luis Obispo
> bgranger at calpoly.edu
> ellisonbg at gmail.com
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20101022/67a1c1e7/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: roundtrip.png
Type: image/png
Size: 30731 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20101022/67a1c1e7/attachment.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: sent.png
Type: image/png
Size: 31114 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20101022/67a1c1e7/attachment-0001.png>