[IPython-dev] First Performance Result

[Brian Granger]

Min,

Thanks for this!  Sorry I have been so quiet, I have been sick for the last
few days.

On Thu, Jul 22, 2010 at 2:22 AM, MinRK <benjaminrk at gmail.com> wrote:

> I have the basic queue built into the controller, and a kernel embedded
> into the Engine, enough to make a simple performance test.
>
> I submitted 32k simple execute requests in a row (round robin to engines,
> explicit multiplexing), then timed the receipt of the results (tic each 1k).
> I did it once with 2 engines, once with 32. (still on a 2-core machine, all
> over tcp on loopback).
>
> Messages went out at an average of 5400 msgs/s, and the results came back
> at ~900 msgs/s.
> So that's 32k jobs submitted in 5.85s, and the last job completed and
> returned its result 43.24s  after the submission of the first one (37.30s
> for 32 engines). On average, a message is sent and received every 1.25 ms.
> When sending very small number of requests (1-10) in this way to just one
> engine, it gets closer to 1.75 ms round trip.
>
>
This is great!  For reference, what is your ping time on localhost?


> In all, it seems to be a good order of magnitude quicker than the Twisted
> implementation for these small messages.
>
>
That is what I would expect.


> Identifying the cost of json for small messages:
>
> Outgoing messages go at 9500/s if I use cPickle for serialization instead
> of json. Round trip to 1 engine for 32k messages: 35s. Round trip to 1
> engine for 32k messages with json: 53s.
>
> It would appear that json is contributing 50% to the overall run time.
>
>
Seems like we know what to do about json now, right?


> With %timeit x.loads(x.dumps(msg))
> on a basic message, I find that json is ~15x slower than cPickle.
> And by these crude estimates, with json, we spend about 35% of our time
> serializing, as opposed to just 2.5% with pickle.
>
> I attached a bar plot of the average replies per second over each 1000 msg
> block, overlaying numbers for 2 engines and for 32. I did the same comparing
> pickle and json for 1 and 2 engines.
>
> The messages are small, but a tiny amount of work is done in the kernel.
> The jobs were submitted like this:
>         for i in xrange(32e3/len(engines)):
>           for eid,key in engines.iteritems():
>             thesession.send(queue, "execute_request",
> dict(code='id=%i'%(int(eid)+i)),ident=str(key))
>
>
>

One thing that is *really* significant is that the requests per/second goes
up with 2 engines connected!  Not sure why this is the case by my guess is
that 0MQ does the queuing/networking in a separate thread and it is able to
overlap logic and communication.  This is wonderful and bodes well for us.

Cheers,

Brian




-- 
Brian E. Granger, Ph.D.
Assistant Professor of Physics
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu
ellisonbg at gmail.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20100725/2b414397/attachment.html>