[pypy-dev] Pypy Benchmark of Tornado.
Phyo Arkar
phyo.arkarlwin at gmail.com
Fri Sep 5 21:57:23 CEST 2014
Yes This really deserve a blog post . I haven't finished writing my
own blog yet tho :D.
On Fri, Sep 5, 2014 at 11:31 PM, Maciej Fijalkowski <fijall at gmail.com> wrote:
> Hi
>
> Can you please put it all in a blog post (ideally with examples), it's
> a lot of useful info.
>
> I'm willing to help
>
> On Fri, Sep 5, 2014 at 10:47 AM, Jolitz Ben - bjolit
> <Ben.Jolitz at acxiom.com> wrote:
>> I don’t have specific suggestions on Mongo, but I can share what I’ve
>> learned in a few months of using PyPy and Tornado.
>>
>> You want to make use of CFFI in PyPy to accelerate operations that would
>> usually be slow in Python, namely encryption and database drivers. But
>> always test first to see if you really need to go to C.
>>
>> /Any/ CPython C-Extensions will torpedo performance. Anything blocking (like
>> the majority of DB drivers) will similarly destroy Tornado performance.
>>
>> Code that is overly dynamic also does lousy on PyPy. If your driver has a
>> ton of paths or makes idiot use of threading.Lock, expect to have an uphill
>> struggle in optimization.
>>
>> When in doubt, ask yourself if the algorithm is appropriate.
>>
>> If you can’t make the Python driver performant and there exists a C-API for
>> it, then it is trivial to wrap it with CFFI.
>>
>> If it doesn’t support nonblocking operations, you can find alternatives. For
>> example in MySQLdb, others have found you can add the Connection._fd to the
>> IOLoop and use it to do a send_query, read_query. If you still can’t find an
>> alternative to a blocking call, you can still mimic nonblocking IO by using
>> pthreads, a work queue and a callback pthread. It’s not perfect and theres a
>> lot you can optimize, but it can easily allow you to delegate long running
>> operations to C.
>>
>> Another thing I learned was to avoid generating Python-side c callback
>> pointers frequently. If C is going to callback into Python with your result
>> to a unique Request, you’re going to need to tag it appropriately. I prefer
>> to use dictionaries and the callback attribute handed to me by gen.Task and
>> pass a unique-enough key to C to callback in with.
>>
>> As with everything in optimization, profile your code first. If you’re
>> losing speed heavily somewhere else, then the above will only serve to
>> distract you.
>>
>> Cheers,
>>
>> Ben
>>
>>
>> From: Phyo Arkar <phyo.arkarlwin at gmail.com>
>> Date: Thursday, September 4, 2014 at 1:55 PM
>> To: Ben Jolitz <Ben.Jolitz at acxiom.com>
>> Cc: pypy-dev <pypy-dev at python.org>
>>
>> Subject: Re: [pypy-dev] Pypy Benchmark of Tornado.
>>
>> Thanks alot Ben,
>> Ok , as PyPy is choice of quora and they also use tornado , i might keep
>> testing on larget projects.
>> How about mongodb performance on PyPy , i heard its slower due to no C
>> Extension (no CFFI) for pypy.
>> Your suggestion will be very appreciated.
>>
>>
>> On Thu, Sep 4, 2014 at 4:56 AM, Jolitz Ben - bjolit <Ben.Jolitz at acxiom.com>
>> wrote:
>>>
>>> I use Tornado and have found PyPy can yield a 30-50% performance increase
>>> for a moderately complex project.
>>>
>>> Ben
>>>
>>> From: Phyo Arkar <phyo.arkarlwin at gmail.com>
>>> Date: Wednesday, September 3, 2014 at 8:21 AM
>>> To: pypy-dev <pypy-dev at python.org>
>>> Subject: Re: [pypy-dev] Pypy Benchmark of Tornado.
>>>
>>> I expect pypy to be faster in those cases but select io is not cpu
>>> intensive thing to do so no real benefit using pypy here i guess.
>>> On Wed, Sep 3, 2014 at 4:43 PM, Phyo Arkar <phyo.arkarlwin at gmail.com>
>>> wrote:
>>> >
>>> > It just return json document with a few thousand characters (1053 bytes)
>>> > $siege -c 400 -t 20s -r 2000 http://localhost:9999/js
>>> >
>>> > Python 2.7.7:
>>> >
>>> > Lifting the server siege... done.
>>> >
>>> > Transactions: 14478 hits
>>> > Availability: 100.00 %
>>> > Elapsed time: 19.10 secs
>>> > Data transferred: 14.54 MB
>>> > Response time: 0.01 secs
>>> > Transaction rate: 758.01 trans/sec
>>> > Throughput: 0.76 MB/sec
>>> > Concurrency: 8.91
>>> > Successful transactions: 14478
>>> > Failed transactions: 0
>>> > Longest transaction: 1.08 seconds
>>> > Shortest transaction: 0.00
>>> >
>>> > pypy-2.3.1 stable:
>>> >
>>> > Transactions: 15149 hits
>>> > Availability: 100.00 %
>>> > Elapsed time: 19.63 secs
>>> > Data transferred: 15.21 MB
>>> > Response time: 0.02 secs
>>> > Transaction rate: 771.73 trans/sec
>>> > Throughput: 0.77 MB/sec
>>> > Concurrency: 11.92
>>> > Successful transactions: 15149
>>> > Failed transactions: 0
>>> > Longest transaction: 1.09 seconds
>>> > Shortest transaction: 0.00
>>> >
>>> >
>>> >
>>> > pypy--c-jit-73283-912dd9df99a8-linux64 (latest nightly build)
>>> >
>>> > Lifting the server siege... done.
>>> >
>>> > Transactions: 14361 hits
>>> > Availability: 100.00 %
>>> > Elapsed time: 19.13 secs
>>> > Data transferred: 14.42 MB
>>> > Response time: 0.03 secs
>>> > Transaction rate: 750.71 trans/sec
>>> > Throughput: 0.75 MB/sec
>>> > Concurrency: 21.53
>>> > Successful transactions: 14361
>>> > Failed transactions: 0
>>> > Longest transaction: 3.03 seconds
>>> > Shortest transaction: 0.00
>>> >
>>> >
>>> >
>>> >
>>> > Pypy Nightly have some request Randomly get to 3.0 Seconds , normally
>>> > those requests (in Cpython) are only ~0.001 to 0.002 sec.
>>> >
>>>
>>> It just return json document with a few thousand characters (1053 bytes)
>>> $siege -c 400 -t 20s -r 2000 http://localhost:9999/js
>>>
>>> Python 2.7.7:
>>>
>>> Lifting the server siege... done.
>>>
>>> Transactions: 14478 hits
>>> Availability: 100.00 %
>>> Elapsed time: 19.10 secs
>>> Data transferred: 14.54 MB
>>> Response time: 0.01 secs
>>> Transaction rate: 758.01 trans/sec
>>> Throughput: 0.76 MB/sec
>>> Concurrency: 8.91
>>> Successful transactions: 14478
>>> Failed transactions: 0
>>> Longest transaction: 1.08 seconds
>>> Shortest transaction: 0.00
>>>
>>> pypy-2.3.1 stable:
>>>
>>> Transactions: 15149 hits
>>> Availability: 100.00 %
>>> Elapsed time: 19.63 secs
>>> Data transferred: 15.21 MB
>>> Response time: 0.02 secs
>>> Transaction rate: 771.73 trans/sec
>>> Throughput: 0.77 MB/sec
>>> Concurrency: 11.92
>>> Successful transactions: 15149
>>> Failed transactions: 0
>>> Longest transaction: 1.09 seconds
>>> Shortest transaction: 0.00
>>>
>>>
>>>
>>> pypy--c-jit-73283-912dd9df99a8-linux64 (latest nightly build)
>>>
>>> Lifting the server siege... done.
>>>
>>> Transactions: 14361 hits
>>> Availability: 100.00 %
>>> Elapsed time: 19.13 secs
>>> Data transferred: 14.42 MB
>>> Response time: 0.03 secs
>>> Transaction rate: 750.71 trans/sec
>>> Throughput: 0.75 MB/sec
>>> Concurrency: 21.53
>>> Successful transactions: 14361
>>> Failed transactions: 0
>>> Longest transaction: 3.03 seconds
>>> Shortest transaction: 0.00
>>>
>>>
>>>
>>>
>>> Pypy Nightly have some request Randomly get to 3.0 Seconds , normally
>>> those requests (in Cpython) are only ~0.001 to 0.002 sec.
>>>
>>>
>>> ***************************************************************************
>>> The information contained in this communication is confidential, is
>>> intended only for the use of the recipient named above, and may be legally
>>> privileged.
>>>
>>> If the reader of this message is not the intended recipient, you are
>>> hereby notified that any dissemination, distribution or copying of this
>>> communication is strictly prohibited.
>>>
>>> If you have received this communication in error, please resend this
>>> communication to the sender and delete the original message or any copy
>>> of it from your computer system.
>>>
>>> Thank You.
>>>
>>> ****************************************************************************
>>>
>>>
>>> _______________________________________________
>>> pypy-dev mailing list
>>> pypy-dev at python.org
>>> https://mail.python.org/mailman/listinfo/pypy-dev
>>>
>>
>>
>> _______________________________________________
>> pypy-dev mailing list
>> pypy-dev at python.org
>> https://mail.python.org/mailman/listinfo/pypy-dev
>>
More information about the pypy-dev
mailing list