Python and the need for speed

[Nathan Ernst]

I think that's fair (and I had intended to mention it). Although, I'm
curious how threading with IO compares to using async/awai (I've not
experience with async/await in Python, just in C#).

Regards,
Nate

On Tue, Apr 11, 2017 at 8:04 PM, MRAB <python at mrabarnett.plus.com> wrote:

> On 2017-04-12 01:28, Nathan Ernst wrote:
> [snip]
>
> I worked on http://www.marketswiki.com/wiki/CMDX - in particular I wrote
>> most of the Migration Utility mentioned to migrate paper CDS trades to
>> standardized CDS contracts against CME. Most of the migration util was
>> written in native Python 2.5 (it was written in 2008) using a single
>> thread. Performance wasn't super critical, but desired. At the end of the
>> project, I was processing ~100K positions per second. Memory usage of the
>> app was constant and processing time of a portfolio was directly linear to
>> the number of positions in the portfolio. Python wasn't the limiting
>> factor
>> for the app - it was the write speed to the database (and we were using
>> the
>> bcp interface of pysybase to write to a Sybase DB).
>>
>> Basically, what I'm getting at is Python *can* be performant. It's also
>> easy to screw up and introduce non-obvious slowness. Threading in Python
>> is
>> bad - don't bother (until we can get rid of the GIL, I doubt the situation
>> will improve).
>>
>> Threading is OK if you're waiting for I/O where the main limiting factor
> is the speed of the I/O, e.g. transferring stuff to/from the internet.
>
> If you have a performance problem with Python, before you blame Python,
>> take a step back and look at your own code (or libraries you're using) and
>> ask yourself: "Is my code optimal?"
>>
>> Yes, Python is not the faster language/runtime in existence. But for
>> probably 99% of the people out there that complain about Python's speed,
>> there's probably plenty of suboptimal or outright wasteful code that they
>> should fix first, before complaining. For the other 1%, Python was
>> probably
>> the wrong choice to begin with.
>>
>> ****************************
>>
>> I don't intend this to be seen or implied as an attack or criticism of
>> anyone. I'm just trying to provide an insight into my experience of using
>> Python.
>>
>> [snip]
> --
> https://mail.python.org/mailman/listinfo/python-list
>