Python and the need for speed

[Brecht Machiels]

On 2017-04-11 14:56:33 +0000, Steve D'Aprano said:

> On Tue, 11 Apr 2017 07:56 pm, Brecht Machiels wrote:
>> 
>> Hey! This random, ignorant blogger has feelings too! :-)
> 
> Hi, and welcome!
> 
> Sorry, I wasn't *specifically* referring to you, except in the sense that
> you aren't a compiler expert.

That's ok. No offense taken.

> But I also think that while making Python faster is a good goal to have, it
> seems that for most people, and most companies, it isn't their priority.

That may very well be correct. I have used Python/NumPy extensively for 
data processing/analysis, and I haven't been disappointed by its 
performance in that area (so far, at least).

However, rinohtype is located in a very different niche and it would 
greatly benefit a lot from a speed boost. Rendering the Sphinx 
documentation (311 pages) takes almost 10 minutes on my i7, which is 
simply too long given the available processing power. And yes, I have 
spent a lot time profiling and optimizing the code. You're always 
welcome to demonstrate my incompetence by means of a pull request, of 
course ;-)

I have toyed with the idea of porting rinohtype to JavaScript, for the 
increased performance (though that is not certain) and the advantage of 
being able to run in the browser. While it may not be an extreme amount 
of work to port rinohtype itself using something like RapydScript, the 
docutils and Sphinx dependencies are not easily replaced.

As for moving to a compiled language (Nim or Rust would be interesting 
choices, IMO), it doesn't make much sense. rinohtype should be easily 
extendable with new document templates for example. An interpreted 
language like Python is ideal for that (not to mention it's lovely 
syntax).

>>> The Python ecosystem is actually quite healthy, if you need to speed up
>>> code there are lots of solutions, and some of them are even good
>>> solutions.
>> 
>> There seem to be no solutions for my use case (rinohtype).
> 
> Have you tried Nuitka?

I have, a long time ago, so it's a bit vague. But if it performed 
marginally better than CPython, I would have remembered. Definitely not 
enough to warrant having to distribute binaries.

>>> Nevertheless, it is interesting to discuss whether or not any
>>> of these features will go mainstream or make it into CPython.
>> 
>> Indeed! I initially wanted to include the following in the article, but
>> decided it would be too controversial. But now that I've been exposed
>> as an ignorant and naive blogger, I might as well share these thoughts.
>> 
>> I have the feeling that a faster Python will never materialise unless
>> the CPython core developers make performance a high priority.
> 
> I think you are both right and wrong.
> 
> You are right in the sense that none of the companies exploring Python
> optimizers have wanted to carry the maintenance burden themselves. Their
> aim is to prove the concept, then push it back into CPython and have the
> core devs maintain it.
> 
> But you're also wrong, in the sense that you're focused on *revolutionary*
> speed-ups rather than *evolutionary* optimizations. CPython is faster today
> than it was in version 1.5, despite doing MUCH more. Python continues to
> shave slow code off and improve performance. No, this isn't likely to make
> Python ten times faster, but its performance does incrementally increase.

I do appreciate the evolutionary speedups of CPython. However, I love 
Python and it's just a bit frustrating that it's not all that it could 
be. Python *could* be faster, it's just not moving in that direction.

But, it may indeed be I'm just one of the few that think that Python is 
not fast enough. If that is the case, it of course makes no sense to 
focus more on performance. However, we don't really know how Python's 
performance is affecting its popularity. It would be good to have some 
kind of information about user's motivations for choosing Python or 
moving away from it. Have any polls been organized in the past to try 
to find out?

I realize that work on CPython (and thus the language itself) is done 
by unpaid volunteers. But I assume that they also want the language to 
thrive, and therefore want to cater to the wishes of the userbase at 
some level.

So perhaps the conclusion to this discussion is that we should first 
try to find out whether performance is an issue for a large part of the 
community (or possible newcomers).

Best regards,
Brecht