Python and the need for speed

[Steven D'Aprano]

On Sun, 09 Apr 2017 13:57:28 +1000, Chris Angelico wrote:

> On Sun, Apr 9, 2017 at 10:20 AM,  <breamoreboy at gmail.com> wrote:
>> I've an idea that http://www.mos6581.org/python_need_for_speed is a
>> week late for April Fool's but just in case I'm sure that some of you
>> may wish to comment.

I'm not sure why Mark thinks that wanting Python to be faster is a joke. 
As the page points out, there are many projects attempting (with greater 
or lesser success) to speed up Python.

I'm happy for Mark if his need for high performance is low, but CPython 
at least is definitely in the middle-tier of performance, and PyPy isn't 
suitable for all use-cases. I don't know anyone who has ever said "this 
interpreter is too fast, can you make it run slower?"


>>From that page:
> 
>> Other candidates for banishment from TurboPython include eval and exec.
> 
> Bye bye namedtuple. 

All that would mean is that the implementation of namedtuple would have 
to change. It would probably require some sort of support from the 
compiler, but that's okay, the compiler can do (nearly) anything.

Something as important and useful as namedtuple would not be dropped from 
this hypothetical TurboPython. It would just shift the implementation 
from pure-Python to something else.

exec() is actually only needed for a *tiny* bit of namedtuple. The 
original implementation by Raymond Hettinger takes the easy way out by 
using exec on the entire class definition, but it is trivially easy to 
restrict that to just the class __new__ method:

https://code.activestate.com/recipes/578918-yet-another-namedtuple


but even that could be avoided with a minimal amount of help from the 
compiler.


> And compile() is going to have to go, 

Indeed.


> since you
> could implement eval/exec by creating a new function:
> 
>>>> runme = r"""
>   print("Hello, world!")
>   import sys sys.stdout.write("I can import modules.\n")
> """
>>>> type(lambda: 1)(compile("def f():" + runme, "exec",
>>>> "exec").co_consts[0], globals())()
> Hello, world!
> I can import modules.
> 
> So if compile() goes, you also lose ast.parse, 

Probably.


> which means you lose introspection tools, 

*Some* introspection tools. Python has many that don't rely on compile or 
ast.parse.

Hell, even Java has some introspection tools, based on reflection, which 
lets you do Python-like things at runtime. Of course if you do this, you 
lose performance.


> plus you lose literal_eval and friends. 

I don't know what "friends" you are referring to, but again, if 
literal_eval is important, the compiler can support it. If you can 
support an entire Python interpreter and compiler in the form of 
compile(), then you can support more restricted subset of the language. 
Writing a parser to evaluate strings, integers, and a few other data 
types is not exactly brain surgery.



> I'm also
> not sure whether the import machinery would have to be rewritten, but a
> quick 'git grep' suggests that it would. Removing eval and exec is not
> as simple as removing them from the standard library.

Well of course not, but removing eval and exec is only a necessary, not 
sufficient, condition, for enabling a huge bunch of compiler optimizations 
and speeding up Python.

This "TurboPython" would require a completely new implementation of the 
Python interpreter to be fast. It's not as if eval and exec are great 
heavy concrete weights chained to the leg of the compiler, and all you 
need do is remove the chain and the compiler suddenly becomes thirty 
times faster.


> In fact, extreme dynamism is baked deep into the language. You'd have to
> make some fairly sweeping language changes to get any real benefits from
> restricting things. 

Well, maybe. As is pointed out many, many times, 99% of Python code 
avoids the sorts of extreme dynamism that keeps things slow. Lots of 
people would be satisfied with a language *really close* to Python that 
was ten or twenty times faster, even if it meant that you couldn't write 
code like this:


answer = input("What's your name?")
exec("name = %r" % answer)
print(name)


Even better would be if the compiler was smart enough to use the 
optimized, fast runtime when the dynamic features aren't used, and fall 
back on a slower implementation only when needed to support the more 
dynamic features.

I wonder how Victor Stinner's FAT Python is going?

http://faster-cpython.readthedocs.io/fat_python.html




> There are better ways to improve performance,

And yet Python lags behind Javascript and PHP for speed...



-- 
Steve