[Python-ideas] Type Hinting - Performance booster ?

[Ron Adam]

On 12/24/2014 06:40 AM, Sturla Molden wrote:
> On 24/12/14 12:45, Stefan Behnel wrote:
>
>> Regarding Cython, it's actually unlikely that it will bring anything. The
>> proposed feature is about specifying Python object types for function
>> arguments and (to a certain extent) local variables, and Cython is already
>> pretty good in guessing those or doing optimistic optimisations that just
>> say "if it looks like you're using a dict, let's generate special code that
>> speeds up dict access and leaves other stuff to a fallback code path".
>
> Numba is also very similar to Cython here. There is a set of types and
> expressions for which it can produce an efficient code path, and otherwise
> it falls back to calling the CPython interpreter. Psyco also used a similar
> strategy. This precisely where the Uladen Swallow went astray because they
> tried to accelerate any code path.
>
> Cython's advantage is that we can mix "down to the iron" C or C++ with
> Python, without having to use or know the Python C API. And of course
> Cython is not only for speed. It is also for making it easier to write C or
> C++ extensions for any thinkable reason.

My thoughts is that making python easier to multi-process on multi-core 
CPUs will be where the biggest performance gains will be.  Think of 100 
core chips in as soon as 5 or 6 years.  (Doubling about every two years, 
you can get 16 core CPU's now.)

It's not uncommon to have multi-term expressions where each term can be 
processed independently of the others.

     result = foo(e1, e2, ... eN)

Where each expression, "eN" may be call's to other functions, which may 
also have multiple terms.  If it can be determined at parse time that they 
can't possibly effect each other, then these terms can be evaluated in 
parallel, and byte code, or compiled code, that does that can be generated.

(I'm not sure it's doable, but it seems like it should be possible.)

And... the same type of data dependence graphs needed to do that, can aid 
in checking correctness of code.  Which is the near term benefit.

Cheers,
    Ron