[Cython] [cython-users] GSoC 2012

[Stefan Behnel]

Robert Bradshaw, 21.03.2012 21:56:
> On Sat, Mar 10, 2012 at 10:44 PM, mark florisson wrote:
> 2.
>> - profile guided optimizations (using python's profilers and/or a
>> custom profiler that collects data such as types etc, which can be
>> used to specialize variables inside loops (or entire functions) with a
>> fallback to normal mode in case the type changes)
> 4.
>> - better type inference, that would be enabled by default and again
>> handle thing like reassignments of variables and fallbacks to the
>> default object type. With entry caching Cython could build a database
>> of types ((extension) classes, functions, variables) used in the
>> modules and functions that are compiled (also def functions), and
>> infer the types used and specialize on those. Maybe a switch should be
>> added to cython to handle circular dependencies, or maybe with the
>> distutils preprocessing it can run all the type inference first and
>> keep track of unresolved entries, and try to fill those in after
>> building the database. For bonus points the user can be allowed to
>> write plugins to aid the process.
> 
> 
> What I would most like to see is the common component in 2 and 4, i.e.
> the ability to generate optimized code for imperfectly-inferred types,
> with transparent fallback to generic code if conditions are not met at
> runtime (during as well as at entering the optimized code path).

Absolutely. The dict iteration change is pointing exactly in that
direction, but there are so many other places where the same thing applies.

Basically, it has separate C implementations for different cases all folded
into an inlined helper function with flag parameters, and then sets some of
the flags to 1/0 constants at compile time and determines others at
runtime. The C compiler can then just drop any inaccessible code and use
the inlined remainings of the function to infer a good way of streamlining
the rest of the surrounding code.

This also turned out to be a perfect way to enable or disable certain
optimisations and/or implementation details for different backends (PyPy
and CPython currently). Much better than actually generating different C
code for them.

Stefan