[Python-ideas] Re: New Tool Proposal

[Dan Stromberg]

On Tue, May 10, 2022 at 3:15 AM Chris Angelico <rosuav at gmail.com> wrote:

> > It is often the case that developer write Code in Python and then
> convert to a C extension module for performance regions.
> >
> > A C extension module has a lot of boiler plate code - for instance the
> Structures required for each class, the functions for Module initialization
> etc.
> >
> > My Idea is a simple tool that uses introspection tools to take a Python
> module and to generate the relevant boiler plate for the module - including
> blank functions for the module classes and for methods. This tool would use
> type annotations (if given) to make sensible choices for parameter and
> attribute types, including using int and float directly rather than
> Internal objects (depending on tool options).
>

Two things to say about this:
1) Sometimes abandoning a pure python module for a C extension for
performance is a mistake - because Pypy is probably going to be much faster
with the pure python module
2) I've had some luck using m4 to maintain a single source file that is
used to automatically generate both pure python and cython.  This is a
little like using cpp in a C project.

For examples of #2, perhaps see:
https://stromberg.dnsalias.org/~strombrg/treap/
https://stromberg.dnsalias.org/svn/rolling_checksum_mod/trunk/
https://stromberg.dnsalias.org/~strombrg/sort-comparison/

It's often nice to keep the lines of the pure-python and cython having a
1-1 relationship, so that tracebacks report useful line numbers either
way.  However, in the treap example I've dispensed with that because some
methods were almost identical but had some boilerplate - and m4 was able to
handle that nicely at the cost of lines being 1-1.

HTH