[Cython] [ENH] Compile-time code execution (macros / templates)

Celelibi celelibi at gmail.com
Mon Feb 22 23:11:43 EST 2021


Hello. I was going to post this as an issue on github, but since it's
the idea isn't complete, I guess this mailing list is better suited.

I read (not in too much details, though) the related CEP, and I feel
they either do too little or too much.

In my idea, there is some syntax (or decorator) and some propagation
rules to decide what to execute during the compilation. It would merge
the need for templates and for constants pre-evaluation.

The uses would be great many.
- It could generate new cdef classes based on an argument (possibly a C
  type).
- It could make sure all the references to the cython module are caught
  during the compilation.
- It could replace the DEF keyword with a pure python syntax.
- It could easily allow any decorator on cdef functions.
- Compile-time decision of a function implementation (common in C using
  the preprocessor).
- And probably many more.


As far as I understand, this would be strictly more general and elegant
than these two proposals:
https://github.com/cython/cython/wiki/enhancements-inlining
https://github.com/cython/cython/wiki/enhancements-methodtemplates

In my idea, the only metaprogramming entry points would be cdef functions,
cdef classes (or cppclasses) and C/C++ types. No access to the AST as
it's unusual in python to do so. But nontheless, it would be quite easy
to have a CdefFunction object that exposes an `ast` attribute.

As such, my proposal would provide a cleaner entry point to
metaprogramming than these two proposals. Together with a pure python
syntax.
https://github.com/cython/cython/wiki/enhancements-metaprogramming
https://github.com/cython/cython/wiki/enhancements-uneval


The compile-time execution doesn't have to support everything.
Especially, the evaulation of `cdef` code could be heavily restricted.
Here are the restriction that apply to CTFE (Compile-Time Function
Evaluation) in the language D.
https://dlang.org/spec/function.html#interpretation


As far as I can tell, it's quite common in python to write load-time
code in modules to achieve a higher level of genericity. Like to support
PY2 and PY3, Windows and Linux, etc. Or to generate classes using
closures.
I think bringing back the habits and culture of python to cython would
be a good thing.


On the syntax side, I'm not sure how to handle cases where the right
hand side of an assignment should be evaluated and the cases where every
use of the variable should be evaluated.

# Need only to evaluate the right hand side
cdef str pi = compute_pi(100)

# Need to evaluate everywhere the variable appear
os = "Linux"
if os == "Windows":
    cdef dostuff():
        return 1
else:
    cdef dostuff():
        return 0


What do you think?
Did I miss something that would make this feature mostly useless?


Best regards,
Celelibi


More information about the cython-devel mailing list