[Cython] Wacky idea: proper macros

[mark florisson]

On 28 April 2012 22:04, Nathaniel Smith <njs at pobox.com> wrote:
> Was chatting with Wes today about the usual problem many of us have
> encountered with needing to use some sort of templating system to
> generate code handling multiple types, operations, etc., and a wacky
> idea occurred to me. So I thought I'd through it out here.
>
> What if we added a simple macro facility to Cython, that worked at the
> AST level? (I.e. I'm talking lisp-style macros, *not* C-style macros.)
> Basically some way to write arbitrary Python code into a .pyx file
> that gets executed at compile time and can transform the AST, plus
> some nice convenience APIs for simple transformations.
>
> E.g., if we steal the illegal token sequence @@ as our marker, we
> could have something like:
>
> @@ # alone on a line, starts a block of Python code
> from Cython.MacroUtil import replace_ctype
> def expand_types(placeholder, typelist):
>  def my_decorator(function_name, ast):
>    functions = {}
>    for typename in typelist:
>      new_name = "%s_%s" % (function_name, typename)
>      functions[name] = replace_ctype(ast, placeholder, typename)
>    return functions
>  return function_decorator
> @@ # this token sequence cannot occur in Python, so it's a safe end-marker
>
> # Compile-time function decorator
> # Results in two cdef functions named sum_double and sum_int
> @@expand_types("T", ["double", "int"])
> cdef T sum(np.ndarray[T] arr):
>  cdef T start = 0;
>  for i in range(arr.size):
>    start += arr[i]
>  return start
>
> I don't know if this is a good idea, but it seems like it'd be very
> easy to do on the Cython side, fairly clean, and be dramatically less
> horrible than all the ad-hoc templating stuff people do now.
> Presumably there'd be strict limits on how much backwards
> compatibility we'd be willing to guarantee for code that went poking
> around in the AST by hand, but a small handful of functions like my
> notional "replace_ctype" would go a long way, and wouldn't impose much
> of a compatibility burden.
>
> -- Nathaniel
> _______________________________________________
> cython-devel mailing list
> cython-devel at python.org
> http://mail.python.org/mailman/listinfo/cython-devel

Have you looked at http://wiki.cython.org/enhancements/metaprogramming ?

In general I would like better meta-programming support, maybe even
allow defining new operators (although I'm not sure any of it is very
pythonic), but for templates I think fused types should be used, or
improved when they fall short. Maybe a plugin system could also help
people.