[Cython] Wacky idea: proper macros

[Nathaniel Smith]

Was chatting with Wes today about the usual problem many of us have
encountered with needing to use some sort of templating system to
generate code handling multiple types, operations, etc., and a wacky
idea occurred to me. So I thought I'd through it out here.

What if we added a simple macro facility to Cython, that worked at the
AST level? (I.e. I'm talking lisp-style macros, *not* C-style macros.)
Basically some way to write arbitrary Python code into a .pyx file
that gets executed at compile time and can transform the AST, plus
some nice convenience APIs for simple transformations.

E.g., if we steal the illegal token sequence @@ as our marker, we
could have something like:

@@ # alone on a line, starts a block of Python code
from Cython.MacroUtil import replace_ctype
def expand_types(placeholder, typelist):
  def my_decorator(function_name, ast):
    functions = {}
    for typename in typelist:
      new_name = "%s_%s" % (function_name, typename)
      functions[name] = replace_ctype(ast, placeholder, typename)
    return functions
  return function_decorator
@@ # this token sequence cannot occur in Python, so it's a safe end-marker

# Compile-time function decorator
# Results in two cdef functions named sum_double and sum_int
@@expand_types("T", ["double", "int"])
cdef T sum(np.ndarray[T] arr):
  cdef T start = 0;
  for i in range(arr.size):
    start += arr[i]
  return start

I don't know if this is a good idea, but it seems like it'd be very
easy to do on the Cython side, fairly clean, and be dramatically less
horrible than all the ad-hoc templating stuff people do now.
Presumably there'd be strict limits on how much backwards
compatibility we'd be willing to guarantee for code that went poking
around in the AST by hand, but a small handful of functions like my
notional "replace_ctype" would go a long way, and wouldn't impose much
of a compatibility burden.

-- Nathaniel