[Cython] Utility Codes and templates

[Robert Bradshaw]

On Fri, Jul 22, 2011 at 3:12 AM, mark florisson
<markflorisson88 at gmail.com> wrote:
> For my work on the _memview branch (and also on fused types) I noticed
> that UtilityCodes started weighing heavily on me in their current
> form, so I wrote a little loader in the _memview branch:
>
> https://github.com/markflorisson88/cython/commit/e13debed2db78680ec0bd8c343433a2b73bd5e64#L2R110
>
> The idea is simple: you put your utility codes in Cython/Utility in
> .pyx, .c, .h files etc, and then load them. It works for both
> prototypes and implementations, for UtilityCode and CythonUtilityCode:

This sounds like it could be a nice way to organize our UtilityCode
snippets. So far we haven't really needed any more templating than
simple substitution, but for what you're doing I can see this being
quite handy. This may also provide a more flexible way forward for
supporting multiple backends.

> myutility.c
>
> // UtilityProto: MyUtility
> header code here
>
> // UtilityCode: MyUtility
> implementation code here
>
> You can add as many other utilities as you like to the same file. You
> can then load it using
>
>    UtilityCode.load_utility_from_file("myutility.c", "MyUtility")

I agree with you that having multiple related, named snippets in same
file is worthwhile. What about

////////////////////// MyUtility.proto ///////////////////////////

and

############ MyCyUtility ##############

so the chunks are easy to see.

> Of course you can pass in any other arguments, like proto_block, name,
> etc. You can additionally pass it a dict for formatting (for both the
> prototypes and the implementation). It will return a UtilityCode
> instance ready for use.
> You can also simply retrieve a utility code as a string, where it
> returns (proto, implementation).
>
> As debated before, an actual template library would be really
> convenient. Dag and I had a discussion about it and he suggested
> Tempita (by Ian Bicking), it is compatible with Python 2 and Python 3,
> and is pure-python. It supports all the good things like iteration,
> template inheritance, etc. Now I'm not sure whether it supports python
> 2.3 as it doesn't compile on my system, but it does support 2.4
> (confirmation for 2.3 would be appreciated). On a side note, I'd be
> perfectly happy to drop support for 2.3, it's kind of a chore.
> The documentation for Tempita can be found here: http://pythonpaste.org/tempita/

Looks like a reasonable choice to me, it's small enough to ship and
looks versatile enough to do everything we need. There's certainly not
a shortage of options, but I'm happy with your and Dag's endorsement.

In terms of delimiters, I prefer some form of strings over comments
(often a value is expected, and Python's comments always extend to the
line break) but sticking with the {{ }} syntax appeals to me even
more. It's pretty standard, and decent syntax hilighters shouldn't
choke on it (well, the surrounding code) too bad.

> That way we might rid ourselves of a lot of code.putln() and move
> those to template utilities instead (or at least prevent writing more
> of those). What do you guys think?

+1

> BTW, I will refrain from moving any utility codes other than the ones
> in MemoryView.py, CythonScope.py and Buffer.py, to avoid any possible
> future conflicts when merging into mainline.
> _______________________________________________
> cython-devel mailing list
> cython-devel at python.org
> http://mail.python.org/mailman/listinfo/cython-devel
>