[Cython] Wacky idea: proper macros

[Wes McKinney]

On Mon, Apr 30, 2012 at 5:36 PM, William Stein <wstein at gmail.com> wrote:
> On Mon, Apr 30, 2012 at 2:32 PM, Dag Sverre Seljebotn
> <d.s.seljebotn at astro.uio.no> wrote:
>>
>>
>> Wes McKinney <wesmckinn at gmail.com> wrote:
>>
>>>On Mon, Apr 30, 2012 at 4:55 PM, Nathaniel Smith <njs at pobox.com> wrote:
>>>> On Mon, Apr 30, 2012 at 9:49 PM, Dag Sverre Seljebotn
>>>> <d.s.seljebotn at astro.uio.no> wrote:
>>>>> JIT is really the way to go. It is one thing that a JIT could
>>>optimize the
>>>>> case where you pass a callback to a function and inline it run-time.
>>>But
>>>>> even if it doesn't get that fancy, it'd be great to just be able to
>>>write
>>>>> something like "cython.eval(s)" and have that be compiled (I guess
>>>you could
>>>>> do that now, but the sheer overhead of the C compiler and all the
>>>.so files
>>>>> involved means nobody would sanely use that as the main way of
>>>stringing
>>>>> together something like pandas).
>>>>
>>>> The overhead of running a fully optimizing compiler over pandas on
>>>> every import is pretty high, though. You can come up with various
>>>> caching mechanisms, but they all mean introducing some kind of
>>>compile
>>>> time/run time distinction. So I'm skeptical we'll just be able to get
>>>> rid of that concept, even in a brave new LLVM/PyPy/Julia world.
>>>>
>>>> -- Nathaniel
>>>> _______________________________________________
>>>> cython-devel mailing list
>>>> cython-devel at python.org
>>>> http://mail.python.org/mailman/listinfo/cython-devel
>>>
>>>I'd be perfectly OK with just having to compile pandas's "data engine"
>>>and generate loads of C/C++ code. JIT-compiling little array
>>>expressions would be cool too. I've got enough of an itch that I might
>>>have to start scratching pretty soon.
>>
>> I think a good start is:
>>
>> Myself I'd look into just using Jinja2 to generate all the Cython code, rather than those horrible Python interpolated strings...that should give you something that's at least rather pleasant for you to work with once you are used to it (even if it is a bit horrible to newcomers to the code base).
>>
>> You can even check in the generated sources.
>>
>> And we've discussed letting cython be smart with templating languages and error report on a line in the original template, such features will certainly accepted once somebody codes it up.
>>
>>  (I can give you me breakdown of how I eliminate other templating languages than Jinja2 for this purpose tomorrow if you are interested).
>
> Can you point us to a good example of you using jinja2 for this purpose?
>
> I'm a big fan of Jinja2 in general (e.g., for HTML)...
>
>>
>> Dag
>>
>>>_______________________________________________
>>>cython-devel mailing list
>>>cython-devel at python.org
>>>http://mail.python.org/mailman/listinfo/cython-devel
>>
>> --
>> Sent from my Android phone with K-9 Mail. Please excuse my brevity.
>> _______________________________________________
>> cython-devel mailing list
>> cython-devel at python.org
>> http://mail.python.org/mailman/listinfo/cython-devel
>
>
>
> --
> William Stein
> Professor of Mathematics
> University of Washington
> http://wstein.org
> _______________________________________________
> cython-devel mailing list
> cython-devel at python.org
> http://mail.python.org/mailman/listinfo/cython-devel

I agree, it'd be cool to see an example or two. I have some ideas for
a mini DSL / code-generation framework that might suit my needs;
jinja2 might be then the right tool for doing the templating /
codegen. If I could cut the amount of Cython code I have in half (and
make it easier to write simple functions, which are currently more
than 50% boilerplate) that would be a big win for me.

- Wes