[Cython] Fwd: Re: [Numpy-discussion] Proposed Roadmap Overview

[Robert Bradshaw]

Python bytecode -> LLVM is a great idea for creating ufuncs, the
overhead of Cython + GCC is atrocious for stuff like this. (I think
Cython could make a good frontent as well, especially if we generated
just the .c code for the function rather than a full extension module
and used a good compiler to generate machine code in-memory
on-the-fly.)

Going too deeply though and you'll be starting to duplicate the work
of Unladen Swallow...

On Mon, Feb 20, 2012 at 8:09 PM, Dag Sverre Seljebotn
<d.s.seljebotn at astro.uio.no> wrote:
> This has got to be the most incredible and interesting turn of events I've
> seen in a while! :-)
>
> Dag
>
> -------- Original Message --------
> Subject:        Re: [Numpy-discussion] Proposed Roadmap Overview
> Date:   Mon, 20 Feb 2012 22:04:00 -0600
> From:   Travis Oliphant <travis at continuum.io>
> Reply-To:       Discussion of Numerical Python <numpy-discussion at scipy.org>
> To:     Discussion of Numerical Python <numpy-discussion at scipy.org>
>
>
>
> Interesting you bring this up. I actually have a working prototype of
> using Python to emit LLVM. I will be showing it at the HPC tutorial that
> I am giving at PyCon. I will be making this available after PyCon to a
> wider audience as open source.
>
> It uses llvm-py (modified to work with LLVM 3.0) and code I wrote to do
> the translation from Python byte-code to LLVM. This LLVM can then be
> "JIT"ed. I have several applications that I would like to use this for.
> It would be possible to write "more of NumPy" using this approach.
> Initially, it makes it *very* easy to create a machine-code ufunc from
> Python code. There are other use-cases of having loops written in Python
> and plugged in to a calculation, filtering, or indexing framework that
> this system will be useful for.
>
> There is still a need for a core data-type object, a core array object,
> and a core calculation object. Maybe some-day these cores can be shrunk
> to a smaller subset and more of something along the lines of LLVM
> generation from Python can be used. But, there is a lot of work to do
> before that is possible. But, a lot of the currently pre-compiled loops
> can be done on the fly instead using this approach. There are several
> things I'm working on in that direction.
>
> This is not PyPy. It certainly uses the same ideas that they are using,
> but instead it fits into the CPython run-time and doesn't require
> changing the whole ecosystem. If you are interested in this work let me
> know. I think I'm going to call the project numpy-llvm, or fast-py, or
> something like that. It is available on github and will be open source
> (but it's still under active development).
>
> Here is an example of the code to create a ufunc using the system (this
> is like vectorize, but it creates machine code and by-passes the
> interpreter and so is 100x faster).
>
> from math import sin, pi
>
> def sinc(x):
> if x==0:
> return 1.0
> else:
> return sin(x*pi)/(pi*x)
>
> from translate import Translate
> t = Translate(sinc)
> t.translate()
> print t.mod
>
> res = t.make_ufunc('sinc')
>
>
> -Travis
>
>
>
> On Feb 20, 2012, at 10:55 AM, Sturla Molden wrote:
>
>> Den 20.02.2012 17:42, skrev Sturla Molden:
>>>
>>> There are still other options than C or C++ that are worth considering.
>>> One would be to write NumPy in Python. E.g. we could use LLVM as a
>>> JIT-compiler and produce the performance critical code we need on the
>>> fly.
>>>
>>>
>>
>> LLVM and its C/C++ frontend Clang are BSD licenced. It compiles faster
>> than GCC and often produces better machine code. They can therefore be
>> used inside an array library. It would give a faster NumPy, and we could
>> keep most of it in Python.
>>
>> Sturla
>>
>> _______________________________________________
>> NumPy-Discussion mailing list
>> NumPy-Discussion at scipy.org <mailto:NumPy-Discussion at scipy.org>
>> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
>
>
> _______________________________________________
> cython-devel mailing list
> cython-devel at python.org
> http://mail.python.org/mailman/listinfo/cython-devel
>