[SciPy-Dev] The state of weave.accelerate?

[Ralf Gommers]

On Sun, Apr 17, 2011 at 3:37 PM, Helio Perroni Filho <xperroni at gmail.com> wrote:
> Dera all,
>
> I have been looking for ways to speed up some portions of Python code
> in my current scipy project. The ideal solution would allow me to keep
> writing all my code in Python 2.7 – even if the "accelerated" parts
> had to written in a subset of the language, as in PyPy's Restricted
> Python – and be easily turned off for debugging.
>
> When I learned of scipy.weave.inline, my first thought was "now if
> only there was a Python-to-C++ translator I could hook up to this!" I
> was intrigued when I found that cryptic reference about
> sicpy.weave.accelerate providing "automatic acceleration of Python
> code"; all the more after searching for more information on it, and
> finding none.
>
> So I decided to take a look at the code. The API seemed simple enough
> (just a decorator for top-level functions), so I tried it on some toy
> scripts I keep around. After hacking around a couple simple bugs (two
> incorrect assertions in bytecodecompiler, a typo defining the
> one-argument form of the range function and the need to add weave's
> directory to the compiler include-path in accelerate_tools) I got
> stuck on the many not-yet-implemented instructions in
> bytecodecompiler: it seems even basic arithmetic is still to be
> included.
>
> I find this frustrating. From the tests I've run it seems accelerate()
> already does an adequate work of generating the necessary C++
> boilerplate code, compiling it to an extension and then wiring the
> result back to the executing script. The only thing missing is getting
> the C++ translation to work – not trivial, I know, but not very hard
> either, considering what's already in place.
>
> Is a lack of manpower holding accelerate() back? If that's the case, I
> volunteer to work on bytecodecompiler, and get it to at least an
> RPython-level of functionality.

Unfortunately all of scipy.weave is not actively developed (only some
basic maintenance the last few years). It is also the only scipy
module that does not (yet?) work with Python 3. It's my impression
that it does its job for legacy code, but for new development people
tend to prefer Cython. Maybe someone who knows more about the weave
history and current state will answer you in more detail.

If you would be willing to work on it that would be great. As you have
already experienced already though, it may be quite some work to fix
the small bugs and modernize the code a bit. I also don't know how the
functionality compares to what's in Shedskin and Nuitka, which are
both (not inline I think) Python to C++ compilers.

> I have read the documentation on the
> Developer Zone, but didn't find how do I create a Trac account. And
> besides that, is there anything else not on that page that I should
> know?

You can create a Trac account at http://projects.scipy.org/scipy/register
Development has moved to git and github, and sending a pull request
there is generally more effective than adding patches in Trac. The
process is described here: http://docs.scipy.org/doc/numpy/dev/

Cheers,
Ralf