[Numpy-discussion] Waf or scons/numscons for a C/Fortran/Cython/Python project -- what's your recommendation?

[David Cournapeau]

On Sun, Jan 17, 2010 at 8:36 AM,  <josef.pktd at gmail.com> wrote:
> On Sat, Jan 16, 2010 at 4:43 PM, Dag Sverre Seljebotn
> <dagss at student.matnat.uio.no> wrote:
>> Kurt Smith wrote:
>>> On Sat, Jan 16, 2010 at 2:38 PM, Dag Sverre Seljebotn
>>> <dagss at student.matnat.uio.no> wrote:
>>>
>>>
>>>> Not that I really know anything about it, but note that one of the
>>>> purposes of David's toydist is to handle the install stage independently
>>>> of the build system used. That is, it is able to create e.g. Python eggs
>>>> without using setuptools.
>>>>
>>>> The thing is, installing Python software is something of a mess, and
>>>> every system would want this done differently (making an Ubuntu package,
>>>> creating a DMG, or creating a Python egg are all different things). So I
>>>> think it makes sense to decouple this from the build in the tools that
>>>> are used.
>>>>
>>>
>>> Yep.  Good points.  I expect once I get the configure/build stages in
>>> a working state, I'll have most of what people need.  The install
>>> stage is less crucial, at least for the first version.  Seems like
>>> people would like the system to just create a .so file in the current
>>> directory, and leave it at that.  If I can get that working on all
>>> platforms I'll be very happy :-)
>>>
>>>
>>>> Of course, toydist is beta, and I dare say you have enough beta
>>>> dependencies for fwrap already :-)
>>>>
>>>
>>> :-)
>>>
>>> Hopefully that can be remedied that in the coming months, at least
>>> from the fparser and memoryview-support-in-Cython side of things.
>>>
>>
>> Obviously I didn't get around to that yet...
>>
>> As for the build systems, some things to consider (I have no clue myself
>> as to waf vs. scons):
>>  - There's already primitive scons support for Cython, but I'm sure it
>> wouldn't be hard to add to waf
>>  - Whatever you pick is likely to become the best supported build system
>> for Cython code in the future, I think, due to our interest in working on it
>>  - Does waf have infrastructure for parsing files and finding
>> dependencies? I know that in Scons one can plug in a "Cython parser",
>> which checks the dependencies (which pxds are used, basically), so that
>> pyx files are rebuilt automatically when pxds they depend on change. I'm
>> sure waf supports something similar, if not I'd say it disqualifies it.
>>
>> My own hunch is that waf looks better, but scons has a larger mind share
>> and Cython support right now in scientific Python, and that both must be
>> supported eventually, so why not do scons first... *shrug*
>>
>> But like you I'm anxious to hear from more non-Cython devs as well on
>> this matter.
>>
>> Dag Sverre
>> _______________________________________________
>> NumPy-Discussion mailing list
>> NumPy-Discussion at scipy.org
>> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>>
>
> >From a very brief look at the waf book, I don't really understand what
> the cross-platform capabilities of waf are
>
> http://freehackers.org/~tnagy/wafbook/single.html :
> "Installing Waf on a system is unnecessary and discouraged: "

The main waf author claims that waf should never be installed, and
always included with your package. I think it makes sense for a lot of
practical cases (and that's how autotools work, mostly:
autoconf/automake are not needed when building something from sources,
because you have a gianting shell script called configure).

I know there has been some effort toward better windows support for
waf - given that waf is written in python, it is hard to see a
architectural reason why waf could not work well on windows. But build
tools depend a lot spawning processes and the likes both efficiently
and reliably, and that's one of the area where windows and unix-like
systems are fundamentally different.

David