[Numpy-discussion] parallel compilation of numpy

[Brian Granger]

David,

Thank you very much for writing this summary up.  It collects a lot of
useful information about the subtle and difficult issues related to
building multi-language python packages.

The reason that Ondrej and I are asking all of these questions is that
we are currently exploring build systems for a couple of projects we
are working on (together).  In general, my sense is that distutils
won't be sufficient for our build needs.  Thus, we are looking at
Scons, cmake, numscons, waf, etc.  Honestly, compared to plain
distutils, *any* of these other solutions seem very nice, even that
each of them has its own downsides.

The big challenge as you mention is that distutils does do certain
things that plain Scons, cmake or waf doesn't:

* Install Python package in a way that is consistent with common
Python practices (site-packages, etc.)
* Integration with setuptools and eggs, which enables things like
namespace packages.
* Windows installers, source tarballs.
* Uploads to pypi.

I wonder if it wouldn't be too difficult to simply implement these
capabilities in raw cmake/Scons/waf and do away with distutils all
together.  But, I don't really know enough about distutils or
cmake/scons/waf to know if this is reasonable to dream about.

Cheers,

Brian

On Wed, Feb 18, 2009 at 9:25 PM, David Cournapeau <cournape at gmail.com> wrote:
> On Thu, Feb 19, 2009 at 9:14 AM, Ondrej Certik <ondrej at certik.cz> wrote:
>> Hi,
>>
>> I have a shiny new computer with 8 cores and numpy still takes forever
>> to compile --- is there a way to compile it in parallel (make -j9)?
>>
>> Do distutils allow that? If not, let's move to some build system that
>> allows that?
>
> Since numscons was mentioned several times recently, and I have not
> answered to every request, I wrote a more complete post about it -
> even with somes advices for people motivated in using cmake :)
>
> http://cournape.wordpress.com/2009/02/19/numscons-current-state-future-alternative-build-tools-for-numpy/
>
> cheers,
>
> David
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion at scipy.org
> http://projects.scipy.org/mailman/listinfo/numpy-discussion
>