[Numpy-discussion] Partially merging scons into trunk ?

[Bryce Hendrix]

David Cournapeau wrote:
> Bryce Hendrix wrote:
>   
>> I've only been following this thread on the perimeter, so I'm not sure
>> if "makefile" migration has been discussed. I have a script I wrote
>> about a year ago when we (Enthought) were looking at using SCons for our
>> internal builds. The script is capable of generating SConscript files
>> from setup.py scripts. It works by faking a build with distutils and
>> uses those build steps to generate the scons targets.
>>
>> The script is not complete, for example 'weave' confuses it, but it
>> could be a good starting point. If anyone is interested in it, I'll post
>> it to the list.
>>   
>>     
> That sounds interesting. Do I understand correctly that you ditched 
> distutils entirely ? That's not the approach I have taken, because I 
> thought it would be too big (before inclusion in numpy, I need to show 
> something remotely impressive, to convince the change worths it), and 
> would prevent doing things iteratively (e.g. building numpy wo distutils 
> would be a huge task). For most subpackages of scipy, which have small 
> setup.py, this would be useful, I guess (for the packages I know, they 
> are independant, at least as far as build issues are concerned).
>
>   

No, what I was investigating was whether we could auto-generate
Sconscript files from setup.py scripts so our builds could reliably only
build/test/document what was necessary. For example, our docs would take
> 10 minutes to re-generate the whole thing, but if only 1 file changed,
building the docs should take < 10 seconds. On top of that, there was a
summer of code project in 2006 to add packaging (msi and rpm were the
targeted formats), and we were looking at packaging in msi, rpm, and egg
formats. I eventually got frustrated with the speed at which the SCons
developers were willing to roll this change into the trunk, so I gave up
and we moved on to a different solution.

> I don't think any automated task would work for numpy:  I spent a good 
> deal of time last WE to be able to build all extensions in numpy with 
> scons, and it is a lot of work, because of bootstrapping issues. If you 
> take into account the code generators, the numpy.core "Makefile" is more 
> than 1000 lines of non trivial code. I finally got my head around it, 
> but I don't think any automated scheme would work in this case.
>   

Understood, especially since we could never auto-generate all of ours.
Just thought I'd offer it, since its just bit-rotting in our internal
repository right now.

Bryce
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20071016/59f89d80/attachment.html>