[Numpy-discussion] Wanted: new release manager for 1.5 and above

[David Cournapeau]

On Fri, Jan 15, 2010 at 11:46 PM, Ralf Gommers
<ralf.gommers at googlemail.com> wrote:
> Hi David,
>
> Here are some questions to get a clearer idea of exactly what's involved in
> / required for making a release.
>
> On Thu, Jan 14, 2010 at 2:34 PM, David Cournapeau <david at silveregg.co.jp>
> wrote:
>>
>> Charles R Harris wrote:
>>
>> >
>> >
>> > What is the setup one needs to build the installers? It might be well to
>> > document that, the dependencies, and the process.
>>
>> Right. The top script is:
>> http://projects.scipy.org/numpy/browser/trunk/release.sh
>>
>> the bulk of the work is in :
>> http://projects.scipy.org/numpy/browser/trunk/pavement.py
>>
>> which describes what is needed to build installers. On mac os x, the
>> release script may be used as is to build every installer + the release
>> notes.
>>
>
> Is it necessary to have OS X to build the dmg installer, or could you build
> that from linux with some modifications to the build script?

You cannot cross compile python extensions, so you have to build
installer on each platform. Mac Os X is the most practical because you
can build windows installers under wine, so you can build both mac and
windows installers from the same machine.

The paver script + the shell script can build everything in one step
thanks to this.

>
> How many combinations do you test manually? All supported Python versions on
> all platforms? Several Linux flavors?

I basically assume that linux works once the branch is stabilized, if
only because that's what most developers use. It is important to test
on the oldest supported python (2.4) and both 32 and 64 bits, though
(especially python 2.4 on 64 bits).

I never test the installers - this is too much work manually. Ideally,
this should be done on a build/test farm.

> For someone new to packaging, how much time would you estimate it takes to
> do a single release? Is most of this time spent testing, or fixing the
> problems you find during testing?

Most of the time is spent on fixing build issues which crop up during
the beta phase. I found difficult to enforce a strict policy on not
changing anything unless critical once in the beta phase. I think we
should improve things in that aspect, and go away from the "but this
is a small fix" mentality - maybe using something like for the linux
kernel, with merge windows, etc... I secretly hope that if we can
regularly change release managers, it will give a sense of why this is
good policy :)

I feel that we have improved things quite a bit since I have started
doing releases: the binary installers are more stable, and build are
mostly automated now. The next step would be automated testing of the
binary installers (in particular testing new numpy against scipy,
etc...), but this is quite a bit of work.

Having a stricter time-based policy would be good as well.

> Do you have an idea about when to start preparing for the release of 1.4.1?

The cython thing is the most problematic bug, and should be fixed
ASAP. I am still not sure whether that would require both a numpy
1.4.1 and a scipy 0.7.1.1 (built against numpy 1.3.0).

cheers,

David