[Numpy-discussion] Buildbot/continuous integration (was Re: Issue Tracking)

[Matthew Brett]

Hi,

On Thu, Feb 16, 2012 at 4:12 PM, Nathaniel Smith <njs at pobox.com> wrote:
> On Thu, Feb 16, 2012 at 11:52 PM, Chris Ball <ceball at gmail.com> wrote:
>> Buildbot is used by some big projects (e.g. Python, Chromium, and
>> Mozilla), but I'm aware that several projects in the scientific/numeric
>> Python ecosystem use Jenkins (including Cython, IPython, and SymPy),
>> often using a hosted Jenkins solution such as Shining Panda. A difficult
>> part of running a Buildbot service is finding hardware for the slaves
>> and keeping them alive, so a hosted solution sounds wonderful (assuming
>> hosted solutions offer an adequate range of operating systems etc).
>
> A quick look at Shining Panda suggests that you get no coverage for
> anything but Linux, which is a good start but rather limiting. IME by
> far the most annoying part of a useful buildbot setup is keeping all
> the build slaves up and working. It's one thing to set up a build
> environment in one OS, it's quite another to keep like 5 of them
> working, each on a different volunteered machine where you don't have
> root and the person who does isn't answering email... the total effort
> isn't large, but it's really poorly suited to the nature of volunteer
> labor, because it needs prompt attention at random intervals. (Also,
> this doesn't become obvious until after one's already gotten
> everything set up, so then you're stuck limping along because who
> wants to start over and build something more maintainable...)
>
> If anyone has existing sysadmin resources then keeping build-slaves
> running is a place where they'd be a huge contribution.

Yup - keeping the slaves running is the big problem.

We do have various slaves running here that at least are all
accessible by me, and Jarrod, and (at a pinch) Fernando, Stefan and
others.

These are:

XP (when I'm not using the machine, which is the large majority of the time)
OSX 10.5
OSX 10.4 PPC
Linux 32 bit
Linux 64 bit

These are all real machines not virtual machines.  I'm happy to give
some reliable person ssh access to the buildslave user on these
machines.  They won't necessarily be available for all time, they are
dotted around campus doing various jobs like being gateways, project
machines, occasional desktops.

See you,

Matthew