[Python-Dev] Automated Python testing (was Re: status of development documentation)

["Martin v. Löwis"]

Jean-Paul Calderone wrote:
> A slave is an entity capable of performing tasks.  It can be 
> asked to perform any task you like, though it may not be able 
> to perform them all if it lacks some requirements.

This is clear in principle. However, what constitutes a "task"?
I see that you can send it shell commands to execute, arbitrary
ones, with environment variables. What else? Can you send it
Python code?

> A builder is a particular job.  It can be performed by any 
> slave (so long as its requirements are met), or by multiple 
> slaves.

That I'm not so sure of. In a builder, I give a single value
"slavename", and I understand that only that single slave will
ever get jobs from the builder - not any slave. I also read
that I can give "slavenames" instead, but that I should do so
only if the slaves are sufficiently similar (for some metric
which I apparently have to define myself).

> A factory defines the work to be done by a builder.  If which 
> compiler is being used is an important part of the purpose of 
> a builder (for example, there might be a builder the purpose of 
> which is to test a Python built with GCC 3.4 and another the 
> purpose of which is to test a Python built with GCC 4.0), then 
> it should be specified in the master configuration.  If it is 
> not important, then it should be left as general as possible, 
> to allow for the most slaves to be able to complete the task.

So would the assumption be that I use the same factory for
multiple builders? I'm gravitating towards a 1:1:1 relationship
between factories, builders, and slaves. For example, on OSX,
I think I have to give --with-suffix=.exe to configure. This
means I have to create a separate factory, which then only
applies to OSX builders (of which I have only one), which
has just a single slavename.

> I have most often seen branches supported by allowing commits 
> to trunk to automatically trigger a build on trunk, while 
> commits to branches do not automatically trigger any builds.

I think I want it differently: commits to release24-maint should
also trigger builds; commits to other branches should not trigger
builds. Does that mean I need twice as many builders? How
do I tell them what branch they should build?

> Builds on branches can then be explicitly requested when a 
> developer wants to know the state of their branch on various 
> platforms/configurations (often before merging to trunk to 
> know if they are introducing any regressions, but valuable 
> at other times as well).

Some people advised me that supporting web-initiated builds
is not a good idea. So I turned off all manual triggering of
builds for now. I would like to give committers permission to
trigger builds, but I'm not sure how to do that.

> I think this means tags/release24-maint won't ever be built 
> automatically (I haven't used AnyBranchScheduler, and I don't 
> know much about schedulers in buildbot in general).  You 
> should be able to manually request a build, but for some 
> reason I don't see the form for doing so on the master website 
> (<http://twistedmatrix.com/buildbot/full-2.3> for an example 
> of what this looks like).  I'm not sure if this is a buildbot
> version problem, or if there is just another piece of 
> configuration that needs to be set.

Ah, no: that's configuration that would need to be removed.
Jeff Pitmann suggested that this feature be disabled
(Waterfall(allowForce=False)). Before I did that, there
was indeed a form to request building of a branch.

Regards,
Martin