[core-workflow] Use buildbot to test tracker patches

[anatoly techtonik]

On Sat, Jul 5, 2014 at 8:43 PM, Ezio Melotti <ezio.melotti at gmail.com> wrote:
> On Sat, Jul 5, 2014 at 8:06 PM, francis <francismb at email.de> wrote:
>> On 07/05/2014 02:46 PM, Ezio Melotti wrote:
>>>
>>>    1) a new "Test with buildbot" (or whatever) button is added to the
>>> tracker next to each patch, and developers/triagers can press it to
>>> trigger the build;
>>
>>
>> Just an small detail/question: does it make a difference to change the
>> "trigger the build"-button with a "continuously check this patch"
>> -checkbox? The idea is not to check when the button is pressed
>> (if I've understood the original idea correctly) but just to set the
>> check and from then on get continuously feedback that's (still) working
>> (e.g. with a small green icon nearby the patch).
>
> The advantage of this is that people can know immediately if a patch
> still applies (assuming someone checked the checkbox before and that
> the check runs often enough).
> However this will waste lot of machine time if no one is looking at
> the patch for a long time, especially if we consider that there are
> over 2000 patches currently on the tracker.

We already know which files patch touches:
https://bitbucket.org/techtonik/python-stdlib/src/567adbfaa3bf1de905949e38a27bee0334081174/modstats.py?at=default#cl-167
so we can automatically detect a subset of patches to try on each
commit. We only need a DB with analyzed patch info to run queries.

Independent pieces of code that need to be implemented as I see it:

1.  [x] patch upload -> [ ] patch detection -> [ ] patch reaction ->
-> [x] patch analysis (get paths) -> [ ] store info (I'd use some key/value)

2.  [ ] post-commit hook to detect broken patches -> [ ] detect paths ->
-> [ ] run query to select patches -> [ ] schedule patch tests against
specific commits

3. [ ] get patch from the test queue -> [ ] checkout revision -> [ ] test
-> [ ] store result to DB

4. [ ] roundup GUI indicator -> [ ] query DB for patch status

> On the technical side there are two issues:
>   1) it's easier to send the request whenever someone press the
> button, rather than having something that keeps track of all the
> patches that need to be checked and either keeps checking it or keeps
> sending requests to the buildbot master;
>   2) the green icon is a good idea, but also requires either a
> different way for the buildbot master to inform the tracker of the
> results, or some way to intercept the email send to the tracker and
> convert it to a green/red icon instead of adding a regular message.  A
> third and simpler option is to just add the icon via JS on the
> client-side, and possibly also collapse buildbot messages so that they
> don't flood the message list.

JS that accesses specific URL about patch status.
http://hostname/patches/file5555/status
{
   status: "ok",   # "check pending..", "broken by commit..", "applied.."
}

The task seems huge, so instead of making half-baked hack, I propose to
add some engineering - write down the roadmap above and concentrate on
that, then allow people to have fun adding meat to the bones.

This will be a good practice to test more focused distributed workflow.
-- 
anatoly t.