[Python-Dev] GH-NNNN vs #NNNN in merge commit

[Nathaniel Smith]

On Thu, Jan 25, 2018 at 1:46 PM, Barry Warsaw <barry at python.org> wrote:
> On Jan 25, 2018, at 13:38, Mariatta Wijaya <mariatta.wijaya at gmail.com> wrote:
>>
>> +1 for the mergebot! :)
>
> Yes, +1 from me too.  As you know, GitLab has the option to “merge when CI completes successfully” and it’s a great workflow.  Once I’ve reviewed and approved the branch, I can hit this button and… we’re done!  Assuming of course that no additional commits get pushed (that’s actually configurable I think), and that CI doesn’t fail.  I’d be very happy to see how close we can get to that with GitHub and a mergebot.

You should definitely check out bors-ng: https://bors.tech/

It's designed to support the workflow that projects like Rust and
Servo use: you tell the bot that a PR is good to merge, and then it
takes over and manages the CI process so as to guarantee that the head
of the master branch has always passed all its tests (the "not rocket
science rule": https://graydon2.dreamwidth.org/1597.html).

So not only does this give you the ability to mark a PR as good before
the CI has finished for that PR, like in GitLab, it also gives you a
few more things:

- one problem with the normal way of using Travis is that the tests on
a PR branch can get stale: the PR worked when it was submitted, but in
the mean time something else got merged to master that breaks it, but
no-one notices until after it's merged. With bors-ng, the bot always
tests exactly the snapshot that will become the new master, so it
catches cases like this before they're merged.

- currently, IIUC, the two times we run automatic tests are when a PR
is submitted, and after it's merged. And since we can't run
non-sandboxed tests like the buildbots automatically on submit, this
means we have to run them after merge, which is suboptimal. This adds
a new test step between approval and merging, so it gives the option
of turning buildbots into a pre-merge check. (Whether this is a good
idea or not I don't know, but it seems like a nice option to have?)

- if PRs are coming in faster than the CI system can keep up, then it
automatically batches up changes to test together, and if the batch
fails it uses binary search to automatically figure out which PR or
PRs are responsible. So if appveyor gets clogged up, or you enable
some slow buildbots, it should still be able to keep things moving.

Anyway, I don't know if it's exactly what cpython wants, but it's at
the least got some really interesting ideas.

-n

-- 
Nathaniel J. Smith -- https://vorpus.org