[core-workflow] Starting the improved workflow discussion again

[Brett Cannon]

On Tue, Jul 21, 2015 at 6:07 AM Maciej Szulik <soltysh at gmail.com> wrote:

> On Tue, Jul 21, 2015 at 4:15 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
>
>> On 21 July 2015 at 05:49, Brett Cannon <bcannon at gmail.com> wrote:
>> > In my ideal workflow scenario, these are the steps a patch would take:
>> >
>> > Issue is created
>> > Issue is triaged to have right affected versions, etc.
>> > Patch is uploaded
>> > CI kicks the patch off for all branches and OSs that are affected
>> > CI flags what branches and OSs did (not) pass or apply cleanly to
>> > If necessary, another patch that works in a specific branch that is
>> affected
>> > is uploaded (obviously requires some way to flag that a patch applies
>> to a
>> > specific branch, deciding how to deal with Misc/NEWS, etc.)
>> > Code review -- with a tool other than Rietveld -- from a core developer
>> with
>> > feedback
>> > New version of patch uploaded, usual CI kicked off
>> > If everything looks good and CI is green, get patch approval from a
>> core dev
>> > Approval submits the patch(es) to the appropriate branches
>> > CI triggered yet again, and if tests fail then patch(es) are
>> automatically
>> > rolled back
>> >
>> > Now I realize this is not about to launch immediately. There are
>> changes to
>> > Roundup in there, a reliable test suite that actually fails only on
>> failures
>> > and not because it's flaky, etc. But the key point here is that
>> everything
>> > that can be automated is, and code reviews can occur entirely through a
>> > browser.
>>
>> I think you're conflating some different issues here, at least two of
>> which are worth separating out from each other:
>>
>> 1. Completely online workflow for documentation editing
>> 2. Pre-commit CI for CPython
>>
>> Both of the current forge.python.org proposals are aimed primarily at
>> the first problem, since they start out with purely documentation
>> repos like the developer guide and the PEPs. Hopefully we can also
>> eventually separate out "version independent" repos for the how to
>> guides and the tutorial. In addition to a completely online process
>> for documentation editing, review, and approval, the other key benefit
>> to these repos would be that *access management* would also be online,
>> rather than requiring shell access to hg.python.org.
>>
>> Documentation projects are a good starting point for this side of
>> things, as they're inherently lower risk. The worst thing
>> documentation can do is give readers bad advice, it can't force them
>> to follow it.
>>
>> This means that for forge.python.org, I think "What about CPython?"
>> should be something we take into account as a "What's next?" for the
>> service, but our near term focus should be on making things like the
>> developer guide and PEPs trivial to suggest edits to, trivial to
>> approve edits to, and trivial to grant approval rights over. Those
>> levels of access (who can suggest edits, who can approve edits, who
>> can approve edit rights for others) should also all be completely
>> transparent and changes in them should be tracked automatically rather
>> than requiring manual updates to a text file.
>>
>> Pre-commit CI for CPython is a different story - as David points out,
>> it is *not* OK to run code on the Buildbot fleet that hasn't been
>> approved by a core developer. Folks are trusting *us* to run code on
>> their systems, not random developers posting patches to
>> bugs.python.org. Noah (quite sensibly) isn't interested in getting the
>> PSF Infrastructure team involved in running random code from the
>> internet either.
>>
>> That's where the system Kushal set up in collaboration with the CentOS
>> folks potentially comes in:
>> https://mail.python.org/pipermail/python-dev/2015-May/140050.html
>>
>> That's just a simple "smoke test" to say "Does the proposed change
>> pass on x86_64 systems running CentOS 7?". If we could combine it with
>> a similar system for running Windows smoke tests in Appveyor, I think
>> we'd flush out a lot of basic cross-platform compatibility issues
>> pre-commit, regardless of whether folks are working locally on a *nix
>> system or a Windows one. (We wouldn't catch *everything*, because
>> Linux is not FreeBSD is not Mac OS X, etc, but we'd catch a lot of
>> them).
>>
>> At the moment, running those requires that we be logged into IRC, be
>> approved to trigger test runs, and indicate which issue we'd like to
>> test.
>>
>> If we instead had a "test" link next to patch files in Roundup, then a
>> core developer, completely online, could:
>>
>> 1. Read over the patch to see if we think its reasonable to smoke test
>> 2. Trigger the smoke test directly from Roundup
>> 3. Receive the results back as Roundup comments, with links to the run
>> logs
>>
>>
> We (openshift) have similar technique developed around vagrant + jenkins
> where
> we can kick of by commenting on a github PR to either run the tests or
> perform
> actual merge. Both of these operation run exactly the same suite of tests,
> but only the merge performs actual, well merging. Obviously only certain
> group
> of core devs has the rights to tag the PRs. Additionally when such
> tag/comment already exists all the new changes (eg. additional changes
> after review)
> are tested/merged and this is something that should be considered in this
> case
> as well (there are some slight nuances, but I don't want to go too much
> into details).
> Does the tag applies always or only once, and if once how you know
> it was applied already? So having a tag on/off future would be desirable
> imho.
> I'd be happy to help triaging this problem as much as my time allows me ;)
>

Thanks for the offer to help. The input of people with experience with a
wide-range of systems will help make sure we don't botch this. =)

-Brett


>
> Maciej
>
>
>> As we gained further familiarity and confidence with the system, we
>> could extend the trust for running pre-commit test runs to anyone that
>> has been granted Developer privileges on the issue tracker, rather
>> than restricting it specifically to core developers. (BTW, we should
>> probably come up with an icon for folks with elevated tracker
>> privileges - at the moment they're just marked as having signed the
>> CLA if they aren't also CPython core developers)
>>
>> Cheers,
>> Nick.
>>
>> --
>> Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
>>
> _______________________________________________
>> core-workflow mailing list
>> core-workflow at python.org
>> https://mail.python.org/mailman/listinfo/core-workflow
>> This list is governed by the PSF Code of Conduct:
>> https://www.python.org/psf/codeofconduct
>>
> _______________________________________________
> core-workflow mailing list
> core-workflow at python.org
> https://mail.python.org/mailman/listinfo/core-workflow
> This list is governed by the PSF Code of Conduct:
> https://www.python.org/psf/codeofconduct
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/core-workflow/attachments/20150721/efaa3d00/attachment-0001.html>