From brett at python.org  Fri Jun  2 15:10:52 2017
From: brett at python.org (Brett Cannon)
Date: Fri, 02 Jun 2017 19:10:52 +0000
Subject: [core-workflow] Requirements to make AppVeyor a required status
 check
Message-ID: <CAP1=2W5Y6rWSyMU2_sVbBsr4qAWVwWvNT9UpdEvy-BDtLN_6-w@mail.gmail.com>

https://github.com/python/core-workflow/issues/41 is tracking the idea of
requiring AppVeyor to pass for a PR to be mergeable so as to prevent people
breaking Windows. Since two commits fixing tests for AppVeyor just landed I
want to see if we can go a week without AppVeyor failing because of
AppVeyor itself. If that threshold is met, I plan to turn on the
requirement.

I'm stating this here mostly so people know to update the issue if a
transient failure is spotted so that it can be fixed and the one-week clock
can be reset.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/core-workflow/attachments/20170602/31d1c291/attachment.html>

From solipsis at pitrou.net  Sat Jun  3 06:17:37 2017
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sat, 3 Jun 2017 12:17:37 +0200
Subject: [core-workflow] Requirements to make AppVeyor a required status
 check
References: <CAP1=2W5Y6rWSyMU2_sVbBsr4qAWVwWvNT9UpdEvy-BDtLN_6-w@mail.gmail.com>
Message-ID: <20170603121737.74a850eb@fsol>

On Fri, 02 Jun 2017 19:10:52 +0000
Brett Cannon <brett at python.org> wrote:
> https://github.com/python/core-workflow/issues/41 is tracking the idea of
> requiring AppVeyor to pass for a PR to be mergeable so as to prevent people
> breaking Windows. Since two commits fixing tests for AppVeyor just landed I
> want to see if we can go a week without AppVeyor failing because of
> AppVeyor itself. If that threshold is met, I plan to turn on the
> requirement.

Is our AppVeyor hook still active? I can't see any build appearing here:
https://github.com/python/cpython/pull/1922

Regards

Antoine.



From brett at python.org  Sat Jun  3 14:36:42 2017
From: brett at python.org (Brett Cannon)
Date: Sat, 03 Jun 2017 18:36:42 +0000
Subject: [core-workflow] Requirements to make AppVeyor a required status
 check
In-Reply-To: <20170603121737.74a850eb@fsol>
References: <CAP1=2W5Y6rWSyMU2_sVbBsr4qAWVwWvNT9UpdEvy-BDtLN_6-w@mail.gmail.com>
 <20170603121737.74a850eb@fsol>
Message-ID: <CAP1=2W42FEbhQ5nM0ArCRyNxcVMEx_keTUeO+HurTjXSHQUK+Q@mail.gmail.com>

On Sat, 3 Jun 2017 at 03:17 Antoine Pitrou <solipsis at pitrou.net> wrote:

> On Fri, 02 Jun 2017 19:10:52 +0000
> Brett Cannon <brett at python.org> wrote:
> > https://github.com/python/core-workflow/issues/41 is tracking the idea
> of
> > requiring AppVeyor to pass for a PR to be mergeable so as to prevent
> people
> > breaking Windows. Since two commits fixing tests for AppVeyor just
> landed I
> > want to see if we can go a week without AppVeyor failing because of
> > AppVeyor itself. If that threshold is met, I plan to turn on the
> > requirement.
>
> Is our AppVeyor hook still active? I can't see any build appearing here:
> https://github.com/python/cpython/pull/1922


Yep as it built your commit:
https://ci.appveyor.com/project/python/cpython/branch/master. So that would
suggest that AppVeyor either had a hiccup and missed your PR  or they got a
bug in their hook.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/core-workflow/attachments/20170603/3e2a0e45/attachment.html>

From brett at python.org  Fri Jun  9 19:36:46 2017
From: brett at python.org (Brett Cannon)
Date: Fri, 09 Jun 2017 23:36:46 +0000
Subject: [core-workflow] Creating a PyPI account for workflow tools
Message-ID: <CAP1=2W52X8kP17ow_6RpUp2=Xft4RRK7cmnC1e6kSsDvgQapUg@mail.gmail.com>

I'm thinking we should create an account to own things like cherry_picker
and blurb for when they get put on PyPI. I'm not sure if it should be
specific to core-workflow or generic to python-dev, though? Anyone have an
opinion on the scope of the account?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/core-workflow/attachments/20170609/708f576b/attachment.html>

From ncoghlan at gmail.com  Sat Jun 10 01:08:31 2017
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 10 Jun 2017 15:08:31 +1000
Subject: [core-workflow] Creating a PyPI account for workflow tools
In-Reply-To: <CAP1=2W52X8kP17ow_6RpUp2=Xft4RRK7cmnC1e6kSsDvgQapUg@mail.gmail.com>
References: <CAP1=2W52X8kP17ow_6RpUp2=Xft4RRK7cmnC1e6kSsDvgQapUg@mail.gmail.com>
Message-ID: <CADiSq7eLc_dSTZuAr6S+PbGpQS_mBmZz0QyC_qjViFJCvx7nzQ@mail.gmail.com>

On 10 June 2017 at 09:36, Brett Cannon <brett at python.org> wrote:
> I'm thinking we should create an account to own things like cherry_picker
> and blurb for when they get put on PyPI. I'm not sure if it should be
> specific to core-workflow or generic to python-dev, though? Anyone have an
> opinion on the scope of the account?

Principle of least privilege suggests a `core-workflow` account would
make sense. It may also be worth talking to Jannis Leidel (founder of
the Jazzband package maintenance collective), since he has some nice
automation set up whereby GitHub tag -> PyPI release without any
manual intervention (that's how I publish contextlib2 releases these
days).

Things like the docs theme could then also happily live under
core-workflow from a release automation perspective.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From mariatta.wijaya at gmail.com  Sun Jun 11 13:04:54 2017
From: mariatta.wijaya at gmail.com (Mariatta Wijaya)
Date: Sun, 11 Jun 2017 10:04:54 -0700
Subject: [core-workflow] Do we still have mention-bot?
In-Reply-To: <CAGbohnZrh1_ip_9picC+K=Jv4a8i=Wi+t7Au9WYiqu9_Kc_8gw@mail.gmail.com>
References: <CAGbohnYyzBrSQ+YZmCxTS5DRrRjvOv9Y1d=pbaiE9+GmMA2CWQ@mail.gmail.com>
 <CAGbohna3RepEfAr+nV0esf+n-_EkGcJM+yaFw5-AtfV8Ax+Lrg@mail.gmail.com>
 <CAGbohnZrh1_ip_9picC+K=Jv4a8i=Wi+t7Au9WYiqu9_Kc_8gw@mail.gmail.com>
Message-ID: <CAGbohnbXbasTN2u9w+b+3FXqVPGS+KrRUyBuy9m7WUe1=FCJag@mail.gmail.com>

The newest PRs in CPython don't have any comment from mention-bot.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/core-workflow/attachments/20170611/2f68a6f3/attachment.html>

From brett at python.org  Mon Jun 12 13:08:41 2017
From: brett at python.org (Brett Cannon)
Date: Mon, 12 Jun 2017 17:08:41 +0000
Subject: [core-workflow] Do we still have mention-bot?
In-Reply-To: <CAGbohnbXbasTN2u9w+b+3FXqVPGS+KrRUyBuy9m7WUe1=FCJag@mail.gmail.com>
References: <CAGbohnYyzBrSQ+YZmCxTS5DRrRjvOv9Y1d=pbaiE9+GmMA2CWQ@mail.gmail.com>
 <CAGbohna3RepEfAr+nV0esf+n-_EkGcJM+yaFw5-AtfV8Ax+Lrg@mail.gmail.com>
 <CAGbohnZrh1_ip_9picC+K=Jv4a8i=Wi+t7Au9WYiqu9_Kc_8gw@mail.gmail.com>
 <CAGbohnbXbasTN2u9w+b+3FXqVPGS+KrRUyBuy9m7WUe1=FCJag@mail.gmail.com>
Message-ID: <CAP1=2W7Fwt4+phPeWbGPJ-hLho5Lge0fx=Z=mFWZVQbS4sfOoA@mail.gmail.com>

Facebook's instance that we use was returning a 500 to the webhooks. I just
redelivered the payloads for the failures on the first page of webhook
events.

On Sun, 11 Jun 2017 at 10:05 Mariatta Wijaya <mariatta.wijaya at gmail.com>
wrote:

> The newest PRs in CPython don't have any comment from mention-bot.
>
> _______________________________________________
> core-workflow mailing list
> core-workflow at python.org
> https://mail.python.org/mailman/listinfo/core-workflow
> This list is governed by the PSF Code of Conduct:
> https://www.python.org/psf/codeofconduct
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/core-workflow/attachments/20170612/385647d9/attachment.html>

From brett at python.org  Sat Jun 17 15:07:50 2017
From: brett at python.org (Brett Cannon)
Date: Sat, 17 Jun 2017 19:07:50 +0000
Subject: [core-workflow] Auto labeling PRs based on what stage they're in
Message-ID: <CAP1=2W6AqaiV26xs0sdD4Dr_2tLKA5MxYMtC=xR501H8Na2h-Q@mail.gmail.com>

I don't know about the rest of you, but I want an easy way to glance at a
pull request and know what it's currently blocking it from being merged (if
I'm wrong then let me know). I opened
https://github.com/python/core-workflow/issues/94 for this idea, but I
figured what the exact stages should be deserves a wider discussion.

The stages I see are:

   1. *Awaiting first review*: no one has reviewed the PR
   2. *Awaiting core review*: a non-core dev has reviewed the PR, but there
   still needs to be that initial core review (still not sure if this is worth
   the distinction or how complex it will make the code, but it may encourage
   people to help in the reviewing process instead of automatically diving
   into coding if their review leads to a visible stage change)
   3. *Awaiting changes*: a review has been done, but changes were
   requested (probably wouldn't reach this stage for non-core reviews; a bit
   tough to block on a non-core review as their asks may not be reasonable and
   so shouldn't automatically signal that what someone that they must do what
   is requested of them in that instance)
   4. *Awaiting Nth review/re-review*: the PR submitter believes all the
   requests have been addressed in all reviews and so is waiting on the
   reviewer(s) who requested changes to review again (will need to provide a
   way to leave a message that signals to Bedevere that the submitter feels
   ready for another review)
   5. *Awaiting merge*: all reviews approve of the PR (we could also drop
   the "awaiting core review" stage and go straight here even for non-core
   reviews that are all approvals, but maybe that's leaning on non-core devs
   too much and giving false hope to PR submitters?)

Am I missing a step?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/core-workflow/attachments/20170617/3c5a6945/attachment.html>

From ncoghlan at gmail.com  Sat Jun 17 21:11:10 2017
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 18 Jun 2017 11:11:10 +1000
Subject: [core-workflow] Auto labeling PRs based on what stage they're in
In-Reply-To: <CAP1=2W6AqaiV26xs0sdD4Dr_2tLKA5MxYMtC=xR501H8Na2h-Q@mail.gmail.com>
References: <CAP1=2W6AqaiV26xs0sdD4Dr_2tLKA5MxYMtC=xR501H8Na2h-Q@mail.gmail.com>
Message-ID: <CADiSq7eQ-+weG-KT58OAFdSqHEha4hDa=0ACNodU-HFrMbKrBQ@mail.gmail.com>

On 18 June 2017 at 05:07, Brett Cannon <brett at python.org> wrote:
> The stages I see are:
>
> 1. Awaiting first review: no one has reviewed the PR

I'd just make this "Awaiting review", and have it advance only for a
positive review with no negative reviews.

> 2. Awaiting core review: a non-core dev has reviewed the PR, but there still needs to be that initial core review (still not sure if this is worth the distinction or how complex it will make the code, but it may encourage people to help in the reviewing process instead of automatically diving into coding if their review leads to a visible stage change)

I think this is worth having for the reason you give - anyone can
advance the state to "awaiting core review", and it gives a visual
indication of "at least 2 humans think this is a reasonable patch" for
core developers looking to pick up pre-filtered proposals.

> 3. Awaiting changes: a review has been done, but changes were requested (probably wouldn't reach this stage for non-core reviews; a bit tough to block on a non-core review as their asks may not be reasonable and so shouldn't automatically signal that what someone that they must do what is requested of them in that instance)

+1 for only getting here based on negative core reviews. An
alternative to requiring a special comment to exit this state would be
to add a "WIP: " prefix to the PR title when the label is added - that
way the submitter can just remove the prefix to indicate that they
have made the requested changes and are ready for another round of
review, while folks could also add the prefix manually to indicate the
change *isn't* ready for review. (GitLab has specific handling for
"WIP:" and "[WIP]" to prevent merges, and it's really handy as a
communications tool when you want feedback on a draft change, but also
want to make it completely clear that the change isn't ready to be
merged yet)

> 4. Awaiting Nth review/re-review: the PR submitter believes all the requests have been addressed in all reviews and so is waiting on the reviewer(s) who requested changes to review again (will need to provide a way to leave a message that signals to Bedevere that the submitter feels ready for another review)

I don't see a need for this state - going back to "Awaiting core
review" should suffice.

> 5. Awaiting merge: all reviews approve of the PR (we could also drop the "awaiting core review" stage and go straight here even for non-core reviews that are all approvals, but maybe that's leaning on non-core devs too much and giving false hope to PR submitters?)

I assume we'd mainly get here based on all positive core reviews, no
pending negative core reviews, but a pending CI run to check
everything is working as expected. At some point in the future, we
could potentially even go to an OpenStack style flow where Bedevere
actually *merges* patches in this state if their CI run succeeds, so
it wouldn't be reasonable to get here based solely on non-core
reviews.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From brett at python.org  Sun Jun 18 12:36:24 2017
From: brett at python.org (Brett Cannon)
Date: Sun, 18 Jun 2017 16:36:24 +0000
Subject: [core-workflow] Auto labeling PRs based on what stage they're in
In-Reply-To: <CADiSq7eQ-+weG-KT58OAFdSqHEha4hDa=0ACNodU-HFrMbKrBQ@mail.gmail.com>
References: <CAP1=2W6AqaiV26xs0sdD4Dr_2tLKA5MxYMtC=xR501H8Na2h-Q@mail.gmail.com>
 <CADiSq7eQ-+weG-KT58OAFdSqHEha4hDa=0ACNodU-HFrMbKrBQ@mail.gmail.com>
Message-ID: <CAP1=2W4oBJiA+fk1+Of4jKqbU=3xFf8riZin6zXfQK5ch8csGw@mail.gmail.com>

On Sat, 17 Jun 2017 at 18:11 Nick Coghlan <ncoghlan at gmail.com> wrote:

> On 18 June 2017 at 05:07, Brett Cannon <brett at python.org> wrote:
> > The stages I see are:
> >
> > 1. Awaiting first review: no one has reviewed the PR
>
> I'd just make this "Awaiting review",


+1 to the simplification of the name.


> and have it advance only for a
> positive review with no negative reviews.
>

But wouldn't that lead to more than one stage being set since this would
perpetually be set until you were ready to merge? As I said later, I don't
know if we want to gate on a non-core devs approval as it might not be
reasonable (e.g. we have had some non-core devs be extremely picky about
PEP 8 conformance).


>
> > 2. Awaiting core review: a non-core dev has reviewed the PR, but there
> still needs to be that initial core review (still not sure if this is worth
> the distinction or how complex it will make the code, but it may encourage
> people to help in the reviewing process instead of automatically diving
> into coding if their review leads to a visible stage change)
>
> I think this is worth having for the reason you give - anyone can
> advance the state to "awaiting core review", and it gives a visual
> indication of "at least 2 humans think this is a reasonable patch" for
> core developers looking to pick up pre-filtered proposals.
>

Don't you mean "one other person" based on the advancement from "awaiting
review"? Sorry if I wasn't clear before, but what I was thinking was that
if a core dev did the initial review then this stage would be skipped. IOW
a single core dev review is all that's needed to get out of any "I need a
review" stage.

If you do mean you want to require two reviewers for all PRs, I'm not sure
if we are there yet. We already have a shortage of reviewers as it is; at
the language summit I mentioned that we were at a +3 delta for open PRs
each day, so slowing things down by requiring two reviews is premature
until we get that growth rate of open PRs under control (not to say I
wouldn't love to get to the point of requiring two reviews, but I think we
need to get a few more pieces in place before we do that, plus we have not
proven that non-core devs are going to step up to help with reviewing
either).


>
> > 3. Awaiting changes: a review has been done, but changes were requested
> (probably wouldn't reach this stage for non-core reviews; a bit tough to
> block on a non-core review as their asks may not be reasonable and so
> shouldn't automatically signal that what someone that they must do what is
> requested of them in that instance)
>
> +1 for only getting here based on negative core reviews. An
> alternative to requiring a special comment to exit this state would be
> to add a "WIP: " prefix to the PR title when the label is added - that
> way the submitter can just remove the prefix to indicate that they
> have made the requested changes and are ready for another round of
> review, while folks could also add the prefix manually to indicate the
> change *isn't* ready for review. (GitLab has specific handling for
> "WIP:" and "[WIP]" to prevent merges, and it's really handy as a
> communications tool when you want feedback on a draft change, but also
> want to make it completely clear that the change isn't ready to be
> merged yet)
>

Guido squashed the title prefix idea at the language summit (it's what I
originally proposed, so this label approach was the compromise).


>
> > 4. Awaiting Nth review/re-review: the PR submitter believes all the
> requests have been addressed in all reviews and so is waiting on the
> reviewer(s) who requested changes to review again (will need to provide a
> way to leave a message that signals to Bedevere that the submitter feels
> ready for another review)
>
> I don't see a need for this state - going back to "Awaiting core
> review" should suffice.
>

So the reason I made this separate is if you're in the middle of working
with someone on a PR and you're already reviewing it, do I really need to
get involved? If I'm perusing the list of open PRs, wouldn't it be better
for me to look for another PR that has zero reviews than pile on to another
PR that already has a core dev reviewing it?

But if you take a view of "more reviews is always good" then there's no
need to distinguish as you suggested. Then you could use the fact that you
were notified by GitHub as the signal that you put in a review (or if
Bedevere makes a comment notifying the change in status and @ mentions you
which would be easy to filter for in emails).


>
> > 5. Awaiting merge: all reviews approve of the PR (we could also drop the
> "awaiting core review" stage and go straight here even for non-core reviews
> that are all approvals, but maybe that's leaning on non-core devs too much
> and giving false hope to PR submitters?)
>
> I assume we'd mainly get here based on all positive core reviews, no
> pending negative core reviews, but a pending CI run to check
> everything is working as expected.


It can also be reached even if the CI is cleared but the PR will require
backporting and the person who plans to do the merge + backport doesn't
have the time ATM (e.g. I did a review of a PR that applies to master and
3.6 on my lunch break, but I'm waiting until I have time to run
cherry_picker at home to do the merge so I don't lose track of the fact
that I need to do the backport).


> At some point in the future, we
> could potentially even go to an OpenStack style flow where Bedevere
> actually *merges* patches in this state if their CI run succeeds, so
> it wouldn't be reasonable to get here based solely on non-core
> reviews.
>

Donald beat you to the suggestion :)
https://github.com/python/core-workflow/issues/29
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/core-workflow/attachments/20170618/ec44ecdb/attachment.html>

From mariatta.wijaya at gmail.com  Sun Jun 18 12:56:04 2017
From: mariatta.wijaya at gmail.com (Mariatta Wijaya)
Date: Sun, 18 Jun 2017 09:56:04 -0700
Subject: [core-workflow] Auto labeling PRs based on what stage they're in
In-Reply-To: <CAP1=2W4oBJiA+fk1+Of4jKqbU=3xFf8riZin6zXfQK5ch8csGw@mail.gmail.com>
References: <CAP1=2W6AqaiV26xs0sdD4Dr_2tLKA5MxYMtC=xR501H8Na2h-Q@mail.gmail.com>
 <CADiSq7eQ-+weG-KT58OAFdSqHEha4hDa=0ACNodU-HFrMbKrBQ@mail.gmail.com>
 <CAP1=2W4oBJiA+fk1+Of4jKqbU=3xFf8riZin6zXfQK5ch8csGw@mail.gmail.com>
Message-ID: <CAGbohnbX0fkJ066Wn8yW9SbkswYs0+O=aQHcHdEGEhomG5zh5w@mail.gmail.com>

On Sun, Jun 18, 2017 at 9:36 AM, Brett Cannon <brett at python.org> wrote:

> At some point in the future, we
>> could potentially even go to an OpenStack style flow where Bedevere
>> actually *merges* patches in this state if their CI run succeeds,
>
>
One concern is about cleaning up the commit message before merging,
something core devs can/should do.
How will this work if the PR get merged automatically by Bedevere?

Mariatta Wijaya
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/core-workflow/attachments/20170618/bead765c/attachment.html>

From brett at python.org  Sun Jun 18 19:02:57 2017
From: brett at python.org (Brett Cannon)
Date: Sun, 18 Jun 2017 23:02:57 +0000
Subject: [core-workflow] Auto labeling PRs based on what stage they're in
In-Reply-To: <CAGbohnbX0fkJ066Wn8yW9SbkswYs0+O=aQHcHdEGEhomG5zh5w@mail.gmail.com>
References: <CAP1=2W6AqaiV26xs0sdD4Dr_2tLKA5MxYMtC=xR501H8Na2h-Q@mail.gmail.com>
 <CADiSq7eQ-+weG-KT58OAFdSqHEha4hDa=0ACNodU-HFrMbKrBQ@mail.gmail.com>
 <CAP1=2W4oBJiA+fk1+Of4jKqbU=3xFf8riZin6zXfQK5ch8csGw@mail.gmail.com>
 <CAGbohnbX0fkJ066Wn8yW9SbkswYs0+O=aQHcHdEGEhomG5zh5w@mail.gmail.com>
Message-ID: <CAP1=2W43i49voLGd7ZnRXijBm5xygiC=T7mcsejyBiZ1oRrUew@mail.gmail.com>

On Sun, Jun 18, 2017, 09:56 Mariatta Wijaya, <mariatta.wijaya at gmail.com>
wrote:

>
> On Sun, Jun 18, 2017 at 9:36 AM, Brett Cannon <brett at python.org> wrote:
>
>> At some point in the future, we
>>> could potentially even go to an OpenStack style flow where Bedevere
>>> actually *merges* patches in this state if their CI run succeeds,
>>
>>
> One concern is about cleaning up the commit message before merging,
> something core devs can/should do.
> How will this work if the PR get merged automatically by Bedevere?
>

If people want to talk about this then let's keep it to the issue on GH as
it's totally speculative ATM and I would like to keep this thread on-topic.

-Brett



> Mariatta Wijaya
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/core-workflow/attachments/20170618/19e1e529/attachment-0001.html>

From ncoghlan at gmail.com  Mon Jun 19 07:42:55 2017
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 19 Jun 2017 21:42:55 +1000
Subject: [core-workflow] Auto labeling PRs based on what stage they're in
In-Reply-To: <CAP1=2W43i49voLGd7ZnRXijBm5xygiC=T7mcsejyBiZ1oRrUew@mail.gmail.com>
References: <CAP1=2W6AqaiV26xs0sdD4Dr_2tLKA5MxYMtC=xR501H8Na2h-Q@mail.gmail.com>
 <CADiSq7eQ-+weG-KT58OAFdSqHEha4hDa=0ACNodU-HFrMbKrBQ@mail.gmail.com>
 <CAP1=2W4oBJiA+fk1+Of4jKqbU=3xFf8riZin6zXfQK5ch8csGw@mail.gmail.com>
 <CAGbohnbX0fkJ066Wn8yW9SbkswYs0+O=aQHcHdEGEhomG5zh5w@mail.gmail.com>
 <CAP1=2W43i49voLGd7ZnRXijBm5xygiC=T7mcsejyBiZ1oRrUew@mail.gmail.com>
Message-ID: <CADiSq7evSS1=sVe=M_8fitpuh_cRW3Od1Xq18p_HM8nWjvpx=w@mail.gmail.com>

On 19 June 2017 at 09:02, Brett Cannon <brett at python.org> wrote:
> On Sun, Jun 18, 2017, 09:56 Mariatta Wijaya, <mariatta.wijaya at gmail.com>
> wrote:
>> On Sun, Jun 18, 2017 at 9:36 AM, Brett Cannon <brett at python.org> wrote:
>>>>
>>>> At some point in the future, we
>>>> could potentially even go to an OpenStack style flow where Bedevere
>>>> actually *merges* patches in this state if their CI run succeeds,
>>
>>
>> One concern is about cleaning up the commit message before merging,
>> something core devs can/should do.
>> How will this work if the PR get merged automatically by Bedevere?
>
> If people want to talk about this then let's keep it to the issue on GH as
> it's totally speculative ATM and I would like to keep this thread on-topic.

Sorry, yeah, that was a complete tangent. I was mainly trying to
explain why I think we genuinely need a core developer's review before
promoting a PR that far in the process.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From ncoghlan at gmail.com  Mon Jun 19 08:08:33 2017
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 19 Jun 2017 22:08:33 +1000
Subject: [core-workflow] Auto labeling PRs based on what stage they're in
In-Reply-To: <CAP1=2W4oBJiA+fk1+Of4jKqbU=3xFf8riZin6zXfQK5ch8csGw@mail.gmail.com>
References: <CAP1=2W6AqaiV26xs0sdD4Dr_2tLKA5MxYMtC=xR501H8Na2h-Q@mail.gmail.com>
 <CADiSq7eQ-+weG-KT58OAFdSqHEha4hDa=0ACNodU-HFrMbKrBQ@mail.gmail.com>
 <CAP1=2W4oBJiA+fk1+Of4jKqbU=3xFf8riZin6zXfQK5ch8csGw@mail.gmail.com>
Message-ID: <CADiSq7e+xB-z9EFpseHXAuwmvYfF+bC=+e1KAR2PMacsQ_=tLw@mail.gmail.com>

On 19 June 2017 at 02:36, Brett Cannon <brett at python.org> wrote:
> On Sat, 17 Jun 2017 at 18:11 Nick Coghlan <ncoghlan at gmail.com> wrote:
>> On 18 June 2017 at 05:07, Brett Cannon <brett at python.org> wrote:
>> > The stages I see are:
>> >
>> > 1. Awaiting first review: no one has reviewed the PR
>>
>> I'd just make this "Awaiting review",
>
> +1 to the simplification of the name.
>> and have it advance only for a
>> positive review with no negative reviews.
>
> But wouldn't that lead to more than one stage being set since this would
> perpetually be set until you were ready to merge? As I said later, I don't
> know if we want to gate on a non-core devs approval as it might not be
> reasonable (e.g. we have had some non-core devs be extremely picky about PEP
> 8 conformance).

I'm also fine with having mixed reviews mean "it's a core dev's problem now".

>> I think this is worth having for the reason you give - anyone can
>> advance the state to "awaiting core review", and it gives a visual
>> indication of "at least 2 humans think this is a reasonable patch" for
>> core developers looking to pick up pre-filtered proposals.
>
> Don't you mean "one other person" based on the advancement from "awaiting
> review"?

2 humans - the original submitter, and whoever did the initial review
(if they weren't a core dev).

> Sorry if I wasn't clear before, but what I was thinking was that if
> a core dev did the initial review then this stage would be skipped. IOW a
> single core dev review is all that's needed to get out of any "I need a
> review" stage.

Right, that's what I was assuming as well.

I also guess that when the submitter *is* a core dev, we'd send a PR
straight to "awaiting merge" unless we had already explicitly added
the "awaiting changes" or "awaiting core review" label.

> If you do mean you want to require two reviewers for all PRs, I'm not sure
> if we are there yet.

No, I didn't mean that (although I can see how you read it that way) -
I still don't think we should even make reviews mandatory for core dev
submitted PRs.

While I do think it would be nice to have the luxury of being able to
afford to implement such a policy, it's the kind of workflow that's
really only feasible when you have multiple folks handling reviews on
a regular basis as part of a full-time job (and even then it can cause
problems in niche areas of a code base where nobody feels particularly
well-equipped to carry out an authoritative review).

>> +1 for only getting here based on negative core reviews. An
>> alternative to requiring a special comment to exit this state would be
>> to add a "WIP: " prefix to the PR title when the label is added - that
>> way the submitter can just remove the prefix to indicate that they
>> have made the requested changes and are ready for another round of
>> review, while folks could also add the prefix manually to indicate the
>> change *isn't* ready for review.
>
> Guido squashed the title prefix idea at the language summit (it's what I
> originally proposed, so this label approach was the compromise).

Ah, fair enough. In that case, I agree that a "Bedevere: ready for
review" comment to request removal of the label make sense.

>> > 4. Awaiting Nth review/re-review: the PR submitter believes all the
>> > requests have been addressed in all reviews and so is waiting on the
>> > reviewer(s) who requested changes to review again (will need to provide a
>> > way to leave a message that signals to Bedevere that the submitter feels
>> > ready for another review)
>>
>> I don't see a need for this state - going back to "Awaiting core
>> review" should suffice.
>
> So the reason I made this separate is if you're in the middle of working
> with someone on a PR and you're already reviewing it, do I really need to
> get involved? If I'm perusing the list of open PRs, wouldn't it be better
> for me to look for another PR that has zero reviews than pile on to another
> PR that already has a core dev reviewing it?

OK, that rationale makes sense. Given that, I'd suggest making the
three review states:

- awaiting review
- awaiting core review
- awaiting change review

>> > 5. Awaiting merge: all reviews approve of the PR (we could also drop the
>> > "awaiting core review" stage and go straight here even for non-core reviews
>> > that are all approvals, but maybe that's leaning on non-core devs too much
>> > and giving false hope to PR submitters?)
>>
>> I assume we'd mainly get here based on all positive core reviews, no
>> pending negative core reviews, but a pending CI run to check
>> everything is working as expected.
>
> It can also be reached even if the CI is cleared but the PR will require
> backporting and the person who plans to do the merge + backport doesn't have
> the time ATM (e.g. I did a review of a PR that applies to master and 3.6 on
> my lunch break, but I'm waiting until I have time to run cherry_picker at
> home to do the merge so I don't lose track of the fact that I need to do the
> backport).

Aye, I can see that.

FWIW, I was doing things that way for a while, but eventually realised
it made more sense for me to make use of the "backport needed" state
on BPO, since that's closer to our target workflow with automated
backports. It also means I can batch up a bunch of relatively
mechanical backports to do via a local checkout later, rather than
having to switch back and forth between code review and backporting
things.

It definitely isn't perfect (Mariatta ended up cleaning up a few of
the backports from the last batch I did after I'd left them pending
for over a week), but given that the original PRs had previously been
lingering for months, I still count it as an improvement over my
initial approach ;)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From brett at python.org  Mon Jun 19 13:39:34 2017
From: brett at python.org (Brett Cannon)
Date: Mon, 19 Jun 2017 17:39:34 +0000
Subject: [core-workflow] Auto labeling PRs based on what stage they're in
In-Reply-To: <CADiSq7e+xB-z9EFpseHXAuwmvYfF+bC=+e1KAR2PMacsQ_=tLw@mail.gmail.com>
References: <CAP1=2W6AqaiV26xs0sdD4Dr_2tLKA5MxYMtC=xR501H8Na2h-Q@mail.gmail.com>
 <CADiSq7eQ-+weG-KT58OAFdSqHEha4hDa=0ACNodU-HFrMbKrBQ@mail.gmail.com>
 <CAP1=2W4oBJiA+fk1+Of4jKqbU=3xFf8riZin6zXfQK5ch8csGw@mail.gmail.com>
 <CADiSq7e+xB-z9EFpseHXAuwmvYfF+bC=+e1KAR2PMacsQ_=tLw@mail.gmail.com>
Message-ID: <CAP1=2W49Y4fVuFmHAcU1vJ-HXRFpZ7yRcjKCRHJXN=1G366OPw@mail.gmail.com>

OK, here is a dot graph that lays out the stages and the flow between them
based on what has been discussed so far. You can use
http://www.webgraphviz.com/ to view the graph in a browser. Anything green
requires/blocks on a core dev, blue requires/blocks on the PR creator, and
orange is open to anyone.

digraph "PR stages" {
  "New PR" [color=blue]
  "Awaiting review" [shape=box, color=orange]
  "Awaiting core review" [shape=box, color=green]
  "Awaiting changes" [shape=box, color=blue]
  "Awaiting change review" [shape=box, color=green]
  "Awaiting merge" [shape=box, color=green]

  "New PR" -> "Awaiting review" [label="PR created by non-core dev",
color=blue]
  "Awaiting review" -> "Awaiting core review" [label="Non-core review",
color=orange]
  "Awaiting core review" -> "Awaiting changes" [label="Core dev requests
changes", color=green]
  "Awaiting changes" -> "Awaiting change review" [label="PR creator
addresses changes", color=blue]
  "Awaiting change review" -> "Awaiting changes" [label="Core dev requests
changes", color=green]
  "Awaiting change review" -> "Awaiting merge" [label="Core dev approves
PR", color=green]


  "Awaiting review" -> "Awaiting merge" [label="Core dev approves PR",
color=green]
  "Awaiting review" -> "Awaiting changes" [label="Core dev requests
changes", color=green]
  "Awaiting core review" -> "Awaiting merge" [label="Core dev approves PR",
color=green]

  "New PR" -> "Awaiting merge" [label="PR by core dev", color=green]


On Mon, 19 Jun 2017 at 05:08 Nick Coghlan <ncoghlan at gmail.com> wrote:

> On 19 June 2017 at 02:36, Brett Cannon <brett at python.org> wrote:
> > On Sat, 17 Jun 2017 at 18:11 Nick Coghlan <ncoghlan at gmail.com> wrote:
> >> On 18 June 2017 at 05:07, Brett Cannon <brett at python.org> wrote:
> >> > The stages I see are:
> >> >
> >> > 1. Awaiting first review: no one has reviewed the PR
> >>
> >> I'd just make this "Awaiting review",
> >
> > +1 to the simplification of the name.
> >> and have it advance only for a
> >> positive review with no negative reviews.
> >
> > But wouldn't that lead to more than one stage being set since this would
> > perpetually be set until you were ready to merge? As I said later, I
> don't
> > know if we want to gate on a non-core devs approval as it might not be
> > reasonable (e.g. we have had some non-core devs be extremely picky about
> PEP
> > 8 conformance).
>
> I'm also fine with having mixed reviews mean "it's a core dev's problem
> now".
>
> >> I think this is worth having for the reason you give - anyone can
> >> advance the state to "awaiting core review", and it gives a visual
> >> indication of "at least 2 humans think this is a reasonable patch" for
> >> core developers looking to pick up pre-filtered proposals.
> >
> > Don't you mean "one other person" based on the advancement from "awaiting
> > review"?
>
> 2 humans - the original submitter, and whoever did the initial review
> (if they weren't a core dev).
>
> > Sorry if I wasn't clear before, but what I was thinking was that if
> > a core dev did the initial review then this stage would be skipped. IOW a
> > single core dev review is all that's needed to get out of any "I need a
> > review" stage.
>
> Right, that's what I was assuming as well.
>
> I also guess that when the submitter *is* a core dev, we'd send a PR
> straight to "awaiting merge" unless we had already explicitly added
> the "awaiting changes" or "awaiting core review" label.
>
> > If you do mean you want to require two reviewers for all PRs, I'm not
> sure
> > if we are there yet.
>
> No, I didn't mean that (although I can see how you read it that way) -
> I still don't think we should even make reviews mandatory for core dev
> submitted PRs.
>
> While I do think it would be nice to have the luxury of being able to
> afford to implement such a policy, it's the kind of workflow that's
> really only feasible when you have multiple folks handling reviews on
> a regular basis as part of a full-time job (and even then it can cause
> problems in niche areas of a code base where nobody feels particularly
> well-equipped to carry out an authoritative review).
>
> >> +1 for only getting here based on negative core reviews. An
> >> alternative to requiring a special comment to exit this state would be
> >> to add a "WIP: " prefix to the PR title when the label is added - that
> >> way the submitter can just remove the prefix to indicate that they
> >> have made the requested changes and are ready for another round of
> >> review, while folks could also add the prefix manually to indicate the
> >> change *isn't* ready for review.
> >
> > Guido squashed the title prefix idea at the language summit (it's what I
> > originally proposed, so this label approach was the compromise).
>
> Ah, fair enough. In that case, I agree that a "Bedevere: ready for
> review" comment to request removal of the label make sense.
>
> >> > 4. Awaiting Nth review/re-review: the PR submitter believes all the
> >> > requests have been addressed in all reviews and so is waiting on the
> >> > reviewer(s) who requested changes to review again (will need to
> provide a
> >> > way to leave a message that signals to Bedevere that the submitter
> feels
> >> > ready for another review)
> >>
> >> I don't see a need for this state - going back to "Awaiting core
> >> review" should suffice.
> >
> > So the reason I made this separate is if you're in the middle of working
> > with someone on a PR and you're already reviewing it, do I really need to
> > get involved? If I'm perusing the list of open PRs, wouldn't it be better
> > for me to look for another PR that has zero reviews than pile on to
> another
> > PR that already has a core dev reviewing it?
>
> OK, that rationale makes sense. Given that, I'd suggest making the
> three review states:
>
> - awaiting review
> - awaiting core review
> - awaiting change review
>
> >> > 5. Awaiting merge: all reviews approve of the PR (we could also drop
> the
> >> > "awaiting core review" stage and go straight here even for non-core
> reviews
> >> > that are all approvals, but maybe that's leaning on non-core devs too
> much
> >> > and giving false hope to PR submitters?)
> >>
> >> I assume we'd mainly get here based on all positive core reviews, no
> >> pending negative core reviews, but a pending CI run to check
> >> everything is working as expected.
> >
> > It can also be reached even if the CI is cleared but the PR will require
> > backporting and the person who plans to do the merge + backport doesn't
> have
> > the time ATM (e.g. I did a review of a PR that applies to master and 3.6
> on
> > my lunch break, but I'm waiting until I have time to run cherry_picker at
> > home to do the merge so I don't lose track of the fact that I need to do
> the
> > backport).
>
> Aye, I can see that.
>
> FWIW, I was doing things that way for a while, but eventually realised
> it made more sense for me to make use of the "backport needed" state
> on BPO, since that's closer to our target workflow with automated
> backports. It also means I can batch up a bunch of relatively
> mechanical backports to do via a local checkout later, rather than
> having to switch back and forth between code review and backporting
> things.
>
> It definitely isn't perfect (Mariatta ended up cleaning up a few of
> the backports from the last batch I did after I'd left them pending
> for over a week), but given that the original PRs had previously been
> lingering for months, I still count it as an improvement over my
> initial approach ;)
>
> Cheers,
> Nick.
>
> --
> Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/core-workflow/attachments/20170619/cdf4d7bc/attachment.html>

From ncoghlan at gmail.com  Mon Jun 19 21:15:27 2017
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 20 Jun 2017 11:15:27 +1000
Subject: [core-workflow] Auto labeling PRs based on what stage they're in
In-Reply-To: <CAP1=2W49Y4fVuFmHAcU1vJ-HXRFpZ7yRcjKCRHJXN=1G366OPw@mail.gmail.com>
References: <CAP1=2W6AqaiV26xs0sdD4Dr_2tLKA5MxYMtC=xR501H8Na2h-Q@mail.gmail.com>
 <CADiSq7eQ-+weG-KT58OAFdSqHEha4hDa=0ACNodU-HFrMbKrBQ@mail.gmail.com>
 <CAP1=2W4oBJiA+fk1+Of4jKqbU=3xFf8riZin6zXfQK5ch8csGw@mail.gmail.com>
 <CADiSq7e+xB-z9EFpseHXAuwmvYfF+bC=+e1KAR2PMacsQ_=tLw@mail.gmail.com>
 <CAP1=2W49Y4fVuFmHAcU1vJ-HXRFpZ7yRcjKCRHJXN=1G366OPw@mail.gmail.com>
Message-ID: <CADiSq7eMyWtDentYRNCmFBX-QNw7=E8O4sdXzyPpCxRhNaSdqA@mail.gmail.com>

On 20 June 2017 at 03:39, Brett Cannon <brett at python.org> wrote:
> OK, here is a dot graph that lays out the stages and the flow between them
> based on what has been discussed so far. You can use
> http://www.webgraphviz.com/ to view the graph in a browser. Anything green
> requires/blocks on a core dev, blue requires/blocks on the PR creator, and
> orange is open to anyone.

This looks good to me. I also think there are a few extra exceptional
flows, but we can probably say "manipulate the labels directly" for
these:

"Awaiting changes" -> "Awaiting merge": Core dev implements changes
themselves and pushes to submitter's branch
Core dev PR -> "Awaiting core review"
Core dev PR -> "Awaiting changes"

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From brett at python.org  Mon Jun 19 21:55:43 2017
From: brett at python.org (Brett Cannon)
Date: Tue, 20 Jun 2017 01:55:43 +0000
Subject: [core-workflow] Auto labeling PRs based on what stage they're in
In-Reply-To: <CADiSq7eMyWtDentYRNCmFBX-QNw7=E8O4sdXzyPpCxRhNaSdqA@mail.gmail.com>
References: <CAP1=2W6AqaiV26xs0sdD4Dr_2tLKA5MxYMtC=xR501H8Na2h-Q@mail.gmail.com>
 <CADiSq7eQ-+weG-KT58OAFdSqHEha4hDa=0ACNodU-HFrMbKrBQ@mail.gmail.com>
 <CAP1=2W4oBJiA+fk1+Of4jKqbU=3xFf8riZin6zXfQK5ch8csGw@mail.gmail.com>
 <CADiSq7e+xB-z9EFpseHXAuwmvYfF+bC=+e1KAR2PMacsQ_=tLw@mail.gmail.com>
 <CAP1=2W49Y4fVuFmHAcU1vJ-HXRFpZ7yRcjKCRHJXN=1G366OPw@mail.gmail.com>
 <CADiSq7eMyWtDentYRNCmFBX-QNw7=E8O4sdXzyPpCxRhNaSdqA@mail.gmail.com>
Message-ID: <CAP1=2W608CKmB-x8VS=bthKL6MurxEYCoEUwQSZdCDB4uzwosg@mail.gmail.com>

On Mon, Jun 19, 2017, 18:15 Nick Coghlan, <ncoghlan at gmail.com> wrote:

> On 20 June 2017 at 03:39, Brett Cannon <brett at python.org> wrote:
> > OK, here is a dot graph that lays out the stages and the flow between
> them
> > based on what has been discussed so far. You can use
> > http://www.webgraphviz.com/ to view the graph in a browser. Anything
> green
> > requires/blocks on a core dev, blue requires/blocks on the PR creator,
> and
> > orange is open to anyone.
>
> This looks good to me. I also think there are a few extra exceptional
> flows, but we can probably say "manipulate the labels directly" for
> these:
>
> "Awaiting changes" -> "Awaiting merge": Core dev implements changes
>

They could change their review to handle this.

themselves and pushes to submitter's branch
> Core dev PR -> "Awaiting core review"
>

Yeah, that could be manual.

Core dev PR -> "Awaiting changes"
>

If this is due to a review from a core dev then it's just like any other
core review as you don't want to ignore such a review.

-Brett


> Cheers,
> Nick.
>
> --
> Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/core-workflow/attachments/20170620/20133181/attachment-0001.html>

From ncoghlan at gmail.com  Mon Jun 19 22:19:16 2017
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 20 Jun 2017 12:19:16 +1000
Subject: [core-workflow] Auto labeling PRs based on what stage they're in
In-Reply-To: <CAP1=2W608CKmB-x8VS=bthKL6MurxEYCoEUwQSZdCDB4uzwosg@mail.gmail.com>
References: <CAP1=2W6AqaiV26xs0sdD4Dr_2tLKA5MxYMtC=xR501H8Na2h-Q@mail.gmail.com>
 <CADiSq7eQ-+weG-KT58OAFdSqHEha4hDa=0ACNodU-HFrMbKrBQ@mail.gmail.com>
 <CAP1=2W4oBJiA+fk1+Of4jKqbU=3xFf8riZin6zXfQK5ch8csGw@mail.gmail.com>
 <CADiSq7e+xB-z9EFpseHXAuwmvYfF+bC=+e1KAR2PMacsQ_=tLw@mail.gmail.com>
 <CAP1=2W49Y4fVuFmHAcU1vJ-HXRFpZ7yRcjKCRHJXN=1G366OPw@mail.gmail.com>
 <CADiSq7eMyWtDentYRNCmFBX-QNw7=E8O4sdXzyPpCxRhNaSdqA@mail.gmail.com>
 <CAP1=2W608CKmB-x8VS=bthKL6MurxEYCoEUwQSZdCDB4uzwosg@mail.gmail.com>
Message-ID: <CADiSq7f5tid858VxbzdrXpDs2g-BY6ECJqjjnkodQ24WwJfRWg@mail.gmail.com>

On 20 June 2017 at 11:55, Brett Cannon <brett at python.org> wrote:
> On Mon, Jun 19, 2017, 18:15 Nick Coghlan, <ncoghlan at gmail.com> wrote:
>> "Awaiting changes" -> "Awaiting merge": Core dev implements changes
>
>
> They could change their review to handle this.

True.

>> Core dev PR -> "Awaiting changes"
>
> If this is due to a review from a core dev then it's just like any other
> core review as you don't want to ignore such a review.

True, but I was thinking of the "WIP" case - GitHub doesn't let you
review your own PRs, so you can't give yourself a negative review to
indicate that the PR is incomplete. However, setting "Awaiting
changes" explicitly will work fine.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From brett at python.org  Fri Jun 23 19:06:08 2017
From: brett at python.org (Brett Cannon)
Date: Fri, 23 Jun 2017 23:06:08 +0000
Subject: [core-workflow] Looking for reviewers on a PR to add a check for
 whitespace
Message-ID: <CAP1=2W6ty6c=h_dWZRndQvevUMEOp4LuYbv+iyCrWj9034jO5Q@mail.gmail.com>

https://github.com/python/core-workflow/issues/3 is tracking adding a check
for proper whitespace in PRs as we had that on hg.python.org but we are
missing it on GitHub. I have a PR at
https://github.com/python/cpython/pull/2367 that I think does the right
thing, but I would appreciate a quick review to make sure I didn't miss
anything since this will block PRs if I get it wrong. :)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/core-workflow/attachments/20170623/e5220507/attachment.html>