From jklymak at uvic.ca Mon Jan 7 14:58:21 2019 From: jklymak at uvic.ca (Jody Klymak) Date: Mon, 7 Jan 2019 11:58:21 -0800 Subject: [Matplotlib-devel] Weekly calll Message-ID: Hi all Starting in a couple of minutes if anyone is around https://paper.dropbox.com/doc/Matplotlib-meeting-agenda--AVNCXbugY4xNG6rChQ80P3GvAg-aAmENlkgepgsMeDZtlsYu From nathan12343 at gmail.com Tue Jan 8 18:13:28 2019 From: nathan12343 at gmail.com (Nathan Goldbaum) Date: Tue, 8 Jan 2019 17:13:28 -0600 Subject: [Matplotlib-devel] [ANN] 2019 Scipy Conference: Call for Proposals Message-ID: SciPy 2019, the 18th annual Scientific Computing with Python conference, will be held July 8-14, 2019 in Austin, Texas. The annual SciPy Conference brings together over 800 participants from industry, academia, and government to showcase their latest projects, learn from skilled users and developers, and collaborate on code development. The call for abstracts for SciPy 2019 for talks, posters and tutorials is now open. The deadline for submissions is February 10, 2019. Conference Website: https://www.scipy2019.scipy.org/ Submission Website: https://easychair.org/conferences/?conf=scipy2019 *Talks and Posters (July 10-12, 2019)* In addition to the general track, this year will have specialized tracks focused on: - Data Driven Discoveries (including Machine Learning and Data Science) - Open Source Communities (Sustainability) *Mini Symposia* - Science Communication through Visualization - Neuroscience and Cognitive Science - Image Processing - Earth, Ocean, Geo and Atmospheric Science There will also be a SciPy Tools Plenary Session each day with 2 to 5 minute updates on tools and libraries. *Tutorials (July 8-9, 2019)* Tutorials should be focused on covering a well-defined topic in a hands-on manner. We are looking for useful techniques or packages, helping new or advanced Python programmers develop better or faster scientific applications. We encourage submissions to be designed to allow at least 50% of the time for hands-on exercises even if this means the subject matter needs to be limited. Tutorials will be 4 hours in duration. In your tutorial application, you can indicate what prerequisite skills and knowledge will be needed for your tutorial, and the approximate expected level of knowledge of your students (i.e., beginner, intermediate, advanced). Instructors of accepted tutorials will receive a stipend. -------------- next part -------------- An HTML attachment was scrubbed... URL: From jklymak at uvic.ca Mon Jan 14 12:44:31 2019 From: jklymak at uvic.ca (Jody Klymak) Date: Mon, 14 Jan 2019 09:44:31 -0800 Subject: [Matplotlib-devel] weekly meeting Message-ID: Hi all, Weekly meeting agenda at: https://paper.dropbox.com/doc/Matplotlib-meeting-agenda--AVqsdFqn2tYkD0LzOQbyHFq5Ag-aAmENlkgepgsMeDZtlsYu Cheers, Jody -- Jody Klymak http://web.uvic.ca/~jklymak/ From anntzer.lee at gmail.com Tue Jan 15 08:59:55 2019 From: anntzer.lee at gmail.com (Antony Lee) Date: Tue, 15 Jan 2019 14:59:55 +0100 Subject: [Matplotlib-devel] Proposal to amend the PR merge rules Message-ID: Hi all, During the weekly dev call, I proposed to amend the PR merge rules, following an initial comment on Github [ https://github.com/matplotlib/matplotlib/pull/13173#issuecomment-453921220]. There was general agreement among the devs present (Tom, Hannah, Jody, and myself), so I'm putting it here for discussion. The objective of this change is to prevent "uncontroversial but uninteresting" PRs from falling into oblivion, and try to decrease the size of the open PR stack. The current rule is that a PR needs positive reviews (from two committers, and excluding the author if a committer) to be merged, except that doc-only PRs (docstrings, rst) only needs a single positive review. I am proposing that, if a (non-doc) PR already has one positive review, but no activity on that PR has occurred for two weeks (exact time interval up to bikeshedding) [Jody suggests: and the PR has 100% code coverage], then a committer (either the first reviewer, or the author if a committer) can suggest that it be merged on the basis of that single review. To do so, the "sponsor" should ping all developers (@matplotlib/developers) on that issue indicating that intent, and add a "single-review-merge" label on the PR (so that these PRs can easily be found). Committers are encouraged to review the PR to accept and merge it or reject it or request changes on it; but they can also just indicate that they consider the PR sufficiently complex that a proper second review is needed before merging, or request an extension, etc. To do so they should still leave a "reject" review, even if just saying "objecting to single-review merged; anyone can dismiss after a second review". However, if within another two weeks, no committer voiced any objection (explicitly, i.e. by rejecting), then the PR can indeed be merged on the basis of that single review. To avoid overwhelming the system, any committer can only "sponsor" a single PR at a time. Thoughts? Antony -------------- next part -------------- An HTML attachment was scrubbed... URL: From nelle.varoquaux at gmail.com Tue Jan 15 12:56:37 2019 From: nelle.varoquaux at gmail.com (Nelle Varoquaux) Date: Tue, 15 Jan 2019 09:56:37 -0800 Subject: [Matplotlib-devel] Proposal to amend the PR merge rules In-Reply-To: References: Message-ID: Hello, Overall, I think this is a good idea, but I would like specifics on what "uncontroversial but uninteresting" PR mean. IMO, that should exclude any PR with API changes. Cheers, N On Tue, 15 Jan 2019 at 06:00, Antony Lee wrote: > Hi all, > > During the weekly dev call, I proposed to amend the PR merge rules, > following an initial comment on Github [ > https://github.com/matplotlib/matplotlib/pull/13173#issuecomment-453921220]. > There was general agreement among the devs present (Tom, Hannah, Jody, and > myself), so I'm putting it here for discussion. The objective of this > change is to prevent "uncontroversial but uninteresting" PRs from falling > into oblivion, and try to decrease the size of the open PR stack. > > The current rule is that a PR needs positive reviews (from two committers, > and excluding the author if a committer) to be merged, except that doc-only > PRs (docstrings, rst) only needs a single positive review. > > I am proposing that, if a (non-doc) PR already has one positive review, > but no activity on that PR has occurred for two weeks (exact time interval > up to bikeshedding) [Jody suggests: and the PR has 100% code coverage], > then a committer (either the first reviewer, or the author if a committer) > can suggest that it be merged on the basis of that single review. To do > so, the "sponsor" should ping all developers (@matplotlib/developers) on > that issue indicating that intent, and add a "single-review-merge" label on > the PR (so that these PRs can easily be found). Committers are encouraged > to review the PR to accept and merge it or reject it or request changes on > it; but they can also just indicate that they consider the PR sufficiently > complex that a proper second review is needed before merging, or request an > extension, etc. To do so they should still leave a "reject" review, even > if just saying "objecting to single-review merged; anyone can dismiss after > a second review". However, if within another two weeks, no committer > voiced any objection (explicitly, i.e. by rejecting), then the PR can > indeed be merged on the basis of that single review. > > To avoid overwhelming the system, any committer can only "sponsor" a > single PR at a time. > > Thoughts? > > Antony > _______________________________________________ > Matplotlib-devel mailing list > Matplotlib-devel at python.org > https://mail.python.org/mailman/listinfo/matplotlib-devel > -------------- next part -------------- An HTML attachment was scrubbed... URL: From pmhobson at gmail.com Tue Jan 15 13:20:24 2019 From: pmhobson at gmail.com (Paul Hobson) Date: Tue, 15 Jan 2019 10:20:24 -0800 Subject: [Matplotlib-devel] Proposal to amend the PR merge rules In-Reply-To: References: Message-ID: I think this would be a good policy as well. Can I get some clarification on the flow? The way I understand it: 1) Someone (committer, contributor, or first-timer) opens a simple PR 2) Committer A reviews it, adds a "single-review-merge" label, then digitally walks away 3) No less than two weeks later, Committer B sees the PR, notices the label, opens it up, and merges without reviewing. Does that capture a successful "single-review-merge" lifecycle? -paul On Tue, Jan 15, 2019 at 9:57 AM Nelle Varoquaux wrote: > Hello, > > Overall, I think this is a good idea, but I would like specifics on what > "uncontroversial but uninteresting" PR mean. IMO, that should exclude any > PR with API changes. > > Cheers, > N > > On Tue, 15 Jan 2019 at 06:00, Antony Lee wrote: > >> Hi all, >> >> During the weekly dev call, I proposed to amend the PR merge rules, >> following an initial comment on Github [ >> https://github.com/matplotlib/matplotlib/pull/13173#issuecomment-453921220]. >> There was general agreement among the devs present (Tom, Hannah, Jody, and >> myself), so I'm putting it here for discussion. The objective of this >> change is to prevent "uncontroversial but uninteresting" PRs from falling >> into oblivion, and try to decrease the size of the open PR stack. >> >> The current rule is that a PR needs positive reviews (from two >> committers, and excluding the author if a committer) to be merged, except >> that doc-only PRs (docstrings, rst) only needs a single positive review. >> >> I am proposing that, if a (non-doc) PR already has one positive review, >> but no activity on that PR has occurred for two weeks (exact time interval >> up to bikeshedding) [Jody suggests: and the PR has 100% code coverage], >> then a committer (either the first reviewer, or the author if a committer) >> can suggest that it be merged on the basis of that single review. To do >> so, the "sponsor" should ping all developers (@matplotlib/developers) on >> that issue indicating that intent, and add a "single-review-merge" label on >> the PR (so that these PRs can easily be found). Committers are encouraged >> to review the PR to accept and merge it or reject it or request changes on >> it; but they can also just indicate that they consider the PR sufficiently >> complex that a proper second review is needed before merging, or request an >> extension, etc. To do so they should still leave a "reject" review, even >> if just saying "objecting to single-review merged; anyone can dismiss after >> a second review". However, if within another two weeks, no committer >> voiced any objection (explicitly, i.e. by rejecting), then the PR can >> indeed be merged on the basis of that single review. >> >> To avoid overwhelming the system, any committer can only "sponsor" a >> single PR at a time. >> >> Thoughts? >> >> Antony >> _______________________________________________ >> Matplotlib-devel mailing list >> Matplotlib-devel at python.org >> https://mail.python.org/mailman/listinfo/matplotlib-devel >> > _______________________________________________ > Matplotlib-devel mailing list > Matplotlib-devel at python.org > https://mail.python.org/mailman/listinfo/matplotlib-devel > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jklymak at uvic.ca Tue Jan 15 13:37:39 2019 From: jklymak at uvic.ca (Jody Klymak) Date: Tue, 15 Jan 2019 10:37:39 -0800 Subject: [Matplotlib-devel] Proposal to amend the PR merge rules In-Reply-To: References: Message-ID: > On 15 Jan 2019, at 10:20, Paul Hobson wrote: > > I think this would be a good policy as well. Can I get some clarification on the flow? The way I understand it: > > 1) Someone (committer, contributor, or first-timer) opens a simple PR > 2) Committer A reviews it, adds a "single-review-merge" label, then digitally walks away > 3) No less than two weeks later, Committer B sees the PR, notices the label, opens it up, and merges without reviewing. If there is a Committer B, I think they should at least look at the PR to make sure it doesn?t have any hidden API changes, or introduce anything hostile. The point here is that getting the third person to look at a PR is proving quite difficult, and if the rest of the folks with the commit bit haven?t looked at a PR for a month, even after being warned, then silence indicates consent. Note that only folks with the commit bit can label PRs, so anyone flagging a PR like this is implicitly trusted to not be wreaking havoc, and not doing this for major changes. Since we all curate our own PRs, I rather expect that the PR contributor will often be the person who adds the flag. Whether we want to allow a self-merge at that point is up for debate, but if not, then folks need to get in the habit of looking at old flagged PRs and merging them. I think a huge problem we have is the LIFO default sort of the github PR queue, and that any PR not on the first page might as well be closed for all the attention it will get without constant nagging. Cheers, Jody > Does that capture a successful "single-review-merge" lifecycle? > -paul > > On Tue, Jan 15, 2019 at 9:57 AM Nelle Varoquaux > wrote: > Hello, > > Overall, I think this is a good idea, but I would like specifics on what "uncontroversial but uninteresting" PR mean. IMO, that should exclude any PR with API changes. > > Cheers, > N > > On Tue, 15 Jan 2019 at 06:00, Antony Lee > wrote: > Hi all, > > During the weekly dev call, I proposed to amend the PR merge rules, following an initial comment on Github [https://github.com/matplotlib/matplotlib/pull/13173#issuecomment-453921220 ]. There was general agreement among the devs present (Tom, Hannah, Jody, and myself), so I'm putting it here for discussion. The objective of this change is to prevent "uncontroversial but uninteresting" PRs from falling into oblivion, and try to decrease the size of the open PR stack. > > The current rule is that a PR needs positive reviews (from two committers, and excluding the author if a committer) to be merged, except that doc-only PRs (docstrings, rst) only needs a single positive review. > > I am proposing that, if a (non-doc) PR already has one positive review, but no activity on that PR has occurred for two weeks (exact time interval up to bikeshedding) [Jody suggests: and the PR has 100% code coverage], then a committer (either the first reviewer, or the author if a committer) can suggest that it be merged on the basis of that single review. To do so, the "sponsor" should ping all developers (@matplotlib/developers) on that issue indicating that intent, and add a "single-review-merge" label on the PR (so that these PRs can easily be found). Committers are encouraged to review the PR to accept and merge it or reject it or request changes on it; but they can also just indicate that they consider the PR sufficiently complex that a proper second review is needed before merging, or request an extension, etc. To do so they should still leave a "reject" review, even if just saying "objecting to single-review merged; anyone can dismiss after a second review". However, if within another two weeks, no committer voiced any objection (explicitly, i.e. by rejecting), then the PR can indeed be merged on the basis of that single review. > > To avoid overwhelming the system, any committer can only "sponsor" a single PR at a time. > > Thoughts? > > Antony > _______________________________________________ > Matplotlib-devel mailing list > Matplotlib-devel at python.org > https://mail.python.org/mailman/listinfo/matplotlib-devel > _______________________________________________ > Matplotlib-devel mailing list > Matplotlib-devel at python.org > https://mail.python.org/mailman/listinfo/matplotlib-devel > _______________________________________________ > Matplotlib-devel mailing list > Matplotlib-devel at python.org > https://mail.python.org/mailman/listinfo/matplotlib-devel -- Jody Klymak http://web.uvic.ca/~jklymak/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From nelle.varoquaux at gmail.com Tue Jan 15 13:45:45 2019 From: nelle.varoquaux at gmail.com (Nelle Varoquaux) Date: Tue, 15 Jan 2019 10:45:45 -0800 Subject: [Matplotlib-devel] Proposal to amend the PR merge rules In-Reply-To: References: Message-ID: The proposed scheme by Paul doesn't seem reasonable to me. Core contributor A or committers needs to actively reach out to other core contributors: labeling is not enough IMO. On Tue, 15 Jan 2019 at 10:37, Jody Klymak wrote: > > > On 15 Jan 2019, at 10:20, Paul Hobson wrote: > > I think this would be a good policy as well. Can I get some clarification > on the flow? The way I understand it: > > 1) Someone (committer, contributor, or first-timer) opens a simple PR > 2) Committer A reviews it, adds a "single-review-merge" label, then > digitally walks away > 3) No less than two weeks later, Committer B sees the PR, notices the > label, opens it up, and merges without reviewing. > > > If there is a Committer B, I think they should at least look at the PR to > make sure it doesn?t have any hidden API changes, or introduce anything > hostile. > > The point here is that getting the third person to look at a PR is proving > quite difficult, and if the rest of the folks with the commit bit haven?t > looked at a PR for a month, even after being warned, then silence indicates > consent. > > Note that only folks with the commit bit can label PRs, so anyone flagging > a PR like this is implicitly trusted to not be wreaking havoc, and not > doing this for major changes. Since we all curate our own PRs, I rather > expect that the PR contributor will often be the person who adds the flag. > Whether we want to allow a self-merge at that point is up for debate, but > if not, then folks need to get in the habit of looking at old flagged PRs > and merging them. > > I think a huge problem we have is the LIFO default sort of the github PR > queue, and that any PR not on the first page might as well be closed for > all the attention it will get without constant nagging. > > Cheers, Jody > > Does that capture a successful "single-review-merge" lifecycle? > -paul > > On Tue, Jan 15, 2019 at 9:57 AM Nelle Varoquaux > wrote: > >> Hello, >> >> Overall, I think this is a good idea, but I would like specifics on what >> "uncontroversial but uninteresting" PR mean. IMO, that should exclude any >> PR with API changes. >> >> Cheers, >> N >> >> On Tue, 15 Jan 2019 at 06:00, Antony Lee wrote: >> >>> Hi all, >>> >>> During the weekly dev call, I proposed to amend the PR merge rules, >>> following an initial comment on Github [ >>> https://github.com/matplotlib/matplotlib/pull/13173#issuecomment-453921220]. >>> There was general agreement among the devs present (Tom, Hannah, Jody, and >>> myself), so I'm putting it here for discussion. The objective of this >>> change is to prevent "uncontroversial but uninteresting" PRs from falling >>> into oblivion, and try to decrease the size of the open PR stack. >>> >>> The current rule is that a PR needs positive reviews (from two >>> committers, and excluding the author if a committer) to be merged, except >>> that doc-only PRs (docstrings, rst) only needs a single positive review. >>> >>> I am proposing that, if a (non-doc) PR already has one positive review, >>> but no activity on that PR has occurred for two weeks (exact time interval >>> up to bikeshedding) [Jody suggests: and the PR has 100% code coverage], >>> then a committer (either the first reviewer, or the author if a committer) >>> can suggest that it be merged on the basis of that single review. To do >>> so, the "sponsor" should ping all developers (@matplotlib/developers) on >>> that issue indicating that intent, and add a "single-review-merge" label on >>> the PR (so that these PRs can easily be found). Committers are encouraged >>> to review the PR to accept and merge it or reject it or request changes on >>> it; but they can also just indicate that they consider the PR sufficiently >>> complex that a proper second review is needed before merging, or request an >>> extension, etc. To do so they should still leave a "reject" review, even >>> if just saying "objecting to single-review merged; anyone can dismiss after >>> a second review". However, if within another two weeks, no committer >>> voiced any objection (explicitly, i.e. by rejecting), then the PR can >>> indeed be merged on the basis of that single review. >>> >>> To avoid overwhelming the system, any committer can only "sponsor" a >>> single PR at a time. >>> >>> Thoughts? >>> >>> Antony >>> _______________________________________________ >>> Matplotlib-devel mailing list >>> Matplotlib-devel at python.org >>> https://mail.python.org/mailman/listinfo/matplotlib-devel >>> >> _______________________________________________ >> Matplotlib-devel mailing list >> Matplotlib-devel at python.org >> https://mail.python.org/mailman/listinfo/matplotlib-devel >> > _______________________________________________ > Matplotlib-devel mailing list > Matplotlib-devel at python.org > https://mail.python.org/mailman/listinfo/matplotlib-devel > > > -- > Jody Klymak > http://web.uvic.ca/~jklymak/ > > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From antony.lee at institutoptique.fr Tue Jan 15 14:11:17 2019 From: antony.lee at institutoptique.fr (Antony Lee) Date: Tue, 15 Jan 2019 20:11:17 +0100 Subject: [Matplotlib-devel] Proposal to amend the PR merge rules In-Reply-To: References: Message-ID: I'm obviously more aware of my own PRs, so here are a few I'd have put up for single-review-merge (note that it's quite possible that they'd have attracted more attention and gotten merged faster under that system; that's also the goal...). Obviously other devs may think about other PRs that could have benefitted from the same process. - https://github.com/matplotlib/matplotlib/pull/9787 adds support for the ttc font format (requested since 2014) for png/svg output (that's provided basically for free by FreeType) but not for pdf/ps (that would require more difficult changes). The entire PR comes down to: * adding "ttc" as an extension that should be treated like "ttf"/"otf", * adding error handling to pdf/ps to signal these formats are not supported there, * tests. The PR sat open for 10 months before a first positive review (because no one cares about fonts, I guess) and took another 3 months to attract a second positive review. - https://github.com/matplotlib/matplotlib/pull/12472 does a fairly trivial fix to fontList.json to make it reusable for Matplotlibs installed in different venvs, got a positive review in one day and waited another 2.5 months for a second positive review. - https://github.com/matplotlib/matplotlib/pull/9867 deduplicates completely duplicated code between the pdf and ps backends, mostly for maintainability's sake; it took 7.5 months (and 5 rebases due to conflicts) to get a first positive review and another 6 (and around as many rebases) to get a second one. ----- As for the specifics (as I see it): - Someone opens a PR, it gets a positive review, no negative review, and no activity occurs for two weeks (significant activity -- excluding trivial chat on the thread). - Sponsor (first reviewer, or author if a committer) *pings* all devs by mentioning @matplotlib/developers on the PR thread with the intent to single-review-merge, and labels the PR accordingly. - The PR can still be reviewed/merged/rejected by the normal review process. However, if no one explicitly opposes the merge within two weeks, anyone can merge it at that point (including by self-merge). I would not exclude all API changes from the process. For example, https://github.com/matplotlib/matplotlib/pull/13173/commits/e90d264f3e5a263ef57243a2af86b46ab74ccc16 deprecates (and prepares for deletion) two parameters to imshow() that have had no effect since 2006. Let's pretend for a second this commit was a single independent PR (and not an example to showcase the parameter-deprecating decorator...); if it started being forgotten I would have proposed it for single-review-merge (again, note that if you're worried about the change, you don't need to reject the PR; you can just say, I think this API change needs to be approved by a second reviewer and that'll block the single-review-merge just as well). Antony On Tue, Jan 15, 2019 at 7:46 PM Nelle Varoquaux wrote: > The proposed scheme by Paul doesn't seem reasonable to me. Core > contributor A or committers needs to actively reach out to other core > contributors: labeling is not enough IMO. > > > On Tue, 15 Jan 2019 at 10:37, Jody Klymak wrote: > >> >> >> On 15 Jan 2019, at 10:20, Paul Hobson wrote: >> >> I think this would be a good policy as well. Can I get some clarification >> on the flow? The way I understand it: >> >> 1) Someone (committer, contributor, or first-timer) opens a simple PR >> 2) Committer A reviews it, adds a "single-review-merge" label, then >> digitally walks away >> 3) No less than two weeks later, Committer B sees the PR, notices the >> label, opens it up, and merges without reviewing. >> >> >> If there is a Committer B, I think they should at least look at the PR to >> make sure it doesn?t have any hidden API changes, or introduce anything >> hostile. >> >> The point here is that getting the third person to look at a PR is >> proving quite difficult, and if the rest of the folks with the commit bit >> haven?t looked at a PR for a month, even after being warned, then silence >> indicates consent. >> >> Note that only folks with the commit bit can label PRs, so anyone >> flagging a PR like this is implicitly trusted to not be wreaking havoc, and >> not doing this for major changes. Since we all curate our own PRs, I >> rather expect that the PR contributor will often be the person who adds the >> flag. Whether we want to allow a self-merge at that point is up for >> debate, but if not, then folks need to get in the habit of looking at old >> flagged PRs and merging them. >> >> I think a huge problem we have is the LIFO default sort of the github PR >> queue, and that any PR not on the first page might as well be closed for >> all the attention it will get without constant nagging. >> >> Cheers, Jody >> >> Does that capture a successful "single-review-merge" lifecycle? >> -paul >> >> On Tue, Jan 15, 2019 at 9:57 AM Nelle Varoquaux < >> nelle.varoquaux at gmail.com> wrote: >> >>> Hello, >>> >>> Overall, I think this is a good idea, but I would like specifics on what >>> "uncontroversial but uninteresting" PR mean. IMO, that should exclude any >>> PR with API changes. >>> >>> Cheers, >>> N >>> >>> On Tue, 15 Jan 2019 at 06:00, Antony Lee wrote: >>> >>>> Hi all, >>>> >>>> During the weekly dev call, I proposed to amend the PR merge rules, >>>> following an initial comment on Github [ >>>> https://github.com/matplotlib/matplotlib/pull/13173#issuecomment-453921220]. >>>> There was general agreement among the devs present (Tom, Hannah, Jody, and >>>> myself), so I'm putting it here for discussion. The objective of this >>>> change is to prevent "uncontroversial but uninteresting" PRs from falling >>>> into oblivion, and try to decrease the size of the open PR stack. >>>> >>>> The current rule is that a PR needs positive reviews (from two >>>> committers, and excluding the author if a committer) to be merged, except >>>> that doc-only PRs (docstrings, rst) only needs a single positive review. >>>> >>>> I am proposing that, if a (non-doc) PR already has one positive review, >>>> but no activity on that PR has occurred for two weeks (exact time interval >>>> up to bikeshedding) [Jody suggests: and the PR has 100% code coverage], >>>> then a committer (either the first reviewer, or the author if a committer) >>>> can suggest that it be merged on the basis of that single review. To do >>>> so, the "sponsor" should ping all developers (@matplotlib/developers) on >>>> that issue indicating that intent, and add a "single-review-merge" label on >>>> the PR (so that these PRs can easily be found). Committers are encouraged >>>> to review the PR to accept and merge it or reject it or request changes on >>>> it; but they can also just indicate that they consider the PR sufficiently >>>> complex that a proper second review is needed before merging, or request an >>>> extension, etc. To do so they should still leave a "reject" review, even >>>> if just saying "objecting to single-review merged; anyone can dismiss after >>>> a second review". However, if within another two weeks, no committer >>>> voiced any objection (explicitly, i.e. by rejecting), then the PR can >>>> indeed be merged on the basis of that single review. >>>> >>>> To avoid overwhelming the system, any committer can only "sponsor" a >>>> single PR at a time. >>>> >>>> Thoughts? >>>> >>>> Antony >>>> _______________________________________________ >>>> Matplotlib-devel mailing list >>>> Matplotlib-devel at python.org >>>> https://mail.python.org/mailman/listinfo/matplotlib-devel >>>> >>> _______________________________________________ >>> Matplotlib-devel mailing list >>> Matplotlib-devel at python.org >>> https://mail.python.org/mailman/listinfo/matplotlib-devel >>> >> _______________________________________________ >> Matplotlib-devel mailing list >> Matplotlib-devel at python.org >> https://mail.python.org/mailman/listinfo/matplotlib-devel >> >> >> -- >> Jody Klymak >> http://web.uvic.ca/~jklymak/ >> >> >> >> >> >> _______________________________________________ > Matplotlib-devel mailing list > Matplotlib-devel at python.org > https://mail.python.org/mailman/listinfo/matplotlib-devel > -------------- next part -------------- An HTML attachment was scrubbed... URL: From efiring at hawaii.edu Tue Jan 15 16:36:06 2019 From: efiring at hawaii.edu (Eric Firing) Date: Tue, 15 Jan 2019 11:36:06 -1000 Subject: [Matplotlib-devel] Proposal to amend the PR merge rules In-Reply-To: References: Message-ID: Antony, Your examples illustrate that the problem is often that we don't have enough people who understand some areas, like fonts and pdf/ps file formats, to get 2 thorough and knowledgeable reviews in a reasonable time. I'm fine with your proposed rule. It should help at least a little bit. Eric On 2019/01/15 9:11 AM, Antony Lee wrote: > I'm obviously more aware of my own PRs, so here are a few I'd have put > up for single-review-merge (note that it's quite possible that they'd > have attracted more attention and gotten merged faster under that > system; that's also the goal...).? Obviously other devs may think about > other PRs that could have benefitted from the same process. > > - https://github.com/matplotlib/matplotlib/pull/9787 adds support for > the ttc font format (requested since 2014) for png/svg output (that's > provided basically for free by FreeType) but not for pdf/ps (that would > require more difficult changes).? The entire PR comes down to: > ? * adding "ttc" as an extension that should be treated like "ttf"/"otf", > ? * adding error handling to pdf/ps to signal these formats are not > supported there, > ? * tests. > ? The PR sat open for 10 months before a first positive review (because > no one cares about fonts, I guess) and took another 3 months to attract > a second positive review. > > - https://github.com/matplotlib/matplotlib/pull/12472 does a fairly > trivial fix to fontList.json to make it reusable for Matplotlibs > installed in different venvs, got a positive review in one day and > waited another 2.5 months for a second positive review. > > - https://github.com/matplotlib/matplotlib/pull/9867 deduplicates > completely duplicated code between the pdf and ps backends, mostly for > maintainability's sake; it took 7.5 months (and 5 rebases due to > conflicts) to get a first positive review and another 6 (and around as > many rebases) to get a second one. > > ----- > > As for the specifics (as I see it): > - Someone opens a PR, it gets a positive review, no negative review, and > no activity occurs for two weeks (significant activity -- excluding > trivial chat on the thread). > - Sponsor (first reviewer, or author if a committer) *pings* all devs by > mentioning @matplotlib/developers on the PR thread with the intent to > single-review-merge, and labels the PR accordingly. > - The PR can still be reviewed/merged/rejected by the normal review > process.? However, if no one explicitly opposes the merge within two > weeks, anyone can merge it at that point (including by self-merge). > > I would not exclude all API changes from the process.? For example, > https://github.com/matplotlib/matplotlib/pull/13173/commits/e90d264f3e5a263ef57243a2af86b46ab74ccc16 > deprecates (and prepares for deletion) two parameters to imshow() that > have had no effect since 2006.? Let's pretend for a second this commit > was a single independent PR (and not an example to showcase the > parameter-deprecating decorator...); if it started being forgotten I > would have proposed it for single-review-merge (again, note that if > you're worried about the change, you don't need to reject the PR; you > can just say, I think this API change needs to be approved by a second > reviewer and that'll block the single-review-merge just as well). > > Antony > > On Tue, Jan 15, 2019 at 7:46 PM Nelle Varoquaux > > wrote: > > The proposed scheme by Paul doesn't seem reasonable to me. Core > contributor A or committers needs to actively reach out to other > core contributors: labeling is not enough IMO. > > > On Tue, 15 Jan 2019 at 10:37, Jody Klymak > wrote: > > > >> On 15 Jan 2019, at 10:20, Paul Hobson > > wrote: >> >> I think this would be a good policy as well. Can I get some >> clarification on the flow? The way I understand it: >> >> 1) Someone (committer, contributor, or first-timer) opens a >> simple PR >> 2) Committer A reviews it, adds a "single-review-merge" label, >> then digitally walks away >> 3) No less than two weeks later, Committer B sees the PR, >> notices the label, opens it up, and merges without reviewing. > > If there is a Committer B, I think they should at least look at > the PR to make sure it doesn?t have any hidden API changes, or > introduce anything hostile. > > The point here is that getting the third person to look at a PR > is proving quite difficult, and if the rest of the folks with > the commit bit haven?t looked at a PR for a month, even after > being warned, then silence indicates consent. > > Note that only folks with the commit bit can label PRs, so > anyone flagging a PR like this is implicitly trusted to not be > wreaking havoc, and not doing this for major changes.? Since we > all curate our own PRs, I rather expect that the PR contributor > will often be the person who adds the flag.? Whether we want to > allow a self-merge at that point is up for debate, but if not, > then folks need to get in the habit of looking at old flagged > PRs and merging them. > > I think a huge problem we have is the LIFO default sort of the > github PR queue, and that any PR not on the first page might as > well be closed for all the attention it will get without > constant nagging. > > Cheers, ? Jody > >> Does that capture a successful "single-review-merge" lifecycle? >> -paul >> >> On Tue, Jan 15, 2019 at 9:57 AM Nelle Varoquaux >> > >> wrote: >> >> Hello, >> >> Overall, I think this is a good idea, but I would like >> specifics on what "uncontroversial but uninteresting" PR >> mean. IMO, that should exclude any PR with API changes. >> >> Cheers, >> N >> >> On Tue, 15 Jan 2019 at 06:00, Antony Lee >> > wrote: >> >> Hi all, >> >> During the weekly dev call, I proposed to amend the PR >> merge rules, following an initial comment on Github >> [https://github.com/matplotlib/matplotlib/pull/13173#issuecomment-453921220]. >> There was general agreement among the devs present >> (Tom, Hannah, Jody, and myself), so I'm putting it >> here for discussion.? The objective of this change is >> to prevent "uncontroversial but uninteresting" PRs >> from falling into oblivion, and try to decrease the >> size of the open PR stack. >> >> The current rule is that a PR needs positive reviews >> (from two committers, and excluding the author if a >> committer) to be merged, except that doc-only PRs >> (docstrings, rst) only needs a single positive review. >> >> I am proposing that, if a (non-doc) PR already has one >> positive review, but no activity on that PR has >> occurred for two weeks (exact time interval up to >> bikeshedding) [Jody suggests: and the PR has 100% code >> coverage], then a committer (either the first >> reviewer, or the author if a committer) can suggest >> that it be merged on the basis of that single review. >> To do so, the "sponsor" should ping all developers >> (@matplotlib/developers) on that issue indicating that >> intent, and add a "single-review-merge" label on the >> PR (so that these PRs can easily be found). >> Committers are encouraged to review the PR to accept >> and merge it or reject it or request changes on it; >> but they can also just indicate that they consider the >> PR sufficiently complex that a proper second review is >> needed before merging, or request an extension, etc. >> To do so they should still leave a "reject" review, >> even if just saying "objecting to single-review >> merged; anyone can dismiss after a second review". >> However, if within another two weeks, no committer >> voiced any objection (explicitly, i.e. by rejecting), >> then the PR can indeed be merged on the basis of that >> single review. >> >> To avoid overwhelming the system, any committer can >> only "sponsor" a single PR at a time. >> >> Thoughts? >> >> Antony >> _______________________________________________ >> Matplotlib-devel mailing list >> Matplotlib-devel at python.org >> >> https://mail.python.org/mailman/listinfo/matplotlib-devel >> >> _______________________________________________ >> Matplotlib-devel mailing list >> Matplotlib-devel at python.org >> >> https://mail.python.org/mailman/listinfo/matplotlib-devel >> >> _______________________________________________ >> Matplotlib-devel mailing list >> Matplotlib-devel at python.org >> https://mail.python.org/mailman/listinfo/matplotlib-devel > > -- > Jody Klymak > http://web.uvic.ca/~jklymak/ > > > > > > _______________________________________________ > Matplotlib-devel mailing list > Matplotlib-devel at python.org > https://mail.python.org/mailman/listinfo/matplotlib-devel > > > _______________________________________________ > Matplotlib-devel mailing list > Matplotlib-devel at python.org > https://mail.python.org/mailman/listinfo/matplotlib-devel > From antony.lee at institutoptique.fr Tue Jan 15 16:44:59 2019 From: antony.lee at institutoptique.fr (Antony Lee) Date: Tue, 15 Jan 2019 22:44:59 +0100 Subject: [Matplotlib-devel] Proposal to amend the PR merge rules In-Reply-To: References: Message-ID: Eric, Neither the ttc font, nor the pdf/ps common code PRs actually touch any "complex" point regarding fonts or pdf/ps. I have described the ttc font in my previous message; the pdf/ps PR is really just, look, these two classes share 80 lines worth of code that's literally copy-pasted, let's just put that in a common parent class and inherit from it. Conversely, one font-related PR for which I actually have low hopes of getting properly reviewed (except on an "sure, I trust you" basis) is https://github.com/matplotlib/matplotlib/pull/12928, which actually touches a rather complex point about font encodings (some of which I had to clarify on the FreeType mailing list). (If anyone wants to have a look at it, though... :p) Antony On Tue, Jan 15, 2019 at 10:36 PM Eric Firing wrote: > Antony, > > Your examples illustrate that the problem is often that we don't have > enough people who understand some areas, like fonts and pdf/ps file > formats, to get 2 thorough and knowledgeable reviews in a reasonable time. > > I'm fine with your proposed rule. It should help at least a little bit. > > Eric > > > On 2019/01/15 9:11 AM, Antony Lee wrote: > > I'm obviously more aware of my own PRs, so here are a few I'd have put > > up for single-review-merge (note that it's quite possible that they'd > > have attracted more attention and gotten merged faster under that > > system; that's also the goal...). Obviously other devs may think about > > other PRs that could have benefitted from the same process. > > > > - https://github.com/matplotlib/matplotlib/pull/9787 adds support for > > the ttc font format (requested since 2014) for png/svg output (that's > > provided basically for free by FreeType) but not for pdf/ps (that would > > require more difficult changes). The entire PR comes down to: > > * adding "ttc" as an extension that should be treated like > "ttf"/"otf", > > * adding error handling to pdf/ps to signal these formats are not > > supported there, > > * tests. > > The PR sat open for 10 months before a first positive review (because > > no one cares about fonts, I guess) and took another 3 months to attract > > a second positive review. > > > > - https://github.com/matplotlib/matplotlib/pull/12472 does a fairly > > trivial fix to fontList.json to make it reusable for Matplotlibs > > installed in different venvs, got a positive review in one day and > > waited another 2.5 months for a second positive review. > > > > - https://github.com/matplotlib/matplotlib/pull/9867 deduplicates > > completely duplicated code between the pdf and ps backends, mostly for > > maintainability's sake; it took 7.5 months (and 5 rebases due to > > conflicts) to get a first positive review and another 6 (and around as > > many rebases) to get a second one. > > > > ----- > > > > As for the specifics (as I see it): > > - Someone opens a PR, it gets a positive review, no negative review, and > > no activity occurs for two weeks (significant activity -- excluding > > trivial chat on the thread). > > - Sponsor (first reviewer, or author if a committer) *pings* all devs by > > mentioning @matplotlib/developers on the PR thread with the intent to > > single-review-merge, and labels the PR accordingly. > > - The PR can still be reviewed/merged/rejected by the normal review > > process. However, if no one explicitly opposes the merge within two > > weeks, anyone can merge it at that point (including by self-merge). > > > > I would not exclude all API changes from the process. For example, > > > https://github.com/matplotlib/matplotlib/pull/13173/commits/e90d264f3e5a263ef57243a2af86b46ab74ccc16 > > deprecates (and prepares for deletion) two parameters to imshow() that > > have had no effect since 2006. Let's pretend for a second this commit > > was a single independent PR (and not an example to showcase the > > parameter-deprecating decorator...); if it started being forgotten I > > would have proposed it for single-review-merge (again, note that if > > you're worried about the change, you don't need to reject the PR; you > > can just say, I think this API change needs to be approved by a second > > reviewer and that'll block the single-review-merge just as well). > > > > Antony > > > > On Tue, Jan 15, 2019 at 7:46 PM Nelle Varoquaux > > > wrote: > > > > The proposed scheme by Paul doesn't seem reasonable to me. Core > > contributor A or committers needs to actively reach out to other > > core contributors: labeling is not enough IMO. > > > > > > On Tue, 15 Jan 2019 at 10:37, Jody Klymak > > wrote: > > > > > > > >> On 15 Jan 2019, at 10:20, Paul Hobson >> > wrote: > >> > >> I think this would be a good policy as well. Can I get some > >> clarification on the flow? The way I understand it: > >> > >> 1) Someone (committer, contributor, or first-timer) opens a > >> simple PR > >> 2) Committer A reviews it, adds a "single-review-merge" label, > >> then digitally walks away > >> 3) No less than two weeks later, Committer B sees the PR, > >> notices the label, opens it up, and merges without reviewing. > > > > If there is a Committer B, I think they should at least look at > > the PR to make sure it doesn?t have any hidden API changes, or > > introduce anything hostile. > > > > The point here is that getting the third person to look at a PR > > is proving quite difficult, and if the rest of the folks with > > the commit bit haven?t looked at a PR for a month, even after > > being warned, then silence indicates consent. > > > > Note that only folks with the commit bit can label PRs, so > > anyone flagging a PR like this is implicitly trusted to not be > > wreaking havoc, and not doing this for major changes. Since we > > all curate our own PRs, I rather expect that the PR contributor > > will often be the person who adds the flag. Whether we want to > > allow a self-merge at that point is up for debate, but if not, > > then folks need to get in the habit of looking at old flagged > > PRs and merging them. > > > > I think a huge problem we have is the LIFO default sort of the > > github PR queue, and that any PR not on the first page might as > > well be closed for all the attention it will get without > > constant nagging. > > > > Cheers, Jody > > > >> Does that capture a successful "single-review-merge" lifecycle? > >> -paul > >> > >> On Tue, Jan 15, 2019 at 9:57 AM Nelle Varoquaux > >> > > >> wrote: > >> > >> Hello, > >> > >> Overall, I think this is a good idea, but I would like > >> specifics on what "uncontroversial but uninteresting" PR > >> mean. IMO, that should exclude any PR with API changes. > >> > >> Cheers, > >> N > >> > >> On Tue, 15 Jan 2019 at 06:00, Antony Lee > >> > > wrote: > >> > >> Hi all, > >> > >> During the weekly dev call, I proposed to amend the PR > >> merge rules, following an initial comment on Github > >> [ > https://github.com/matplotlib/matplotlib/pull/13173#issuecomment-453921220]. > > >> There was general agreement among the devs present > >> (Tom, Hannah, Jody, and myself), so I'm putting it > >> here for discussion. The objective of this change is > >> to prevent "uncontroversial but uninteresting" PRs > >> from falling into oblivion, and try to decrease the > >> size of the open PR stack. > >> > >> The current rule is that a PR needs positive reviews > >> (from two committers, and excluding the author if a > >> committer) to be merged, except that doc-only PRs > >> (docstrings, rst) only needs a single positive review. > >> > >> I am proposing that, if a (non-doc) PR already has one > >> positive review, but no activity on that PR has > >> occurred for two weeks (exact time interval up to > >> bikeshedding) [Jody suggests: and the PR has 100% code > >> coverage], then a committer (either the first > >> reviewer, or the author if a committer) can suggest > >> that it be merged on the basis of that single review. > >> To do so, the "sponsor" should ping all developers > >> (@matplotlib/developers) on that issue indicating that > >> intent, and add a "single-review-merge" label on the > >> PR (so that these PRs can easily be found). > >> Committers are encouraged to review the PR to accept > >> and merge it or reject it or request changes on it; > >> but they can also just indicate that they consider the > >> PR sufficiently complex that a proper second review is > >> needed before merging, or request an extension, etc. > >> To do so they should still leave a "reject" review, > >> even if just saying "objecting to single-review > >> merged; anyone can dismiss after a second review". > >> However, if within another two weeks, no committer > >> voiced any objection (explicitly, i.e. by rejecting), > >> then the PR can indeed be merged on the basis of that > >> single review. > >> > >> To avoid overwhelming the system, any committer can > >> only "sponsor" a single PR at a time. > >> > >> Thoughts? > >> > >> Antony > >> _______________________________________________ > >> Matplotlib-devel mailing list > >> Matplotlib-devel at python.org > >> > >> > https://mail.python.org/mailman/listinfo/matplotlib-devel > >> > >> _______________________________________________ > >> Matplotlib-devel mailing list > >> Matplotlib-devel at python.org > >> > >> https://mail.python.org/mailman/listinfo/matplotlib-devel > >> > >> _______________________________________________ > >> Matplotlib-devel mailing list > >> Matplotlib-devel at python.org > > >> https://mail.python.org/mailman/listinfo/matplotlib-devel > > > > -- > > Jody Klymak > > http://web.uvic.ca/~jklymak/ > > > > > > > > > > > > _______________________________________________ > > Matplotlib-devel mailing list > > Matplotlib-devel at python.org > > https://mail.python.org/mailman/listinfo/matplotlib-devel > > > > > > _______________________________________________ > > Matplotlib-devel mailing list > > Matplotlib-devel at python.org > > https://mail.python.org/mailman/listinfo/matplotlib-devel > > > > _______________________________________________ > Matplotlib-devel mailing list > Matplotlib-devel at python.org > https://mail.python.org/mailman/listinfo/matplotlib-devel > -------------- next part -------------- An HTML attachment was scrubbed... URL: From efiring at hawaii.edu Tue Jan 15 18:18:12 2019 From: efiring at hawaii.edu (Eric Firing) Date: Tue, 15 Jan 2019 13:18:12 -1000 Subject: [Matplotlib-devel] Proposal to amend the PR merge rules In-Reply-To: References: Message-ID: <92fabc0d-6d6e-7781-3a9e-c95f114c4828@hawaii.edu> On 2019/01/15 11:44 AM, Antony Lee wrote: > Neither the ttc font, nor the pdf/ps common code PRs actually touch any > "complex" point regarding fonts or pdf/ps. I understand. Nevertheless, there is an understandable tendency for someone like me to look at such a PR and back off, hoping someone more familiar with those areas will take care of it. I think your proposal will help with this problem. Eric From tcaswell at gmail.com Tue Jan 15 18:18:52 2019 From: tcaswell at gmail.com (Thomas Caswell) Date: Tue, 15 Jan 2019 18:18:52 -0500 Subject: [Matplotlib-devel] Proposal to amend the PR merge rules In-Reply-To: References: Message-ID: I think a distinction between "no one cares" and "no one has bandwidth" needs to be made. If you look at the 'pulse' page, we merging 100+ PRs a month ( https://github.com/matplotlib/matplotlib/pulse/monthly) so reviews are happening, the issue is just that some are falling through the cracks. Tom On Tue, Jan 15, 2019 at 4:45 PM Antony Lee wrote: > Eric, > Neither the ttc font, nor the pdf/ps common code PRs actually touch any > "complex" point regarding fonts or pdf/ps. I have described the ttc font > in my previous message; the pdf/ps PR is really just, look, these two > classes share 80 lines worth of code that's literally copy-pasted, let's > just put that in a common parent class and inherit from it. > Conversely, one font-related PR for which I actually have low hopes of > getting properly reviewed (except on an "sure, I trust you" basis) is > https://github.com/matplotlib/matplotlib/pull/12928, which actually > touches a rather complex point about font encodings (some of which I had to > clarify on the FreeType mailing list). (If anyone wants to have a look at > it, though... :p) > Antony > > On Tue, Jan 15, 2019 at 10:36 PM Eric Firing wrote: > >> Antony, >> >> Your examples illustrate that the problem is often that we don't have >> enough people who understand some areas, like fonts and pdf/ps file >> formats, to get 2 thorough and knowledgeable reviews in a reasonable time. >> >> I'm fine with your proposed rule. It should help at least a little bit. >> >> Eric >> >> >> On 2019/01/15 9:11 AM, Antony Lee wrote: >> > I'm obviously more aware of my own PRs, so here are a few I'd have put >> > up for single-review-merge (note that it's quite possible that they'd >> > have attracted more attention and gotten merged faster under that >> > system; that's also the goal...). Obviously other devs may think about >> > other PRs that could have benefitted from the same process. >> > >> > - https://github.com/matplotlib/matplotlib/pull/9787 adds support for >> > the ttc font format (requested since 2014) for png/svg output (that's >> > provided basically for free by FreeType) but not for pdf/ps (that would >> > require more difficult changes). The entire PR comes down to: >> > * adding "ttc" as an extension that should be treated like >> "ttf"/"otf", >> > * adding error handling to pdf/ps to signal these formats are not >> > supported there, >> > * tests. >> > The PR sat open for 10 months before a first positive review >> (because >> > no one cares about fonts, I guess) and took another 3 months to attract >> > a second positive review. >> > >> > - https://github.com/matplotlib/matplotlib/pull/12472 does a fairly >> > trivial fix to fontList.json to make it reusable for Matplotlibs >> > installed in different venvs, got a positive review in one day and >> > waited another 2.5 months for a second positive review. >> > >> > - https://github.com/matplotlib/matplotlib/pull/9867 deduplicates >> > completely duplicated code between the pdf and ps backends, mostly for >> > maintainability's sake; it took 7.5 months (and 5 rebases due to >> > conflicts) to get a first positive review and another 6 (and around as >> > many rebases) to get a second one. >> > >> > ----- >> > >> > As for the specifics (as I see it): >> > - Someone opens a PR, it gets a positive review, no negative review, >> and >> > no activity occurs for two weeks (significant activity -- excluding >> > trivial chat on the thread). >> > - Sponsor (first reviewer, or author if a committer) *pings* all devs >> by >> > mentioning @matplotlib/developers on the PR thread with the intent to >> > single-review-merge, and labels the PR accordingly. >> > - The PR can still be reviewed/merged/rejected by the normal review >> > process. However, if no one explicitly opposes the merge within two >> > weeks, anyone can merge it at that point (including by self-merge). >> > >> > I would not exclude all API changes from the process. For example, >> > >> https://github.com/matplotlib/matplotlib/pull/13173/commits/e90d264f3e5a263ef57243a2af86b46ab74ccc16 >> > deprecates (and prepares for deletion) two parameters to imshow() that >> > have had no effect since 2006. Let's pretend for a second this commit >> > was a single independent PR (and not an example to showcase the >> > parameter-deprecating decorator...); if it started being forgotten I >> > would have proposed it for single-review-merge (again, note that if >> > you're worried about the change, you don't need to reject the PR; you >> > can just say, I think this API change needs to be approved by a second >> > reviewer and that'll block the single-review-merge just as well). >> > >> > Antony >> > >> > On Tue, Jan 15, 2019 at 7:46 PM Nelle Varoquaux >> > > wrote: >> > >> > The proposed scheme by Paul doesn't seem reasonable to me. Core >> > contributor A or committers needs to actively reach out to other >> > core contributors: labeling is not enough IMO. >> > >> > >> > On Tue, 15 Jan 2019 at 10:37, Jody Klymak > > > wrote: >> > >> > >> > >> >> On 15 Jan 2019, at 10:20, Paul Hobson > >> > wrote: >> >> >> >> I think this would be a good policy as well. Can I get some >> >> clarification on the flow? The way I understand it: >> >> >> >> 1) Someone (committer, contributor, or first-timer) opens a >> >> simple PR >> >> 2) Committer A reviews it, adds a "single-review-merge" label, >> >> then digitally walks away >> >> 3) No less than two weeks later, Committer B sees the PR, >> >> notices the label, opens it up, and merges without reviewing. >> > >> > If there is a Committer B, I think they should at least look at >> > the PR to make sure it doesn?t have any hidden API changes, or >> > introduce anything hostile. >> > >> > The point here is that getting the third person to look at a PR >> > is proving quite difficult, and if the rest of the folks with >> > the commit bit haven?t looked at a PR for a month, even after >> > being warned, then silence indicates consent. >> > >> > Note that only folks with the commit bit can label PRs, so >> > anyone flagging a PR like this is implicitly trusted to not be >> > wreaking havoc, and not doing this for major changes. Since we >> > all curate our own PRs, I rather expect that the PR contributor >> > will often be the person who adds the flag. Whether we want to >> > allow a self-merge at that point is up for debate, but if not, >> > then folks need to get in the habit of looking at old flagged >> > PRs and merging them. >> > >> > I think a huge problem we have is the LIFO default sort of the >> > github PR queue, and that any PR not on the first page might as >> > well be closed for all the attention it will get without >> > constant nagging. >> > >> > Cheers, Jody >> > >> >> Does that capture a successful "single-review-merge" lifecycle? >> >> -paul >> >> >> >> On Tue, Jan 15, 2019 at 9:57 AM Nelle Varoquaux >> >> > >> >> wrote: >> >> >> >> Hello, >> >> >> >> Overall, I think this is a good idea, but I would like >> >> specifics on what "uncontroversial but uninteresting" PR >> >> mean. IMO, that should exclude any PR with API changes. >> >> >> >> Cheers, >> >> N >> >> >> >> On Tue, 15 Jan 2019 at 06:00, Antony Lee >> >> > >> wrote: >> >> >> >> Hi all, >> >> >> >> During the weekly dev call, I proposed to amend the PR >> >> merge rules, following an initial comment on Github >> >> [ >> https://github.com/matplotlib/matplotlib/pull/13173#issuecomment-453921220]. >> >> >> There was general agreement among the devs present >> >> (Tom, Hannah, Jody, and myself), so I'm putting it >> >> here for discussion. The objective of this change is >> >> to prevent "uncontroversial but uninteresting" PRs >> >> from falling into oblivion, and try to decrease the >> >> size of the open PR stack. >> >> >> >> The current rule is that a PR needs positive reviews >> >> (from two committers, and excluding the author if a >> >> committer) to be merged, except that doc-only PRs >> >> (docstrings, rst) only needs a single positive review. >> >> >> >> I am proposing that, if a (non-doc) PR already has one >> >> positive review, but no activity on that PR has >> >> occurred for two weeks (exact time interval up to >> >> bikeshedding) [Jody suggests: and the PR has 100% code >> >> coverage], then a committer (either the first >> >> reviewer, or the author if a committer) can suggest >> >> that it be merged on the basis of that single review. >> >> To do so, the "sponsor" should ping all developers >> >> (@matplotlib/developers) on that issue indicating that >> >> intent, and add a "single-review-merge" label on the >> >> PR (so that these PRs can easily be found). >> >> Committers are encouraged to review the PR to accept >> >> and merge it or reject it or request changes on it; >> >> but they can also just indicate that they consider the >> >> PR sufficiently complex that a proper second review is >> >> needed before merging, or request an extension, etc. >> >> To do so they should still leave a "reject" review, >> >> even if just saying "objecting to single-review >> >> merged; anyone can dismiss after a second review". >> >> However, if within another two weeks, no committer >> >> voiced any objection (explicitly, i.e. by rejecting), >> >> then the PR can indeed be merged on the basis of that >> >> single review. >> >> >> >> To avoid overwhelming the system, any committer can >> >> only "sponsor" a single PR at a time. >> >> >> >> Thoughts? >> >> >> >> Antony >> >> _______________________________________________ >> >> Matplotlib-devel mailing list >> >> Matplotlib-devel at python.org >> >> >> >> >> https://mail.python.org/mailman/listinfo/matplotlib-devel >> >> >> >> _______________________________________________ >> >> Matplotlib-devel mailing list >> >> Matplotlib-devel at python.org >> >> >> >> https://mail.python.org/mailman/listinfo/matplotlib-devel >> >> >> >> _______________________________________________ >> >> Matplotlib-devel mailing list >> >> Matplotlib-devel at python.org > Matplotlib-devel at python.org> >> >> https://mail.python.org/mailman/listinfo/matplotlib-devel >> > >> > -- >> > Jody Klymak >> > http://web.uvic.ca/~jklymak/ >> > >> > >> > >> > >> > >> > _______________________________________________ >> > Matplotlib-devel mailing list >> > Matplotlib-devel at python.org >> > https://mail.python.org/mailman/listinfo/matplotlib-devel >> > >> > >> > _______________________________________________ >> > Matplotlib-devel mailing list >> > Matplotlib-devel at python.org >> > https://mail.python.org/mailman/listinfo/matplotlib-devel >> > >> >> _______________________________________________ >> Matplotlib-devel mailing list >> Matplotlib-devel at python.org >> https://mail.python.org/mailman/listinfo/matplotlib-devel >> > _______________________________________________ > Matplotlib-devel mailing list > Matplotlib-devel at python.org > https://mail.python.org/mailman/listinfo/matplotlib-devel > -- Thomas Caswell tcaswell at gmail.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From nelle.varoquaux at gmail.com Tue Jan 15 18:45:54 2019 From: nelle.varoquaux at gmail.com (Nelle Varoquaux) Date: Tue, 15 Jan 2019 15:45:54 -0800 Subject: [Matplotlib-devel] Proposal to amend the PR merge rules In-Reply-To: References: Message-ID: For the way GitHub sorts pull requests, it is indeed a problem, but there's a way to sort by "most recently updated." I sometimes use it to make sure that I review recently updated pull request that don't appear on the first page. It seems to me that this "uncontroversial but uninteresting" rule is currently not based on objective criteria. I'm personally not in favor of removing two core contributors approval for anything that modifies the API. We have had too many close calls on PR almost merged that had backward incompatible changes: I realize that some core developers believe that we are not moving fast enough, on a project with hundreds of thousands of users, changes to the API should not be considered lightly. Here's an attempt at better defining an "uncontroversial but uninteresting" rule based on objective criteria: - Has not been commented on by more than one core developer; - Doesn't change the *public* API; - Doesn't add or remove a functionality; - Doesn't add or remove a dependency; - Is fully tested; - All parameters are fully documented; - Doesn't fix a release-critical bug; - Is less than XX lines of code. (Long PR are harder to review, and thus the likelihood of one core dev not noticing a problem is higher). Threshold needs to be defined. To be honest, it also doesn't help that some people have 50+ pull requests open (Antony? :p). It also doesn't help that some PR contain the same code (13117, 13128) and are being hold off until the other one is merged (also, why is 13117 being hold off for a PR that has been opened afterwards?). IMO, if a PR can be merged, it should be merged. But we shouldn't make too many exceptions to the two core contributors rules. It's easier to have two core contributor review a pull request, than to try to fix API issues on code that has been merged and released. Having two pairs of eyes on a PR is also a good way to avoid having a bus factor of 1 on some areas of the library. On Tue, 15 Jan 2019 at 15:19, Thomas Caswell wrote: > I think a distinction between "no one cares" and "no one has bandwidth" > needs to be made. > > If you look at the 'pulse' page, we merging 100+ PRs a month ( > https://github.com/matplotlib/matplotlib/pulse/monthly) so reviews are > happening, the issue is just that some are falling through the cracks. > > Tom > > On Tue, Jan 15, 2019 at 4:45 PM Antony Lee > wrote: > >> Eric, >> Neither the ttc font, nor the pdf/ps common code PRs actually touch any >> "complex" point regarding fonts or pdf/ps. I have described the ttc font >> in my previous message; the pdf/ps PR is really just, look, these two >> classes share 80 lines worth of code that's literally copy-pasted, let's >> just put that in a common parent class and inherit from it. >> Conversely, one font-related PR for which I actually have low hopes of >> getting properly reviewed (except on an "sure, I trust you" basis) is >> https://github.com/matplotlib/matplotlib/pull/12928, which actually >> touches a rather complex point about font encodings (some of which I had to >> clarify on the FreeType mailing list). (If anyone wants to have a look at >> it, though... :p) >> Antony >> >> On Tue, Jan 15, 2019 at 10:36 PM Eric Firing wrote: >> >>> Antony, >>> >>> Your examples illustrate that the problem is often that we don't have >>> enough people who understand some areas, like fonts and pdf/ps file >>> formats, to get 2 thorough and knowledgeable reviews in a reasonable >>> time. >>> >>> I'm fine with your proposed rule. It should help at least a little bit. >>> >>> Eric >>> >>> >>> On 2019/01/15 9:11 AM, Antony Lee wrote: >>> > I'm obviously more aware of my own PRs, so here are a few I'd have put >>> > up for single-review-merge (note that it's quite possible that they'd >>> > have attracted more attention and gotten merged faster under that >>> > system; that's also the goal...). Obviously other devs may think >>> about >>> > other PRs that could have benefitted from the same process. >>> > >>> > - https://github.com/matplotlib/matplotlib/pull/9787 adds support for >>> > the ttc font format (requested since 2014) for png/svg output (that's >>> > provided basically for free by FreeType) but not for pdf/ps (that >>> would >>> > require more difficult changes). The entire PR comes down to: >>> > * adding "ttc" as an extension that should be treated like >>> "ttf"/"otf", >>> > * adding error handling to pdf/ps to signal these formats are not >>> > supported there, >>> > * tests. >>> > The PR sat open for 10 months before a first positive review >>> (because >>> > no one cares about fonts, I guess) and took another 3 months to >>> attract >>> > a second positive review. >>> > >>> > - https://github.com/matplotlib/matplotlib/pull/12472 does a fairly >>> > trivial fix to fontList.json to make it reusable for Matplotlibs >>> > installed in different venvs, got a positive review in one day and >>> > waited another 2.5 months for a second positive review. >>> > >>> > - https://github.com/matplotlib/matplotlib/pull/9867 deduplicates >>> > completely duplicated code between the pdf and ps backends, mostly for >>> > maintainability's sake; it took 7.5 months (and 5 rebases due to >>> > conflicts) to get a first positive review and another 6 (and around as >>> > many rebases) to get a second one. >>> > >>> > ----- >>> > >>> > As for the specifics (as I see it): >>> > - Someone opens a PR, it gets a positive review, no negative review, >>> and >>> > no activity occurs for two weeks (significant activity -- excluding >>> > trivial chat on the thread). >>> > - Sponsor (first reviewer, or author if a committer) *pings* all devs >>> by >>> > mentioning @matplotlib/developers on the PR thread with the intent to >>> > single-review-merge, and labels the PR accordingly. >>> > - The PR can still be reviewed/merged/rejected by the normal review >>> > process. However, if no one explicitly opposes the merge within two >>> > weeks, anyone can merge it at that point (including by self-merge). >>> > >>> > I would not exclude all API changes from the process. For example, >>> > >>> https://github.com/matplotlib/matplotlib/pull/13173/commits/e90d264f3e5a263ef57243a2af86b46ab74ccc16 >>> > deprecates (and prepares for deletion) two parameters to imshow() that >>> > have had no effect since 2006. Let's pretend for a second this commit >>> > was a single independent PR (and not an example to showcase the >>> > parameter-deprecating decorator...); if it started being forgotten I >>> > would have proposed it for single-review-merge (again, note that if >>> > you're worried about the change, you don't need to reject the PR; you >>> > can just say, I think this API change needs to be approved by a second >>> > reviewer and that'll block the single-review-merge just as well). >>> > >>> > Antony >>> > >>> > On Tue, Jan 15, 2019 at 7:46 PM Nelle Varoquaux >>> > > wrote: >>> > >>> > The proposed scheme by Paul doesn't seem reasonable to me. Core >>> > contributor A or committers needs to actively reach out to other >>> > core contributors: labeling is not enough IMO. >>> > >>> > >>> > On Tue, 15 Jan 2019 at 10:37, Jody Klymak >> > > wrote: >>> > >>> > >>> > >>> >> On 15 Jan 2019, at 10:20, Paul Hobson >> >> > wrote: >>> >> >>> >> I think this would be a good policy as well. Can I get some >>> >> clarification on the flow? The way I understand it: >>> >> >>> >> 1) Someone (committer, contributor, or first-timer) opens a >>> >> simple PR >>> >> 2) Committer A reviews it, adds a "single-review-merge" label, >>> >> then digitally walks away >>> >> 3) No less than two weeks later, Committer B sees the PR, >>> >> notices the label, opens it up, and merges without reviewing. >>> > >>> > If there is a Committer B, I think they should at least look at >>> > the PR to make sure it doesn?t have any hidden API changes, or >>> > introduce anything hostile. >>> > >>> > The point here is that getting the third person to look at a PR >>> > is proving quite difficult, and if the rest of the folks with >>> > the commit bit haven?t looked at a PR for a month, even after >>> > being warned, then silence indicates consent. >>> > >>> > Note that only folks with the commit bit can label PRs, so >>> > anyone flagging a PR like this is implicitly trusted to not be >>> > wreaking havoc, and not doing this for major changes. Since we >>> > all curate our own PRs, I rather expect that the PR contributor >>> > will often be the person who adds the flag. Whether we want to >>> > allow a self-merge at that point is up for debate, but if not, >>> > then folks need to get in the habit of looking at old flagged >>> > PRs and merging them. >>> > >>> > I think a huge problem we have is the LIFO default sort of the >>> > github PR queue, and that any PR not on the first page might as >>> > well be closed for all the attention it will get without >>> > constant nagging. >>> > >>> > Cheers, Jody >>> > >>> >> Does that capture a successful "single-review-merge" >>> lifecycle? >>> >> -paul >>> >> >>> >> On Tue, Jan 15, 2019 at 9:57 AM Nelle Varoquaux >>> >> >> >> >>> >> wrote: >>> >> >>> >> Hello, >>> >> >>> >> Overall, I think this is a good idea, but I would like >>> >> specifics on what "uncontroversial but uninteresting" PR >>> >> mean. IMO, that should exclude any PR with API changes. >>> >> >>> >> Cheers, >>> >> N >>> >> >>> >> On Tue, 15 Jan 2019 at 06:00, Antony Lee >>> >> > >>> wrote: >>> >> >>> >> Hi all, >>> >> >>> >> During the weekly dev call, I proposed to amend the PR >>> >> merge rules, following an initial comment on Github >>> >> [ >>> https://github.com/matplotlib/matplotlib/pull/13173#issuecomment-453921220]. >>> >>> >> There was general agreement among the devs present >>> >> (Tom, Hannah, Jody, and myself), so I'm putting it >>> >> here for discussion. The objective of this change is >>> >> to prevent "uncontroversial but uninteresting" PRs >>> >> from falling into oblivion, and try to decrease the >>> >> size of the open PR stack. >>> >> >>> >> The current rule is that a PR needs positive reviews >>> >> (from two committers, and excluding the author if a >>> >> committer) to be merged, except that doc-only PRs >>> >> (docstrings, rst) only needs a single positive review. >>> >> >>> >> I am proposing that, if a (non-doc) PR already has one >>> >> positive review, but no activity on that PR has >>> >> occurred for two weeks (exact time interval up to >>> >> bikeshedding) [Jody suggests: and the PR has 100% code >>> >> coverage], then a committer (either the first >>> >> reviewer, or the author if a committer) can suggest >>> >> that it be merged on the basis of that single review. >>> >> To do so, the "sponsor" should ping all developers >>> >> (@matplotlib/developers) on that issue indicating that >>> >> intent, and add a "single-review-merge" label on the >>> >> PR (so that these PRs can easily be found). >>> >> Committers are encouraged to review the PR to accept >>> >> and merge it or reject it or request changes on it; >>> >> but they can also just indicate that they consider the >>> >> PR sufficiently complex that a proper second review is >>> >> needed before merging, or request an extension, etc. >>> >> To do so they should still leave a "reject" review, >>> >> even if just saying "objecting to single-review >>> >> merged; anyone can dismiss after a second review". >>> >> However, if within another two weeks, no committer >>> >> voiced any objection (explicitly, i.e. by rejecting), >>> >> then the PR can indeed be merged on the basis of that >>> >> single review. >>> >> >>> >> To avoid overwhelming the system, any committer can >>> >> only "sponsor" a single PR at a time. >>> >> >>> >> Thoughts? >>> >> >>> >> Antony >>> >> _______________________________________________ >>> >> Matplotlib-devel mailing list >>> >> Matplotlib-devel at python.org >>> >> >>> >> >>> https://mail.python.org/mailman/listinfo/matplotlib-devel >>> >> >>> >> _______________________________________________ >>> >> Matplotlib-devel mailing list >>> >> Matplotlib-devel at python.org >>> >> >>> >> https://mail.python.org/mailman/listinfo/matplotlib-devel >>> >> >>> >> _______________________________________________ >>> >> Matplotlib-devel mailing list >>> >> Matplotlib-devel at python.org >> Matplotlib-devel at python.org> >>> >> https://mail.python.org/mailman/listinfo/matplotlib-devel >>> > >>> > -- >>> > Jody Klymak >>> > http://web.uvic.ca/~jklymak/ >>> > >>> > >>> > >>> > >>> > >>> > _______________________________________________ >>> > Matplotlib-devel mailing list >>> > Matplotlib-devel at python.org >>> > https://mail.python.org/mailman/listinfo/matplotlib-devel >>> > >>> > >>> > _______________________________________________ >>> > Matplotlib-devel mailing list >>> > Matplotlib-devel at python.org >>> > https://mail.python.org/mailman/listinfo/matplotlib-devel >>> > >>> >>> _______________________________________________ >>> Matplotlib-devel mailing list >>> Matplotlib-devel at python.org >>> https://mail.python.org/mailman/listinfo/matplotlib-devel >>> >> _______________________________________________ >> Matplotlib-devel mailing list >> Matplotlib-devel at python.org >> https://mail.python.org/mailman/listinfo/matplotlib-devel >> > > > -- > Thomas Caswell > tcaswell at gmail.com > _______________________________________________ > Matplotlib-devel mailing list > Matplotlib-devel at python.org > https://mail.python.org/mailman/listinfo/matplotlib-devel > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dstansby at gmail.com Wed Jan 16 04:34:22 2019 From: dstansby at gmail.com (David Stansby) Date: Wed, 16 Jan 2019 09:34:22 +0000 Subject: [Matplotlib-devel] Proposal to amend the PR merge rules In-Reply-To: References: Message-ID: I think it is good remember that (as far as I know) everyone submitting PRs and everyone reviewing them isn't paid and does so in their spare time; therefore is no obligation for anyone to review anything on any particular timescale. One thing that hasn't been discussed is how to increase the reviewing capacity of the community, which I guess would involve working out how to attract and retain more regular contributors. re. the proposed changes, I'm +1 modulo objective criteria being agreed upon. Personally I agree with Nelle's list, with the addition "has 100% code coverage". David On Tue, 15 Jan 2019, 23:46 Nelle Varoquaux For the way GitHub sorts pull requests, it is indeed a problem, but > there's a way to sort by "most recently updated." I sometimes use it to > make sure that I review recently updated pull request that don't appear on > the first page. > > It seems to me that this "uncontroversial but uninteresting" rule is > currently not based on objective criteria. I'm personally not in favor of > removing two core contributors approval for anything that modifies the > API. We have had too many close calls on PR almost merged that had > backward incompatible changes: I realize that some core developers believe > that we are not moving fast enough, on a project with hundreds of thousands > of users, changes to the API should not be considered lightly. > > Here's an attempt at better defining an "uncontroversial but > uninteresting" rule based on objective criteria: > - Has not been commented on by more than one core developer; > - Doesn't change the *public* API; > - Doesn't add or remove a functionality; > - Doesn't add or remove a dependency; > - Is fully tested; > - All parameters are fully documented; > - Doesn't fix a release-critical bug; > - Is less than XX lines of code. (Long PR are harder to review, and thus > the likelihood of one core dev not noticing a problem is higher). Threshold > needs to be defined. > > To be honest, it also doesn't help that some people have 50+ pull requests > open (Antony? :p). It also doesn't help that some PR contain the same code > (13117, 13128) and are being hold off until the other one is merged (also, > why is 13117 being hold off for a PR that has been opened afterwards?). > IMO, if a PR can be merged, it should be merged. But we shouldn't make too > many exceptions to the two core contributors rules. It's easier to have two > core contributor review a pull request, than to try to fix API issues on > code that has been merged and released. Having two pairs of eyes on a PR is > also a good way to avoid having a bus factor of 1 on some areas of the > library. > > > On Tue, 15 Jan 2019 at 15:19, Thomas Caswell wrote: > >> I think a distinction between "no one cares" and "no one has bandwidth" >> needs to be made. >> >> If you look at the 'pulse' page, we merging 100+ PRs a month ( >> https://github.com/matplotlib/matplotlib/pulse/monthly) so reviews are >> happening, the issue is just that some are falling through the cracks. >> >> Tom >> >> On Tue, Jan 15, 2019 at 4:45 PM Antony Lee >> wrote: >> >>> Eric, >>> Neither the ttc font, nor the pdf/ps common code PRs actually touch any >>> "complex" point regarding fonts or pdf/ps. I have described the ttc font >>> in my previous message; the pdf/ps PR is really just, look, these two >>> classes share 80 lines worth of code that's literally copy-pasted, let's >>> just put that in a common parent class and inherit from it. >>> Conversely, one font-related PR for which I actually have low hopes of >>> getting properly reviewed (except on an "sure, I trust you" basis) is >>> https://github.com/matplotlib/matplotlib/pull/12928, which actually >>> touches a rather complex point about font encodings (some of which I had to >>> clarify on the FreeType mailing list). (If anyone wants to have a look at >>> it, though... :p) >>> Antony >>> >>> On Tue, Jan 15, 2019 at 10:36 PM Eric Firing wrote: >>> >>>> Antony, >>>> >>>> Your examples illustrate that the problem is often that we don't have >>>> enough people who understand some areas, like fonts and pdf/ps file >>>> formats, to get 2 thorough and knowledgeable reviews in a reasonable >>>> time. >>>> >>>> I'm fine with your proposed rule. It should help at least a little bit. >>>> >>>> Eric >>>> >>>> >>>> On 2019/01/15 9:11 AM, Antony Lee wrote: >>>> > I'm obviously more aware of my own PRs, so here are a few I'd have >>>> put >>>> > up for single-review-merge (note that it's quite possible that they'd >>>> > have attracted more attention and gotten merged faster under that >>>> > system; that's also the goal...). Obviously other devs may think >>>> about >>>> > other PRs that could have benefitted from the same process. >>>> > >>>> > - https://github.com/matplotlib/matplotlib/pull/9787 adds support >>>> for >>>> > the ttc font format (requested since 2014) for png/svg output (that's >>>> > provided basically for free by FreeType) but not for pdf/ps (that >>>> would >>>> > require more difficult changes). The entire PR comes down to: >>>> > * adding "ttc" as an extension that should be treated like >>>> "ttf"/"otf", >>>> > * adding error handling to pdf/ps to signal these formats are not >>>> > supported there, >>>> > * tests. >>>> > The PR sat open for 10 months before a first positive review >>>> (because >>>> > no one cares about fonts, I guess) and took another 3 months to >>>> attract >>>> > a second positive review. >>>> > >>>> > - https://github.com/matplotlib/matplotlib/pull/12472 does a fairly >>>> > trivial fix to fontList.json to make it reusable for Matplotlibs >>>> > installed in different venvs, got a positive review in one day and >>>> > waited another 2.5 months for a second positive review. >>>> > >>>> > - https://github.com/matplotlib/matplotlib/pull/9867 deduplicates >>>> > completely duplicated code between the pdf and ps backends, mostly >>>> for >>>> > maintainability's sake; it took 7.5 months (and 5 rebases due to >>>> > conflicts) to get a first positive review and another 6 (and around >>>> as >>>> > many rebases) to get a second one. >>>> > >>>> > ----- >>>> > >>>> > As for the specifics (as I see it): >>>> > - Someone opens a PR, it gets a positive review, no negative review, >>>> and >>>> > no activity occurs for two weeks (significant activity -- excluding >>>> > trivial chat on the thread). >>>> > - Sponsor (first reviewer, or author if a committer) *pings* all devs >>>> by >>>> > mentioning @matplotlib/developers on the PR thread with the intent to >>>> > single-review-merge, and labels the PR accordingly. >>>> > - The PR can still be reviewed/merged/rejected by the normal review >>>> > process. However, if no one explicitly opposes the merge within two >>>> > weeks, anyone can merge it at that point (including by self-merge). >>>> > >>>> > I would not exclude all API changes from the process. For example, >>>> > >>>> https://github.com/matplotlib/matplotlib/pull/13173/commits/e90d264f3e5a263ef57243a2af86b46ab74ccc16 >>>> > deprecates (and prepares for deletion) two parameters to imshow() >>>> that >>>> > have had no effect since 2006. Let's pretend for a second this >>>> commit >>>> > was a single independent PR (and not an example to showcase the >>>> > parameter-deprecating decorator...); if it started being forgotten I >>>> > would have proposed it for single-review-merge (again, note that if >>>> > you're worried about the change, you don't need to reject the PR; you >>>> > can just say, I think this API change needs to be approved by a >>>> second >>>> > reviewer and that'll block the single-review-merge just as well). >>>> > >>>> > Antony >>>> > >>>> > On Tue, Jan 15, 2019 at 7:46 PM Nelle Varoquaux >>>> > > wrote: >>>> > >>>> > The proposed scheme by Paul doesn't seem reasonable to me. Core >>>> > contributor A or committers needs to actively reach out to other >>>> > core contributors: labeling is not enough IMO. >>>> > >>>> > >>>> > On Tue, 15 Jan 2019 at 10:37, Jody Klymak >>> > > wrote: >>>> > >>>> > >>>> > >>>> >> On 15 Jan 2019, at 10:20, Paul Hobson >>> >> > wrote: >>>> >> >>>> >> I think this would be a good policy as well. Can I get some >>>> >> clarification on the flow? The way I understand it: >>>> >> >>>> >> 1) Someone (committer, contributor, or first-timer) opens a >>>> >> simple PR >>>> >> 2) Committer A reviews it, adds a "single-review-merge" >>>> label, >>>> >> then digitally walks away >>>> >> 3) No less than two weeks later, Committer B sees the PR, >>>> >> notices the label, opens it up, and merges without reviewing. >>>> > >>>> > If there is a Committer B, I think they should at least look >>>> at >>>> > the PR to make sure it doesn?t have any hidden API changes, or >>>> > introduce anything hostile. >>>> > >>>> > The point here is that getting the third person to look at a >>>> PR >>>> > is proving quite difficult, and if the rest of the folks with >>>> > the commit bit haven?t looked at a PR for a month, even after >>>> > being warned, then silence indicates consent. >>>> > >>>> > Note that only folks with the commit bit can label PRs, so >>>> > anyone flagging a PR like this is implicitly trusted to not be >>>> > wreaking havoc, and not doing this for major changes. Since >>>> we >>>> > all curate our own PRs, I rather expect that the PR >>>> contributor >>>> > will often be the person who adds the flag. Whether we want >>>> to >>>> > allow a self-merge at that point is up for debate, but if not, >>>> > then folks need to get in the habit of looking at old flagged >>>> > PRs and merging them. >>>> > >>>> > I think a huge problem we have is the LIFO default sort of the >>>> > github PR queue, and that any PR not on the first page might >>>> as >>>> > well be closed for all the attention it will get without >>>> > constant nagging. >>>> > >>>> > Cheers, Jody >>>> > >>>> >> Does that capture a successful "single-review-merge" >>>> lifecycle? >>>> >> -paul >>>> >> >>>> >> On Tue, Jan 15, 2019 at 9:57 AM Nelle Varoquaux >>>> >> >>> >> >>>> >> wrote: >>>> >> >>>> >> Hello, >>>> >> >>>> >> Overall, I think this is a good idea, but I would like >>>> >> specifics on what "uncontroversial but uninteresting" PR >>>> >> mean. IMO, that should exclude any PR with API changes. >>>> >> >>>> >> Cheers, >>>> >> N >>>> >> >>>> >> On Tue, 15 Jan 2019 at 06:00, Antony Lee >>>> >> > >>>> wrote: >>>> >> >>>> >> Hi all, >>>> >> >>>> >> During the weekly dev call, I proposed to amend the >>>> PR >>>> >> merge rules, following an initial comment on Github >>>> >> [ >>>> https://github.com/matplotlib/matplotlib/pull/13173#issuecomment-453921220]. >>>> >>>> >> There was general agreement among the devs present >>>> >> (Tom, Hannah, Jody, and myself), so I'm putting it >>>> >> here for discussion. The objective of this change is >>>> >> to prevent "uncontroversial but uninteresting" PRs >>>> >> from falling into oblivion, and try to decrease the >>>> >> size of the open PR stack. >>>> >> >>>> >> The current rule is that a PR needs positive reviews >>>> >> (from two committers, and excluding the author if a >>>> >> committer) to be merged, except that doc-only PRs >>>> >> (docstrings, rst) only needs a single positive >>>> review. >>>> >> >>>> >> I am proposing that, if a (non-doc) PR already has >>>> one >>>> >> positive review, but no activity on that PR has >>>> >> occurred for two weeks (exact time interval up to >>>> >> bikeshedding) [Jody suggests: and the PR has 100% >>>> code >>>> >> coverage], then a committer (either the first >>>> >> reviewer, or the author if a committer) can suggest >>>> >> that it be merged on the basis of that single >>>> review. >>>> >> To do so, the "sponsor" should ping all developers >>>> >> (@matplotlib/developers) on that issue indicating >>>> that >>>> >> intent, and add a "single-review-merge" label on the >>>> >> PR (so that these PRs can easily be found). >>>> >> Committers are encouraged to review the PR to accept >>>> >> and merge it or reject it or request changes on it; >>>> >> but they can also just indicate that they consider >>>> the >>>> >> PR sufficiently complex that a proper second review >>>> is >>>> >> needed before merging, or request an extension, etc. >>>> >> To do so they should still leave a "reject" review, >>>> >> even if just saying "objecting to single-review >>>> >> merged; anyone can dismiss after a second review". >>>> >> However, if within another two weeks, no committer >>>> >> voiced any objection (explicitly, i.e. by rejecting), >>>> >> then the PR can indeed be merged on the basis of that >>>> >> single review. >>>> >> >>>> >> To avoid overwhelming the system, any committer can >>>> >> only "sponsor" a single PR at a time. >>>> >> >>>> >> Thoughts? >>>> >> >>>> >> Antony >>>> >> _______________________________________________ >>>> >> Matplotlib-devel mailing list >>>> >> Matplotlib-devel at python.org >>>> >> >>>> >> >>>> https://mail.python.org/mailman/listinfo/matplotlib-devel >>>> >> >>>> >> _______________________________________________ >>>> >> Matplotlib-devel mailing list >>>> >> Matplotlib-devel at python.org >>>> >> >>>> >> >>>> https://mail.python.org/mailman/listinfo/matplotlib-devel >>>> >> >>>> >> _______________________________________________ >>>> >> Matplotlib-devel mailing list >>>> >> Matplotlib-devel at python.org >>> Matplotlib-devel at python.org> >>>> >> https://mail.python.org/mailman/listinfo/matplotlib-devel >>>> > >>>> > -- >>>> > Jody Klymak >>>> > http://web.uvic.ca/~jklymak/ >>>> > >>>> > >>>> > >>>> > >>>> > >>>> > _______________________________________________ >>>> > Matplotlib-devel mailing list >>>> > Matplotlib-devel at python.org >>>> > https://mail.python.org/mailman/listinfo/matplotlib-devel >>>> > >>>> > >>>> > _______________________________________________ >>>> > Matplotlib-devel mailing list >>>> > Matplotlib-devel at python.org >>>> > https://mail.python.org/mailman/listinfo/matplotlib-devel >>>> > >>>> >>>> _______________________________________________ >>>> Matplotlib-devel mailing list >>>> Matplotlib-devel at python.org >>>> https://mail.python.org/mailman/listinfo/matplotlib-devel >>>> >>> _______________________________________________ >>> Matplotlib-devel mailing list >>> Matplotlib-devel at python.org >>> https://mail.python.org/mailman/listinfo/matplotlib-devel >>> >> >> >> -- >> Thomas Caswell >> tcaswell at gmail.com >> _______________________________________________ >> Matplotlib-devel mailing list >> Matplotlib-devel at python.org >> https://mail.python.org/mailman/listinfo/matplotlib-devel >> > _______________________________________________ > Matplotlib-devel mailing list > Matplotlib-devel at python.org > https://mail.python.org/mailman/listinfo/matplotlib-devel > -------------- next part -------------- An HTML attachment was scrubbed... URL: From antony.lee at institutoptique.fr Wed Jan 16 06:10:46 2019 From: antony.lee at institutoptique.fr (Antony Lee) Date: Wed, 16 Jan 2019 12:10:46 +0100 Subject: [Matplotlib-devel] Proposal to amend the PR merge rules In-Reply-To: References: Message-ID: Replying to Nelle's points first: > It seems to me that this "uncontroversial but uninteresting" rule is > currently not based on objective criteria. I'm personally not in favor of > removing two core contributors approval for anything that modifies the > API. We have had too many close calls on PR almost merged that had > backward incompatible changes: I realize that some core developers believe > that we are not moving fast enough, on a project with hundreds of thousands > of users, changes to the API should not be considered lightly. > To be clear (and even though it is probably common knowledge that I would like to evolve the API faster), there is no "secret agenda" of using this to sneak API breaks through the system. > Here's an attempt at better defining an "uncontroversial but > uninteresting" rule based on objective criteria: > - Has not been commented on by more than one core developer; > Sounds reasonable (with a reasonably flexible interpretation -- similarly to the "no activity since two weeks, excluding trivial chat"), basically "no other core dev expressed interest in reviewing it". > - Doesn't change the public API; > - Doesn't add or remove a functionality; > - Doesn't add or remove a dependency; > I expect all changes of API to go through the normal route of deprecation and API change note, which even the first reviewer should make sure are present before sponsoring the PR for single-review-merge (as an aside, I guess the first reviewer could even say, "approving, but I don't want to take the responsibility for being the single reviewer"...). So now you have a two week period where any other core dev passing by can just see whether there is an API change note and make their own judgment, including deciding to block the PR if necessary, based on that change note. I guess it is reasonable to say "API deprecations or changes and new functionality should be clearly documented" to help that "skim-through" review. As for *adding* functionality, I guess you would have rejected the ttc-font PR on that premise? (Actually I now see that the additional functionality was actually not documented :)) Note that this is not some obscure edge case that no one cares about; it's a feature that has been requested for years on the tracker. - Is fully tested; > Also proposed by Jody and now by David, sounds good to me. > - All parameters are fully documented; > (sounds contradictory with "doesn't add a functionality"... if there is no new functionality it's unlikely there's a new parameter) > - Doesn't fix a release-critical bug; > These tend to all be merged just before the next release is cut anyways (review dynamics are a bit different in these periods), so I wouldn't worry about it, but sure. > - Is less than XX lines of code. (Long PR are harder to review, and thus > the likelihood of one core dev not noticing a problem is higher). Threshold > needs to be defined. > I don't actually think this is a good rule; the pdf/ps merge PR is +138 lines, -182 lines but as explained above the change is fairly trivial; obviously I could have split it into smaller chunks but if anything that would have made it harder to review. Conversely (to use an example I have already raised elsewhere) https://github.com/matplotlib/matplotlib/pull/11530 (making sure that LocationEvent.x and LocationEvent.y are integers (as they refer to pixel locations) rather than floats) is 2 lines of changes in the codebase and 20 lines of test, was reviewed by 2.5 core devs (I left a comment in passing) and we all missed that there was "problem" with it (it broke mplcursors and some other pieces of code I have that assume that it is possible to synthetize LocationEvents pointing to arbitrary places in the canvas). (Not to derail this discussion into how to evaluate API breaks once again... but I think these kinds of changes are much more insidious than, say, removing a function, as the latter would just trigger an AttributeError that is easily noticed and at worst you just go back to the previous Matplotlib release, whereas the LocationEvent change was only caught because it was covered by the mplcursors test suite; otherwise I'd have been silently generating wrong results due to it...) > To be honest, it also doesn't help that some people have 50+ pull requests > open (Antony? :p). I guess if the way to fix Matplotlib is to not contribute to Matplotlib anymore, we have a bigger problem. In fact, I have some pretty big patches fixing long-standing issues (e.g. for #8550 I have a pure-Python reimplementation of the fontconfig font weight deduction algorithm, translated from fontconfig's C implementation) that I don't even bother turning into PRs, as I know they have no chance of being reviewed and merged under the current system at least, because no one would care (see also the bus-factor point below). > It also doesn't help that some PR contain the same code (13117, 13128) and > are being hold off until the other one is merged (also, why is 13117 being > hold off for a PR that has been opened afterwards?). IMO, if a PR can be > merged, it should be merged. That's discussed in https://github.com/matplotlib/matplotlib/pull/13117#discussion_r245503693 (FTR I actually agreed on merging 13117 directly, but apparently that's an API break...) > But we shouldn't make too many exceptions to the two core contributors > rules. It's easier to have two core contributor review a pull request, than > to try to fix API issues on code that has been merged and released. Having > two pairs of eyes on a PR is also a good way to avoid having a bus factor > of 1 on some areas of the library. There are already huge and central chunks of the library (Agg internals, PDF/PS) that have a bus factor close to zero. If anything, I hope that this "drive-by review" system will encourage reviewers to start looking at places they are less familiar with ("oh yes, I guess this change makes sense" -- see Eric's reply above) and decrease that. And then to David's (the code-coverage point is already covered above): I think it is good remember that (as far as I know) everyone submitting PRs > and everyone reviewing them isn't paid and does so in their spare time; > therefore is no obligation for anyone to review anything on any particular > timescale. One thing that hasn't been discussed is how to increase the > reviewing capacity of the community, which I guess would involve working > out how to attract and retain more regular contributors. Given the premises "no one is paid to review PRs" and "the reviewing capacity is too small" (which I agree with), I think there are two reasonable solutions: - Make PRs easier to merge (this proposal), - Increasing to number of core devs (your proposal, but it's not clear to me whether this number can increase significantly in a short period of time...). Antony On Wed, Jan 16, 2019 at 10:34 AM David Stansby wrote: > I think it is good remember that (as far as I know) everyone submitting > PRs and everyone reviewing them isn't paid and does so in their spare time; > therefore is no obligation for anyone to review anything on any particular > timescale. One thing that hasn't been discussed is how to increase the > reviewing capacity of the community, which I guess would involve working > out how to attract and retain more regular contributors. > > re. the proposed changes, I'm +1 modulo objective criteria being agreed > upon. Personally I agree with Nelle's list, with the addition "has 100% > code coverage". > > David > > > > On Tue, 15 Jan 2019, 23:46 Nelle Varoquaux wrote: > >> For the way GitHub sorts pull requests, it is indeed a problem, but >> there's a way to sort by "most recently updated." I sometimes use it to >> make sure that I review recently updated pull request that don't appear on >> the first page. >> >> It seems to me that this "uncontroversial but uninteresting" rule is >> currently not based on objective criteria. I'm personally not in favor of >> removing two core contributors approval for anything that modifies the >> API. We have had too many close calls on PR almost merged that had >> backward incompatible changes: I realize that some core developers believe >> that we are not moving fast enough, on a project with hundreds of thousands >> of users, changes to the API should not be considered lightly. >> >> Here's an attempt at better defining an "uncontroversial but >> uninteresting" rule based on objective criteria: >> - Has not been commented on by more than one core developer; >> - Doesn't change the *public* API; >> - Doesn't add or remove a functionality; >> - Doesn't add or remove a dependency; >> - Is fully tested; >> - All parameters are fully documented; >> - Doesn't fix a release-critical bug; >> - Is less than XX lines of code. (Long PR are harder to review, and thus >> the likelihood of one core dev not noticing a problem is higher). Threshold >> needs to be defined. >> >> To be honest, it also doesn't help that some people have 50+ pull >> requests open (Antony? :p). It also doesn't help that some PR contain the >> same code (13117, 13128) and are being hold off until the other one is >> merged (also, why is 13117 being hold off for a PR that has been opened >> afterwards?). IMO, if a PR can be merged, it should be merged. But we >> shouldn't make too many exceptions to the two core contributors rules. It's >> easier to have two core contributor review a pull request, than to try to >> fix API issues on code that has been merged and released. Having two pairs >> of eyes on a PR is also a good way to avoid having a bus factor of 1 on >> some areas of the library. >> >> >> On Tue, 15 Jan 2019 at 15:19, Thomas Caswell wrote: >> >>> I think a distinction between "no one cares" and "no one has bandwidth" >>> needs to be made. >>> >>> If you look at the 'pulse' page, we merging 100+ PRs a month ( >>> https://github.com/matplotlib/matplotlib/pulse/monthly) so reviews are >>> happening, the issue is just that some are falling through the cracks. >>> >>> Tom >>> >>> On Tue, Jan 15, 2019 at 4:45 PM Antony Lee < >>> antony.lee at institutoptique.fr> wrote: >>> >>>> Eric, >>>> Neither the ttc font, nor the pdf/ps common code PRs actually touch any >>>> "complex" point regarding fonts or pdf/ps. I have described the ttc font >>>> in my previous message; the pdf/ps PR is really just, look, these two >>>> classes share 80 lines worth of code that's literally copy-pasted, let's >>>> just put that in a common parent class and inherit from it. >>>> Conversely, one font-related PR for which I actually have low hopes of >>>> getting properly reviewed (except on an "sure, I trust you" basis) is >>>> https://github.com/matplotlib/matplotlib/pull/12928, which actually >>>> touches a rather complex point about font encodings (some of which I had to >>>> clarify on the FreeType mailing list). (If anyone wants to have a look at >>>> it, though... :p) >>>> Antony >>>> >>>> On Tue, Jan 15, 2019 at 10:36 PM Eric Firing >>>> wrote: >>>> >>>>> Antony, >>>>> >>>>> Your examples illustrate that the problem is often that we don't have >>>>> enough people who understand some areas, like fonts and pdf/ps file >>>>> formats, to get 2 thorough and knowledgeable reviews in a reasonable >>>>> time. >>>>> >>>>> I'm fine with your proposed rule. It should help at least a little >>>>> bit. >>>>> >>>>> Eric >>>>> >>>>> >>>>> On 2019/01/15 9:11 AM, Antony Lee wrote: >>>>> > I'm obviously more aware of my own PRs, so here are a few I'd have >>>>> put >>>>> > up for single-review-merge (note that it's quite possible that >>>>> they'd >>>>> > have attracted more attention and gotten merged faster under that >>>>> > system; that's also the goal...). Obviously other devs may think >>>>> about >>>>> > other PRs that could have benefitted from the same process. >>>>> > >>>>> > - https://github.com/matplotlib/matplotlib/pull/9787 adds support >>>>> for >>>>> > the ttc font format (requested since 2014) for png/svg output >>>>> (that's >>>>> > provided basically for free by FreeType) but not for pdf/ps (that >>>>> would >>>>> > require more difficult changes). The entire PR comes down to: >>>>> > * adding "ttc" as an extension that should be treated like >>>>> "ttf"/"otf", >>>>> > * adding error handling to pdf/ps to signal these formats are not >>>>> > supported there, >>>>> > * tests. >>>>> > The PR sat open for 10 months before a first positive review >>>>> (because >>>>> > no one cares about fonts, I guess) and took another 3 months to >>>>> attract >>>>> > a second positive review. >>>>> > >>>>> > - https://github.com/matplotlib/matplotlib/pull/12472 does a fairly >>>>> > trivial fix to fontList.json to make it reusable for Matplotlibs >>>>> > installed in different venvs, got a positive review in one day and >>>>> > waited another 2.5 months for a second positive review. >>>>> > >>>>> > - https://github.com/matplotlib/matplotlib/pull/9867 deduplicates >>>>> > completely duplicated code between the pdf and ps backends, mostly >>>>> for >>>>> > maintainability's sake; it took 7.5 months (and 5 rebases due to >>>>> > conflicts) to get a first positive review and another 6 (and around >>>>> as >>>>> > many rebases) to get a second one. >>>>> > >>>>> > ----- >>>>> > >>>>> > As for the specifics (as I see it): >>>>> > - Someone opens a PR, it gets a positive review, no negative review, >>>>> and >>>>> > no activity occurs for two weeks (significant activity -- excluding >>>>> > trivial chat on the thread). >>>>> > - Sponsor (first reviewer, or author if a committer) *pings* all >>>>> devs by >>>>> > mentioning @matplotlib/developers on the PR thread with the intent >>>>> to >>>>> > single-review-merge, and labels the PR accordingly. >>>>> > - The PR can still be reviewed/merged/rejected by the normal review >>>>> > process. However, if no one explicitly opposes the merge within two >>>>> > weeks, anyone can merge it at that point (including by self-merge). >>>>> > >>>>> > I would not exclude all API changes from the process. For example, >>>>> > >>>>> https://github.com/matplotlib/matplotlib/pull/13173/commits/e90d264f3e5a263ef57243a2af86b46ab74ccc16 >>>>> > deprecates (and prepares for deletion) two parameters to imshow() >>>>> that >>>>> > have had no effect since 2006. Let's pretend for a second this >>>>> commit >>>>> > was a single independent PR (and not an example to showcase the >>>>> > parameter-deprecating decorator...); if it started being forgotten I >>>>> > would have proposed it for single-review-merge (again, note that if >>>>> > you're worried about the change, you don't need to reject the PR; >>>>> you >>>>> > can just say, I think this API change needs to be approved by a >>>>> second >>>>> > reviewer and that'll block the single-review-merge just as well). >>>>> > >>>>> > Antony >>>>> > >>>>> > On Tue, Jan 15, 2019 at 7:46 PM Nelle Varoquaux >>>>> > > >>>>> wrote: >>>>> > >>>>> > The proposed scheme by Paul doesn't seem reasonable to me. Core >>>>> > contributor A or committers needs to actively reach out to other >>>>> > core contributors: labeling is not enough IMO. >>>>> > >>>>> > >>>>> > On Tue, 15 Jan 2019 at 10:37, Jody Klymak >>>> > > wrote: >>>>> > >>>>> > >>>>> > >>>>> >> On 15 Jan 2019, at 10:20, Paul Hobson >>>> >> > wrote: >>>>> >> >>>>> >> I think this would be a good policy as well. Can I get some >>>>> >> clarification on the flow? The way I understand it: >>>>> >> >>>>> >> 1) Someone (committer, contributor, or first-timer) opens a >>>>> >> simple PR >>>>> >> 2) Committer A reviews it, adds a "single-review-merge" >>>>> label, >>>>> >> then digitally walks away >>>>> >> 3) No less than two weeks later, Committer B sees the PR, >>>>> >> notices the label, opens it up, and merges without >>>>> reviewing. >>>>> > >>>>> > If there is a Committer B, I think they should at least look >>>>> at >>>>> > the PR to make sure it doesn?t have any hidden API changes, >>>>> or >>>>> > introduce anything hostile. >>>>> > >>>>> > The point here is that getting the third person to look at a >>>>> PR >>>>> > is proving quite difficult, and if the rest of the folks with >>>>> > the commit bit haven?t looked at a PR for a month, even after >>>>> > being warned, then silence indicates consent. >>>>> > >>>>> > Note that only folks with the commit bit can label PRs, so >>>>> > anyone flagging a PR like this is implicitly trusted to not >>>>> be >>>>> > wreaking havoc, and not doing this for major changes. Since >>>>> we >>>>> > all curate our own PRs, I rather expect that the PR >>>>> contributor >>>>> > will often be the person who adds the flag. Whether we want >>>>> to >>>>> > allow a self-merge at that point is up for debate, but if >>>>> not, >>>>> > then folks need to get in the habit of looking at old flagged >>>>> > PRs and merging them. >>>>> > >>>>> > I think a huge problem we have is the LIFO default sort of >>>>> the >>>>> > github PR queue, and that any PR not on the first page might >>>>> as >>>>> > well be closed for all the attention it will get without >>>>> > constant nagging. >>>>> > >>>>> > Cheers, Jody >>>>> > >>>>> >> Does that capture a successful "single-review-merge" >>>>> lifecycle? >>>>> >> -paul >>>>> >> >>>>> >> On Tue, Jan 15, 2019 at 9:57 AM Nelle Varoquaux >>>>> >> >>>> nelle.varoquaux at gmail.com>> >>>>> >> wrote: >>>>> >> >>>>> >> Hello, >>>>> >> >>>>> >> Overall, I think this is a good idea, but I would like >>>>> >> specifics on what "uncontroversial but uninteresting" PR >>>>> >> mean. IMO, that should exclude any PR with API changes. >>>>> >> >>>>> >> Cheers, >>>>> >> N >>>>> >> >>>>> >> On Tue, 15 Jan 2019 at 06:00, Antony Lee >>>>> >> > >>>>> wrote: >>>>> >> >>>>> >> Hi all, >>>>> >> >>>>> >> During the weekly dev call, I proposed to amend the >>>>> PR >>>>> >> merge rules, following an initial comment on Github >>>>> >> [ >>>>> https://github.com/matplotlib/matplotlib/pull/13173#issuecomment-453921220]. >>>>> >>>>> >> There was general agreement among the devs present >>>>> >> (Tom, Hannah, Jody, and myself), so I'm putting it >>>>> >> here for discussion. The objective of this change >>>>> is >>>>> >> to prevent "uncontroversial but uninteresting" PRs >>>>> >> from falling into oblivion, and try to decrease the >>>>> >> size of the open PR stack. >>>>> >> >>>>> >> The current rule is that a PR needs positive reviews >>>>> >> (from two committers, and excluding the author if a >>>>> >> committer) to be merged, except that doc-only PRs >>>>> >> (docstrings, rst) only needs a single positive >>>>> review. >>>>> >> >>>>> >> I am proposing that, if a (non-doc) PR already has >>>>> one >>>>> >> positive review, but no activity on that PR has >>>>> >> occurred for two weeks (exact time interval up to >>>>> >> bikeshedding) [Jody suggests: and the PR has 100% >>>>> code >>>>> >> coverage], then a committer (either the first >>>>> >> reviewer, or the author if a committer) can suggest >>>>> >> that it be merged on the basis of that single >>>>> review. >>>>> >> To do so, the "sponsor" should ping all developers >>>>> >> (@matplotlib/developers) on that issue indicating >>>>> that >>>>> >> intent, and add a "single-review-merge" label on the >>>>> >> PR (so that these PRs can easily be found). >>>>> >> Committers are encouraged to review the PR to accept >>>>> >> and merge it or reject it or request changes on it; >>>>> >> but they can also just indicate that they consider >>>>> the >>>>> >> PR sufficiently complex that a proper second review >>>>> is >>>>> >> needed before merging, or request an extension, >>>>> etc. >>>>> >> To do so they should still leave a "reject" review, >>>>> >> even if just saying "objecting to single-review >>>>> >> merged; anyone can dismiss after a second review". >>>>> >> However, if within another two weeks, no committer >>>>> >> voiced any objection (explicitly, i.e. by >>>>> rejecting), >>>>> >> then the PR can indeed be merged on the basis of >>>>> that >>>>> >> single review. >>>>> >> >>>>> >> To avoid overwhelming the system, any committer can >>>>> >> only "sponsor" a single PR at a time. >>>>> >> >>>>> >> Thoughts? >>>>> >> >>>>> >> Antony >>>>> >> _______________________________________________ >>>>> >> Matplotlib-devel mailing list >>>>> >> Matplotlib-devel at python.org >>>>> >> >>>>> >> >>>>> https://mail.python.org/mailman/listinfo/matplotlib-devel >>>>> >> >>>>> >> _______________________________________________ >>>>> >> Matplotlib-devel mailing list >>>>> >> Matplotlib-devel at python.org >>>>> >> >>>>> >> >>>>> https://mail.python.org/mailman/listinfo/matplotlib-devel >>>>> >> >>>>> >> _______________________________________________ >>>>> >> Matplotlib-devel mailing list >>>>> >> Matplotlib-devel at python.org >>>> Matplotlib-devel at python.org> >>>>> >> https://mail.python.org/mailman/listinfo/matplotlib-devel >>>>> > >>>>> > -- >>>>> > Jody Klymak >>>>> > http://web.uvic.ca/~jklymak/ >>>>> > >>>>> > >>>>> > >>>>> > >>>>> > >>>>> > _______________________________________________ >>>>> > Matplotlib-devel mailing list >>>>> > Matplotlib-devel at python.org >>>>> > https://mail.python.org/mailman/listinfo/matplotlib-devel >>>>> > >>>>> > >>>>> > _______________________________________________ >>>>> > Matplotlib-devel mailing list >>>>> > Matplotlib-devel at python.org >>>>> > https://mail.python.org/mailman/listinfo/matplotlib-devel >>>>> > >>>>> >>>>> _______________________________________________ >>>>> Matplotlib-devel mailing list >>>>> Matplotlib-devel at python.org >>>>> https://mail.python.org/mailman/listinfo/matplotlib-devel >>>>> >>>> _______________________________________________ >>>> Matplotlib-devel mailing list >>>> Matplotlib-devel at python.org >>>> https://mail.python.org/mailman/listinfo/matplotlib-devel >>>> >>> >>> >>> -- >>> Thomas Caswell >>> tcaswell at gmail.com >>> _______________________________________________ >>> Matplotlib-devel mailing list >>> Matplotlib-devel at python.org >>> https://mail.python.org/mailman/listinfo/matplotlib-devel >>> >> _______________________________________________ >> Matplotlib-devel mailing list >> Matplotlib-devel at python.org >> https://mail.python.org/mailman/listinfo/matplotlib-devel >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From tcaswell at gmail.com Wed Jan 16 09:32:24 2019 From: tcaswell at gmail.com (Thomas Caswell) Date: Wed, 16 Jan 2019 09:32:24 -0500 Subject: [Matplotlib-devel] Proposal to amend the PR merge rules In-Reply-To: References: Message-ID: I want to push back strongly on the notion that "did not review --> does not care". As as already been discussed above there are many other reasons someone does not comment (for example "I _do_ care about this, but I {do not have time, am too tired and cranky, ...} to properly review this now"). It seems to me that there are two classes of PRs that are getting grouped together here: small PRs that just fall through the cracks and big PRs that are hard to review, let's stay focused on the small PRs. One habit related thing that may help is if we all bookmark: https://github.com/matplotlib/matplotlib/pulls?q=is%3Apr+is%3Aopen+review%3Aapproved+sort%3Aupdated-desc as where to start when we sit down to do some Matplotlib work. I would also remind everyone that if you have a commit bit, then we trust you to use it! As we write down the list of things that need to be true for single-review merging, it seems the effort to determine that a given PR checks (or does not) all of the boxes is equal to the effort to just review the PR. This make me think that the biggest benefit of this change would be to get more eyes on PRs than any policy change. To a fair degree this comes down to a project management / resource allocation problem (which are hard even in situations where everyone is paid!) Tom On Wed, Jan 16, 2019 at 6:11 AM Antony Lee wrote: > Replying to Nelle's points first: > > >> It seems to me that this "uncontroversial but uninteresting" rule is >> currently not based on objective criteria. I'm personally not in favor of >> removing two core contributors approval for anything that modifies the >> API. We have had too many close calls on PR almost merged that had >> backward incompatible changes: I realize that some core developers believe >> that we are not moving fast enough, on a project with hundreds of thousands >> of users, changes to the API should not be considered lightly. >> > > To be clear (and even though it is probably common knowledge that I would > like to evolve the API faster), there is no "secret agenda" of using this > to sneak API breaks through the system. > > >> Here's an attempt at better defining an "uncontroversial but >> uninteresting" rule based on objective criteria: >> - Has not been commented on by more than one core developer; >> > > Sounds reasonable (with a reasonably flexible interpretation -- similarly > to the "no activity since two weeks, excluding trivial chat"), basically > "no other core dev expressed interest in reviewing it". > > >> - Doesn't change the public API; >> - Doesn't add or remove a functionality; >> - Doesn't add or remove a dependency; >> > > I expect all changes of API to go through the normal route of deprecation > and API change note, which even the first reviewer should make sure are > present before sponsoring the PR for single-review-merge (as an aside, I > guess the first reviewer could even say, "approving, but I don't want to > take the responsibility for being the single reviewer"...). So now you > have a two week period where any other core dev passing by can just see > whether there is an API change note and make their own judgment, including > deciding to block the PR if necessary, based on that change note. > I guess it is reasonable to say "API deprecations or changes and new > functionality should be clearly documented" to help that "skim-through" > review. > As for *adding* functionality, I guess you would have rejected the > ttc-font PR on that premise? (Actually I now see that the additional > functionality was actually not documented :)) Note that this is not some > obscure edge case that no one cares about; it's a feature that has been > requested for years on the tracker. > > - Is fully tested; >> > > Also proposed by Jody and now by David, sounds good to me. > > >> - All parameters are fully documented; >> > > (sounds contradictory with "doesn't add a functionality"... if there is no > new functionality it's unlikely there's a new parameter) > > >> - Doesn't fix a release-critical bug; >> > > These tend to all be merged just before the next release is cut anyways > (review dynamics are a bit different in these periods), so I wouldn't worry > about it, but sure. > > >> - Is less than XX lines of code. (Long PR are harder to review, and thus >> the likelihood of one core dev not noticing a problem is higher). Threshold >> needs to be defined. >> > > I don't actually think this is a good rule; the pdf/ps merge PR is +138 > lines, -182 lines but as explained above the change is fairly trivial; > obviously I could have split it into smaller chunks but if anything that > would have made it harder to review. > Conversely (to use an example I have already raised elsewhere) > https://github.com/matplotlib/matplotlib/pull/11530 (making sure that > LocationEvent.x and LocationEvent.y are integers (as they refer to pixel > locations) rather than floats) is 2 lines of changes in the codebase and 20 > lines of test, was reviewed by 2.5 core devs (I left a comment in passing) > and we all missed that there was "problem" with it (it broke mplcursors and > some other pieces of code I have that assume that it is possible to > synthetize LocationEvents pointing to arbitrary places in the canvas). > (Not to derail this discussion into how to evaluate API breaks once > again... but I think these kinds of changes are much more insidious than, > say, removing a function, as the latter would just trigger an > AttributeError that is easily noticed and at worst you just go back to the > previous Matplotlib release, whereas the LocationEvent change was only > caught because it was covered by the mplcursors test suite; otherwise I'd > have been silently generating wrong results due to it...) > > >> To be honest, it also doesn't help that some people have 50+ pull >> requests open (Antony? :p). > > > I guess if the way to fix Matplotlib is to not contribute to Matplotlib > anymore, we have a bigger problem. In fact, I have some pretty big patches > fixing long-standing issues (e.g. for #8550 I have a pure-Python > reimplementation of the fontconfig font weight deduction algorithm, > translated from fontconfig's C implementation) that I don't even bother > turning into PRs, as I know they have no chance of being reviewed and > merged under the current system at least, because no one would care (see > also the bus-factor point below). > > >> It also doesn't help that some PR contain the same code (13117, 13128) >> and are being hold off until the other one is merged (also, why is 13117 >> being hold off for a PR that has been opened afterwards?). IMO, if a PR can >> be merged, it should be merged. > > > That's discussed in > https://github.com/matplotlib/matplotlib/pull/13117#discussion_r245503693 > (FTR I actually agreed on merging 13117 directly, but apparently that's an > API break...) > > >> But we shouldn't make too many exceptions to the two core contributors >> rules. It's easier to have two core contributor review a pull request, than >> to try to fix API issues on code that has been merged and released. Having >> two pairs of eyes on a PR is also a good way to avoid having a bus factor >> of 1 on some areas of the library. > > > There are already huge and central chunks of the library (Agg internals, > PDF/PS) that have a bus factor close to zero. If anything, I hope that > this "drive-by review" system will encourage reviewers to start looking at > places they are less familiar with ("oh yes, I guess this change makes > sense" -- see Eric's reply above) and decrease that. > > And then to David's (the code-coverage point is already covered above): > > I think it is good remember that (as far as I know) everyone submitting >> PRs and everyone reviewing them isn't paid and does so in their spare time; >> therefore is no obligation for anyone to review anything on any particular >> timescale. One thing that hasn't been discussed is how to increase the >> reviewing capacity of the community, which I guess would involve working >> out how to attract and retain more regular contributors. > > > Given the premises "no one is paid to review PRs" and "the reviewing > capacity is too small" (which I agree with), I think there are two > reasonable solutions: > - Make PRs easier to merge (this proposal), > - Increasing to number of core devs (your proposal, but it's not clear to > me whether this number can increase significantly in a short period of > time...). > > Antony > > On Wed, Jan 16, 2019 at 10:34 AM David Stansby wrote: > >> I think it is good remember that (as far as I know) everyone submitting >> PRs and everyone reviewing them isn't paid and does so in their spare time; >> therefore is no obligation for anyone to review anything on any particular >> timescale. One thing that hasn't been discussed is how to increase the >> reviewing capacity of the community, which I guess would involve working >> out how to attract and retain more regular contributors. >> >> re. the proposed changes, I'm +1 modulo objective criteria being agreed >> upon. Personally I agree with Nelle's list, with the addition "has 100% >> code coverage". >> >> David >> >> >> >> On Tue, 15 Jan 2019, 23:46 Nelle Varoquaux > wrote: >> >>> For the way GitHub sorts pull requests, it is indeed a problem, but >>> there's a way to sort by "most recently updated." I sometimes use it to >>> make sure that I review recently updated pull request that don't appear on >>> the first page. >>> >>> It seems to me that this "uncontroversial but uninteresting" rule is >>> currently not based on objective criteria. I'm personally not in favor of >>> removing two core contributors approval for anything that modifies the >>> API. We have had too many close calls on PR almost merged that had >>> backward incompatible changes: I realize that some core developers believe >>> that we are not moving fast enough, on a project with hundreds of thousands >>> of users, changes to the API should not be considered lightly. >>> >>> Here's an attempt at better defining an "uncontroversial but >>> uninteresting" rule based on objective criteria: >>> - Has not been commented on by more than one core developer; >>> - Doesn't change the *public* API; >>> - Doesn't add or remove a functionality; >>> - Doesn't add or remove a dependency; >>> - Is fully tested; >>> - All parameters are fully documented; >>> - Doesn't fix a release-critical bug; >>> - Is less than XX lines of code. (Long PR are harder to review, and thus >>> the likelihood of one core dev not noticing a problem is higher). Threshold >>> needs to be defined. >>> >>> To be honest, it also doesn't help that some people have 50+ pull >>> requests open (Antony? :p). It also doesn't help that some PR contain the >>> same code (13117, 13128) and are being hold off until the other one is >>> merged (also, why is 13117 being hold off for a PR that has been opened >>> afterwards?). IMO, if a PR can be merged, it should be merged. But we >>> shouldn't make too many exceptions to the two core contributors rules. It's >>> easier to have two core contributor review a pull request, than to try to >>> fix API issues on code that has been merged and released. Having two pairs >>> of eyes on a PR is also a good way to avoid having a bus factor of 1 on >>> some areas of the library. >>> >>> >>> On Tue, 15 Jan 2019 at 15:19, Thomas Caswell wrote: >>> >>>> I think a distinction between "no one cares" and "no one has bandwidth" >>>> needs to be made. >>>> >>>> If you look at the 'pulse' page, we merging 100+ PRs a month ( >>>> https://github.com/matplotlib/matplotlib/pulse/monthly) so reviews are >>>> happening, the issue is just that some are falling through the cracks. >>>> >>>> Tom >>>> >>>> On Tue, Jan 15, 2019 at 4:45 PM Antony Lee < >>>> antony.lee at institutoptique.fr> wrote: >>>> >>>>> Eric, >>>>> Neither the ttc font, nor the pdf/ps common code PRs actually touch >>>>> any "complex" point regarding fonts or pdf/ps. I have described the ttc >>>>> font in my previous message; the pdf/ps PR is really just, look, these two >>>>> classes share 80 lines worth of code that's literally copy-pasted, let's >>>>> just put that in a common parent class and inherit from it. >>>>> Conversely, one font-related PR for which I actually have low hopes of >>>>> getting properly reviewed (except on an "sure, I trust you" basis) is >>>>> https://github.com/matplotlib/matplotlib/pull/12928, which actually >>>>> touches a rather complex point about font encodings (some of which I had to >>>>> clarify on the FreeType mailing list). (If anyone wants to have a look at >>>>> it, though... :p) >>>>> Antony >>>>> >>>>> On Tue, Jan 15, 2019 at 10:36 PM Eric Firing >>>>> wrote: >>>>> >>>>>> Antony, >>>>>> >>>>>> Your examples illustrate that the problem is often that we don't have >>>>>> enough people who understand some areas, like fonts and pdf/ps file >>>>>> formats, to get 2 thorough and knowledgeable reviews in a reasonable >>>>>> time. >>>>>> >>>>>> I'm fine with your proposed rule. It should help at least a little >>>>>> bit. >>>>>> >>>>>> Eric >>>>>> >>>>>> >>>>>> On 2019/01/15 9:11 AM, Antony Lee wrote: >>>>>> > I'm obviously more aware of my own PRs, so here are a few I'd have >>>>>> put >>>>>> > up for single-review-merge (note that it's quite possible that >>>>>> they'd >>>>>> > have attracted more attention and gotten merged faster under that >>>>>> > system; that's also the goal...). Obviously other devs may think >>>>>> about >>>>>> > other PRs that could have benefitted from the same process. >>>>>> > >>>>>> > - https://github.com/matplotlib/matplotlib/pull/9787 adds support >>>>>> for >>>>>> > the ttc font format (requested since 2014) for png/svg output >>>>>> (that's >>>>>> > provided basically for free by FreeType) but not for pdf/ps (that >>>>>> would >>>>>> > require more difficult changes). The entire PR comes down to: >>>>>> > * adding "ttc" as an extension that should be treated like >>>>>> "ttf"/"otf", >>>>>> > * adding error handling to pdf/ps to signal these formats are >>>>>> not >>>>>> > supported there, >>>>>> > * tests. >>>>>> > The PR sat open for 10 months before a first positive review >>>>>> (because >>>>>> > no one cares about fonts, I guess) and took another 3 months to >>>>>> attract >>>>>> > a second positive review. >>>>>> > >>>>>> > - https://github.com/matplotlib/matplotlib/pull/12472 does a >>>>>> fairly >>>>>> > trivial fix to fontList.json to make it reusable for Matplotlibs >>>>>> > installed in different venvs, got a positive review in one day and >>>>>> > waited another 2.5 months for a second positive review. >>>>>> > >>>>>> > - https://github.com/matplotlib/matplotlib/pull/9867 deduplicates >>>>>> > completely duplicated code between the pdf and ps backends, mostly >>>>>> for >>>>>> > maintainability's sake; it took 7.5 months (and 5 rebases due to >>>>>> > conflicts) to get a first positive review and another 6 (and around >>>>>> as >>>>>> > many rebases) to get a second one. >>>>>> > >>>>>> > ----- >>>>>> > >>>>>> > As for the specifics (as I see it): >>>>>> > - Someone opens a PR, it gets a positive review, no negative >>>>>> review, and >>>>>> > no activity occurs for two weeks (significant activity -- excluding >>>>>> > trivial chat on the thread). >>>>>> > - Sponsor (first reviewer, or author if a committer) *pings* all >>>>>> devs by >>>>>> > mentioning @matplotlib/developers on the PR thread with the intent >>>>>> to >>>>>> > single-review-merge, and labels the PR accordingly. >>>>>> > - The PR can still be reviewed/merged/rejected by the normal review >>>>>> > process. However, if no one explicitly opposes the merge within >>>>>> two >>>>>> > weeks, anyone can merge it at that point (including by self-merge). >>>>>> > >>>>>> > I would not exclude all API changes from the process. For example, >>>>>> > >>>>>> https://github.com/matplotlib/matplotlib/pull/13173/commits/e90d264f3e5a263ef57243a2af86b46ab74ccc16 >>>>>> > deprecates (and prepares for deletion) two parameters to imshow() >>>>>> that >>>>>> > have had no effect since 2006. Let's pretend for a second this >>>>>> commit >>>>>> > was a single independent PR (and not an example to showcase the >>>>>> > parameter-deprecating decorator...); if it started being forgotten >>>>>> I >>>>>> > would have proposed it for single-review-merge (again, note that if >>>>>> > you're worried about the change, you don't need to reject the PR; >>>>>> you >>>>>> > can just say, I think this API change needs to be approved by a >>>>>> second >>>>>> > reviewer and that'll block the single-review-merge just as well). >>>>>> > >>>>>> > Antony >>>>>> > >>>>>> > On Tue, Jan 15, 2019 at 7:46 PM Nelle Varoquaux >>>>>> > > >>>>>> wrote: >>>>>> > >>>>>> > The proposed scheme by Paul doesn't seem reasonable to me. Core >>>>>> > contributor A or committers needs to actively reach out to other >>>>>> > core contributors: labeling is not enough IMO. >>>>>> > >>>>>> > >>>>>> > On Tue, 15 Jan 2019 at 10:37, Jody Klymak >>>>> > > wrote: >>>>>> > >>>>>> > >>>>>> > >>>>>> >> On 15 Jan 2019, at 10:20, Paul Hobson >>>>> >> > wrote: >>>>>> >> >>>>>> >> I think this would be a good policy as well. Can I get some >>>>>> >> clarification on the flow? The way I understand it: >>>>>> >> >>>>>> >> 1) Someone (committer, contributor, or first-timer) opens a >>>>>> >> simple PR >>>>>> >> 2) Committer A reviews it, adds a "single-review-merge" >>>>>> label, >>>>>> >> then digitally walks away >>>>>> >> 3) No less than two weeks later, Committer B sees the PR, >>>>>> >> notices the label, opens it up, and merges without >>>>>> reviewing. >>>>>> > >>>>>> > If there is a Committer B, I think they should at least >>>>>> look at >>>>>> > the PR to make sure it doesn?t have any hidden API changes, >>>>>> or >>>>>> > introduce anything hostile. >>>>>> > >>>>>> > The point here is that getting the third person to look at >>>>>> a PR >>>>>> > is proving quite difficult, and if the rest of the folks >>>>>> with >>>>>> > the commit bit haven?t looked at a PR for a month, even >>>>>> after >>>>>> > being warned, then silence indicates consent. >>>>>> > >>>>>> > Note that only folks with the commit bit can label PRs, so >>>>>> > anyone flagging a PR like this is implicitly trusted to not >>>>>> be >>>>>> > wreaking havoc, and not doing this for major changes. >>>>>> Since we >>>>>> > all curate our own PRs, I rather expect that the PR >>>>>> contributor >>>>>> > will often be the person who adds the flag. Whether we >>>>>> want to >>>>>> > allow a self-merge at that point is up for debate, but if >>>>>> not, >>>>>> > then folks need to get in the habit of looking at old >>>>>> flagged >>>>>> > PRs and merging them. >>>>>> > >>>>>> > I think a huge problem we have is the LIFO default sort of >>>>>> the >>>>>> > github PR queue, and that any PR not on the first page >>>>>> might as >>>>>> > well be closed for all the attention it will get without >>>>>> > constant nagging. >>>>>> > >>>>>> > Cheers, Jody >>>>>> > >>>>>> >> Does that capture a successful "single-review-merge" >>>>>> lifecycle? >>>>>> >> -paul >>>>>> >> >>>>>> >> On Tue, Jan 15, 2019 at 9:57 AM Nelle Varoquaux >>>>>> >> >>>>> nelle.varoquaux at gmail.com>> >>>>>> >> wrote: >>>>>> >> >>>>>> >> Hello, >>>>>> >> >>>>>> >> Overall, I think this is a good idea, but I would like >>>>>> >> specifics on what "uncontroversial but uninteresting" >>>>>> PR >>>>>> >> mean. IMO, that should exclude any PR with API changes. >>>>>> >> >>>>>> >> Cheers, >>>>>> >> N >>>>>> >> >>>>>> >> On Tue, 15 Jan 2019 at 06:00, Antony Lee >>>>>> >> > >>>>>> wrote: >>>>>> >> >>>>>> >> Hi all, >>>>>> >> >>>>>> >> During the weekly dev call, I proposed to amend >>>>>> the PR >>>>>> >> merge rules, following an initial comment on Github >>>>>> >> [ >>>>>> https://github.com/matplotlib/matplotlib/pull/13173#issuecomment-453921220]. >>>>>> >>>>>> >> There was general agreement among the devs present >>>>>> >> (Tom, Hannah, Jody, and myself), so I'm putting it >>>>>> >> here for discussion. The objective of this change >>>>>> is >>>>>> >> to prevent "uncontroversial but uninteresting" PRs >>>>>> >> from falling into oblivion, and try to decrease the >>>>>> >> size of the open PR stack. >>>>>> >> >>>>>> >> The current rule is that a PR needs positive >>>>>> reviews >>>>>> >> (from two committers, and excluding the author if a >>>>>> >> committer) to be merged, except that doc-only PRs >>>>>> >> (docstrings, rst) only needs a single positive >>>>>> review. >>>>>> >> >>>>>> >> I am proposing that, if a (non-doc) PR already has >>>>>> one >>>>>> >> positive review, but no activity on that PR has >>>>>> >> occurred for two weeks (exact time interval up to >>>>>> >> bikeshedding) [Jody suggests: and the PR has 100% >>>>>> code >>>>>> >> coverage], then a committer (either the first >>>>>> >> reviewer, or the author if a committer) can suggest >>>>>> >> that it be merged on the basis of that single >>>>>> review. >>>>>> >> To do so, the "sponsor" should ping all developers >>>>>> >> (@matplotlib/developers) on that issue indicating >>>>>> that >>>>>> >> intent, and add a "single-review-merge" label on >>>>>> the >>>>>> >> PR (so that these PRs can easily be found). >>>>>> >> Committers are encouraged to review the PR to >>>>>> accept >>>>>> >> and merge it or reject it or request changes on it; >>>>>> >> but they can also just indicate that they consider >>>>>> the >>>>>> >> PR sufficiently complex that a proper second >>>>>> review is >>>>>> >> needed before merging, or request an extension, >>>>>> etc. >>>>>> >> To do so they should still leave a "reject" review, >>>>>> >> even if just saying "objecting to single-review >>>>>> >> merged; anyone can dismiss after a second review". >>>>>> >> However, if within another two weeks, no committer >>>>>> >> voiced any objection (explicitly, i.e. by >>>>>> rejecting), >>>>>> >> then the PR can indeed be merged on the basis of >>>>>> that >>>>>> >> single review. >>>>>> >> >>>>>> >> To avoid overwhelming the system, any committer can >>>>>> >> only "sponsor" a single PR at a time. >>>>>> >> >>>>>> >> Thoughts? >>>>>> >> >>>>>> >> Antony >>>>>> >> _______________________________________________ >>>>>> >> Matplotlib-devel mailing list >>>>>> >> Matplotlib-devel at python.org >>>>>> >> >>>>>> >> >>>>>> https://mail.python.org/mailman/listinfo/matplotlib-devel >>>>>> >> >>>>>> >> _______________________________________________ >>>>>> >> Matplotlib-devel mailing list >>>>>> >> Matplotlib-devel at python.org >>>>>> >> >>>>>> >> >>>>>> https://mail.python.org/mailman/listinfo/matplotlib-devel >>>>>> >> >>>>>> >> _______________________________________________ >>>>>> >> Matplotlib-devel mailing list >>>>>> >> Matplotlib-devel at python.org >>>>> Matplotlib-devel at python.org> >>>>>> >> https://mail.python.org/mailman/listinfo/matplotlib-devel >>>>>> > >>>>>> > -- >>>>>> > Jody Klymak >>>>>> > http://web.uvic.ca/~jklymak/ >>>>>> > >>>>>> > >>>>>> > >>>>>> > >>>>>> > >>>>>> > _______________________________________________ >>>>>> > Matplotlib-devel mailing list >>>>>> > Matplotlib-devel at python.org >>>>> > >>>>>> > https://mail.python.org/mailman/listinfo/matplotlib-devel >>>>>> > >>>>>> > >>>>>> > _______________________________________________ >>>>>> > Matplotlib-devel mailing list >>>>>> > Matplotlib-devel at python.org >>>>>> > https://mail.python.org/mailman/listinfo/matplotlib-devel >>>>>> > >>>>>> >>>>>> _______________________________________________ >>>>>> Matplotlib-devel mailing list >>>>>> Matplotlib-devel at python.org >>>>>> https://mail.python.org/mailman/listinfo/matplotlib-devel >>>>>> >>>>> _______________________________________________ >>>>> Matplotlib-devel mailing list >>>>> Matplotlib-devel at python.org >>>>> https://mail.python.org/mailman/listinfo/matplotlib-devel >>>>> >>>> >>>> >>>> -- >>>> Thomas Caswell >>>> tcaswell at gmail.com >>>> _______________________________________________ >>>> Matplotlib-devel mailing list >>>> Matplotlib-devel at python.org >>>> https://mail.python.org/mailman/listinfo/matplotlib-devel >>>> >>> _______________________________________________ >>> Matplotlib-devel mailing list >>> Matplotlib-devel at python.org >>> https://mail.python.org/mailman/listinfo/matplotlib-devel >>> >> -- Thomas Caswell tcaswell at gmail.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From jklymak at uvic.ca Wed Jan 16 11:08:40 2019 From: jklymak at uvic.ca (Jody Klymak) Date: Wed, 16 Jan 2019 08:08:40 -0800 Subject: [Matplotlib-devel] Proposal to amend the PR merge rules In-Reply-To: References: Message-ID: <24D956B0-E974-4F82-B074-BECE828F22AB@uvic.ca> There is a mathematical necessity here; if you have the commit bit and submit 50 PRs a month, you need to review 100 PRs a month, and merge or close 50 of those PRs a month. If all the folks with commit bit do this, then obviously PRs will pile up. Cheers, Jody > On Jan 16, 2019, at 6:32 AM, Thomas Caswell wrote: > > I want to push back strongly on the notion that "did not review --> does not care". As as already been discussed above there are many other reasons someone does not comment (for example "I _do_ care about this, but I {do not have time, am too tired and cranky, ...} to properly review this now"). > > It seems to me that there are two classes of PRs that are getting grouped together here: small PRs that just fall through the cracks and big PRs that are hard to review, let's stay focused on the small PRs. > > One habit related thing that may help is if we all bookmark: > > https://github.com/matplotlib/matplotlib/pulls?q=is%3Apr+is%3Aopen+review%3Aapproved+sort%3Aupdated-desc > > as where to start when we sit down to do some Matplotlib work. I would also remind everyone that if you have a commit bit, then we trust you to use it! > > As we write down the list of things that need to be true for single-review merging, it seems the effort to determine that a given PR checks (or does not) all of the boxes is equal to the effort to just review the PR. This make me think that the biggest benefit of this change would be to get more eyes on PRs than any policy change. > > To a fair degree this comes down to a project management / resource allocation problem (which are hard even in situations where everyone is paid!) > > Tom > > On Wed, Jan 16, 2019 at 6:11 AM Antony Lee > wrote: > Replying to Nelle's points first: > > It seems to me that this "uncontroversial but uninteresting" rule is currently not based on objective criteria. I'm personally not in favor of removing two core contributors approval for anything that modifies the API. We have had too many close calls on PR almost merged that had backward incompatible changes: I realize that some core developers believe that we are not moving fast enough, on a project with hundreds of thousands of users, changes to the API should not be considered lightly. > > To be clear (and even though it is probably common knowledge that I would like to evolve the API faster), there is no "secret agenda" of using this to sneak API breaks through the system. > > Here's an attempt at better defining an "uncontroversial but uninteresting" rule based on objective criteria: > - Has not been commented on by more than one core developer; > > Sounds reasonable (with a reasonably flexible interpretation -- similarly to the "no activity since two weeks, excluding trivial chat"), basically "no other core dev expressed interest in reviewing it". > > - Doesn't change the public API; > - Doesn't add or remove a functionality; > - Doesn't add or remove a dependency; > > I expect all changes of API to go through the normal route of deprecation and API change note, which even the first reviewer should make sure are present before sponsoring the PR for single-review-merge (as an aside, I guess the first reviewer could even say, "approving, but I don't want to take the responsibility for being the single reviewer"...). So now you have a two week period where any other core dev passing by can just see whether there is an API change note and make their own judgment, including deciding to block the PR if necessary, based on that change note. > I guess it is reasonable to say "API deprecations or changes and new functionality should be clearly documented" to help that "skim-through" review. > As for *adding* functionality, I guess you would have rejected the ttc-font PR on that premise? (Actually I now see that the additional functionality was actually not documented :)) Note that this is not some obscure edge case that no one cares about; it's a feature that has been requested for years on the tracker. > > - Is fully tested; > > Also proposed by Jody and now by David, sounds good to me. > > - All parameters are fully documented; > > (sounds contradictory with "doesn't add a functionality"... if there is no new functionality it's unlikely there's a new parameter) > > - Doesn't fix a release-critical bug; > > These tend to all be merged just before the next release is cut anyways (review dynamics are a bit different in these periods), so I wouldn't worry about it, but sure. > > - Is less than XX lines of code. (Long PR are harder to review, and thus the likelihood of one core dev not noticing a problem is higher). Threshold needs to be defined. > > I don't actually think this is a good rule; the pdf/ps merge PR is +138 lines, -182 lines but as explained above the change is fairly trivial; obviously I could have split it into smaller chunks but if anything that would have made it harder to review. > Conversely (to use an example I have already raised elsewhere) https://github.com/matplotlib/matplotlib/pull/11530 (making sure that LocationEvent.x and LocationEvent.y are integers (as they refer to pixel locations) rather than floats) is 2 lines of changes in the codebase and 20 lines of test, was reviewed by 2.5 core devs (I left a comment in passing) and we all missed that there was "problem" with it (it broke mplcursors and some other pieces of code I have that assume that it is possible to synthetize LocationEvents pointing to arbitrary places in the canvas). (Not to derail this discussion into how to evaluate API breaks once again... but I think these kinds of changes are much more insidious than, say, removing a function, as the latter would just trigger an AttributeError that is easily noticed and at worst you just go back to the previous Matplotlib release, whereas the LocationEvent change was only caught because it was covered by the mplcursors test suite; otherwise I'd have been silently generating wrong results due to it...) > > To be honest, it also doesn't help that some people have 50+ pull requests open (Antony? :p). > > I guess if the way to fix Matplotlib is to not contribute to Matplotlib anymore, we have a bigger problem. In fact, I have some pretty big patches fixing long-standing issues (e.g. for #8550 I have a pure-Python reimplementation of the fontconfig font weight deduction algorithm, translated from fontconfig's C implementation) that I don't even bother turning into PRs, as I know they have no chance of being reviewed and merged under the current system at least, because no one would care (see also the bus-factor point below). > > It also doesn't help that some PR contain the same code (13117, 13128) and are being hold off until the other one is merged (also, why is 13117 being hold off for a PR that has been opened afterwards?). IMO, if a PR can be merged, it should be merged. > > That's discussed in https://github.com/matplotlib/matplotlib/pull/13117#discussion_r245503693 (FTR I actually agreed on merging 13117 directly, but apparently that's an API break...) > > But we shouldn't make too many exceptions to the two core contributors rules. It's easier to have two core contributor review a pull request, than to try to fix API issues on code that has been merged and released. Having two pairs of eyes on a PR is also a good way to avoid having a bus factor of 1 on some areas of the library. > > There are already huge and central chunks of the library (Agg internals, PDF/PS) that have a bus factor close to zero. If anything, I hope that this "drive-by review" system will encourage reviewers to start looking at places they are less familiar with ("oh yes, I guess this change makes sense" -- see Eric's reply above) and decrease that. > > And then to David's (the code-coverage point is already covered above): > > I think it is good remember that (as far as I know) everyone submitting PRs and everyone reviewing them isn't paid and does so in their spare time; therefore is no obligation for anyone to review anything on any particular timescale. One thing that hasn't been discussed is how to increase the reviewing capacity of the community, which I guess would involve working out how to attract and retain more regular contributors. > > Given the premises "no one is paid to review PRs" and "the reviewing capacity is too small" (which I agree with), I think there are two reasonable solutions: > - Make PRs easier to merge (this proposal), > - Increasing to number of core devs (your proposal, but it's not clear to me whether this number can increase significantly in a short period of time...). > > Antony > > On Wed, Jan 16, 2019 at 10:34 AM David Stansby > wrote: > I think it is good remember that (as far as I know) everyone submitting PRs and everyone reviewing them isn't paid and does so in their spare time; therefore is no obligation for anyone to review anything on any particular timescale. One thing that hasn't been discussed is how to increase the reviewing capacity of the community, which I guess would involve working out how to attract and retain more regular contributors. > > re. the proposed changes, I'm +1 modulo objective criteria being agreed upon. Personally I agree with Nelle's list, with the addition "has 100% code coverage". > > David > > > > On Tue, 15 Jan 2019, 23:46 Nelle Varoquaux wrote: > For the way GitHub sorts pull requests, it is indeed a problem, but there's a way to sort by "most recently updated." I sometimes use it to make sure that I review recently updated pull request that don't appear on the first page. > > It seems to me that this "uncontroversial but uninteresting" rule is currently not based on objective criteria. I'm personally not in favor of removing two core contributors approval for anything that modifies the API. We have had too many close calls on PR almost merged that had backward incompatible changes: I realize that some core developers believe that we are not moving fast enough, on a project with hundreds of thousands of users, changes to the API should not be considered lightly. > > Here's an attempt at better defining an "uncontroversial but uninteresting" rule based on objective criteria: > - Has not been commented on by more than one core developer; > - Doesn't change the public API; > - Doesn't add or remove a functionality; > - Doesn't add or remove a dependency; > - Is fully tested; > - All parameters are fully documented; > - Doesn't fix a release-critical bug; > - Is less than XX lines of code. (Long PR are harder to review, and thus the likelihood of one core dev not noticing a problem is higher). Threshold needs to be defined. > > To be honest, it also doesn't help that some people have 50+ pull requests open (Antony? :p). It also doesn't help that some PR contain the same code (13117, 13128) and are being hold off until the other one is merged (also, why is 13117 being hold off for a PR that has been opened afterwards?). IMO, if a PR can be merged, it should be merged. But we shouldn't make too many exceptions to the two core contributors rules. It's easier to have two core contributor review a pull request, than to try to fix API issues on code that has been merged and released. Having two pairs of eyes on a PR is also a good way to avoid having a bus factor of 1 on some areas of the library. > > > On Tue, 15 Jan 2019 at 15:19, Thomas Caswell > wrote: > I think a distinction between "no one cares" and "no one has bandwidth" needs to be made. > > If you look at the 'pulse' page, we merging 100+ PRs a month (https://github.com/matplotlib/matplotlib/pulse/monthly ) so reviews are happening, the issue is just that some are falling through the cracks. > > Tom > > On Tue, Jan 15, 2019 at 4:45 PM Antony Lee > wrote: > Eric, > Neither the ttc font, nor the pdf/ps common code PRs actually touch any "complex" point regarding fonts or pdf/ps. I have described the ttc font in my previous message; the pdf/ps PR is really just, look, these two classes share 80 lines worth of code that's literally copy-pasted, let's just put that in a common parent class and inherit from it. > Conversely, one font-related PR for which I actually have low hopes of getting properly reviewed (except on an "sure, I trust you" basis) is https://github.com/matplotlib/matplotlib/pull/12928 , which actually touches a rather complex point about font encodings (some of which I had to clarify on the FreeType mailing list). (If anyone wants to have a look at it, though... :p) > Antony > > On Tue, Jan 15, 2019 at 10:36 PM Eric Firing > wrote: > Antony, > > Your examples illustrate that the problem is often that we don't have > enough people who understand some areas, like fonts and pdf/ps file > formats, to get 2 thorough and knowledgeable reviews in a reasonable time. > > I'm fine with your proposed rule. It should help at least a little bit. > > Eric > > > On 2019/01/15 9:11 AM, Antony Lee wrote: > > I'm obviously more aware of my own PRs, so here are a few I'd have put > > up for single-review-merge (note that it's quite possible that they'd > > have attracted more attention and gotten merged faster under that > > system; that's also the goal...). Obviously other devs may think about > > other PRs that could have benefitted from the same process. > > > > - https://github.com/matplotlib/matplotlib/pull/9787 adds support for > > the ttc font format (requested since 2014) for png/svg output (that's > > provided basically for free by FreeType) but not for pdf/ps (that would > > require more difficult changes). The entire PR comes down to: > > * adding "ttc" as an extension that should be treated like "ttf"/"otf", > > * adding error handling to pdf/ps to signal these formats are not > > supported there, > > * tests. > > The PR sat open for 10 months before a first positive review (because > > no one cares about fonts, I guess) and took another 3 months to attract > > a second positive review. > > > > - https://github.com/matplotlib/matplotlib/pull/12472 does a fairly > > trivial fix to fontList.json to make it reusable for Matplotlibs > > installed in different venvs, got a positive review in one day and > > waited another 2.5 months for a second positive review. > > > > - https://github.com/matplotlib/matplotlib/pull/9867 deduplicates > > completely duplicated code between the pdf and ps backends, mostly for > > maintainability's sake; it took 7.5 months (and 5 rebases due to > > conflicts) to get a first positive review and another 6 (and around as > > many rebases) to get a second one. > > > > ----- > > > > As for the specifics (as I see it): > > - Someone opens a PR, it gets a positive review, no negative review, and > > no activity occurs for two weeks (significant activity -- excluding > > trivial chat on the thread). > > - Sponsor (first reviewer, or author if a committer) *pings* all devs by > > mentioning @matplotlib/developers on the PR thread with the intent to > > single-review-merge, and labels the PR accordingly. > > - The PR can still be reviewed/merged/rejected by the normal review > > process. However, if no one explicitly opposes the merge within two > > weeks, anyone can merge it at that point (including by self-merge). > > > > I would not exclude all API changes from the process. For example, > > https://github.com/matplotlib/matplotlib/pull/13173/commits/e90d264f3e5a263ef57243a2af86b46ab74ccc16 > > deprecates (and prepares for deletion) two parameters to imshow() that > > have had no effect since 2006. Let's pretend for a second this commit > > was a single independent PR (and not an example to showcase the > > parameter-deprecating decorator...); if it started being forgotten I > > would have proposed it for single-review-merge (again, note that if > > you're worried about the change, you don't need to reject the PR; you > > can just say, I think this API change needs to be approved by a second > > reviewer and that'll block the single-review-merge just as well). > > > > Antony > > > > On Tue, Jan 15, 2019 at 7:46 PM Nelle Varoquaux > > >> wrote: > > > > The proposed scheme by Paul doesn't seem reasonable to me. Core > > contributor A or committers needs to actively reach out to other > > core contributors: labeling is not enough IMO. > > > > > > On Tue, 15 Jan 2019 at 10:37, Jody Klymak > > >> wrote: > > > > > > > >> On 15 Jan 2019, at 10:20, Paul Hobson > >> >> wrote: > >> > >> I think this would be a good policy as well. Can I get some > >> clarification on the flow? The way I understand it: > >> > >> 1) Someone (committer, contributor, or first-timer) opens a > >> simple PR > >> 2) Committer A reviews it, adds a "single-review-merge" label, > >> then digitally walks away > >> 3) No less than two weeks later, Committer B sees the PR, > >> notices the label, opens it up, and merges without reviewing. > > > > If there is a Committer B, I think they should at least look at > > the PR to make sure it doesn?t have any hidden API changes, or > > introduce anything hostile. > > > > The point here is that getting the third person to look at a PR > > is proving quite difficult, and if the rest of the folks with > > the commit bit haven?t looked at a PR for a month, even after > > being warned, then silence indicates consent. > > > > Note that only folks with the commit bit can label PRs, so > > anyone flagging a PR like this is implicitly trusted to not be > > wreaking havoc, and not doing this for major changes. Since we > > all curate our own PRs, I rather expect that the PR contributor > > will often be the person who adds the flag. Whether we want to > > allow a self-merge at that point is up for debate, but if not, > > then folks need to get in the habit of looking at old flagged > > PRs and merging them. > > > > I think a huge problem we have is the LIFO default sort of the > > github PR queue, and that any PR not on the first page might as > > well be closed for all the attention it will get without > > constant nagging. > > > > Cheers, Jody > > > >> Does that capture a successful "single-review-merge" lifecycle? > >> -paul > >> > >> On Tue, Jan 15, 2019 at 9:57 AM Nelle Varoquaux > >> >> > >> wrote: > >> > >> Hello, > >> > >> Overall, I think this is a good idea, but I would like > >> specifics on what "uncontroversial but uninteresting" PR > >> mean. IMO, that should exclude any PR with API changes. > >> > >> Cheers, > >> N > >> > >> On Tue, 15 Jan 2019 at 06:00, Antony Lee > >> >> wrote: > >> > >> Hi all, > >> > >> During the weekly dev call, I proposed to amend the PR > >> merge rules, following an initial comment on Github > >> [https://github.com/matplotlib/matplotlib/pull/13173#issuecomment-453921220 ]. > >> There was general agreement among the devs present > >> (Tom, Hannah, Jody, and myself), so I'm putting it > >> here for discussion. The objective of this change is > >> to prevent "uncontroversial but uninteresting" PRs > >> from falling into oblivion, and try to decrease the > >> size of the open PR stack. > >> > >> The current rule is that a PR needs positive reviews > >> (from two committers, and excluding the author if a > >> committer) to be merged, except that doc-only PRs > >> (docstrings, rst) only needs a single positive review. > >> > >> I am proposing that, if a (non-doc) PR already has one > >> positive review, but no activity on that PR has > >> occurred for two weeks (exact time interval up to > >> bikeshedding) [Jody suggests: and the PR has 100% code > >> coverage], then a committer (either the first > >> reviewer, or the author if a committer) can suggest > >> that it be merged on the basis of that single review. > >> To do so, the "sponsor" should ping all developers > >> (@matplotlib/developers) on that issue indicating that > >> intent, and add a "single-review-merge" label on the > >> PR (so that these PRs can easily be found). > >> Committers are encouraged to review the PR to accept > >> and merge it or reject it or request changes on it; > >> but they can also just indicate that they consider the > >> PR sufficiently complex that a proper second review is > >> needed before merging, or request an extension, etc. > >> To do so they should still leave a "reject" review, > >> even if just saying "objecting to single-review > >> merged; anyone can dismiss after a second review". > >> However, if within another two weeks, no committer > >> voiced any objection (explicitly, i.e. by rejecting), > >> then the PR can indeed be merged on the basis of that > >> single review. > >> > >> To avoid overwhelming the system, any committer can > >> only "sponsor" a single PR at a time. > >> > >> Thoughts? > >> > >> Antony > >> _______________________________________________ > >> Matplotlib-devel mailing list > >> Matplotlib-devel at python.org > >> > > >> https://mail.python.org/mailman/listinfo/matplotlib-devel > >> > >> _______________________________________________ > >> Matplotlib-devel mailing list > >> Matplotlib-devel at python.org > >> > > >> https://mail.python.org/mailman/listinfo/matplotlib-devel > >> > >> _______________________________________________ > >> Matplotlib-devel mailing list > >> Matplotlib-devel at python.org > > >> https://mail.python.org/mailman/listinfo/matplotlib-devel > > > > -- > > Jody Klymak > > http://web.uvic.ca/~jklymak/ > > > > > > > > > > > > _______________________________________________ > > Matplotlib-devel mailing list > > Matplotlib-devel at python.org > > > https://mail.python.org/mailman/listinfo/matplotlib-devel > > > > > > _______________________________________________ > > Matplotlib-devel mailing list > > Matplotlib-devel at python.org > > https://mail.python.org/mailman/listinfo/matplotlib-devel > > > > _______________________________________________ > Matplotlib-devel mailing list > Matplotlib-devel at python.org > https://mail.python.org/mailman/listinfo/matplotlib-devel > _______________________________________________ > Matplotlib-devel mailing list > Matplotlib-devel at python.org > https://mail.python.org/mailman/listinfo/matplotlib-devel > > > -- > Thomas Caswell > tcaswell at gmail.com _______________________________________________ > Matplotlib-devel mailing list > Matplotlib-devel at python.org > https://mail.python.org/mailman/listinfo/matplotlib-devel > _______________________________________________ > Matplotlib-devel mailing list > Matplotlib-devel at python.org > https://mail.python.org/mailman/listinfo/matplotlib-devel > > > -- > Thomas Caswell > tcaswell at gmail.com _______________________________________________ > Matplotlib-devel mailing list > Matplotlib-devel at python.org > https://mail.python.org/mailman/listinfo/matplotlib-devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From nelle.varoquaux at gmail.com Wed Jan 16 12:25:23 2019 From: nelle.varoquaux at gmail.com (Nelle Varoquaux) Date: Wed, 16 Jan 2019 09:25:23 -0800 Subject: [Matplotlib-devel] Proposal to amend the PR merge rules In-Reply-To: References: Message-ID: I didn't mean to imply that you had a secret agenda in changing the API in a backward incompatible way. But mistakes happen, and looking for backward incompatibility has not been a top priority for some core developers recently, and over time it creates a culture where looking carefully for backward incompatibility is not part of our reviewing process. Having two core developers review changes limits the chance of such pull requests being merged with backward incompatibility. I also think that every API change, including the addition of keywords, needs to be considered carefully. Part of the issues we are having in terms of maintenance are that too many functions are public and many of those don't have a consistent API with the rest of the library. To summarize the points that have been agreed upon and the points that still need to be discussed. I've removed the one about documentation, but if we accept API changes in this system, it should probably added back as a point to be discussed. Agreed upon: - Has not been commented on by more than one core developer; - Doesn't fix a release-critical bug; - Is fully tested. Still needs to be discussed: 1. Doesn't change the public API; 2. Doesn't add or remove a functionality; 3. Doesn't add or remove a dependency; 4. Is less than XX lines of code. (Long PR are harder to review, and thus the likelihood of one core dev not noticing a problem is higher). Threshold needs to be defined. I'd also like to add: 5. All CI passes (because we have, in the past, ignored some errors on Travis when we felt tests were flaky) I maintain that any pull requests that add or remove a functionality should be reviewed by 2 core developers. I guess our BDFL may have to make the final call on that one :) My argument for 4. is that long pull requests are harder to review, even if they are trivial (there are studies on this.). It is thus easier to make a mistake in reviewing. In terms of number of pull request per core dev, it is just that more pull requests dilute our effort in reviewing. The more someone has pull request open, the less likely they are to be reviewed and the more likely that developer is to "forget" about them. I don't think we should have a rule on the number of pull request open per person, but on the different project I contribute to, I have a rule of thumb to not have more than 5 pull requests opened. It forces me to finish up the work on the pull requests I have opened before working on something else. As for David's point, it just happens that some collaborators and I are working on understanding factors for newcomer retention and developer burnout. We've got preliminary results that we are not ready to share but we are hoping that it'll help on that point. Cheers, N On Wed, 16 Jan 2019 at 03:11, Antony Lee wrote: > Replying to Nelle's points first: > > >> It seems to me that this "uncontroversial but uninteresting" rule is >> currently not based on objective criteria. I'm personally not in favor of >> removing two core contributors approval for anything that modifies the >> API. We have had too many close calls on PR almost merged that had >> backward incompatible changes: I realize that some core developers believe >> that we are not moving fast enough, on a project with hundreds of thousands >> of users, changes to the API should not be considered lightly. >> > > To be clear (and even though it is probably common knowledge that I would > like to evolve the API faster), there is no "secret agenda" of using this > to sneak API breaks through the system. > > >> Here's an attempt at better defining an "uncontroversial but >> uninteresting" rule based on objective criteria: >> - Has not been commented on by more than one core developer; >> > > Sounds reasonable (with a reasonably flexible interpretation -- similarly > to the "no activity since two weeks, excluding trivial chat"), basically > "no other core dev expressed interest in reviewing it". > > >> - Doesn't change the public API; >> - Doesn't add or remove a functionality; >> - Doesn't add or remove a dependency; >> > > I expect all changes of API to go through the normal route of deprecation > and API change note, which even the first reviewer should make sure are > present before sponsoring the PR for single-review-merge (as an aside, I > guess the first reviewer could even say, "approving, but I don't want to > take the responsibility for being the single reviewer"...). So now you > have a two week period where any other core dev passing by can just see > whether there is an API change note and make their own judgment, including > deciding to block the PR if necessary, based on that change note. > I guess it is reasonable to say "API deprecations or changes and new > functionality should be clearly documented" to help that "skim-through" > review. > As for *adding* functionality, I guess you would have rejected the > ttc-font PR on that premise? (Actually I now see that the additional > functionality was actually not documented :)) Note that this is not some > obscure edge case that no one cares about; it's a feature that has been > requested for years on the tracker. > > - Is fully tested; >> > > Also proposed by Jody and now by David, sounds good to me. > > >> - All parameters are fully documented; >> > > (sounds contradictory with "doesn't add a functionality"... if there is no > new functionality it's unlikely there's a new parameter) > > >> - Doesn't fix a release-critical bug; >> > > These tend to all be merged just before the next release is cut anyways > (review dynamics are a bit different in these periods), so I wouldn't worry > about it, but sure. > > >> - Is less than XX lines of code. (Long PR are harder to review, and thus >> the likelihood of one core dev not noticing a problem is higher). Threshold >> needs to be defined. >> > > I don't actually think this is a good rule; the pdf/ps merge PR is +138 > lines, -182 lines but as explained above the change is fairly trivial; > obviously I could have split it into smaller chunks but if anything that > would have made it harder to review. > Conversely (to use an example I have already raised elsewhere) > https://github.com/matplotlib/matplotlib/pull/11530 (making sure that > LocationEvent.x and LocationEvent.y are integers (as they refer to pixel > locations) rather than floats) is 2 lines of changes in the codebase and 20 > lines of test, was reviewed by 2.5 core devs (I left a comment in passing) > and we all missed that there was "problem" with it (it broke mplcursors and > some other pieces of code I have that assume that it is possible to > synthetize LocationEvents pointing to arbitrary places in the canvas). > (Not to derail this discussion into how to evaluate API breaks once > again... but I think these kinds of changes are much more insidious than, > say, removing a function, as the latter would just trigger an > AttributeError that is easily noticed and at worst you just go back to the > previous Matplotlib release, whereas the LocationEvent change was only > caught because it was covered by the mplcursors test suite; otherwise I'd > have been silently generating wrong results due to it...) > > >> To be honest, it also doesn't help that some people have 50+ pull >> requests open (Antony? :p). > > > I guess if the way to fix Matplotlib is to not contribute to Matplotlib > anymore, we have a bigger problem. In fact, I have some pretty big patches > fixing long-standing issues (e.g. for #8550 I have a pure-Python > reimplementation of the fontconfig font weight deduction algorithm, > translated from fontconfig's C implementation) that I don't even bother > turning into PRs, as I know they have no chance of being reviewed and > merged under the current system at least, because no one would care (see > also the bus-factor point below). > > >> It also doesn't help that some PR contain the same code (13117, 13128) >> and are being hold off until the other one is merged (also, why is 13117 >> being hold off for a PR that has been opened afterwards?). IMO, if a PR can >> be merged, it should be merged. > > > That's discussed in > https://github.com/matplotlib/matplotlib/pull/13117#discussion_r245503693 > (FTR I actually agreed on merging 13117 directly, but apparently that's an > API break...) > > >> But we shouldn't make too many exceptions to the two core contributors >> rules. It's easier to have two core contributor review a pull request, than >> to try to fix API issues on code that has been merged and released. Having >> two pairs of eyes on a PR is also a good way to avoid having a bus factor >> of 1 on some areas of the library. > > > There are already huge and central chunks of the library (Agg internals, > PDF/PS) that have a bus factor close to zero. If anything, I hope that > this "drive-by review" system will encourage reviewers to start looking at > places they are less familiar with ("oh yes, I guess this change makes > sense" -- see Eric's reply above) and decrease that. > > And then to David's (the code-coverage point is already covered above): > > I think it is good remember that (as far as I know) everyone submitting >> PRs and everyone reviewing them isn't paid and does so in their spare time; >> therefore is no obligation for anyone to review anything on any particular >> timescale. One thing that hasn't been discussed is how to increase the >> reviewing capacity of the community, which I guess would involve working >> out how to attract and retain more regular contributors. > > > Given the premises "no one is paid to review PRs" and "the reviewing > capacity is too small" (which I agree with), I think there are two > reasonable solutions: > - Make PRs easier to merge (this proposal), > - Increasing to number of core devs (your proposal, but it's not clear to > me whether this number can increase significantly in a short period of > time...). > > Antony > > On Wed, Jan 16, 2019 at 10:34 AM David Stansby wrote: > >> I think it is good remember that (as far as I know) everyone submitting >> PRs and everyone reviewing them isn't paid and does so in their spare time; >> therefore is no obligation for anyone to review anything on any particular >> timescale. One thing that hasn't been discussed is how to increase the >> reviewing capacity of the community, which I guess would involve working >> out how to attract and retain more regular contributors. >> >> re. the proposed changes, I'm +1 modulo objective criteria being agreed >> upon. Personally I agree with Nelle's list, with the addition "has 100% >> code coverage". >> >> David >> >> >> >> On Tue, 15 Jan 2019, 23:46 Nelle Varoquaux > wrote: >> >>> For the way GitHub sorts pull requests, it is indeed a problem, but >>> there's a way to sort by "most recently updated." I sometimes use it to >>> make sure that I review recently updated pull request that don't appear on >>> the first page. >>> >>> It seems to me that this "uncontroversial but uninteresting" rule is >>> currently not based on objective criteria. I'm personally not in favor of >>> removing two core contributors approval for anything that modifies the >>> API. We have had too many close calls on PR almost merged that had >>> backward incompatible changes: I realize that some core developers believe >>> that we are not moving fast enough, on a project with hundreds of thousands >>> of users, changes to the API should not be considered lightly. >>> >>> Here's an attempt at better defining an "uncontroversial but >>> uninteresting" rule based on objective criteria: >>> - Has not been commented on by more than one core developer; >>> - Doesn't change the *public* API; >>> - Doesn't add or remove a functionality; >>> - Doesn't add or remove a dependency; >>> - Is fully tested; >>> - All parameters are fully documented; >>> - Doesn't fix a release-critical bug; >>> - Is less than XX lines of code. (Long PR are harder to review, and thus >>> the likelihood of one core dev not noticing a problem is higher). Threshold >>> needs to be defined. >>> >>> To be honest, it also doesn't help that some people have 50+ pull >>> requests open (Antony? :p). It also doesn't help that some PR contain the >>> same code (13117, 13128) and are being hold off until the other one is >>> merged (also, why is 13117 being hold off for a PR that has been opened >>> afterwards?). IMO, if a PR can be merged, it should be merged. But we >>> shouldn't make too many exceptions to the two core contributors rules. It's >>> easier to have two core contributor review a pull request, than to try to >>> fix API issues on code that has been merged and released. Having two pairs >>> of eyes on a PR is also a good way to avoid having a bus factor of 1 on >>> some areas of the library. >>> >>> >>> On Tue, 15 Jan 2019 at 15:19, Thomas Caswell wrote: >>> >>>> I think a distinction between "no one cares" and "no one has bandwidth" >>>> needs to be made. >>>> >>>> If you look at the 'pulse' page, we merging 100+ PRs a month ( >>>> https://github.com/matplotlib/matplotlib/pulse/monthly) so reviews are >>>> happening, the issue is just that some are falling through the cracks. >>>> >>>> Tom >>>> >>>> On Tue, Jan 15, 2019 at 4:45 PM Antony Lee < >>>> antony.lee at institutoptique.fr> wrote: >>>> >>>>> Eric, >>>>> Neither the ttc font, nor the pdf/ps common code PRs actually touch >>>>> any "complex" point regarding fonts or pdf/ps. I have described the ttc >>>>> font in my previous message; the pdf/ps PR is really just, look, these two >>>>> classes share 80 lines worth of code that's literally copy-pasted, let's >>>>> just put that in a common parent class and inherit from it. >>>>> Conversely, one font-related PR for which I actually have low hopes of >>>>> getting properly reviewed (except on an "sure, I trust you" basis) is >>>>> https://github.com/matplotlib/matplotlib/pull/12928, which actually >>>>> touches a rather complex point about font encodings (some of which I had to >>>>> clarify on the FreeType mailing list). (If anyone wants to have a look at >>>>> it, though... :p) >>>>> Antony >>>>> >>>>> On Tue, Jan 15, 2019 at 10:36 PM Eric Firing >>>>> wrote: >>>>> >>>>>> Antony, >>>>>> >>>>>> Your examples illustrate that the problem is often that we don't have >>>>>> enough people who understand some areas, like fonts and pdf/ps file >>>>>> formats, to get 2 thorough and knowledgeable reviews in a reasonable >>>>>> time. >>>>>> >>>>>> I'm fine with your proposed rule. It should help at least a little >>>>>> bit. >>>>>> >>>>>> Eric >>>>>> >>>>>> >>>>>> On 2019/01/15 9:11 AM, Antony Lee wrote: >>>>>> > I'm obviously more aware of my own PRs, so here are a few I'd have >>>>>> put >>>>>> > up for single-review-merge (note that it's quite possible that >>>>>> they'd >>>>>> > have attracted more attention and gotten merged faster under that >>>>>> > system; that's also the goal...). Obviously other devs may think >>>>>> about >>>>>> > other PRs that could have benefitted from the same process. >>>>>> > >>>>>> > - https://github.com/matplotlib/matplotlib/pull/9787 adds support >>>>>> for >>>>>> > the ttc font format (requested since 2014) for png/svg output >>>>>> (that's >>>>>> > provided basically for free by FreeType) but not for pdf/ps (that >>>>>> would >>>>>> > require more difficult changes). The entire PR comes down to: >>>>>> > * adding "ttc" as an extension that should be treated like >>>>>> "ttf"/"otf", >>>>>> > * adding error handling to pdf/ps to signal these formats are >>>>>> not >>>>>> > supported there, >>>>>> > * tests. >>>>>> > The PR sat open for 10 months before a first positive review >>>>>> (because >>>>>> > no one cares about fonts, I guess) and took another 3 months to >>>>>> attract >>>>>> > a second positive review. >>>>>> > >>>>>> > - https://github.com/matplotlib/matplotlib/pull/12472 does a >>>>>> fairly >>>>>> > trivial fix to fontList.json to make it reusable for Matplotlibs >>>>>> > installed in different venvs, got a positive review in one day and >>>>>> > waited another 2.5 months for a second positive review. >>>>>> > >>>>>> > - https://github.com/matplotlib/matplotlib/pull/9867 deduplicates >>>>>> > completely duplicated code between the pdf and ps backends, mostly >>>>>> for >>>>>> > maintainability's sake; it took 7.5 months (and 5 rebases due to >>>>>> > conflicts) to get a first positive review and another 6 (and around >>>>>> as >>>>>> > many rebases) to get a second one. >>>>>> > >>>>>> > ----- >>>>>> > >>>>>> > As for the specifics (as I see it): >>>>>> > - Someone opens a PR, it gets a positive review, no negative >>>>>> review, and >>>>>> > no activity occurs for two weeks (significant activity -- excluding >>>>>> > trivial chat on the thread). >>>>>> > - Sponsor (first reviewer, or author if a committer) *pings* all >>>>>> devs by >>>>>> > mentioning @matplotlib/developers on the PR thread with the intent >>>>>> to >>>>>> > single-review-merge, and labels the PR accordingly. >>>>>> > - The PR can still be reviewed/merged/rejected by the normal review >>>>>> > process. However, if no one explicitly opposes the merge within >>>>>> two >>>>>> > weeks, anyone can merge it at that point (including by self-merge). >>>>>> > >>>>>> > I would not exclude all API changes from the process. For example, >>>>>> > >>>>>> https://github.com/matplotlib/matplotlib/pull/13173/commits/e90d264f3e5a263ef57243a2af86b46ab74ccc16 >>>>>> > deprecates (and prepares for deletion) two parameters to imshow() >>>>>> that >>>>>> > have had no effect since 2006. Let's pretend for a second this >>>>>> commit >>>>>> > was a single independent PR (and not an example to showcase the >>>>>> > parameter-deprecating decorator...); if it started being forgotten >>>>>> I >>>>>> > would have proposed it for single-review-merge (again, note that if >>>>>> > you're worried about the change, you don't need to reject the PR; >>>>>> you >>>>>> > can just say, I think this API change needs to be approved by a >>>>>> second >>>>>> > reviewer and that'll block the single-review-merge just as well). >>>>>> > >>>>>> > Antony >>>>>> > >>>>>> > On Tue, Jan 15, 2019 at 7:46 PM Nelle Varoquaux >>>>>> > > >>>>>> wrote: >>>>>> > >>>>>> > The proposed scheme by Paul doesn't seem reasonable to me. Core >>>>>> > contributor A or committers needs to actively reach out to other >>>>>> > core contributors: labeling is not enough IMO. >>>>>> > >>>>>> > >>>>>> > On Tue, 15 Jan 2019 at 10:37, Jody Klymak >>>>> > > wrote: >>>>>> > >>>>>> > >>>>>> > >>>>>> >> On 15 Jan 2019, at 10:20, Paul Hobson >>>>> >> > wrote: >>>>>> >> >>>>>> >> I think this would be a good policy as well. Can I get some >>>>>> >> clarification on the flow? The way I understand it: >>>>>> >> >>>>>> >> 1) Someone (committer, contributor, or first-timer) opens a >>>>>> >> simple PR >>>>>> >> 2) Committer A reviews it, adds a "single-review-merge" >>>>>> label, >>>>>> >> then digitally walks away >>>>>> >> 3) No less than two weeks later, Committer B sees the PR, >>>>>> >> notices the label, opens it up, and merges without >>>>>> reviewing. >>>>>> > >>>>>> > If there is a Committer B, I think they should at least >>>>>> look at >>>>>> > the PR to make sure it doesn?t have any hidden API changes, >>>>>> or >>>>>> > introduce anything hostile. >>>>>> > >>>>>> > The point here is that getting the third person to look at >>>>>> a PR >>>>>> > is proving quite difficult, and if the rest of the folks >>>>>> with >>>>>> > the commit bit haven?t looked at a PR for a month, even >>>>>> after >>>>>> > being warned, then silence indicates consent. >>>>>> > >>>>>> > Note that only folks with the commit bit can label PRs, so >>>>>> > anyone flagging a PR like this is implicitly trusted to not >>>>>> be >>>>>> > wreaking havoc, and not doing this for major changes. >>>>>> Since we >>>>>> > all curate our own PRs, I rather expect that the PR >>>>>> contributor >>>>>> > will often be the person who adds the flag. Whether we >>>>>> want to >>>>>> > allow a self-merge at that point is up for debate, but if >>>>>> not, >>>>>> > then folks need to get in the habit of looking at old >>>>>> flagged >>>>>> > PRs and merging them. >>>>>> > >>>>>> > I think a huge problem we have is the LIFO default sort of >>>>>> the >>>>>> > github PR queue, and that any PR not on the first page >>>>>> might as >>>>>> > well be closed for all the attention it will get without >>>>>> > constant nagging. >>>>>> > >>>>>> > Cheers, Jody >>>>>> > >>>>>> >> Does that capture a successful "single-review-merge" >>>>>> lifecycle? >>>>>> >> -paul >>>>>> >> >>>>>> >> On Tue, Jan 15, 2019 at 9:57 AM Nelle Varoquaux >>>>>> >> >>>>> nelle.varoquaux at gmail.com>> >>>>>> >> wrote: >>>>>> >> >>>>>> >> Hello, >>>>>> >> >>>>>> >> Overall, I think this is a good idea, but I would like >>>>>> >> specifics on what "uncontroversial but uninteresting" >>>>>> PR >>>>>> >> mean. IMO, that should exclude any PR with API changes. >>>>>> >> >>>>>> >> Cheers, >>>>>> >> N >>>>>> >> >>>>>> >> On Tue, 15 Jan 2019 at 06:00, Antony Lee >>>>>> >> > >>>>>> wrote: >>>>>> >> >>>>>> >> Hi all, >>>>>> >> >>>>>> >> During the weekly dev call, I proposed to amend >>>>>> the PR >>>>>> >> merge rules, following an initial comment on Github >>>>>> >> [ >>>>>> https://github.com/matplotlib/matplotlib/pull/13173#issuecomment-453921220]. >>>>>> >>>>>> >> There was general agreement among the devs present >>>>>> >> (Tom, Hannah, Jody, and myself), so I'm putting it >>>>>> >> here for discussion. The objective of this change >>>>>> is >>>>>> >> to prevent "uncontroversial but uninteresting" PRs >>>>>> >> from falling into oblivion, and try to decrease the >>>>>> >> size of the open PR stack. >>>>>> >> >>>>>> >> The current rule is that a PR needs positive >>>>>> reviews >>>>>> >> (from two committers, and excluding the author if a >>>>>> >> committer) to be merged, except that doc-only PRs >>>>>> >> (docstrings, rst) only needs a single positive >>>>>> review. >>>>>> >> >>>>>> >> I am proposing that, if a (non-doc) PR already has >>>>>> one >>>>>> >> positive review, but no activity on that PR has >>>>>> >> occurred for two weeks (exact time interval up to >>>>>> >> bikeshedding) [Jody suggests: and the PR has 100% >>>>>> code >>>>>> >> coverage], then a committer (either the first >>>>>> >> reviewer, or the author if a committer) can suggest >>>>>> >> that it be merged on the basis of that single >>>>>> review. >>>>>> >> To do so, the "sponsor" should ping all developers >>>>>> >> (@matplotlib/developers) on that issue indicating >>>>>> that >>>>>> >> intent, and add a "single-review-merge" label on >>>>>> the >>>>>> >> PR (so that these PRs can easily be found). >>>>>> >> Committers are encouraged to review the PR to >>>>>> accept >>>>>> >> and merge it or reject it or request changes on it; >>>>>> >> but they can also just indicate that they consider >>>>>> the >>>>>> >> PR sufficiently complex that a proper second >>>>>> review is >>>>>> >> needed before merging, or request an extension, >>>>>> etc. >>>>>> >> To do so they should still leave a "reject" review, >>>>>> >> even if just saying "objecting to single-review >>>>>> >> merged; anyone can dismiss after a second review". >>>>>> >> However, if within another two weeks, no committer >>>>>> >> voiced any objection (explicitly, i.e. by >>>>>> rejecting), >>>>>> >> then the PR can indeed be merged on the basis of >>>>>> that >>>>>> >> single review. >>>>>> >> >>>>>> >> To avoid overwhelming the system, any committer can >>>>>> >> only "sponsor" a single PR at a time. >>>>>> >> >>>>>> >> Thoughts? >>>>>> >> >>>>>> >> Antony >>>>>> >> _______________________________________________ >>>>>> >> Matplotlib-devel mailing list >>>>>> >> Matplotlib-devel at python.org >>>>>> >> >>>>>> >> >>>>>> https://mail.python.org/mailman/listinfo/matplotlib-devel >>>>>> >> >>>>>> >> _______________________________________________ >>>>>> >> Matplotlib-devel mailing list >>>>>> >> Matplotlib-devel at python.org >>>>>> >> >>>>>> >> >>>>>> https://mail.python.org/mailman/listinfo/matplotlib-devel >>>>>> >> >>>>>> >> _______________________________________________ >>>>>> >> Matplotlib-devel mailing list >>>>>> >> Matplotlib-devel at python.org >>>>> Matplotlib-devel at python.org> >>>>>> >> https://mail.python.org/mailman/listinfo/matplotlib-devel >>>>>> > >>>>>> > -- >>>>>> > Jody Klymak >>>>>> > http://web.uvic.ca/~jklymak/ >>>>>> > >>>>>> > >>>>>> > >>>>>> > >>>>>> > >>>>>> > _______________________________________________ >>>>>> > Matplotlib-devel mailing list >>>>>> > Matplotlib-devel at python.org >>>>> > >>>>>> > https://mail.python.org/mailman/listinfo/matplotlib-devel >>>>>> > >>>>>> > >>>>>> > _______________________________________________ >>>>>> > Matplotlib-devel mailing list >>>>>> > Matplotlib-devel at python.org >>>>>> > https://mail.python.org/mailman/listinfo/matplotlib-devel >>>>>> > >>>>>> >>>>>> _______________________________________________ >>>>>> Matplotlib-devel mailing list >>>>>> Matplotlib-devel at python.org >>>>>> https://mail.python.org/mailman/listinfo/matplotlib-devel >>>>>> >>>>> _______________________________________________ >>>>> Matplotlib-devel mailing list >>>>> Matplotlib-devel at python.org >>>>> https://mail.python.org/mailman/listinfo/matplotlib-devel >>>>> >>>> >>>> >>>> -- >>>> Thomas Caswell >>>> tcaswell at gmail.com >>>> _______________________________________________ >>>> Matplotlib-devel mailing list >>>> Matplotlib-devel at python.org >>>> https://mail.python.org/mailman/listinfo/matplotlib-devel >>>> >>> _______________________________________________ >>> Matplotlib-devel mailing list >>> Matplotlib-devel at python.org >>> https://mail.python.org/mailman/listinfo/matplotlib-devel >>> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From antony.lee at institutoptique.fr Wed Jan 16 12:53:16 2019 From: antony.lee at institutoptique.fr (Antony Lee) Date: Wed, 16 Jan 2019 18:53:16 +0100 Subject: [Matplotlib-devel] Proposal to amend the PR merge rules In-Reply-To: References: Message-ID: On Wed, Jan 16, 2019 at 6:26 PM Nelle Varoquaux wrote: > > I didn't mean to imply that you had a secret agenda in changing the API in > a backward incompatible way. But mistakes happen, and looking for backward > incompatibility has not been a top priority for some core developers > recently, and over time it creates a culture where looking carefully for > backward incompatibility is not part of our reviewing process. Having two > core developers review changes limits the chance of such pull requests > being merged with backward incompatibility. I also think that every API > change, including the addition of keywords, needs to be considered > carefully. Part of the issues we are having in terms of maintenance are > that too many functions are public and many of those don't have a > consistent API with the rest of the library. > Again, you may note that I have fairly few PRs that introduce new functionality; most of the API changes I PR'd actually deprecate and later remove functions that "should not have been public" (I have sort of given up on making things consistent in an backcompatible manner). So in that respect I am mostly trying to improve maintanability... > To summarize the points that have been agreed upon and the points that > still need to be discussed. I've removed the one about documentation, but > if we accept API changes in this system, it should probably added back as a > point to be discussed. > > Agreed upon: > - Has not been commented on by more than one core developer; > - Doesn't fix a release-critical bug; > - Is fully tested. > > Still needs to be discussed: > 1. Doesn't change the public API; > 2. Doesn't add or remove a functionality; > 3. Doesn't add or remove a dependency; > 4. Is less than XX lines of code. (Long PR are harder to review, and thus > the likelihood of one core dev not noticing a problem is higher). Threshold > needs to be defined. > > I'd also like to add: > 5. All CI passes (because we have, in the past, ignored some errors on > Travis when we felt tests were flaky) > Agreed on 5. > I maintain that any pull requests that add or remove a functionality > should be reviewed by 2 core developers. I guess our BDFL may have to make > the final call on that one :) > Happy to let Tom decide, but again I would say that "add or remove a functionality" is not always an objective feature either (e.g., does the LocationEvent-integer PR qualify?). > My argument for 4. is that long pull requests are harder to review, even > if they are trivial (there are studies on this.). It is thus easier to make > a mistake in reviewing. > As I already mentioned, there have been cases of 2-line PRs (+ 20 lines of *new* tests) making backwards incompatible changes, whereas some 100+ lines PR are essentially copy-paste trivialities, so I really don't buy the argument. > In terms of number of pull request per core dev, it is just that more pull > requests dilute our effort in reviewing. The more someone has pull request > open, the less likely they are to be reviewed and the more likely that > developer is to "forget" about them. I don't think we should have a rule on > the number of pull request open per person, but on the different project I > contribute to, I have a rule of thumb to not have more than 5 pull requests > opened. It forces me to finish up the work on the pull requests I have > opened before working on something else. > Frankly it's not clear what "finish up the work" is supposed to mean. If you look at the PRs I've mentioned above they have been merged with (close to) no changes (except for repeated rebases on master...) after the 6-to-12-month review period, so essentially all the work was done from the beginning. > As for David's point, it just happens that some collaborators and I are > working on understanding factors for newcomer retention and developer > burnout. We've got preliminary results that we are not ready to share but > we are hoping that it'll help on that point. > Happy to hear anything that could be done to help on that point. But I guess I'll take advantage of this to reply to Tom's remark as well [Tom] I want to push back strongly on the notion that "did not review --> > does not care". As as already been discussed above there are many other > reasons someone does not comment (for example "I _do_ care about this, but > I {do not have time, am too tired and cranky, ...} to properly review this > now"). I'm not pointing fingers at anyone not doing their review share, and once again totally agree with David's point regarding no one being paid. However, you have to realize that the consequences of insufficient review volume is that real improvements are not getting in (either because the PRs stall out, or (in my case) because I don't even bother PR'ing them anymore and just live-patch matplotlib locally), and saying "it's no-one's fault" (which is true) is not going to help. Cheers, > N > Antony -------------- next part -------------- An HTML attachment was scrubbed... URL: From dstansby at gmail.com Wed Jan 16 13:36:27 2019 From: dstansby at gmail.com (David Stansby) Date: Wed, 16 Jan 2019 18:36:27 +0000 Subject: [Matplotlib-devel] Proposal to amend the PR merge rules In-Reply-To: References: Message-ID: I think real improvements not getting in at the rate they are submitted is just how it works; if I started submitting a PR a day for the next 50 days I would expect a good chunk of them sit there un-reviewed based on our current rate of merging things, and that's fine. The developers as a whole have a total reviewing rate, which is unpredictable and finite by the very nature of this being an open-source project where no-one is obliged to do anything. If we want to adjust reviewing rules to increase 'productivity' that seems fine to me, but I don't think we should be doing it with the end goal of being able to review at the rate that code is submitted (or any particular other rate). re. large diff PRs, I find 100+ line PRs that are just copy and paste really hard to review - it's pretty dull for me trying to work out if code has been copy/pasted exactly without any errors (if there's an easier way to review stuff like this instead of checking every line please let me know!). David On Wed, 16 Jan 2019 at 17:54, Antony Lee wrote: > > > On Wed, Jan 16, 2019 at 6:26 PM Nelle Varoquaux > wrote: > >> >> I didn't mean to imply that you had a secret agenda in changing the API >> in a backward incompatible way. But mistakes happen, and looking for >> backward incompatibility has not been a top priority for some core >> developers recently, and over time it creates a culture where looking >> carefully for backward incompatibility is not part of our reviewing >> process. Having two core developers review changes limits the chance of >> such pull requests being merged with backward incompatibility. I also think >> that every API change, including the addition of keywords, needs to be >> considered carefully. Part of the issues we are having in terms of >> maintenance are that too many functions are public and many of those don't >> have a consistent API with the rest of the library. >> > > Again, you may note that I have fairly few PRs that introduce new > functionality; most of the API changes I PR'd actually deprecate and later > remove functions that "should not have been public" (I have sort of given > up on making things consistent in an backcompatible manner). So in that > respect I am mostly trying to improve maintanability... > > >> To summarize the points that have been agreed upon and the points that >> still need to be discussed. I've removed the one about documentation, but >> if we accept API changes in this system, it should probably added back as a >> point to be discussed. >> >> Agreed upon: >> - Has not been commented on by more than one core developer; >> - Doesn't fix a release-critical bug; >> - Is fully tested. >> >> Still needs to be discussed: >> 1. Doesn't change the public API; >> 2. Doesn't add or remove a functionality; >> 3. Doesn't add or remove a dependency; >> 4. Is less than XX lines of code. (Long PR are harder to review, and thus >> the likelihood of one core dev not noticing a problem is higher). Threshold >> needs to be defined. >> >> I'd also like to add: >> 5. All CI passes (because we have, in the past, ignored some errors on >> Travis when we felt tests were flaky) >> > > Agreed on 5. > > >> I maintain that any pull requests that add or remove a functionality >> should be reviewed by 2 core developers. I guess our BDFL may have to make >> the final call on that one :) >> > > Happy to let Tom decide, but again I would say that "add or remove a > functionality" is not always an objective feature either (e.g., does the > LocationEvent-integer PR qualify?). > > >> My argument for 4. is that long pull requests are harder to review, even >> if they are trivial (there are studies on this.). It is thus easier to make >> a mistake in reviewing. >> > > As I already mentioned, there have been cases of 2-line PRs (+ 20 lines of > *new* tests) making backwards incompatible changes, whereas some 100+ lines > PR are essentially copy-paste trivialities, so I really don't buy the > argument. > > >> In terms of number of pull request per core dev, it is just that more >> pull requests dilute our effort in reviewing. The more someone has pull >> request open, the less likely they are to be reviewed and the more likely >> that developer is to "forget" about them. I don't think we should have a >> rule on the number of pull request open per person, but on the different >> project I contribute to, I have a rule of thumb to not have more than 5 >> pull requests opened. It forces me to finish up the work on the pull >> requests I have opened before working on something else. >> > > Frankly it's not clear what "finish up the work" is supposed to mean. If > you look at the PRs I've mentioned above they have been merged with (close > to) no changes (except for repeated rebases on master...) after the > 6-to-12-month review period, so essentially all the work was done from the > beginning. > > >> As for David's point, it just happens that some collaborators and I are >> working on understanding factors for newcomer retention and developer >> burnout. We've got preliminary results that we are not ready to share but >> we are hoping that it'll help on that point. >> > > Happy to hear anything that could be done to help on that point. But I > guess I'll take advantage of this to reply to Tom's remark as well > > [Tom] I want to push back strongly on the notion that "did not review --> >> does not care". As as already been discussed above there are many other >> reasons someone does not comment (for example "I _do_ care about this, but >> I {do not have time, am too tired and cranky, ...} to properly review this >> now"). > > > I'm not pointing fingers at anyone not doing their review share, and once > again totally agree with David's point regarding no one being paid. > However, you have to realize that the consequences of insufficient review > volume is that real improvements are not getting in (either because the PRs > stall out, or (in my case) because I don't even bother PR'ing them anymore > and just live-patch matplotlib locally), and saying "it's no-one's fault" > (which is true) is not going to help. > > Cheers, >> N >> > > Antony > -------------- next part -------------- An HTML attachment was scrubbed... URL: From antony.lee at institutoptique.fr Wed Jan 16 19:48:02 2019 From: antony.lee at institutoptique.fr (Antony Lee) Date: Thu, 17 Jan 2019 01:48:02 +0100 Subject: [Matplotlib-devel] Proposal to amend the PR merge rules In-Reply-To: References: Message-ID: On Wed, Jan 16, 2019 at 7:36 PM David Stansby wrote: > I think real improvements not getting in at the rate they are submitted is > just how it works; if I started submitting a PR a day for the next 50 days > I would expect a good chunk of them sit there un-reviewed based on our > current rate of merging things, and that's fine. The developers as a whole > have a total reviewing rate, which is unpredictable and finite by the very > nature of this being an open-source project where no-one is obliged to do > anything. If we want to adjust reviewing rules to increase 'productivity' > that seems fine to me, but I don't think we should be doing it with the end > goal of being able to review at the rate that code is submitted (or any > particular other rate). > The end goal is not to set a specific rate, but it is indeed to try to raise that rate up. Obviously we could also throw our hands up and say nothing can be done about it, but that seems a bit defeatist. Right now, I would bet that a lot of PRs go in with exactly two reviewers having gone through them and no other reviewer having even skimmed them. If anything, I believe that the system I propose will increase the number of core-devs that'll have at least skimmed through the PR (all you need to do, if you want to participate, is to check your github notifications once every other week, whereas right now you'd actually need to pay attention to each PR as it is being submitted), and it is quite likely that 1 in-depth reviewer + 3-4 skim-throughs works as well, if not better, than 2 "in-depth" reviews (especially when the second one is sometimes "oh, that looks fine and it's already been approved once, let's just commit it"). > re. large diff PRs, I find 100+ line PRs that are just copy and paste > really hard to review - it's pretty dull for me trying to work out if code > has been copy/pasted exactly without any errors (if there's an easier way > to review stuff like this instead of checking every line please let me > know!). > Frankly, for a PR like that, I would just check that no function is missing and that everything is test-covered (without changes to the tests), and (in case you want to be sure) skim through the code to see that the PR author has not added a sneaky vulnerability that allows them to take over the user's computer... (In other words, I don't think a line-by-line check matters for such cases.) Antony -------------- next part -------------- An HTML attachment was scrubbed... URL: From dstansby at gmail.com Thu Jan 17 06:25:19 2019 From: dstansby at gmail.com (David Stansby) Date: Thu, 17 Jan 2019 11:25:19 +0000 Subject: [Matplotlib-devel] Proposal to amend the PR merge rules In-Reply-To: References: Message-ID: Apologies, my previous email was unnecessarily negative... I think the new system is a good idea, so here's where we seem to be. In terms of process: 1) PR opened 2) PR gets 1 positive review 3) 2 weeks after review the PR gets *no other significant activity, *and *qualifies for simpler review* 4) PR opener or reviewer 1 pings all Matplotlib devs 5) If 2 weeks after pinging no other significant activity has occurred, PR can be merged by PR opener or reviewer 1 Does everyone agree on this? The bits I've put in bold still need to be defined; I don't have time now, but can provide a summary of these later. David On Thu, 17 Jan 2019 at 00:48, Antony Lee wrote: > On Wed, Jan 16, 2019 at 7:36 PM David Stansby wrote: > >> I think real improvements not getting in at the rate they are submitted >> is just how it works; if I started submitting a PR a day for the next 50 >> days I would expect a good chunk of them sit there un-reviewed based on our >> current rate of merging things, and that's fine. The developers as a whole >> have a total reviewing rate, which is unpredictable and finite by the very >> nature of this being an open-source project where no-one is obliged to do >> anything. If we want to adjust reviewing rules to increase 'productivity' >> that seems fine to me, but I don't think we should be doing it with the end >> goal of being able to review at the rate that code is submitted (or any >> particular other rate). >> > > The end goal is not to set a specific rate, but it is indeed to try to > raise that rate up. Obviously we could also throw our hands up and say > nothing can be done about it, but that seems a bit defeatist. > > Right now, I would bet that a lot of PRs go in with exactly two reviewers > having gone through them and no other reviewer having even skimmed them. > If anything, I believe that the system I propose will increase the number > of core-devs that'll have at least skimmed through the PR (all you need to > do, if you want to participate, is to check your github notifications once > every other week, whereas right now you'd actually need to pay attention to > each PR as it is being submitted), and it is quite likely that 1 in-depth > reviewer + 3-4 skim-throughs works as well, if not better, than 2 > "in-depth" reviews (especially when the second one is sometimes "oh, that > looks fine and it's already been approved once, let's just commit it"). > > >> re. large diff PRs, I find 100+ line PRs that are just copy and paste >> really hard to review - it's pretty dull for me trying to work out if code >> has been copy/pasted exactly without any errors (if there's an easier way >> to review stuff like this instead of checking every line please let me >> know!). >> > > Frankly, for a PR like that, I would just check that no function is > missing and that everything is test-covered (without changes to the tests), > and (in case you want to be sure) skim through the code to see that the PR > author has not added a sneaky vulnerability that allows them to take over > the user's computer... (In other words, I don't think a line-by-line check > matters for such cases.) > > Antony > -------------- next part -------------- An HTML attachment was scrubbed... URL: From antony.lee at institutoptique.fr Thu Jan 17 06:40:10 2019 From: antony.lee at institutoptique.fr (Antony Lee) Date: Thu, 17 Jan 2019 12:40:10 +0100 Subject: [Matplotlib-devel] Proposal to amend the PR merge rules In-Reply-To: References: Message-ID: Agreed. On Thu, Jan 17, 2019, 12:25 PM David Stansby Apologies, my previous email was unnecessarily negative... > > I think the new system is a good idea, so here's where we seem to be. In > terms of process: > > 1) PR opened > 2) PR gets 1 positive review > 3) 2 weeks after review the PR gets *no other significant activity, *and *qualifies > for simpler review* > 4) PR opener or reviewer 1 pings all Matplotlib devs > 5) If 2 weeks after pinging no other significant activity has occurred, PR > can be merged by PR opener or reviewer 1 > > Does everyone agree on this? The bits I've put in bold still need to be > defined; I don't have time now, but can provide a summary of these later. > > David > > > > On Thu, 17 Jan 2019 at 00:48, Antony Lee > wrote: > >> On Wed, Jan 16, 2019 at 7:36 PM David Stansby wrote: >> >>> I think real improvements not getting in at the rate they are submitted >>> is just how it works; if I started submitting a PR a day for the next 50 >>> days I would expect a good chunk of them sit there un-reviewed based on our >>> current rate of merging things, and that's fine. The developers as a whole >>> have a total reviewing rate, which is unpredictable and finite by the very >>> nature of this being an open-source project where no-one is obliged to do >>> anything. If we want to adjust reviewing rules to increase 'productivity' >>> that seems fine to me, but I don't think we should be doing it with the end >>> goal of being able to review at the rate that code is submitted (or any >>> particular other rate). >>> >> >> The end goal is not to set a specific rate, but it is indeed to try to >> raise that rate up. Obviously we could also throw our hands up and say >> nothing can be done about it, but that seems a bit defeatist. >> >> Right now, I would bet that a lot of PRs go in with exactly two reviewers >> having gone through them and no other reviewer having even skimmed them. >> If anything, I believe that the system I propose will increase the number >> of core-devs that'll have at least skimmed through the PR (all you need to >> do, if you want to participate, is to check your github notifications once >> every other week, whereas right now you'd actually need to pay attention to >> each PR as it is being submitted), and it is quite likely that 1 in-depth >> reviewer + 3-4 skim-throughs works as well, if not better, than 2 >> "in-depth" reviews (especially when the second one is sometimes "oh, that >> looks fine and it's already been approved once, let's just commit it"). >> >> >>> re. large diff PRs, I find 100+ line PRs that are just copy and paste >>> really hard to review - it's pretty dull for me trying to work out if code >>> has been copy/pasted exactly without any errors (if there's an easier way >>> to review stuff like this instead of checking every line please let me >>> know!). >>> >> >> Frankly, for a PR like that, I would just check that no function is >> missing and that everything is test-covered (without changes to the tests), >> and (in case you want to be sure) skim through the code to see that the PR >> author has not added a sneaky vulnerability that allows them to take over >> the user's computer... (In other words, I don't think a line-by-line check >> matters for such cases.) >> >> Antony >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From nelle.varoquaux at gmail.com Sun Jan 20 17:58:44 2019 From: nelle.varoquaux at gmail.com (Nelle Varoquaux) Date: Sun, 20 Jan 2019 14:58:44 -0800 Subject: [Matplotlib-devel] Students of ASPP contributing to Matplotlib Message-ID: Hello everyone, A bit of a last minute heads up? I'm at the advanced scientific programming in python summer school, and the instructor of the git & github lesson was looking for a package for students to do a small contribution (to see the workflow). As in 2016, I suggested small documentation changes to Matplotlib: we have selected 15 examples that could be improved, either in the title, either merging two examples that showcase similar, or just a better description. This means you might see 15 pull requests appear on the Matplotlib repository, by people who have experience in scientific Python, but no experience in contributing to open-source software. The students are pair-programming, so the authors of commits of each the pull request might not reflect an actual developer. The changes should be minimal, and I expect to be able to review and merge pull requests fairly quickly if the CI passes. If not, help in reviewing and merging is appreciated (but don't feel obliged to)! Thanks, Nelle -------------- next part -------------- An HTML attachment was scrubbed... URL: From pmhobson at gmail.com Mon Jan 21 11:58:10 2019 From: pmhobson at gmail.com (Paul Hobson) Date: Mon, 21 Jan 2019 08:58:10 -0800 Subject: [Matplotlib-devel] Students of ASPP contributing to Matplotlib In-Reply-To: References: Message-ID: Thanks for the heads up, Nelle. Sounds like a great class. -p On Sun, Jan 20, 2019 at 2:59 PM Nelle Varoquaux wrote: > Hello everyone, > > A bit of a last minute heads up? > I'm at the advanced scientific programming in python summer school, and > the instructor of the git & github lesson was looking for a package for > students to do a small contribution (to see the workflow). As in 2016, I > suggested small documentation changes to Matplotlib: we have selected 15 > examples that could be improved, either in the title, either merging two > examples that showcase similar, or just a better description. > This means you might see 15 pull requests appear on the Matplotlib > repository, by people who have experience in scientific Python, but no > experience in contributing to open-source software. The students are > pair-programming, so the authors of commits of each the pull request might > not reflect an actual developer. The changes should be minimal, and I > expect to be able to review and merge pull requests fairly quickly if the > CI passes. If not, help in reviewing and merging is appreciated (but don't > feel obliged to)! > > Thanks, > Nelle > _______________________________________________ > Matplotlib-devel mailing list > Matplotlib-devel at python.org > https://mail.python.org/mailman/listinfo/matplotlib-devel > -------------- next part -------------- An HTML attachment was scrubbed... URL: From nelle.varoquaux at gmail.com Mon Jan 21 15:43:38 2019 From: nelle.varoquaux at gmail.com (Nelle Varoquaux) Date: Tue, 22 Jan 2019 07:43:38 +1100 Subject: [Matplotlib-devel] Students of ASPP contributing to Matplotlib In-Reply-To: References: Message-ID: They didn't go far enough in the lesson. Considering how much we are hammering them with work during the rest of the week, I doubt that any of the students will continue working on the assignments. Thanks, N On Tue, Jan 22, 2019, 3:58 AM Paul Hobson Thanks for the heads up, Nelle. Sounds like a great class. > -p > > On Sun, Jan 20, 2019 at 2:59 PM Nelle Varoquaux > wrote: > >> Hello everyone, >> >> A bit of a last minute heads up? >> I'm at the advanced scientific programming in python summer school, and >> the instructor of the git & github lesson was looking for a package for >> students to do a small contribution (to see the workflow). As in 2016, I >> suggested small documentation changes to Matplotlib: we have selected 15 >> examples that could be improved, either in the title, either merging two >> examples that showcase similar, or just a better description. >> This means you might see 15 pull requests appear on the Matplotlib >> repository, by people who have experience in scientific Python, but no >> experience in contributing to open-source software. The students are >> pair-programming, so the authors of commits of each the pull request might >> not reflect an actual developer. The changes should be minimal, and I >> expect to be able to review and merge pull requests fairly quickly if the >> CI passes. If not, help in reviewing and merging is appreciated (but don't >> feel obliged to)! >> >> Thanks, >> Nelle >> _______________________________________________ >> Matplotlib-devel mailing list >> Matplotlib-devel at python.org >> https://mail.python.org/mailman/listinfo/matplotlib-devel >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From tcaswell at gmail.com Mon Jan 21 16:22:47 2019 From: tcaswell at gmail.com (Thomas Caswell) Date: Mon, 21 Jan 2019 16:22:47 -0500 Subject: [Matplotlib-devel] Summer stundents Message-ID: Folks, On this weeks call we discussed if we want to take summer students this year (either through GSOC or running the JDH summer fellowship again). Is anyone able and interested in mentoring this summer? Notes from the call: https://paper.dropbox.com/doc/Matplotlib-meeting-agenda--AWH9pSDlOSvrxXBVPR8mLuxUAg-aAmENlkgepgsMeDZtlsYu#:h2=GSOC-&-JDH Tom -- Thomas Caswell tcaswell at gmail.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From tcaswell at gmail.com Mon Jan 21 16:26:29 2019 From: tcaswell at gmail.com (Thomas Caswell) Date: Mon, 21 Jan 2019 16:26:29 -0500 Subject: [Matplotlib-devel] Documnetation summit Message-ID: Folks, We are planning to have a documentation summit in late spring in Boulder. The work product of this summit is a roadmap of what we want to do with the documentation going forward and a proposal to hire a technical writer to implement it. Who is interested in attending? We do have a small budget for travel. notes from call: https://paper.dropbox.com/doc/Matplotlib-meeting-agenda--AWH9pSDlOSvrxXBVPR8mLuxUAg-aAmENlkgepgsMeDZtlsYu#:h2=documentation-summit Tom -- Thomas Caswell tcaswell at gmail.com -------------- next part -------------- An HTML attachment was scrubbed... URL: