From ezio.melotti at gmail.com Thu Mar 12 19:12:01 2015 From: ezio.melotti at gmail.com (Ezio Melotti) Date: Thu, 12 Mar 2015 20:12:01 +0200 Subject: [core-workflow] bugs.python.org-related GSoC project Message-ID: Hi, Since the PSF is looking for ideas and mentors for Google Summer of Code, I was thinking about proposing and mentoring a project that aims at further improving our bug tracker. I'm writing to core-workflow in order to understand what features we want and which ones have higher priority, so that I can put together a coherent proposal. There are currently two PEPs that are proposing changes to our infrastructure: * PEP 462 - Core development workflow automation for CPython (https://www.python.org/dev/peps/pep-0462/) * PEP 474 - Creating forge.python.org (https://www.python.org/dev/peps/pep-0474/) These proposals will likely require changes to the bug tracker if we want to have a solid integration with the new tools and they might make for a good project, however it might still be too early to know what we will actually need. There are other changes that have been proposed in the past, in particular: * Some features previously discussed on this ML and summarized at https://wiki.python.org/moin/TrackerDevelopmentPlanning * Some other miscellaneous features listed at https://wiki.python.org/moin/DesiredTrackerFeatures * Better integration with Rietveld (e.g. add messages to roundup when someone posts a review) * Smarter branch detection in patches (sometimes patches don't get the review link) * Patch analysis (information such as affected files could be extracted from the files and used by Roundup) * Reviewing and applying patches submitted on the meta-tracker * Fix other issues listed on the meta-tracker These (or a subset of these) might also make for a good project, albeit less focused. After discussing with Nick on IRC about this, I also sent an email to roundup-devel about another possible project to be developed by Roundup under the umbrella of the PSF: adding a RESTful interface (see http://issues.roundup-tracker.org/issue2550734). This will also help us with any future work might want to do to integrate our bug tracker with other tools. If you have any feedback let me know. Best Regards, Ezio Melotti From rachel at trustrachel.com Thu Mar 12 23:49:29 2015 From: rachel at trustrachel.com (Rachel Sanders) Date: Thu, 12 Mar 2015 15:49:29 -0700 Subject: [core-workflow] bugs.python.org-related GSoC project In-Reply-To: References: Message-ID: That sounds great. Would you be interested in an apprentice mentor? I haven't done any python core dev work (yet) but I'm an engineer and I've mentored plenty of interns before and enjoy it. On Thu, Mar 12, 2015 at 11:12 AM, Ezio Melotti wrote: > Hi, > Since the PSF is looking for ideas and mentors for Google Summer of > Code, I was thinking about proposing and mentoring a project that aims > at further improving our bug tracker. > > I'm writing to core-workflow in order to understand what features we > want and which ones have higher priority, so that I can put together a > coherent proposal. > > There are currently two PEPs that are proposing changes to our > infrastructure: > * PEP 462 - Core development workflow automation for CPython > (https://www.python.org/dev/peps/pep-0462/) > * PEP 474 - Creating forge.python.org > (https://www.python.org/dev/peps/pep-0474/) > > These proposals will likely require changes to the bug tracker if we > want to have a solid integration with the new tools and they might > make for a good project, however it might still be too early to know > what we will actually need. > > There are other changes that have been proposed in the past, in particular: > * Some features previously discussed on this ML and summarized at > https://wiki.python.org/moin/TrackerDevelopmentPlanning > * Some other miscellaneous features listed at > https://wiki.python.org/moin/DesiredTrackerFeatures > * Better integration with Rietveld (e.g. add messages to roundup > when someone posts a review) > * Smarter branch detection in patches (sometimes patches don't get > the review link) > * Patch analysis (information such as affected files could be > extracted from the files and used by Roundup) > * Reviewing and applying patches submitted on the meta-tracker > * Fix other issues listed on the meta-tracker > > These (or a subset of these) might also make for a good project, > albeit less focused. > > After discussing with Nick on IRC about this, I also sent an email to > roundup-devel about another possible project to be developed by > Roundup under the umbrella of the PSF: adding a RESTful interface (see > http://issues.roundup-tracker.org/issue2550734). This will also help > us with any future work might want to do to integrate our bug tracker > with other tools. > > If you have any feedback let me know. > > Best Regards, > Ezio Melotti > _______________________________________________ > core-workflow mailing list > core-workflow at python.org > https://mail.python.org/mailman/listinfo/core-workflow > This list is governed by the PSF Code of Conduct: > https://www.python.org/psf/codeofconduct > -------------- next part -------------- An HTML attachment was scrubbed... URL: From taleinat at gmail.com Fri Mar 13 01:34:43 2015 From: taleinat at gmail.com (Tal Einat) Date: Fri, 13 Mar 2015 02:34:43 +0200 Subject: [core-workflow] bugs.python.org-related GSoC project In-Reply-To: References: Message-ID: +1 This seems like a good fit for GSoC, including with regard to documenting the work done and getting improvements integrated upstream. On Thu, Mar 12, 2015 at 8:12 PM, Ezio Melotti wrote: > Hi, > Since the PSF is looking for ideas and mentors for Google Summer of > Code, I was thinking about proposing and mentoring a project that aims > at further improving our bug tracker. > > I'm writing to core-workflow in order to understand what features we > want and which ones have higher priority, so that I can put together a > coherent proposal. > > There are currently two PEPs that are proposing changes to our > infrastructure: > * PEP 462 - Core development workflow automation for CPython > (https://www.python.org/dev/peps/pep-0462/) > * PEP 474 - Creating forge.python.org > (https://www.python.org/dev/peps/pep-0474/) > > These proposals will likely require changes to the bug tracker if we > want to have a solid integration with the new tools and they might > make for a good project, however it might still be too early to know > what we will actually need. > > There are other changes that have been proposed in the past, in particular: > * Some features previously discussed on this ML and summarized at > https://wiki.python.org/moin/TrackerDevelopmentPlanning > * Some other miscellaneous features listed at > https://wiki.python.org/moin/DesiredTrackerFeatures > * Better integration with Rietveld (e.g. add messages to roundup > when someone posts a review) > * Smarter branch detection in patches (sometimes patches don't get > the review link) > * Patch analysis (information such as affected files could be > extracted from the files and used by Roundup) > * Reviewing and applying patches submitted on the meta-tracker > * Fix other issues listed on the meta-tracker > > These (or a subset of these) might also make for a good project, > albeit less focused. > > After discussing with Nick on IRC about this, I also sent an email to > roundup-devel about another possible project to be developed by > Roundup under the umbrella of the PSF: adding a RESTful interface (see > http://issues.roundup-tracker.org/issue2550734). This will also help > us with any future work might want to do to integrate our bug tracker > with other tools. > > If you have any feedback let me know. > > Best Regards, > Ezio Melotti > _______________________________________________ > core-workflow mailing list > core-workflow at python.org > https://mail.python.org/mailman/listinfo/core-workflow > This list is governed by the PSF Code of Conduct: > https://www.python.org/psf/codeofconduct > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bcannon at gmail.com Fri Mar 13 13:08:41 2015 From: bcannon at gmail.com (Brett Cannon) Date: Fri, 13 Mar 2015 12:08:41 +0000 Subject: [core-workflow] bugs.python.org-related GSoC project In-Reply-To: References: Message-ID: Another idea would be dropping Rietveld for some other code review tool. Guido has mentioned we should probably switch off since our copy of Rietveld no longer tracks upstream. On Thu, Mar 12, 2015 at 2:12 PM Ezio Melotti wrote: > Hi, > Since the PSF is looking for ideas and mentors for Google Summer of > Code, I was thinking about proposing and mentoring a project that aims > at further improving our bug tracker. > > I'm writing to core-workflow in order to understand what features we > want and which ones have higher priority, so that I can put together a > coherent proposal. > > There are currently two PEPs that are proposing changes to our > infrastructure: > * PEP 462 - Core development workflow automation for CPython > (https://www.python.org/dev/peps/pep-0462/) > * PEP 474 - Creating forge.python.org > (https://www.python.org/dev/peps/pep-0474/) > > These proposals will likely require changes to the bug tracker if we > want to have a solid integration with the new tools and they might > make for a good project, however it might still be too early to know > what we will actually need. > > There are other changes that have been proposed in the past, in particular: > * Some features previously discussed on this ML and summarized at > https://wiki.python.org/moin/TrackerDevelopmentPlanning > * Some other miscellaneous features listed at > https://wiki.python.org/moin/DesiredTrackerFeatures > * Better integration with Rietveld (e.g. add messages to roundup > when someone posts a review) > * Smarter branch detection in patches (sometimes patches don't get > the review link) > * Patch analysis (information such as affected files could be > extracted from the files and used by Roundup) > * Reviewing and applying patches submitted on the meta-tracker > * Fix other issues listed on the meta-tracker > > These (or a subset of these) might also make for a good project, > albeit less focused. > > After discussing with Nick on IRC about this, I also sent an email to > roundup-devel about another possible project to be developed by > Roundup under the umbrella of the PSF: adding a RESTful interface (see > http://issues.roundup-tracker.org/issue2550734). This will also help > us with any future work might want to do to integrate our bug tracker > with other tools. > > If you have any feedback let me know. > > Best Regards, > Ezio Melotti > _______________________________________________ > core-workflow mailing list > core-workflow at python.org > https://mail.python.org/mailman/listinfo/core-workflow > This list is governed by the PSF Code of Conduct: > https://www.python.org/psf/codeofconduct > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ncoghlan at gmail.com Fri Mar 13 13:26:29 2015 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 13 Mar 2015 22:26:29 +1000 Subject: [core-workflow] bugs.python.org-related GSoC project In-Reply-To: References: Message-ID: On 13 March 2015 at 22:08, Brett Cannon wrote: > Another idea would be dropping Rietveld for some other code review tool. > Guido has mentioned we should probably switch off since our copy of Rietveld > no longer tracks upstream. That's probably not a good idea, given that both PEP 474 and PEP 481 suggest introducing new code review capable services as forge.python.org (Kallithea and Phabricator respectively) so regardless of how that competition turns out, there'll be a potential replacement for Rietveld incoming. Since both those PEPs suggest leaving the main CPython workflow alone for the time being, and there's nothing actually *broken* with the Rietveld integration, it could be worth pursuing some of the simpler changers Ezio suggested, like pinging the tracker when a review is filed, trying harder to find a base branch, or (one we discussed on IRC) better defining a workflow for generating patches directly from a BitBucket Mercurial clone could still be worthwhile. Sure, we're likely to stop using Rietveld in favour of the winner of the forge.python.org analysis at some point in the future, but that point is likely to be quite some time away where CPython is concerned. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From ezio.melotti at gmail.com Fri Mar 13 17:15:47 2015 From: ezio.melotti at gmail.com (Ezio Melotti) Date: Fri, 13 Mar 2015 18:15:47 +0200 Subject: [core-workflow] bugs.python.org-related GSoC project In-Reply-To: References: Message-ID: On Fri, Mar 13, 2015 at 2:26 PM, Nick Coghlan wrote: > On 13 March 2015 at 22:08, Brett Cannon wrote: >> Another idea would be dropping Rietveld for some other code review tool. >> Guido has mentioned we should probably switch off since our copy of Rietveld >> no longer tracks upstream. > > That's probably not a good idea, given that both PEP 474 and PEP 481 > suggest introducing new code review capable services as > forge.python.org (Kallithea and Phabricator respectively) so > regardless of how that competition turns out, there'll be a potential > replacement for Rietveld incoming. > If Rietveld is going to eventually be replaced, it might be better to focus the efforts on the bug tracker itself and avoid "large-scale" projects targeting Rietveld (e.g. updating it with upstream). > Since both those PEPs suggest leaving the main CPython workflow alone > for the time being, and there's nothing actually *broken* with the > Rietveld integration, it could be worth pursuing some of the simpler > changers Ezio suggested, like pinging the tracker when a review is > filed, trying harder to find a base branch, These simpler features could still be implemented and they are probably worth the effort, even if Rietveld will be gone in a couple of years. > or (one we discussed on > IRC) better defining a workflow for generating patches directly from a > BitBucket Mercurial clone could still be worthwhile. > Determining the desired workflow(s) is one of the reasons why I am writing this email. With the current workflow someone posts a patch on the bug tracker and after a discussion/review a core-dev commits and pushes it. Patch generation from a remote repo and patch review with Rietveld are also supported. If people desire more options, they could be added to the current workflow (e.g. a better integration with BitBucket, pull requests support, a Mercurial extension that allow contributors to post patches to b.p.o directly). AFAIU the introduction of Kallithea/Phabricator and possibly other tools will likely change our workflow: we might start using pull requests instead of/in addition to patches, use other review tools instead of Rietveld, and even commit patches directly from the bug tracker or other tools. Understanding in which direction we want to go will allow me to put together a project that, once completed, will have long-term benefits for our workflow. Perhaps I should post this to python-dev too and get feedback from a wider audience. > Sure, we're likely to stop using Rietveld in favour of the winner of > the forge.python.org analysis at some point in the future, but that > point is likely to be quite some time away where CPython is concerned. > Having a student investigating how Kallithea and Phabricator will interact with Roundup and start developing a proof-of-concept integration and/or tools that we already know will be needed might also be an idea. Best Regards, Ezio Melotti > Cheers, > Nick. > > -- > Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From ezio.melotti at gmail.com Fri Mar 13 17:19:38 2015 From: ezio.melotti at gmail.com (Ezio Melotti) Date: Fri, 13 Mar 2015 18:19:38 +0200 Subject: [core-workflow] bugs.python.org-related GSoC project In-Reply-To: References: Message-ID: On Fri, Mar 13, 2015 at 12:49 AM, Rachel Sanders wrote: > That sounds great. Would you be interested in an apprentice mentor? I > haven't done any python core dev work (yet) but I'm an engineer and I've > mentored plenty of interns before and enjoy it. > This is my first time mentoring for GSoC, so it might very well be that you already have more experience than me. In general I expect to interact with the student mainly on IRC (on #python-dev at freenode) and on the bug tracker. There other devs can also help and contribute, so if you are around your help will definitely be welcomed. You could also apply officially as a backup mentor, even though I'm not sure what this exactly entails. Best Regards, Ezio Melotti From bcannon at gmail.com Fri Mar 13 17:29:01 2015 From: bcannon at gmail.com (Brett Cannon) Date: Fri, 13 Mar 2015 16:29:01 +0000 Subject: [core-workflow] bugs.python.org-related GSoC project In-Reply-To: References: Message-ID: On Fri, Mar 13, 2015 at 12:15 PM Ezio Melotti wrote: > On Fri, Mar 13, 2015 at 2:26 PM, Nick Coghlan wrote: > > On 13 March 2015 at 22:08, Brett Cannon wrote: > >> Another idea would be dropping Rietveld for some other code review tool. > >> Guido has mentioned we should probably switch off since our copy of > Rietveld > >> no longer tracks upstream. > > > > That's probably not a good idea, given that both PEP 474 and PEP 481 > > suggest introducing new code review capable services as > > forge.python.org (Kallithea and Phabricator respectively) so > > regardless of how that competition turns out, there'll be a potential > > replacement for Rietveld incoming. > > > > If Rietveld is going to eventually be replaced, it might be better to > focus the efforts on the bug tracker itself and avoid "large-scale" > projects targeting Rietveld (e.g. updating it with upstream). > SGTM > > > Since both those PEPs suggest leaving the main CPython workflow alone > > for the time being, and there's nothing actually *broken* with the > > Rietveld integration, it could be worth pursuing some of the simpler > > changers Ezio suggested, like pinging the tracker when a review is > > filed, trying harder to find a base branch, > > These simpler features could still be implemented and they are > probably worth the effort, even if Rietveld will be gone in a couple > of years. > > > or (one we discussed on > > IRC) better defining a workflow for generating patches directly from a > > BitBucket Mercurial clone could still be worthwhile. > > > > Determining the desired workflow(s) is one of the reasons why I am > writing this email. > With the current workflow someone posts a patch on the bug tracker and > after a discussion/review a core-dev commits and pushes it. Patch > generation from a remote repo and patch review with Rietveld are also > supported. > If people desire more options, they could be added to the current > workflow (e.g. a better integration with BitBucket, pull requests > support, a Mercurial extension that allow contributors to post patches > to b.p.o directly). > As Nick said, current plans are to try workflows with ancillary repos first -- e.g. peps repo -- and then hopefully eventually graduate to helping with CPython. So making it easier to pull patches from Bitbucket and GitHub would be still be very useful. Same goes for the workflow cleanup that R. David Murray proposed when this was first started. > AFAIU the introduction of Kallithea/Phabricator and possibly other > tools will likely change our workflow: we might start using pull > requests instead of/in addition to patches, use other review tools > instead of Rietveld, and even commit patches directly from the bug > tracker or other tools. > Yes, but the workflow will bifurcate initially with CPython not changing and everything else shifting. That means making things easier for CPython today with its current workflow will still be beneficial. My areas of focus would be: * workflow simplification in the issue tracker like what R. David outlined previously * Push button patch generation from a GitHub repo * Some tool that will update a checkout (or somehow make sure a clone is clean for patching), grab a patch from an issue, apply it, run the test suite, and then ask if the committer wants to commit the patch and submit it (assuming everything else worked out in favour of committing the patch); essentially script what the fancy workflows being proposed using Phabricator/Kallithea do with the assumption the code was already reviewed in the issue tracker and deemed worthy of being committed > > Understanding in which direction we want to go will allow me to put > together a project that, once completed, will have long-term benefits > for our workflow. > Perhaps I should post this to python-dev too and get feedback from a > wider audience. > If you want, but I would assume everyone who cares is here at least for an initial discussion. > > > Sure, we're likely to stop using Rietveld in favour of the winner of > > the forge.python.org analysis at some point in the future, but that > > point is likely to be quite some time away where CPython is concerned. > > > > Having a student investigating how Kallithea and Phabricator will > interact with Roundup and start developing a proof-of-concept > integration and/or tools that we already know will be needed might > also be an idea. > > Yes, especially if I can make a decision fast enough to know which one to focus on. -Brett > Best Regards, > Ezio Melotti > > > Cheers, > > Nick. > > > > -- > > Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia > -------------- next part -------------- An HTML attachment was scrubbed... URL: From willingc at willingconsulting.com Fri Mar 13 19:39:18 2015 From: willingc at willingconsulting.com (Carol Willing) Date: Fri, 13 Mar 2015 11:39:18 -0700 Subject: [core-workflow] bugs.python.org-related GSoC project In-Reply-To: References: Message-ID: <55032ED6.7050400@willingconsulting.com> Hello Ezio and others, Sorry about the blank message. I'm happy to help informally for GSOC if needed. Rachel would be great as a supporting mentor (or whatever the correct wording is). Thoughts on projects and focus are in line. Thanks, Ezio, for mentoring :-) Carol On 13/03/2015 09:29, Brett Cannon wrote: > > > My areas of focus would be: > > * workflow simplification in the issue tracker like what R. David > outlined previously This one, while valuable, seems less focused for a student since likely there will be different opinions on the earlier outlined stuff by R. David. > * Push button patch generation from a GitHub repo This looks like a well bounded project for a student. > * Some tool that will update a checkout (or somehow make sure a clone > is clean for patching), grab a patch from an issue, apply it, run the > test suite, and then ask if the committer wants to commit the patch > and submit it (assuming everything else worked out in favour of > committing the patch); This looks like a good student project with valuable experience for the user. > essentially script what the fancy workflows being proposed using > Phabricator/Kallithea do with the assumption the code was already > reviewed in the issue tracker and deemed worthy of being committed > > Understanding in which direction we want to go will allow me to put > together a project that, once completed, will have long-term benefits > for our workflow. > Perhaps I should post this to python-dev too and get feedback from a > wider audience. > > > If you want, but I would assume everyone who cares is here at least > for an initial discussion. > > > > Sure, we're likely to stop using Rietveld in favour of the winner of > > the forge.python.org analysis at some > point in the future, but that > > point is likely to be quite some time away where CPython is > concerned. > > > > Having a student investigating how Kallithea and Phabricator will > interact with Roundup and start developing a proof-of-concept > integration and/or tools that we already know will be needed might > also be an idea. > > > Yes, especially if I can make a decision fast enough to know which one > to focus on. > > -Brett > > Best Regards, > Ezio Melotti > > > Cheers, > > Nick. > > > > -- > > Nick Coghlan | ncoghlan at gmail.com > | Brisbane, Australia > > > > _______________________________________________ > core-workflow mailing list > core-workflow at python.org > https://mail.python.org/mailman/listinfo/core-workflow > This list is governed by the PSF Code of Conduct: https://www.python.org/psf/codeofconduct -- *Carol Willing* Developer | Willing Consulting https://willingconsulting.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From ncoghlan at gmail.com Mon Mar 16 12:52:04 2015 From: ncoghlan at gmail.com (Nick Coghlan) Date: Mon, 16 Mar 2015 21:52:04 +1000 Subject: [core-workflow] bugs.python.org-related GSoC project In-Reply-To: <55032ED6.7050400@willingconsulting.com> References: <55032ED6.7050400@willingconsulting.com> Message-ID: On 14 March 2015 at 04:39, Carol Willing wrote: > * Push button patch generation from a GitHub repo > > This looks like a well bounded project for a student. It occurs to me also that this could potentially be done in a hosting service independent way by using Marcin Kuzminski's vcs module to actually generate the patch directly from the remote repo: https://pypi.python.org/pypi/vcs Added bonus: that library is the basis of the Git & Mercurial support in Rhodecode and Kallithea as well, so a patch generation utility based on it would potentially be useful for those projects as well. To avoid have to enter the full URL every time, we could potentially have something where a user could specify a CPython clone URL in their user preferences, and then prepopulate the VCS URL for patch generation based on that and a branch name based on the issue number like "issue12345". > * Some tool that will update a checkout (or somehow make sure a clone is > clean for patching), grab a patch from an issue, apply it, run the test > suite, and then ask if the committer wants to commit the patch and submit it > (assuming everything else worked out in favour of committing the patch); > > This looks like a good student project with valuable experience for the > user. Pierre-Yves David (one of the Mercurial devs) has been working on a Mercurial extension for that at https://bitbucket.org/ncoghlan/cpydev/src/default/cpyhg.py?at=default He's hoping to spend more time on it soon so folks will have something to try out at the PyCon sprints, so I wouldn't bet on this idea still being around as a candidate project by the time GSoC rolls around. > essentially script what the fancy workflows being proposed using > Phabricator/Kallithea do with the assumption the code was already reviewed > in the issue tracker and deemed worthy of being committed Yeah, our general inspiration for the idea is actually OpenStack's git/Gerrit review plugin: https://github.com/openstack-infra/git-review >> Understanding in which direction we want to go will allow me to put >> together a project that, once completed, will have long-term benefits >> for our workflow. >> Perhaps I should post this to python-dev too and get feedback from a >> wider audience. > > If you want, but I would assume everyone who cares is here at least for an > initial discussion. I've found one neat trick for this kind of scenario is to devise standalone projects that are likely to be useful regardless of what happens in the broader context. REST-API-in-upstream-Roundup qualifies, and I think a vcs based utility library for easily generating patches from remote repos would likely also be useful, independently of its integration into our Roundup instance. >> > Sure, we're likely to stop using Rietveld in favour of the winner of >> > the forge.python.org analysis at some point in the future, but that >> > point is likely to be quite some time away where CPython is concerned. >> > >> >> Having a student investigating how Kallithea and Phabricator will >> interact with Roundup and start developing a proof-of-concept >> integration and/or tools that we already know will be needed might >> also be an idea. >> > > Yes, especially if I can make a decision fast enough to know which one to > focus on. For the record, the reason it was fairly straightforward for me to wrangle some work time to spend on containerisation and related development workflow improvements for open source services like Kallithea was publicly announced last Friday: http://connect.redhat.com/zones/containers :) The relevant upstream bits are also there already (https://github.com/projectatomic/adb-atomic-developer-bundle) but there's still some work to do to on that front to integrate community Linux platforms in addition to RHEL. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From barry at python.org Mon Mar 16 16:36:11 2015 From: barry at python.org (Barry Warsaw) Date: Mon, 16 Mar 2015 11:36:11 -0400 Subject: [core-workflow] bugs.python.org-related GSoC project References: <55032ED6.7050400@willingconsulting.com> Message-ID: <20150316113611.63dbe625@anarchist.wooz.org> On Mar 16, 2015, at 09:52 PM, Nick Coghlan wrote: >I've found one neat trick for this kind of scenario is to devise >standalone projects that are likely to be useful regardless of what >happens in the broader context. REST-API-in-upstream-Roundup >qualifies As Nick knows, we've had great success with REST in Mailman 3. Having a REST API for Roundup would be very cool, and given our experience it feels GSoC-sized in effort, especially including the filling out of a robust API. Cheers, -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 819 bytes Desc: OpenPGP digital signature URL: From saimadhavheblikar at gmail.com Mon Mar 16 17:37:16 2015 From: saimadhavheblikar at gmail.com (Saimadhav Heblikar) Date: Mon, 16 Mar 2015 22:07:16 +0530 Subject: [core-workflow] bugs.python.org-related GSoC project In-Reply-To: <20150316113611.63dbe625@anarchist.wooz.org> References: <55032ED6.7050400@willingconsulting.com> <20150316113611.63dbe625@anarchist.wooz.org> Message-ID: On 16 March 2015 at 21:06, Barry Warsaw wrote: > On Mar 16, 2015, at 09:52 PM, Nick Coghlan wrote: > >>I've found one neat trick for this kind of scenario is to devise >>standalone projects that are likely to be useful regardless of what >>happens in the broader context. REST-API-in-upstream-Roundup >>qualifies > Hi, I am currently looking at the design of Roundup[1]. Would a standalone project as mentioned above be bugs.python.org roundup specific or would it target upstream? Which would serve *us* better? In either case, would someone familiar with bugs.python.org roundup point me to the differences between bugs.python.org from the upstream version?(if any) [1] - http://roundup.sourceforge.net/docs/design.html -- Regards Saimadhav Heblikar From ezio.melotti at gmail.com Mon Mar 16 18:27:35 2015 From: ezio.melotti at gmail.com (Ezio Melotti) Date: Mon, 16 Mar 2015 19:27:35 +0200 Subject: [core-workflow] bugs.python.org-related GSoC project In-Reply-To: References: <55032ED6.7050400@willingconsulting.com> <20150316113611.63dbe625@anarchist.wooz.org> Message-ID: On Mon, Mar 16, 2015 at 6:37 PM, Saimadhav Heblikar wrote: > On 16 March 2015 at 21:06, Barry Warsaw wrote: >> On Mar 16, 2015, at 09:52 PM, Nick Coghlan wrote: >> >>>I've found one neat trick for this kind of scenario is to devise >>>standalone projects that are likely to be useful regardless of what >>>happens in the broader context. REST-API-in-upstream-Roundup >>>qualifies >> > > Hi, > > I am currently looking at the design of Roundup[1]. Would a standalone > project as mentioned above be bugs.python.org roundup specific or > would it target upstream? Upstream. I sent a mail to roundup-devel asking if anyone wants to mentor the project but no one stepped forward. If you would like to tackle this project you could write there -- maybe someone will volunteer as a mentor once they see students are willing to work on it. > Which would serve *us* better? Once it's committed upstream we can easily pull it and use it right away. > In either case, would someone familiar with bugs.python.org roundup > point me to the differences between bugs.python.org from the upstream > version?(if any) Here is our Roundup clone: https://hg.python.org/tracker/roundup/ There are a few minor differences that are kept in a separate branch called "python-dev". Eventually these differences should either be ported upstream or removed altogether, but no one had time to do it yet. bugs.python.org is a specific instance of Roundup and the code is at https://hg.python.org/tracker/python-dev/ You can also find more info at https://wiki.python.org/moin/TrackerDevelopment Best Regards, Ezio Melotti > > [1] - http://roundup.sourceforge.net/docs/design.html > > > > -- > Regards > Saimadhav Heblikar > _______________________________________________ > core-workflow mailing list > core-workflow at python.org > https://mail.python.org/mailman/listinfo/core-workflow > This list is governed by the PSF Code of Conduct: https://www.python.org/psf/codeofconduct From rdmurray at bitdance.com Mon Mar 16 18:32:58 2015 From: rdmurray at bitdance.com (R. David Murray) Date: Mon, 16 Mar 2015 13:32:58 -0400 Subject: [core-workflow] bugs.python.org-related GSoC project In-Reply-To: References: <55032ED6.7050400@willingconsulting.com> <20150316113611.63dbe625@anarchist.wooz.org> Message-ID: <20150316173258.8FB45250ED7@webabinitio.net> On Mon, 16 Mar 2015 22:07:16 +0530, Saimadhav Heblikar wrote: > I am currently looking at the design of Roundup[1]. Would a standalone > project as mentioned above be bugs.python.org roundup specific or > would it target upstream? Which would serve *us* better? Most of what we have been talking about should target roundup. Following Nick's thoughts, even the patch generation could be made a Roundup option rather than CPython specific. > In either case, would someone familiar with bugs.python.org roundup > point me to the differences between bugs.python.org from the upstream > version?(if any) There shouldn't be any core Roundup differences that have not been also submitted upstream. The bugs.python.org *instance* is, of course, bugs.python.org specific. That's not a lot of code; though the open id and Rietveld support MVL added is non-trivial. The repo for the instance is at hg.python.org/tracker/python-dev. tracker/roundup is our copy of Roundup itself. --David From ezio.melotti at gmail.com Mon Mar 16 23:06:43 2015 From: ezio.melotti at gmail.com (Ezio Melotti) Date: Tue, 17 Mar 2015 00:06:43 +0200 Subject: [core-workflow] bugs.python.org-related GSoC project In-Reply-To: References: <55032ED6.7050400@willingconsulting.com> Message-ID: On Mon, Mar 16, 2015 at 1:52 PM, Nick Coghlan wrote: > On 14 March 2015 at 04:39, Carol Willing wrote: >> * Push button patch generation from a GitHub repo >> >> This looks like a well bounded project for a student. > I agree, however I think it could be completed in maybe a couple of weeks, so it is not enough to be the only goal of the project. For comparison, https://hg.python.org/tracker/python-dev/rev/59532d2c180b seems to be the equivalent code for Mercurial, so it's just a matter to do something similar for git. > It occurs to me also that this could potentially be done in a hosting > service independent way by using Marcin Kuzminski's vcs module to > actually generate the patch directly from the remote repo: > https://pypi.python.org/pypi/vcs > This is also an option -- it would mean that instead of writing a git equivalent of what I linked above, it would just need to be refactored using vcs. (FWIW the doc for vcs is at http://pythonhosted.org/vcs/quickstart.html) > Added bonus: that library is the basis of the Git & Mercurial support > in Rhodecode and Kallithea as well, so a patch generation utility > based on it would potentially be useful for those projects as well. > > To avoid have to enter the full URL every time, we could potentially > have something where a user could specify a CPython clone URL in their > user preferences, and then prepopulate the VCS URL for patch > generation based on that and a branch name based on the issue number > like "issue12345". > This can be done, but I would say it's lower priority. >> * Some tool that will update a checkout (or somehow make sure a clone is >> clean for patching), grab a patch from an issue, apply it, run the test >> suite, and then ask if the committer wants to commit the patch and submit it >> (assuming everything else worked out in favour of committing the patch); >> >> This looks like a good student project with valuable experience for the >> user. > > Pierre-Yves David (one of the Mercurial devs) has been working on a > Mercurial extension for that at > https://bitbucket.org/ncoghlan/cpydev/src/default/cpyhg.py?at=default > > He's hoping to spend more time on it soon so folks will have something > to try out at the PyCon sprints, so I wouldn't bet on this idea still > being around as a candidate project by the time GSoC rolls around. > I'm not sure if Brett was suggesting to do this on the client side (i.e. a tool used by committers on their machines) or something on the b.p.o side since both have been considered and discussed in the past. If it's aimed to committers/contributors (like the one Nick linked) then we have to see if people wants something similar. Personally I find importing a patch from the tracker (hg imp --no-c url_of_the_patch), running the tests (./python -m test), and committing it (hg ci -m "message") trivial, so I would have little use for this tool. I might find it more interesting if it allowed me to post patches to bpo without having to save a .diff and upload it manually, or if it had some kind of interaction with the other tools that we will use. If it is intended to be used on the server side, then it might be more interesting. The basic idea is that when a patch is submitted, the tracker will try to apply it on the branches specified on the "version" field, and then compile / run the tests / build the doc (depending on the content of the patch) and report back. IIUC this would be a simplified version of what Kallithea will do, so eventually it might end up being replace. >> essentially script what the fancy workflows being proposed using >> Phabricator/Kallithea do with the assumption the code was already reviewed >> in the issue tracker and deemed worthy of being committed > > Yeah, our general inspiration for the idea is actually OpenStack's > git/Gerrit review plugin: > https://github.com/openstack-infra/git-review > >>> Understanding in which direction we want to go will allow me to put >>> together a project that, once completed, will have long-term benefits >>> for our workflow. >>> Perhaps I should post this to python-dev too and get feedback from a >>> wider audience. >> >> If you want, but I would assume everyone who cares is here at least for an >> initial discussion. > > I've found one neat trick for this kind of scenario is to devise > standalone projects that are likely to be useful regardless of what > happens in the broader context. REST-API-in-upstream-Roundup > qualifies, and I think a vcs based utility library for easily > generating patches from remote repos would likely also be useful, > independently of its integration into our Roundup instance. > The problem with this is that creating a fully-featured general purpose tool is much harder. For example our Rietveld integration is far from being mature enough to be included upstream, but it has been serving us quite well for a few years now even though it's a half-baked implementation. Best Regards, Ezio Melotti >>> > Sure, we're likely to stop using Rietveld in favour of the winner of >>> > the forge.python.org analysis at some point in the future, but that >>> > point is likely to be quite some time away where CPython is concerned. >>> > >>> >>> Having a student investigating how Kallithea and Phabricator will >>> interact with Roundup and start developing a proof-of-concept >>> integration and/or tools that we already know will be needed might >>> also be an idea. >>> >> >> Yes, especially if I can make a decision fast enough to know which one to >> focus on. > > For the record, the reason it was fairly straightforward for me to > wrangle some work time to spend on containerisation and related > development workflow improvements for open source services like > Kallithea was publicly announced last Friday: > http://connect.redhat.com/zones/containers :) > > The relevant upstream bits are also there already > (https://github.com/projectatomic/adb-atomic-developer-bundle) but > there's still some work to do to on that front to integrate community > Linux platforms in addition to RHEL. > > Cheers, > Nick. > > -- > Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From bcannon at gmail.com Tue Mar 17 15:03:14 2015 From: bcannon at gmail.com (Brett Cannon) Date: Tue, 17 Mar 2015 14:03:14 +0000 Subject: [core-workflow] bugs.python.org-related GSoC project In-Reply-To: References: <55032ED6.7050400@willingconsulting.com> Message-ID: On Mon, Mar 16, 2015 at 6:06 PM Ezio Melotti wrote: > On Mon, Mar 16, 2015 at 1:52 PM, Nick Coghlan wrote: > > On 14 March 2015 at 04:39, Carol Willing > wrote: > >> * Push button patch generation from a GitHub repo > >> > >> This looks like a well bounded project for a student. > > > > I agree, however I think it could be completed in maybe a couple of > weeks, so it is not enough to be the only goal of the project. > For comparison, > https://hg.python.org/tracker/python-dev/rev/59532d2c180b seems to be > the equivalent code for Mercurial, so it's just a matter to do > something similar for git. > > > It occurs to me also that this could potentially be done in a hosting > > service independent way by using Marcin Kuzminski's vcs module to > > actually generate the patch directly from the remote repo: > > https://pypi.python.org/pypi/vcs > > > > This is also an option -- it would mean that instead of writing a git > equivalent of what I linked above, it would just need to be refactored > using vcs. > (FWIW the doc for vcs is at http://pythonhosted.org/vcs/quickstart.html) > > > Added bonus: that library is the basis of the Git & Mercurial support > > in Rhodecode and Kallithea as well, so a patch generation utility > > based on it would potentially be useful for those projects as well. > > > > To avoid have to enter the full URL every time, we could potentially > > have something where a user could specify a CPython clone URL in their > > user preferences, and then prepopulate the VCS URL for patch > > generation based on that and a branch name based on the issue number > > like "issue12345". > > > > This can be done, but I would say it's lower priority. > > >> * Some tool that will update a checkout (or somehow make sure a clone is > >> clean for patching), grab a patch from an issue, apply it, run the test > >> suite, and then ask if the committer wants to commit the patch and > submit it > >> (assuming everything else worked out in favour of committing the patch); > >> > >> This looks like a good student project with valuable experience for the > >> user. > > > > Pierre-Yves David (one of the Mercurial devs) has been working on a > > Mercurial extension for that at > > https://bitbucket.org/ncoghlan/cpydev/src/default/cpyhg.py?at=default > > > > He's hoping to spend more time on it soon so folks will have something > > to try out at the PyCon sprints, so I wouldn't bet on this idea still > > being around as a candidate project by the time GSoC rolls around. > > > > I'm not sure if Brett was suggesting to do this on the client side > (i.e. a tool used by committers on their machines) or something on the > b.p.o side since both have been considered and discussed in the past. > > If it's aimed to committers/contributors (like the one Nick linked) > then we have to see if people wants something similar. Personally I > find importing a patch from the tracker (hg imp --no-c > url_of_the_patch), running the tests (./python -m test), and > committing it (hg ci -m "message") trivial, so I would have little use > for this tool. I might find it more interesting if it allowed me to > post patches to bpo without having to save a .diff and upload it > manually, or if it had some kind of interaction with the other tools > that we will use. > This is what I meant as I was suggesting low-hanging fruit solutions that don't require significant tooling. It just seems like the next logical step to making patch committal that much simpler and faster. > > If it is intended to be used on the server side, then it might be more > interesting. The basic idea is that when a patch is submitted, the > tracker will try to apply it on the branches specified on the > "version" field, and then compile / run the tests / build the doc > (depending on the content of the patch) and report back. IIUC this > would be a simplified version of what Kallithea will do, so eventually > it might end up being replace. > I was not suggesting this since Nick's proposal as well as Donald's cover the server-side idea as it would essentially become a third proposal for push button patch submission. -Brett > > >> essentially script what the fancy workflows being proposed using > >> Phabricator/Kallithea do with the assumption the code was already > reviewed > >> in the issue tracker and deemed worthy of being committed > > > > Yeah, our general inspiration for the idea is actually OpenStack's > > git/Gerrit review plugin: > > https://github.com/openstack-infra/git-review > > > >>> Understanding in which direction we want to go will allow me to put > >>> together a project that, once completed, will have long-term benefits > >>> for our workflow. > >>> Perhaps I should post this to python-dev too and get feedback from a > >>> wider audience. > >> > >> If you want, but I would assume everyone who cares is here at least for > an > >> initial discussion. > > > > I've found one neat trick for this kind of scenario is to devise > > standalone projects that are likely to be useful regardless of what > > happens in the broader context. REST-API-in-upstream-Roundup > > qualifies, and I think a vcs based utility library for easily > > generating patches from remote repos would likely also be useful, > > independently of its integration into our Roundup instance. > > > > The problem with this is that creating a fully-featured general > purpose tool is much harder. For example our Rietveld integration is > far from being mature enough to be included upstream, but it has been > serving us quite well for a few years now even though it's a > half-baked implementation. > > Best Regards, > Ezio Melotti > > >>> > Sure, we're likely to stop using Rietveld in favour of the winner of > >>> > the forge.python.org analysis at some point in the future, but that > >>> > point is likely to be quite some time away where CPython is > concerned. > >>> > > >>> > >>> Having a student investigating how Kallithea and Phabricator will > >>> interact with Roundup and start developing a proof-of-concept > >>> integration and/or tools that we already know will be needed might > >>> also be an idea. > >>> > >> > >> Yes, especially if I can make a decision fast enough to know which one > to > >> focus on. > > > > For the record, the reason it was fairly straightforward for me to > > wrangle some work time to spend on containerisation and related > > development workflow improvements for open source services like > > Kallithea was publicly announced last Friday: > > http://connect.redhat.com/zones/containers :) > > > > The relevant upstream bits are also there already > > (https://github.com/projectatomic/adb-atomic-developer-bundle) but > > there's still some work to do to on that front to integrate community > > Linux platforms in addition to RHEL. > > > > Cheers, > > Nick. > > > > -- > > Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia > _______________________________________________ > core-workflow mailing list > core-workflow at python.org > https://mail.python.org/mailman/listinfo/core-workflow > This list is governed by the PSF Code of Conduct: > https://www.python.org/psf/codeofconduct > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ncoghlan at gmail.com Wed Mar 18 15:29:02 2015 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 19 Mar 2015 00:29:02 +1000 Subject: [core-workflow] bugs.python.org-related GSoC project In-Reply-To: References: <55032ED6.7050400@willingconsulting.com> Message-ID: On 18 March 2015 at 00:03, Brett Cannon wrote: > On Mon, Mar 16, 2015 at 6:06 PM Ezio Melotti wrote: >> I'm not sure if Brett was suggesting to do this on the client side >> (i.e. a tool used by committers on their machines) or something on the >> b.p.o side since both have been considered and discussed in the past. >> >> If it's aimed to committers/contributors (like the one Nick linked) >> then we have to see if people wants something similar. Personally I >> find importing a patch from the tracker (hg imp --no-c >> url_of_the_patch), running the tests (./python -m test), and >> committing it (hg ci -m "message") trivial, so I would have little use >> for this tool. I might find it more interesting if it allowed me to >> post patches to bpo without having to save a .diff and upload it >> manually, or if it had some kind of interaction with the other tools >> that we will use. > > > This is what I meant as I was suggesting low-hanging fruit solutions that > don't require significant tooling. It just seems like the next logical step > to making patch committal that much simpler and faster. Right, the idea would be to have a relatively prescriptive extension where we can say "Don't know Mercurial? Just do this". Gerrit, GitHub and OpenStack's gitreview offer that kind of experience for using git - you don't need to learn all of git, just stay within the bounds of the defined workflow for the project/service. Learning to use Mercurial independently of the extension would still be a good long term idea, this would just give us a way to get helpers into the hands of folks that are either new to Mercurial in general, or just new to our workflow in particular. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From ncoghlan at gmail.com Wed Mar 18 15:37:49 2015 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 19 Mar 2015 00:37:49 +1000 Subject: [core-workflow] bugs.python.org-related GSoC project In-Reply-To: <20150316113611.63dbe625@anarchist.wooz.org> References: <55032ED6.7050400@willingconsulting.com> <20150316113611.63dbe625@anarchist.wooz.org> Message-ID: On 17 March 2015 at 01:36, Barry Warsaw wrote: > On Mar 16, 2015, at 09:52 PM, Nick Coghlan wrote: > >>I've found one neat trick for this kind of scenario is to devise >>standalone projects that are likely to be useful regardless of what >>happens in the broader context. REST-API-in-upstream-Roundup >>qualifies > > As Nick knows, we've had great success with REST in Mailman 3. Having a REST > API for Roundup would be very cool, and given our experience it feels > GSoC-sized in effort, especially including the filling out of a robust API. I actually need to take an internal email I wrote regarding the definition of the overall REST API design for beaker-project.org and turn it into a public blog post. I believe the reason "REST backend, JavaScript client front end" works so well is that it lets you have a single "implementation model" (the REST web service) that exposes the raw data model in a form that's easy for *developers* to work with, while supporting multiple "representation models" that different groups of users interact with (your JavaScript UX, command line tools, UI elements in other web services): http://uxoslo.com/2014/01/14/ux-hints-why-mental-models-matter/ In the case of Roundup, the data model is actually very REST friendly, as the existing XML-RPC interface already embodies the "collections of resources" approach. In theory, it should "just" be a matter of exposing those collections through an appropriate set of APIs (and figuring out things like access management, etc). Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From barry at python.org Wed Mar 18 17:35:57 2015 From: barry at python.org (Barry Warsaw) Date: Wed, 18 Mar 2015 12:35:57 -0400 Subject: [core-workflow] bugs.python.org-related GSoC project References: <55032ED6.7050400@willingconsulting.com> <20150316113611.63dbe625@anarchist.wooz.org> Message-ID: <20150318123557.64765a83@anarchist.wooz.org> On Mar 19, 2015, at 12:37 AM, Nick Coghlan wrote: >In the case of Roundup, the data model is actually very REST friendly, >as the existing XML-RPC interface already embodies the "collections of >resources" approach. In theory, it should "just" be a matter of >exposing those collections through an appropriate set of APIs (and >figuring out things like access management, etc). This is probably getting off-topic, but MM3 took the approach that the core engine's REST API deliberately doesn't do access management. We call it an "administrative API" and only run it on localhost (configurable of course, but we tell admins never to run it on a public IP). This opens up a wide range of very interesting possibilities. Our web ui is written entirely against the API so while you could use the one we'll release, you're not tied to it at all. Another interesting piece is the public REST proxy, which *will* be available on a public IP. This is where access management is implemented. It uses information from the core, but has its own model for restricting and controlling access. This means it can be implemented independently, deployed or not deployed independently, and even completely replaces by downstream integrators. Anyway, we're really happy with the way this architecture has turned out. Cheers, -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 819 bytes Desc: OpenPGP digital signature URL: From northlomax at gmail.com Fri Mar 20 18:12:33 2015 From: northlomax at gmail.com (=?UTF-8?B?SsO4cm4gTG9tYXg=?=) Date: Fri, 20 Mar 2015 18:12:33 +0100 Subject: [core-workflow] GSoC idea: bug.python.org improvements Message-ID: <550C5501.4050105@gmail.com> Hi I'm a potential GSoC participant, and I'm interested in the bug tracker improvement idea[1]. I had a brief look at what has already been discussed on the mailing list and liked what I read. I don't know anything about the code-review tools mentioned (Kallithea and Phabricator), but I would be up to investigating how they could be integrated. But I really like the idea of integrating a REST API to roundup, but I'm not sure if that would be a python-core project or a distinct roundup project (requireing mentors from the Roundup devs)? But I'm very interested in doing a project with the python core team, so if there are any other interesting ideas I'm definalty up to it. I should perhaps ask around on the python-dev ML? [1]https://wiki.python.org/moin/SummerOfCode/2015/python-core From ezio.melotti at gmail.com Sat Mar 21 00:23:17 2015 From: ezio.melotti at gmail.com (Ezio Melotti) Date: Sat, 21 Mar 2015 01:23:17 +0200 Subject: [core-workflow] GSoC idea: bug.python.org improvements In-Reply-To: <550C5501.4050105@gmail.com> References: <550C5501.4050105@gmail.com> Message-ID: On Fri, Mar 20, 2015 at 7:12 PM, J?rn Lomax wrote: > Hi > > I'm a potential GSoC participant, and I'm interested in the bug tracker > improvement idea[1]. I had a brief look at what has already been discussed > on the mailing list and liked what I read. > > I don't know anything about the code-review tools mentioned (Kallithea and > Phabricator), but I would be up to investigating how they could be > integrated. But I really like the idea of integrating a REST API to roundup, > but I'm not sure if that would be a python-core project or a distinct > roundup project (requireing mentors from the Roundup devs)? > Adding a REST API to Roundup would be a separate project, and it's currently being discussed on the roundup-devel ML (http://sourceforge.net/p/roundup/mailman/roundup-devel/). No one volunteered to mentor the project yet, but maybe someone will step forward if they see interested students. Best Regards, Ezio Melotti > But I'm very interested in doing a project with the python core team, so if > there are any other interesting ideas I'm definalty up to it. I should > perhaps ask around on the python-dev ML? > > [1]https://wiki.python.org/moin/SummerOfCode/2015/python-core From northlomax at gmail.com Sat Mar 21 08:29:14 2015 From: northlomax at gmail.com (=?UTF-8?B?SsO4cm4gTG9tYXg=?=) Date: Sat, 21 Mar 2015 08:29:14 +0100 Subject: [core-workflow] GSoC idea: bug.python.org improvements In-Reply-To: References: <550C5501.4050105@gmail.com> Message-ID: <550D1DCA.80207@gmail.com> On 21.03.2015 00:23, Ezio Melotti wrote: > On Fri, Mar 20, 2015 at 7:12 PM, J?rn Lomax wrote: >> Hi >> >> I'm a potential GSoC participant, and I'm interested in the bug tracker >> improvement idea[1]. I had a brief look at what has already been discussed >> on the mailing list and liked what I read. >> >> I don't know anything about the code-review tools mentioned (Kallithea and >> Phabricator), but I would be up to investigating how they could be >> integrated. But I really like the idea of integrating a REST API to roundup, >> but I'm not sure if that would be a python-core project or a distinct >> roundup project (requireing mentors from the Roundup devs)? >> > Adding a REST API to Roundup would be a separate project, and it's > currently being discussed on the roundup-devel ML > (http://sourceforge.net/p/roundup/mailman/roundup-devel/). > No one volunteered to mentor the project yet, but maybe someone will > step forward if they see interested students. > > Best Regards, > Ezio Melotti > >> But I'm very interested in doing a project with the python core team, so if >> there are any other interesting ideas I'm definalty up to it. I should >> perhaps ask around on the python-dev ML? >> >> [1]https://wiki.python.org/moin/SummerOfCode/2015/python-core So is there any of the ideas discussed for bug.python.org you think might be a viable GSoC project? From ncoghlan at gmail.com Sat Mar 21 14:13:13 2015 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sat, 21 Mar 2015 23:13:13 +1000 Subject: [core-workflow] GSoC idea: bug.python.org improvements In-Reply-To: <550D1DCA.80207@gmail.com> References: <550C5501.4050105@gmail.com> <550D1DCA.80207@gmail.com> Message-ID: On 21 March 2015 at 17:29, J?rn Lomax wrote: > > > On 21.03.2015 00:23, Ezio Melotti wrote: >> >> On Fri, Mar 20, 2015 at 7:12 PM, J?rn Lomax wrote: >>> >>> Hi >>> >>> I'm a potential GSoC participant, and I'm interested in the bug tracker >>> improvement idea[1]. I had a brief look at what has already been >>> discussed >>> on the mailing list and liked what I read. >>> >>> I don't know anything about the code-review tools mentioned (Kallithea >>> and >>> Phabricator), but I would be up to investigating how they could be >>> integrated. But I really like the idea of integrating a REST API to >>> roundup, >>> but I'm not sure if that would be a python-core project or a distinct >>> roundup project (requireing mentors from the Roundup devs)? >>> >> Adding a REST API to Roundup would be a separate project, and it's >> currently being discussed on the roundup-devel ML >> (http://sourceforge.net/p/roundup/mailman/roundup-devel/). >> No one volunteered to mentor the project yet, but maybe someone will >> step forward if they see interested students. >> >> Best Regards, >> Ezio Melotti >> >>> But I'm very interested in doing a project with the python core team, so >>> if >>> there are any other interesting ideas I'm definalty up to it. I should >>> perhaps ask around on the python-dev ML? >>> >>> [1]https://wiki.python.org/moin/SummerOfCode/2015/python-core > > So is there any of the ideas discussed for bug.python.org you think might be > a viable GSoC project? I don't think we've discussed it anywhere yet (unless I mentioned it to Ezio on IRC), but there are some issues around dependency display that could conceivably be handled downstream. The main one is showing which bugs a given bug is *blocking* - that is, those which depend on the bug you're currently looking at. Another nice-to-have from my perspective would be the ability to add a new comment without having to scroll back to the top of the discussion. Unfortunately, I suspect those would also qualify as being projects best tackled under the Roundup banner, and I don't think it would be fair to ask a GSoC student to do that without a mentor that was already part of the upstream Roundup community that could facilitate getting their patches to a point where they were ready to be merged. Regards, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From rdmurray at bitdance.com Sat Mar 21 20:33:39 2015 From: rdmurray at bitdance.com (R. David Murray) Date: Sat, 21 Mar 2015 15:33:39 -0400 Subject: [core-workflow] GSoC idea: bug.python.org improvements In-Reply-To: References: <550C5501.4050105@gmail.com> <550D1DCA.80207@gmail.com> Message-ID: <20150321193339.B120DB1415E@webabinitio.net> On Sat, 21 Mar 2015 23:13:13 +1000, Nick Coghlan wrote: > I don't think we've discussed it anywhere yet (unless I mentioned it > to Ezio on IRC), but there are some issues around dependency display > that could conceivably be handled downstream. The main one is showing > which bugs a given bug is *blocking* - that is, those which depend on > the bug you're currently looking at. > > Another nice-to-have from my perspective would be the ability to add a > new comment without having to scroll back to the top of the > discussion. > > Unfortunately, I suspect those would also qualify as being projects > best tackled under the Roundup banner, and I don't think it would be Nope. Both of those are instance-only modifications, not core Roundup. The 'dependencies' field doesn't even exist in the stock roundup instance template. Comment box at the bottom has been previously suggested and was rejected in favor of hotkeys that let you scroll back and forth easily. I think that decision could be revisited, though. Both of these are "small" modifications, so you'd need a bunch of additional items to make it a GSoC project. And, you need a mentor...that really seems to be the sticking point right now. I myself will simply have no time this summer to do it, otherwise I would. --David From ezio.melotti at gmail.com Sun Mar 22 05:11:46 2015 From: ezio.melotti at gmail.com (Ezio Melotti) Date: Sun, 22 Mar 2015 06:11:46 +0200 Subject: [core-workflow] GSoC idea: bug.python.org improvements In-Reply-To: <20150321193339.B120DB1415E@webabinitio.net> References: <550C5501.4050105@gmail.com> <550D1DCA.80207@gmail.com> <20150321193339.B120DB1415E@webabinitio.net> Message-ID: On Sat, Mar 21, 2015 at 9:33 PM, R. David Murray wrote: > On Sat, 21 Mar 2015 23:13:13 +1000, Nick Coghlan wrote: >> I don't think we've discussed it anywhere yet (unless I mentioned it >> to Ezio on IRC), but there are some issues around dependency display >> that could conceivably be handled downstream. The main one is showing >> which bugs a given bug is *blocking* - that is, those which depend on >> the bug you're currently looking at. >> >> Another nice-to-have from my perspective would be the ability to add a >> new comment without having to scroll back to the top of the >> discussion. >> >> Unfortunately, I suspect those would also qualify as being projects >> best tackled under the Roundup banner, and I don't think it would be > > Nope. Both of those are instance-only modifications, not core Roundup. > The 'dependencies' field doesn't even exist in the stock roundup > instance template. > > Comment box at the bottom has been previously suggested and was > rejected in favor of hotkeys that let you scroll back and forth easily. > I think that decision could be revisited, though. > FTR, you can press 'r' (for reply) to jump to the reply box at the top, 'esc' to unfocus it, and 'l' (lowercase L, for last) to jump to the last message. There are other shortcuts that you can see by pressing '?' or by clicking 'Keyb. shortcuts (?)' at the end of the sidebar. Note that these shortcuts only work in the issues pages. > Both of these are "small" modifications, so you'd need a bunch > of additional items to make it a GSoC project. And, you need a > mentor...that really seems to be the sticking point right now. > I myself will simply have no time this summer to do it, otherwise > I would. > I've been talking with a few people and this seems to be the situation right now: * There is the bugs.python.org project that I suggested and that I'm willing to mentor, and some students already showed interest (so I would advise them to also have a backup project in case they don't get accepted for this); * since there are several things that can be fixed/improved and none of them seems to have an particularly high priority, I think they can be decided together with the students (different students might prefer to work on different ideas in different order, and that shouldn't matter as long as they do enough work). * There is the Roundup project about adding a REST API that has been proposed upstream, however no Roundup mentor has stepped forward; * since some of the Roundup devs said they would be available to help, one option would be to find a core Python mentor for this project (Nick?) and then the student can still interact with the Roundup guys (including me) and get help from other devs as well. Best Regards, Ezio Melotti > --David From ncoghlan at gmail.com Sun Mar 22 08:50:56 2015 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sun, 22 Mar 2015 17:50:56 +1000 Subject: [core-workflow] GSoC idea: bug.python.org improvements In-Reply-To: References: <550C5501.4050105@gmail.com> <550D1DCA.80207@gmail.com> <20150321193339.B120DB1415E@webabinitio.net> Message-ID: On 22 March 2015 at 14:11, Ezio Melotti wrote: > On Sat, Mar 21, 2015 at 9:33 PM, R. David Murray wrote: >> On Sat, 21 Mar 2015 23:13:13 +1000, Nick Coghlan wrote: >>> I don't think we've discussed it anywhere yet (unless I mentioned it >>> to Ezio on IRC), but there are some issues around dependency display >>> that could conceivably be handled downstream. The main one is showing >>> which bugs a given bug is *blocking* - that is, those which depend on >>> the bug you're currently looking at. >>> >>> Another nice-to-have from my perspective would be the ability to add a >>> new comment without having to scroll back to the top of the >>> discussion. >>> >>> Unfortunately, I suspect those would also qualify as being projects >>> best tackled under the Roundup banner, and I don't think it would be >> >> Nope. Both of those are instance-only modifications, not core Roundup. >> The 'dependencies' field doesn't even exist in the stock roundup >> instance template. >> >> Comment box at the bottom has been previously suggested and was >> rejected in favor of hotkeys that let you scroll back and forth easily. >> I think that decision could be revisited, though. >> > > FTR, you can press 'r' (for reply) to jump to the reply box at the > top, 'esc' to unfocus it, and 'l' (lowercase L, for last) to jump to > the last message. > There are other shortcuts that you can see by pressing '?' or by > clicking 'Keyb. shortcuts (?)' at the end of the sidebar. Note that > these shortcuts only work in the issues pages. I almost never use hotkeys beyond Ctrl-C/Ctrl-X/Ctrl-V - if something doesn't have a GUI element associated with it, it generally doesn't exist as far as I'm concerned :) I did just remember another small item though - hooking up searching by Real Name in addition to Username when adding someone to the nosy list (not everyone uses the first.last naming convention for their username, especially if they have an already established username they're bringing over from other services) > * since some of the Roundup devs said they would be available to > help, one option would be to find a core Python mentor for this > project (Nick?) and then the student can still interact with the > Roundup guys (including me) and get help from other devs as well. I'm already trying to figure out how to gracefully hand at least some of my current projects over to other people, so I won't be volunteering for anything new any time soon :) Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From northlomax at gmail.com Sun Mar 22 13:02:24 2015 From: northlomax at gmail.com (=?UTF-8?B?SsO4cm4gTG9tYXg=?=) Date: Sun, 22 Mar 2015 13:02:24 +0100 Subject: [core-workflow] GSoC idea: bug.python.org improvements In-Reply-To: References: <550C5501.4050105@gmail.com> <550D1DCA.80207@gmail.com> <20150321193339.B120DB1415E@webabinitio.net> Message-ID: <550EAF50.6080006@gmail.com> On 22.03.2015 08:50, Nick Coghlan wrote: > On 22 March 2015 at 14:11, Ezio Melotti wrote: >> On Sat, Mar 21, 2015 at 9:33 PM, R. David Murray wrote: >>> On Sat, 21 Mar 2015 23:13:13 +1000, Nick Coghlan wrote: >>>> I don't think we've discussed it anywhere yet (unless I mentioned it >>>> to Ezio on IRC), but there are some issues around dependency display >>>> that could conceivably be handled downstream. The main one is showing >>>> which bugs a given bug is *blocking* - that is, those which depend on >>>> the bug you're currently looking at. >>>> >>>> Another nice-to-have from my perspective would be the ability to add a >>>> new comment without having to scroll back to the top of the >>>> discussion. >>>> >>>> Unfortunately, I suspect those would also qualify as being projects >>>> best tackled under the Roundup banner, and I don't think it would be >>> Nope. Both of those are instance-only modifications, not core Roundup. >>> The 'dependencies' field doesn't even exist in the stock roundup >>> instance template. >>> >>> Comment box at the bottom has been previously suggested and was >>> rejected in favor of hotkeys that let you scroll back and forth easily. >>> I think that decision could be revisited, though. >>> >> FTR, you can press 'r' (for reply) to jump to the reply box at the >> top, 'esc' to unfocus it, and 'l' (lowercase L, for last) to jump to >> the last message. >> There are other shortcuts that you can see by pressing '?' or by >> clicking 'Keyb. shortcuts (?)' at the end of the sidebar. Note that >> these shortcuts only work in the issues pages. > I almost never use hotkeys beyond Ctrl-C/Ctrl-X/Ctrl-V - if something > doesn't have a GUI element associated with it, it generally doesn't > exist as far as I'm concerned :) > > I did just remember another small item though - hooking up searching > by Real Name in addition to Username when adding someone to the nosy > list (not everyone uses the first.last naming convention for their > username, especially if they have an already established username > they're bringing over from other services) > >> * since some of the Roundup devs said they would be available to >> help, one option would be to find a core Python mentor for this >> project (Nick?) and then the student can still interact with the >> Roundup guys (including me) and get help from other devs as well. > I'm already trying to figure out how to gracefully hand at least some > of my current projects over to other people, so I won't be > volunteering for anything new any time soon :) > > Cheers, > Nick. > So as a student, if I'm writing a proposal to the project I should just mark it as bug tracker improvements and fixes? It just seem a little hard to point to very specific goals to complete, and the project would gain more goals and task goes by. I don't have any problem with it, I'm a very play by ear person so that works for me, I just don't want the project proposal to look to thin because not all the ideas are fleshed out yet Regards, J?rn Lomax From ezio.melotti at gmail.com Sun Mar 22 14:19:53 2015 From: ezio.melotti at gmail.com (Ezio Melotti) Date: Sun, 22 Mar 2015 15:19:53 +0200 Subject: [core-workflow] GSoC idea: bug.python.org improvements In-Reply-To: <550EAF50.6080006@gmail.com> References: <550C5501.4050105@gmail.com> <550D1DCA.80207@gmail.com> <20150321193339.B120DB1415E@webabinitio.net> <550EAF50.6080006@gmail.com> Message-ID: On Sun, Mar 22, 2015 at 2:02 PM, J?rn Lomax wrote: > > So as a student, if I'm writing a proposal to the project I should just > mark it as bug tracker improvements and fixes? Yes. > It just seem a little hard to > point to very specific goals to complete, and the project would gain more > goals and task goes by. I don't have any problem with it, I'm a very play > by ear person so that works for me, I just don't want the project proposal > to look to thin because not all the ideas are fleshed out yet > I'm planning to make a list of these ideas as soon as I have some time, and let students pick a substantial-enough subsets of ideas for their proposals. Best Regards, Ezio Melotti > Regards, > J?rn Lomax > From ezio.melotti at gmail.com Sun Mar 22 14:16:03 2015 From: ezio.melotti at gmail.com (Ezio Melotti) Date: Sun, 22 Mar 2015 15:16:03 +0200 Subject: [core-workflow] GSoC idea: bug.python.org improvements In-Reply-To: References: <550C5501.4050105@gmail.com> <550D1DCA.80207@gmail.com> <20150321193339.B120DB1415E@webabinitio.net> Message-ID: On Sun, Mar 22, 2015 at 9:50 AM, Nick Coghlan wrote: > On 22 March 2015 at 14:11, Ezio Melotti wrote: >> On Sat, Mar 21, 2015 at 9:33 PM, R. David Murray wrote: >>> On Sat, 21 Mar 2015 23:13:13 +1000, Nick Coghlan wrote: >>>> I don't think we've discussed it anywhere yet (unless I mentioned it >>>> to Ezio on IRC), but there are some issues around dependency display >>>> that could conceivably be handled downstream. The main one is showing >>>> which bugs a given bug is *blocking* - that is, those which depend on >>>> the bug you're currently looking at. >>>> >>>> Another nice-to-have from my perspective would be the ability to add a >>>> new comment without having to scroll back to the top of the >>>> discussion. >>>> >>>> Unfortunately, I suspect those would also qualify as being projects >>>> best tackled under the Roundup banner, and I don't think it would be >>> >>> Nope. Both of those are instance-only modifications, not core Roundup. >>> The 'dependencies' field doesn't even exist in the stock roundup >>> instance template. >>> >>> Comment box at the bottom has been previously suggested and was >>> rejected in favor of hotkeys that let you scroll back and forth easily. >>> I think that decision could be revisited, though. >>> >> >> FTR, you can press 'r' (for reply) to jump to the reply box at the >> top, 'esc' to unfocus it, and 'l' (lowercase L, for last) to jump to >> the last message. >> There are other shortcuts that you can see by pressing '?' or by >> clicking 'Keyb. shortcuts (?)' at the end of the sidebar. Note that >> these shortcuts only work in the issues pages. > > I almost never use hotkeys beyond Ctrl-C/Ctrl-X/Ctrl-V - if something > doesn't have a GUI element associated with it, it generally doesn't > exist as far as I'm concerned :) > > I did just remember another small item though - hooking up searching > by Real Name in addition to Username when adding someone to the nosy > list (not everyone uses the first.last naming convention for their > username, especially if they have an already established username > they're bringing over from other services) > That should already work, but the autocomplete only includes people in the expert list and committers. >> * since some of the Roundup devs said they would be available to >> help, one option would be to find a core Python mentor for this >> project (Nick?) and then the student can still interact with the >> Roundup guys (including me) and get help from other devs as well. > > I'm already trying to figure out how to gracefully hand at least some > of my current projects over to other people, so I won't be > volunteering for anything new any time soon :) > If you figure that out let me know, or else I'll have to take the cloning route again :) > Cheers, > Nick. > > -- > Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From techtonik at gmail.com Mon Mar 23 08:10:50 2015 From: techtonik at gmail.com (anatoly techtonik) Date: Mon, 23 Mar 2015 10:10:50 +0300 Subject: [core-workflow] bugs.python.org-related GSoC project In-Reply-To: References: Message-ID: On Thu, Mar 12, 2015 at 9:12 PM, Ezio Melotti wrote: > Hi, > Since the PSF is looking for ideas and mentors for Google Summer of > Code, I was thinking about proposing and mentoring a project that aims > at further improving our bug tracker. > > I'm writing to core-workflow in order to understand what features we > want and which ones have higher priority, so that I can put together a > coherent proposal. I have a quite coherent vision about that, and my opinion is that creating a flexible system around working Python infrastructure is not less important than implementing the stuff as a project. > There are currently two PEPs that are proposing changes to our infrastructure: > * PEP 462 - Core development workflow automation for CPython > (https://www.python.org/dev/peps/pep-0462/) > * PEP 474 - Creating forge.python.org > (https://www.python.org/dev/peps/pep-0474/) If there is an opportunity to have a person who will be dedicating his/her full time to integrate the latest news in software development practices from academia, then instead of going with waterfall model of: [ ask questions ] -> [ get a plan ] -> [ build a site ] <--------------------- GSoC 2015 ---------------------> https://en.wikipedia.org/wiki/Waterfall_model#Criticism Let a person try alternative development methodology and try it over a few rounds with flexibility that allows to change the result in the middle. [check]->[design]->[build]->[check]->[design]->[build]->[check]->... <-------------------------- GSoC 2015 --------------------------> > These proposals will likely require changes to the bug tracker if we > want to have a solid integration with the new tools and they might > make for a good project, however it might still be too early to know > what we will actually need. Yes, it is really hard to know what is needed without trying, so planning exact changes beforehand may lead to a system that will not be used as much. > There are other changes that have been proposed in the past, in particular: > * Some features previously discussed on this ML and summarized at > https://wiki.python.org/moin/TrackerDevelopmentPlanning > * Some other miscellaneous features listed at > https://wiki.python.org/moin/DesiredTrackerFeatures > * Better integration with Rietveld (e.g. add messages to roundup > when someone posts a review) > * Smarter branch detection in patches (sometimes patches don't get > the review link) > * Patch analysis (information such as affected files could be > extracted from the files and used by Roundup) Proof of concept is done, needs transforming pydotorg a bit to automate maintenance process and integrate with Roundup UI. https://bitbucket.org/techtonik/python-stdlib > * Reviewing and applying patches submitted on the meta-tracker > * Fix other issues listed on the meta-tracker > > These (or a subset of these) might also make for a good project, > albeit less focused. > > After discussing with Nick on IRC about this, I also sent an email to > roundup-devel about another possible project to be developed by > Roundup under the umbrella of the PSF: adding a RESTful interface (see > http://issues.roundup-tracker.org/issue2550734). This will also help > us with any future work might want to do to integrate our bug tracker > with other tools. > > If you have any feedback let me know. Right now we have the problem that Roundup UI work barrier is to high due to the usage of TAL that is not as known and popular as Jinja2/Django style templates (researching this could be a great opportunity to get an analyst position in any top company). And then there is a big encoding problem with Roundup and Jinja2 (which is already there). Jinja2 uses unicode internally, Roundup stores data in DB in "utf-8", but Python still uses "ascii" internally, so when internationalized "utf-8" input goes from web to Roundup and (without saving to DB) to Jinja2, the Jinja2 chokes. For me the obvious solution (after meditating over it for some weeks) is to release Python 2.8 with "utf-8" strings, or enable "per module encoding" in it or I don't know. Because Armin said that it it impossible to hack Jinja2. https://stackoverflow.com/questions/28642781/hack-jinja2-to-encode-from-utf-8-instead-of-ascii So, solving that problem alone could be a useful outcome of GSoC project. -- anatoly t. -------------- next part -------------- An HTML attachment was scrubbed... URL: From techtonik at gmail.com Mon Mar 23 08:13:24 2015 From: techtonik at gmail.com (anatoly techtonik) Date: Mon, 23 Mar 2015 10:13:24 +0300 Subject: [core-workflow] bugs.python.org-related GSoC project In-Reply-To: References: Message-ID: On Fri, Mar 13, 2015 at 3:08 PM, Brett Cannon wrote: > Another idea would be dropping Rietveld for some other code review tool. > Guido has mentioned we should probably switch off since our copy of > Rietveld no longer tracks upstream. > Developing an interface to integrate bug trackers with code review tools. That will be helpful for other users of Roundup as well. Current integration is pydotorg only and there should be some analysis to know why and how to make it more useful for other people. -------------- next part -------------- An HTML attachment was scrubbed... URL: From techtonik at gmail.com Mon Mar 23 13:06:23 2015 From: techtonik at gmail.com (anatoly techtonik) Date: Mon, 23 Mar 2015 15:06:23 +0300 Subject: [core-workflow] web API to get a list of all module in stdlib Message-ID: Hi, I am doing an exercise as a part of agile ux data mining team, and I need to get a list of Python modules: https://stackoverflow.com/questions/6463918/how-can-i-get-a-list-of-all-the-python-standard-library-modules But this gives only the modules that were compiled into specific interpreter, and I need a list of modules that are de-facto included in stdlib standard. I also need this for all Python versions, and be able to fetch it as csv, json or html table format over webm so that result of my work could be validated and experiment repeated as necessary. I see the data as the necessary step to organize a work around "externally evolving standard library", so a way to query it should be somewhat sustainable and obvious. It might be possible to generate something from docs, like: https://docs.python.org/2.7.2/dataset/modules.json This way you get static information without ability to version or refresh the info (still good to have anyway to compare docs and other sources). Or it may be a dedicated URL: https://api.python.org/2.7.2/stdlib/modules/ The result is HTML be default. ?format=csv - result is csv ?format=yaml I need in particular: - module name - files that comprise module sources - os supported So, basically I need an official support for this: https://bitbucket.org/techtonik/python-stdlib/src/092af75da07cb264070115fb9a970e27b1e57f72/stdlib.json?at=default Because I don't have means to maintain this myself and feel tired trying to think about how it can be maintained from outside. If I have this mapping, I can make a diagram how many patches per module are sitting there on the tracker, and it may open a can of worms for many other fishy stats that will be attractive for people to work on. Actually, the code that sorts patches by modules is already there in that repository. It is also unlicensed to get it free from restrictions placed by copyright law over distributed development, so it doesn't require me or you to sign CLA to further develop it. So, where is the first class info about the module structure of stdlib? Where this info should be fetched from if accessed automatically from the web? How it should be kept up to date for all Python versions? -- anatoly t. From ezio.melotti at gmail.com Mon Mar 23 16:04:58 2015 From: ezio.melotti at gmail.com (Ezio Melotti) Date: Mon, 23 Mar 2015 17:04:58 +0200 Subject: [core-workflow] bugs.python.org-related GSoC project In-Reply-To: References: <55032ED6.7050400@willingconsulting.com> <20150316113611.63dbe625@anarchist.wooz.org> Message-ID: Hi, On Wed, Mar 18, 2015 at 4:37 PM, Nick Coghlan wrote: > On 17 March 2015 at 01:36, Barry Warsaw wrote: >> On Mar 16, 2015, at 09:52 PM, Nick Coghlan wrote: >> >>>I've found one neat trick for this kind of scenario is to devise >>>standalone projects that are likely to be useful regardless of what >>>happens in the broader context. REST-API-in-upstream-Roundup >>>qualifies >> >> As Nick knows, we've had great success with REST in Mailman 3. Having a REST >> API for Roundup would be very cool, and given our experience it feels >> GSoC-sized in effort, especially including the filling out of a robust API. > > I actually need to take an internal email I wrote regarding the > definition of the overall REST API design for beaker-project.org and > turn it into a public blog post. I believe the reason "REST backend, > JavaScript client front end" works so well is that it lets you have a > single "implementation model" (the REST web service) that exposes the > raw data model in a form that's easy for *developers* to work with, > while supporting multiple "representation models" that different > groups of users interact with (your JavaScript UX, command line tools, > UI elements in other web services): > http://uxoslo.com/2014/01/14/ux-hints-why-mental-models-matter/ > > In the case of Roundup, the data model is actually very REST friendly, > as the existing XML-RPC interface already embodies the "collections of > resources" approach. In theory, it should "just" be a matter of > exposing those collections through an appropriate set of APIs (and > figuring out things like access management, etc). > Just a quick update: 1) I updated the wiki page [0] and added some of the things we discussed here to the project; 2) I couldn't find any Roundup dev willing to mentor the REST API project, however I might have found someone else, so I added the project to the wiki as well [0]. The idea is to work on bugs.python.org first, and once the API is stable we can think about porting it upstream. I would encourage the students to start submitting their proposals for the projects ASAP, since the deadline is approaching (the 27th of March). Best Regards, Ezio Melotti [0]: https://wiki.python.org/moin/SummerOfCode/2015/python-core > Cheers, > Nick. > > -- > Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia > _______________________________________________ > core-workflow mailing list > core-workflow at python.org > https://mail.python.org/mailman/listinfo/core-workflow > This list is governed by the PSF Code of Conduct: https://www.python.org/psf/codeofconduct From northlomax at gmail.com Mon Mar 23 16:38:28 2015 From: northlomax at gmail.com (=?windows-1252?Q?J=F8rn_Lomax?=) Date: Mon, 23 Mar 2015 16:38:28 +0100 Subject: [core-workflow] bugs.python.org-related GSoC project In-Reply-To: References: <55032ED6.7050400@willingconsulting.com> <20150316113611.63dbe625@anarchist.wooz.org> Message-ID: <55103374.6060904@gmail.com> On 23.03.2015 16:04, Ezio Melotti wrote: > Hi, > > On Wed, Mar 18, 2015 at 4:37 PM, Nick Coghlan wrote: >> On 17 March 2015 at 01:36, Barry Warsaw wrote: >>> On Mar 16, 2015, at 09:52 PM, Nick Coghlan wrote: >>> >>>> I've found one neat trick for this kind of scenario is to devise >>>> standalone projects that are likely to be useful regardless of what >>>> happens in the broader context. REST-API-in-upstream-Roundup >>>> qualifies >>> As Nick knows, we've had great success with REST in Mailman 3. Having a REST >>> API for Roundup would be very cool, and given our experience it feels >>> GSoC-sized in effort, especially including the filling out of a robust API. >> I actually need to take an internal email I wrote regarding the >> definition of the overall REST API design for beaker-project.org and >> turn it into a public blog post. I believe the reason "REST backend, >> JavaScript client front end" works so well is that it lets you have a >> single "implementation model" (the REST web service) that exposes the >> raw data model in a form that's easy for *developers* to work with, >> while supporting multiple "representation models" that different >> groups of users interact with (your JavaScript UX, command line tools, >> UI elements in other web services): >> http://uxoslo.com/2014/01/14/ux-hints-why-mental-models-matter/ >> >> In the case of Roundup, the data model is actually very REST friendly, >> as the existing XML-RPC interface already embodies the "collections of >> resources" approach. In theory, it should "just" be a matter of >> exposing those collections through an appropriate set of APIs (and >> figuring out things like access management, etc). >> > Just a quick update: > 1) I updated the wiki page [0] and added some of the things we > discussed here to the project; > 2) I couldn't find any Roundup dev willing to mentor the REST API > project, however I might have found someone else, so I added the > project to the wiki as well [0]. The idea is to work on > bugs.python.org first, and once the API is stable we can think about > porting it upstream. > > I would encourage the students to start submitting their proposals for > the projects ASAP, since the deadline is approaching (the 27th of > March). > > Best Regards, > Ezio Melotti > > [0]: https://wiki.python.org/moin/SummerOfCode/2015/python-core > >> Cheers, >> Nick. >> >> -- >> Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia >> _______________________________________________ >> core-workflow mailing list >> core-workflow at python.org >> https://mail.python.org/mailman/listinfo/core-workflow >> This list is governed by the PSF Code of Conduct: https://www.python.org/psf/codeofconduct > _______________________________________________ > core-workflow mailing list > core-workflow at python.org > https://mail.python.org/mailman/listinfo/core-workflow > This list is governed by the PSF Code of Conduct: https://www.python.org/psf/codeofconduct I have submitted one proposal for this project[1]. The text is not final and is still being updated. I'll add some of the ideas from the ML later today. I'm happy to take any comments to help improve the proposal. I might create an proposal for adding a REST API to Rondup too, but I'll have to see how much free time I have on my hands this week. regards, J?rn Lomax [1]https://www.google-melange.com/gsoc/proposal/review/student/google/gsoc2015/lurimax/5657382461898752 From ezio.melotti at gmail.com Mon Mar 23 17:06:21 2015 From: ezio.melotti at gmail.com (Ezio Melotti) Date: Mon, 23 Mar 2015 18:06:21 +0200 Subject: [core-workflow] bugs.python.org-related GSoC project In-Reply-To: <55103374.6060904@gmail.com> References: <55032ED6.7050400@willingconsulting.com> <20150316113611.63dbe625@anarchist.wooz.org> <55103374.6060904@gmail.com> Message-ID: Hi, On Mon, Mar 23, 2015 at 5:38 PM, J?rn Lomax wrote: > > I have submitted one proposal for this project[1]. The text is not final and > is still being updated. I'll add some of the ideas from the ML later today. > I'm happy to take any comments to help improve the proposal. > I left a comment on melange. > I might create an proposal for adding a REST API to Rondup too, but I'll > have to see how much free time I have on my hands this week. > Please do, even if it's incomplete I believe you can update it after the deadline, as long as you submitted it before. Best Regards, Ezio Melotti > regards, > J?rn Lomax > > [1]https://www.google-melange.com/gsoc/proposal/review/student/google/gsoc2015/lurimax/5657382461898752 > From northlomax at gmail.com Mon Mar 23 19:19:07 2015 From: northlomax at gmail.com (=?UTF-8?B?SsO4cm4gTG9tYXg=?=) Date: Mon, 23 Mar 2015 19:19:07 +0100 Subject: [core-workflow] bugs.python.org-related GSoC project In-Reply-To: References: <55032ED6.7050400@willingconsulting.com> <20150316113611.63dbe625@anarchist.wooz.org> <55103374.6060904@gmail.com> Message-ID: <5510591B.5050101@gmail.com> On 23.03.2015 17:06, Ezio Melotti wrote: > Hi, > > On Mon, Mar 23, 2015 at 5:38 PM, J?rn Lomax wrote: >> I have submitted one proposal for this project[1]. The text is not final and >> is still being updated. I'll add some of the ideas from the ML later today. >> I'm happy to take any comments to help improve the proposal. >> > I left a comment on melange. I can't seem to find the comment >> I might create an proposal for adding a REST API to Rondup too, but I'll >> have to see how much free time I have on my hands this week. >> > Please do, even if it's incomplete I believe you can update it after > the deadline, as long as you submitted it before. I wrote a very simple proposal[1]. Again, it's not complete and I will be fleshing it more out during the week > Best Regards, > Ezio Melotti > >> regards, >> J?rn Lomax >> >> [1]https://www.google-melange.com/gsoc/proposal/review/student/google/gsoc2015/lurimax/5657382461898752 >> From northlomax at gmail.com Mon Mar 23 19:23:19 2015 From: northlomax at gmail.com (=?UTF-8?B?SsO4cm4gTG9tYXg=?=) Date: Mon, 23 Mar 2015 19:23:19 +0100 Subject: [core-workflow] bugs.python.org-related GSoC project In-Reply-To: <5510591B.5050101@gmail.com> References: <55032ED6.7050400@willingconsulting.com> <20150316113611.63dbe625@anarchist.wooz.org> <55103374.6060904@gmail.com> <5510591B.5050101@gmail.com> Message-ID: <55105A17.50606@gmail.com> On 23.03.2015 19:19, J?rn Lomax wrote: > > > On 23.03.2015 17:06, Ezio Melotti wrote: >> Hi, >> >> On Mon, Mar 23, 2015 at 5:38 PM, J?rn Lomax >> wrote: >>> I have submitted one proposal for this project[1]. The text is not >>> final and >>> is still being updated. I'll add some of the ideas from the ML later >>> today. >>> I'm happy to take any comments to help improve the proposal. >>> >> I left a comment on melange. > I can't seem to find the comment >>> I might create an proposal for adding a REST API to Rondup too, but >>> I'll >>> have to see how much free time I have on my hands this week. >>> >> Please do, even if it's incomplete I believe you can update it after >> the deadline, as long as you submitted it before. > > I wrote a very simple proposal[1]. Again, it's not complete and I will > be fleshing it more out during the week >> Best Regards, >> Ezio Melotti >> >>> regards, >>> J?rn Lomax >>> >>> [1]https://www.google-melange.com/gsoc/proposal/review/student/google/gsoc2015/lurimax/5657382461898752 >>> >>> > now complete with link [1]https://www.google-melange.com/gsoc/proposal/edit/google/gsoc2015/lurimax/5693417237512192?validated=True# From ezio.melotti at gmail.com Mon Mar 23 19:40:26 2015 From: ezio.melotti at gmail.com (Ezio Melotti) Date: Mon, 23 Mar 2015 20:40:26 +0200 Subject: [core-workflow] bugs.python.org-related GSoC project In-Reply-To: <5510591B.5050101@gmail.com> References: <55032ED6.7050400@willingconsulting.com> <20150316113611.63dbe625@anarchist.wooz.org> <55103374.6060904@gmail.com> <5510591B.5050101@gmail.com> Message-ID: On Mon, Mar 23, 2015 at 8:19 PM, J?rn Lomax wrote: > > > On 23.03.2015 17:06, Ezio Melotti wrote: >> >> Hi, >> >> On Mon, Mar 23, 2015 at 5:38 PM, J?rn Lomax wrote: >>> >>> I have submitted one proposal for this project[1]. The text is not final >>> and >>> is still being updated. I'll add some of the ideas from the ML later >>> today. >>> I'm happy to take any comments to help improve the proposal. >>> >> I left a comment on melange. > > I can't seem to find the comment Sorry, that was my fault -- I accidentally set the comment as private. I now resubmitted it and you should be able to see it. >>> >>> I might create an proposal for adding a REST API to Rondup too, but I'll >>> have to see how much free time I have on my hands this week. >>> >> Please do, even if it's incomplete I believe you can update it after >> the deadline, as long as you submitted it before. > > > I wrote a very simple proposal[1]. Again, it's not complete and I will be > fleshing it more out during the week > That's OK, I will wait for updates before adding further comments. Best Regards, Ezio Melotti >> Best Regards, >> Ezio Melotti >> >>> regards, >>> J?rn Lomax >>> >>> >>> [1]https://www.google-melange.com/gsoc/proposal/review/student/google/gsoc2015/lurimax/5657382461898752 >>> > From pierre-yves.david at ens-lyon.org Thu Mar 26 01:03:53 2015 From: pierre-yves.david at ens-lyon.org (Pierre-Yves David) Date: Wed, 25 Mar 2015 17:03:53 -0700 Subject: [core-workflow] bugs.python.org-related GSoC project In-Reply-To: References: <55032ED6.7050400@willingconsulting.com> Message-ID: <55134CE9.40408@ens-lyon.org> On 03/16/2015 03:06 PM, Ezio Melotti wrote: >> Pierre-Yves David (one of the Mercurial devs) has been working on a >> >Mercurial extension for that at >> >https://bitbucket.org/ncoghlan/cpydev/src/default/cpyhg.py?at=default >> > >> >He's hoping to spend more time on it soon so folks will have something >> >to try out at the PyCon sprints, so I wouldn't bet on this idea still >> >being around as a candidate project by the time GSoC rolls around. >> > > I'm not sure if Brett was suggesting to do this on the client side > (i.e. a tool used by committers on their machines) or something on the > b.p.o side since both have been considered and discussed in the past. > > If it's aimed to committers/contributors (like the one Nick linked) > then we have to see if people wants something similar. Personally I > find importing a patch from the tracker (hg imp --no-c > url_of_the_patch), running the tests (./python -m test), and > committing it (hg ci -m "message") trivial, so I would have little use > for this tool. I might find it more interesting if it allowed me to > post patches to bpo without having to save a .diff and upload it > manually, or if it had some kind of interaction with the other tools > that we will use. The idea here is to have a very simple extension talking to roundup and allowing: - apply latest patch from an issue `hg cpy-get ` - upload patches to an issue `hg cpy-put ` (I've not strong opinion on command names) The first one (get) is really easy to do and will reduce the overhead of looking at a patch. The second one is not hard either as long as we have the appropriate API roundup side. For about a year, I've been using one line command to fetch and submit series of patches for the Mercurial project itself and it is really convenient. I gave a small look at the tool again this monday but got blocked by permission issue on the current API (did not spend too much time to look at it) Cheers, -- Pierre-Yves David From berker.peksag at gmail.com Thu Mar 26 01:28:18 2015 From: berker.peksag at gmail.com (=?UTF-8?Q?Berker_Peksa=C4=9F?=) Date: Thu, 26 Mar 2015 02:28:18 +0200 Subject: [core-workflow] bugs.python.org-related GSoC project In-Reply-To: <55134CE9.40408@ens-lyon.org> References: <55032ED6.7050400@willingconsulting.com> <55134CE9.40408@ens-lyon.org> Message-ID: On Thu, Mar 26, 2015 at 2:03 AM, Pierre-Yves David wrote: > The idea here is to have a very simple extension talking to roundup and > allowing: > > - apply latest patch from an issue > > `hg cpy-get ` > > - upload patches to an issue > > `hg cpy-put ` > > (I've not strong opinion on command names) > > The first one (get) is really easy to do and will reduce the overhead of > looking at a patch. The second one is not hard either as long as we have the > appropriate API roundup side. I was managed to upload a patch using Roundup's XMLRPC API to Roundup a while ago. --Berker From i at introo.me Thu Mar 26 01:54:45 2015 From: i at introo.me (Shiyao Ma) Date: Wed, 25 Mar 2015 20:54:45 -0400 Subject: [core-workflow] bugs.python.org-related GSoC project In-Reply-To: <55134CE9.40408@ens-lyon.org> References: <55032ED6.7050400@willingconsulting.com> <55134CE9.40408@ens-lyon.org> Message-ID: On Wed, Mar 25, 2015 at 8:03 PM, Pierre-Yves David wrote: > I gave a small look at the tool again this monday but got blocked by > permission issue on the current API (did not spend too much time to look at > it) What permission issue are you experiencing? I just had a look on the xml-rpc http://roundup.sourceforge.net/docs/xmlrpc.html AFAIK, the security issue stands out. SSL is not supported by default, so some proxying should be done. Meanwhile, it would be better if it could support RSA (or alike) for authentication. Either storing the password in plaintext locally or typing it everytime when needed is a little laborious. (Though we could use gnome-keyring or osx keychain, but it's another topic). The b.p.o is also under HTTP not HTTPS, might be another issue on the agenda. Regards. -- ????????????????http://introo.me? From ncoghlan at gmail.com Thu Mar 26 08:10:20 2015 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 26 Mar 2015 17:10:20 +1000 Subject: [core-workflow] GSoC idea: bug.python.org improvements In-Reply-To: References: <550C5501.4050105@gmail.com> <550D1DCA.80207@gmail.com> <20150321193339.B120DB1415E@webabinitio.net> Message-ID: On 22 March 2015 at 23:16, Ezio Melotti wrote: > On Sun, Mar 22, 2015 at 9:50 AM, Nick Coghlan wrote: >> I did just remember another small item though - hooking up searching >> by Real Name in addition to Username when adding someone to the nosy >> list (not everyone uses the first.last naming convention for their >> username, especially if they have an already established username >> they're bringing over from other services) >> > > That should already work, but the autocomplete only includes people in > the expert list and committers. Aye, that part is great, the specific part I'm referring to is the search dialogue that pops up when you click on the "(list)" link. I also wondered whether there might be a few enhancements that could be gathered under a "Improved mentoring support" theme, like being able to check how many patches someone has uploaded vs how many of the related issues have been resolved, or helping mentors keep a record of who's patches they've accepted recently. At the moment we can track commits fairly well, but there isn't really much of a link back to the uploaded patches outside the NEWS file and commit message, which automated tools tend not to read. (We could also look at using "hg commit -u" more to credit patch authors in the metadata, and something https://bitbucket.org/ede/committer/src/default/hgcommitter.py to track the committer info. On the other hand, it may not be worth doing that given the expected future work on forge.python.org based workflows, regardless of whether that's Kallithea or Phabricator based) >>> * since some of the Roundup devs said they would be available to >>> help, one option would be to find a core Python mentor for this >>> project (Nick?) and then the student can still interact with the >>> Roundup guys (including me) and get help from other devs as well. >> >> I'm already trying to figure out how to gracefully hand at least some >> of my current projects over to other people, so I won't be >> volunteering for anything new any time soon :) > > If you figure that out let me know, or else I'll have to take the > cloning route again :) Taking into account the many things I'd like to eventually see fixed upstream when figuring out what I want to do with my professional career helps, but I can certainly see the attraction of the cloning option :) Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From ezio.melotti at gmail.com Thu Mar 26 10:05:37 2015 From: ezio.melotti at gmail.com (Ezio Melotti) Date: Thu, 26 Mar 2015 11:05:37 +0200 Subject: [core-workflow] GSoC idea: bug.python.org improvements In-Reply-To: References: <550C5501.4050105@gmail.com> <550D1DCA.80207@gmail.com> <20150321193339.B120DB1415E@webabinitio.net> Message-ID: On Thu, Mar 26, 2015 at 9:10 AM, Nick Coghlan wrote: > On 22 March 2015 at 23:16, Ezio Melotti wrote: >> On Sun, Mar 22, 2015 at 9:50 AM, Nick Coghlan wrote: >>> I did just remember another small item though - hooking up searching >>> by Real Name in addition to Username when adding someone to the nosy >>> list (not everyone uses the first.last naming convention for their >>> username, especially if they have an already established username >>> they're bringing over from other services) >>> >> >> That should already work, but the autocomplete only includes people in >> the expert list and committers. > > Aye, that part is great, the specific part I'm referring to is the > search dialogue that pops up when you click on the "(list)" link. > I haven't really used that since I added the autocomplete -- I even considered removing it before forgetting about its existence. Once the REST API for Roundup is in place, these things can easily be improved (and new things can also be added -- e.g. searching related issues as you type the title of a new one). > I also wondered whether there might be a few enhancements that could > be gathered under a "Improved mentoring support" theme, like being > able to check how many patches someone has uploaded vs how many of the > related issues have been resolved, or helping mentors keep a record of > who's patches they've accepted recently. > This will likely require two changes: 1) add more metadata in the db (possibly determined automatically by Roundup/HG/Kallithea/etc.); 2) retrieving and presenting them to the user (this will be easier once the REST API is there). FWIW there are already plans about analyzing patches and adding to the DB the list of affected files and possibly other fields to signal if the patch contains tests and docs. > At the moment we can track commits fairly well, but there isn't really > much of a link back to the uploaded patches outside the NEWS file and > commit message, which automated tools tend not to read. (We could also > look at using "hg commit -u" more to credit patch authors in the > metadata, and something > https://bitbucket.org/ede/committer/src/default/hgcommitter.py to > track the committer info. On the other hand, it may not be worth doing > that given the expected future work on forge.python.org based > workflows, regardless of whether that's Kallithea or Phabricator > based) > This is actually something that I wanted to bring up on python-dev. Currently our workflow is mostly patch-based, but adding support for pull requests from BitBucket/GitHub is one of the things that we are considering adding during GSoC. In addition, the GSoC students will work on a separate clone of the tracker, likely hosted on BitBucket. While we could still use the patch-based approach for GSoC, this would be a good chance to experiment with a more modern approach and also test on the meta-tracker both the new workflow and the code that will enable it. If successful, the same approach can also be adopted for CPython. One of the main issues is, as you mentioned, how to track both the committer and the author. This is both a technical and a "philosophical" issue -- that's why I wanted to bring it up on python-dev. Another issue is establishing a policy regarding branches and rebases (rebasements?). These issues might eventually be solved by Kallithea/Phabricator, but I expect a transition period where different workflows will be used. I would like our repo to be still intact by the time we settle on the new workflow. IIUC hgcommitter adds extra metadata to the changeset, and if we go down this route we might also consider adding metadata for the issue number and tweak hgweb to display both. Best Regards, Ezio Melotti >>>> * since some of the Roundup devs said they would be available to >>>> help, one option would be to find a core Python mentor for this >>>> project (Nick?) and then the student can still interact with the >>>> Roundup guys (including me) and get help from other devs as well. >>> >>> I'm already trying to figure out how to gracefully hand at least some >>> of my current projects over to other people, so I won't be >>> volunteering for anything new any time soon :) >> >> If you figure that out let me know, or else I'll have to take the >> cloning route again :) > > Taking into account the many things I'd like to eventually see fixed > upstream when figuring out what I want to do with my professional > career helps, but I can certainly see the attraction of the cloning > option :) > > Cheers, > Nick. > > -- > Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From ncoghlan at gmail.com Thu Mar 26 11:37:46 2015 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 26 Mar 2015 20:37:46 +1000 Subject: [core-workflow] GSoC idea: bug.python.org improvements In-Reply-To: References: <550C5501.4050105@gmail.com> <550D1DCA.80207@gmail.com> <20150321193339.B120DB1415E@webabinitio.net> Message-ID: On 26 March 2015 at 19:05, Ezio Melotti wrote: > This is actually something that I wanted to bring up on python-dev. > Currently our workflow is mostly patch-based, but adding support for > pull requests from BitBucket/GitHub is one of the things that we are > considering adding during GSoC. In addition, the GSoC students will > work on a separate clone of the tracker, likely hosted on BitBucket. > While we could still use the patch-based approach for GSoC, this would > be a good chance to experiment with a more modern approach and also > test on the meta-tracker both the new workflow and the code that will > enable it. If successful, the same approach can also be adopted for > CPython. I managed to forget about the idea of allowing roundup patch generation direct from GitHub/BitBucket, so in that case it definitely makes sense to pursue some of this for the current bugs.python.org workflow. It especially makes sense as both forge.python.org proposal target the support repos first (devguide, peps, etc), so regardless of which of those moves forward, it's going to be a long time before it impacts the CPython workflows. By contrast, improvements to Roundup's integration with other services can start being helpful as soon as they get deployed. > One of the main issues is, as you mentioned, how to track both the > committer and the author. This is both a technical and a > "philosophical" issue -- that's why I wanted to bring it up on > python-dev. I think there's plenty of precedent from the Git/Gerrit world here, but agree there should be a discussion to check for any disagreement with us following that precedent. > Another issue is establishing a policy regarding branches and rebases > (rebasements?). In terms of actually *make* changes, I think we'd still want changes to effectively involve apply patches until we're able to adopt a more capable repo hosting service with integrated review management. > These issues might eventually be solved by Kallithea/Phabricator, but > I expect a transition period where different workflows will be used. I > would like our repo to be still intact by the time we settle on the > new workflow. Agreed. > IIUC hgcommitter adds extra metadata to the changeset, and if we go > down this route we might also consider adding metadata for the issue > number and tweak hgweb to display both. The other possible direction to go is the direction Gerrit and the Linux kernel go, which is to just have a footer on commit messages for relevant key:value data fields, rather than using VCS metadata. This has the benefit of being easy to port to new VCS's later, since "committer & commit message" is setting the "what metadata does the VCS track?" quite low. So your commit message might look like: Apples are no longer counted as oranges Apples were being counted as oranges in some cases. They're now always counted as apples. Issue: 12345 Contributed-by: Pat Chauthor Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From ezio.melotti at gmail.com Thu Mar 26 13:42:56 2015 From: ezio.melotti at gmail.com (Ezio Melotti) Date: Thu, 26 Mar 2015 14:42:56 +0200 Subject: [core-workflow] GSoC idea: bug.python.org improvements In-Reply-To: References: <550C5501.4050105@gmail.com> <550D1DCA.80207@gmail.com> <20150321193339.B120DB1415E@webabinitio.net> Message-ID: On Thu, Mar 26, 2015 at 12:37 PM, Nick Coghlan wrote: > On 26 March 2015 at 19:05, Ezio Melotti wrote: >> This is actually something that I wanted to bring up on python-dev. >> Currently our workflow is mostly patch-based, but adding support for >> pull requests from BitBucket/GitHub is one of the things that we are >> considering adding during GSoC. In addition, the GSoC students will >> work on a separate clone of the tracker, likely hosted on BitBucket. >> While we could still use the patch-based approach for GSoC, this would >> be a good chance to experiment with a more modern approach and also >> test on the meta-tracker both the new workflow and the code that will >> enable it. If successful, the same approach can also be adopted for >> CPython. > > I managed to forget about the idea of allowing roundup patch > generation direct from GitHub/BitBucket, so in that case it definitely > makes sense to pursue some of this for the current bugs.python.org > workflow. Note that there are 3 distinct but related features here: 1) given a link to a separate clone/branch (e.g. on BitBucket/GitHub), compute the differences and create a diff file (this is already implemented for hg thanks to MvL); 2) add a button to commit patches directly from the bug tracker (this might happen only once we switch to Kallithea/Phabricator); 3) automatically detect/update pull requests against the official BitBucket/GitHub CPython mirrors. There are also two major approaches: 1) diff-based (we get a diff and apply/commit it ourself, we are the "user" in the cs); 2) changeset-based (we get a changeset and merge it, the contributor is the "user" in the cs); (user is the name of the metadata field used by HG to store the committer in the changeset.) For the second approach, the changeset can be provided either as a patch created with hg export, or a URL of a clone. In both cases I can either import/pull on my local clone and push to the main repo, or Roundup/Kallithea/Phabricator could do the same (possibly directly on the main repo). Switching to the second approach changes the semantics of the user field, since it will start representing the contributor, not the core-dev/committer. This will affect hg blame/annotate, and will also make things such as finding out how many patches have been committed by a given core-dev more difficult. If we decide not to change the semantics, we will have to edit the user field (assuming that's possible) and save the contributor name somewhere else. > It especially makes sense as both forge.python.org proposal > target the support repos first (devguide, peps, etc), so regardless of > which of those moves forward, it's going to be a long time before it > impacts the CPython workflows. By contrast, improvements to Roundup's > integration with other services can start being helpful as soon as > they get deployed. > Note that currently nothing is preventing us to switch to the second approach, and some patches have already been imported with the contributor name as "user". >> One of the main issues is, as you mentioned, how to track both the >> committer and the author. This is both a technical and a >> "philosophical" issue -- that's why I wanted to bring it up on >> python-dev. > > I think there's plenty of precedent from the Git/Gerrit world here, > but agree there should be a discussion to check for any disagreement > with us following that precedent. > With git both fields are available, but not on mercurial (unless we use the extension you linked). This has already been discussed a few times in the past, see e.g.: https://mail.python.org/pipermail/python-dev/2011-November/114540.html >> Another issue is establishing a policy regarding branches and rebases >> (rebasements?). > > In terms of actually *make* changes, I think we'd still want changes > to effectively involve apply patches until we're able to adopt a more > capable repo hosting service with integrated review management. > While this is certainly the easiest way, it could also be possible to pull/import some changesets, tweak a few things (add Misc/ACKS and Misc/NEWS) and make a new commit (and possibly collapse/rebase as well). This doesn't need any new tool. >> These issues might eventually be solved by Kallithea/Phabricator, but >> I expect a transition period where different workflows will be used. I >> would like our repo to be still intact by the time we settle on the >> new workflow. > > Agreed. > >> IIUC hgcommitter adds extra metadata to the changeset, and if we go >> down this route we might also consider adding metadata for the issue >> number and tweak hgweb to display both. > > The other possible direction to go is the direction Gerrit and the > Linux kernel go, which is to just have a footer on commit messages for > relevant key:value data fields, rather than using VCS metadata. This > has the benefit of being easy to port to new VCS's later, since > "committer & commit message" is setting the "what metadata does the > VCS track?" quite low. > > So your commit message might look like: > > Apples are no longer counted as oranges > > Apples were being counted as oranges in some cases. They're now > always counted as apples. > > Issue: 12345 > Contributed-by: Pat Chauthor > This is also an option, but it requires everyone to be consistent. Having these in the commit message will also make them less accessible (including for the tool we will use to convert the repo to the next quantum VCS). Best Regards, Ezio Melotti > Cheers, > Nick. > > -- > Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia