From steve at holdenweb.com Sat Dec 1 09:54:00 2018 From: steve at holdenweb.com (Steve Holden) Date: Sat, 1 Dec 2018 14:54:00 +0000 Subject: [Python-Dev] Inclusion of lz4 bindings in stdlib? In-Reply-To: <1921b844-f1a9-6434-b367-7aa118ee187e@g.nevcal.com> References: <20181128174343.6bc8760b@fsol> <20181128212736.GW4319@ando.pearwood.info> <1921b844-f1a9-6434-b367-7aa118ee187e@g.nevcal.com> Message-ID: We* should probably do more collectively to point people at production-quality third-party modules, as I believe we currently do with pipenv which, while not a part of the standard library, is still recommended in the documentation as the preferred method of dependency management. We should also be even more strident when a library module is a basic version, not to be used for production purposes. This inevitably means, however, that there will be lag in the documentation, which generally speaking lags current best practices. Steve Holden * I am not a significant contributor to the code base. On Fri, Nov 30, 2018 at 9:02 PM Glenn Linderman wrote: > On 11/29/2018 2:10 PM, Andrew Svetlov wrote: > > Neither http.client nor http.server doesn't support compression > (gzip/compress/deflate) at all. > I doubt if we want to add this feature: for client better to use requests > or, well, aiohttp. > The same for servers: almost any production ready web server from PyPI > supports compression. > > What production ready web servers exist on PyPi? Are there any that don't > bring lots of baggage, their own enhanced way of doing things? The nice > thing about the http.server is that it does things in a standard-conforming > way, the bad thing about it is that it doesn't implement all the standards, > and isn't maintained very well. > > From just reading PyPi, it is hard to discover whether a particular > package is production-ready or not. > > I had used CherryPy for a while, but at the time it didn't support Python > 3, and to use the same scripts behind CherryPy or Apache CGI (my deployment > target, because that was what web hosts provided) became difficult for > complex scripts.... so I reverted to http.server with a few private > extensions (private because no one merged the bugs I reported some 3 > versions of Python-development-process ago; back then I submitted patches, > but I haven't had time to keep up with the churn of technologies Pythondev > has used since Python 3 came out, which is when I started using Python, and > I'm sure the submitted patches have bit-rotted by now). > > When I google "python web server" the first hit is the doc page for > http.server, the second is a wiki page that mentions CherryPy and a bunch > of others, but the descriptions, while terse, mostly point out some special > capabilities of the server, making it seem like you not only get a web > server, but a philosophy. I just want a web server. The last one, Waitress, > is the only one that doesn't seem to have a philosophy in its description. > > So it would be nice if http.server and http.client could get some basic > improvements to be complete, or if the docs could point to a replacement > that is a complete server, but without a philosophy or framework > (bloatware) to have to learn and/or work around. > > Glenn > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/steve%40holdenweb.com > -------------- next part -------------- An HTML attachment was scrubbed... URL: From njs at pobox.com Sat Dec 1 12:38:14 2018 From: njs at pobox.com (Nathaniel Smith) Date: Sat, 1 Dec 2018 09:38:14 -0800 Subject: [Python-Dev] Inclusion of lz4 bindings in stdlib? In-Reply-To: References: <20181128174343.6bc8760b@fsol> <20181128212736.GW4319@ando.pearwood.info> <1921b844-f1a9-6434-b367-7aa118ee187e@g.nevcal.com> Message-ID: On Sat, Dec 1, 2018, 06:56 Steve Holden We* should probably do more collectively to point people at > production-quality third-party modules, as I believe we currently do with > pipenv which, while not a part of the standard library, is still > recommended in the documentation as the preferred method of dependency > management. > Small correction: the only "official" recommendation for pipenv is that packaging.python.org (which is maintained by pypa, not python-dev) contains several tutorials, and one of them discusses how to use pipenv. For a while Kenneth used this as justification for telling everyone pipenv was "the officially recommended install tool", and this created a lot of ill will, so the pipenv team has been working on rolling that back. A better precedent is requests. There was a long discussion a few years ago about whether requests should move to the stdlib, and the outcome was that it didn't, but the urllib docs got a note added recommending the use of requests, which you can see here: https://docs.python.org/3/library/urllib.request.html#module-urllib.request Personally I would have phrased the note more strongly, but my perspective is skewed by having tried to understand the internals. I'm glad urllib has helped a lot of people solve their problems, but there's also a lot of ways that it's flat out broken. Anyway, I agree that there are probably other places where the docs could use this technique. -n -------------- next part -------------- An HTML attachment was scrubbed... URL: From chris at simplistix.co.uk Sun Dec 2 09:42:24 2018 From: chris at simplistix.co.uk (Chris Withers) Date: Sun, 2 Dec 2018 14:42:24 +0000 Subject: [Python-Dev] getting merge rights back on github Message-ID: <390ddce3-9d42-d270-4495-69f6bd567fc8@simplistix.co.uk> Hi All, It's been quite a long time since I last used my python commit rights, and it appears they've evaporated in the move to GitHub. I'd like to get back into helping out, particularly with unittest.mock where I've recently started helping out as a maintainer over on https://github.com/testing-cabal/mock. Please let me know what I need to do, Chris From chris at withers.org Sun Dec 2 09:37:09 2018 From: chris at withers.org (Chris Withers) Date: Sun, 2 Dec 2018 14:37:09 +0000 Subject: [Python-Dev] getting merge rights back on github Message-ID: <878a07ed-c4a6-5d8d-6f6f-1b92f8e3442b@withers.org> Hi All, It's been quite a long time since I last used my python commit rights, and it appears they've evaporated in the move to GitHub. I'd like to get back into helping out, particularly with unittest.mock where I've recently started helping out as a maintainer over on https://github.com/testing-cabal/mock. Please let me know what I need to do, Chris From brett at python.org Sun Dec 2 20:55:25 2018 From: brett at python.org (Brett Cannon) Date: Sun, 2 Dec 2018 17:55:25 -0800 Subject: [Python-Dev] getting merge rights back on github In-Reply-To: <878a07ed-c4a6-5d8d-6f6f-1b92f8e3442b@withers.org> References: <878a07ed-c4a6-5d8d-6f6f-1b92f8e3442b@withers.org> Message-ID: This is being dealt with. On Sun, 2 Dec 2018 at 08:28, Chris Withers wrote: > Hi All, > > It's been quite a long time since I last used my python commit rights, > and it appears they've evaporated in the move to GitHub. > > I'd like to get back into helping out, particularly with unittest.mock > where I've recently started helping out as a maintainer over on > https://github.com/testing-cabal/mock. > > Please let me know what I need to do, > > Chris > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/brett%40python.org > -------------- next part -------------- An HTML attachment was scrubbed... URL: From vstinner at redhat.com Mon Dec 3 05:26:52 2018 From: vstinner at redhat.com (Victor Stinner) Date: Mon, 3 Dec 2018 11:26:52 +0100 Subject: [Python-Dev] getting merge rights back on github In-Reply-To: <390ddce3-9d42-d270-4495-69f6bd567fc8@simplistix.co.uk> References: <390ddce3-9d42-d270-4495-69f6bd567fc8@simplistix.co.uk> Message-ID: Hi, unittest.mock definitely needs your help :-) I merged a few changes last months, but I had to rely on Mario Corchero (who authored the new seal() method) for the review, since I don't know well this module. Michael Foord (original author) doesn't seem available for reviews. More reviewers shouldn't hurt :-) If you search for "mock" in pull requests: https://github.com/python/cpython/pulls?utf8=%E2%9C%93&q=is%3Apr+is%3Aopen+mock I see at least 5 open PRs: https://github.com/python/cpython/pull/10555 https://github.com/python/cpython/pull/9296 https://github.com/python/cpython/pull/7491 https://github.com/python/cpython/pull/4476 https://github.com/python/cpython/pull/1982 You can also search into the bug tracker ;-) unittest.mock changes that I merged last months: commit 552be9d7e64f91b8e4ba5b29cd5dcc442d56f92c Author: Mario Corchero Date: Tue Oct 17 12:35:11 2017 +0100 bpo-30541: Add new method to seal mocks (GH61923) The new method allows the developer to control when to stop the feature of mocks that automagically creates new mocks when accessing an attribute that was not declared before Signed-off-by: Mario Corchero commit 6c4fab0f4b95410a1a964a75dcdd953697eff089 Author: John Reese Date: Tue May 22 13:01:10 2018 -0700 bpo-33516: Add support for __round__ in MagicMock (GH-6880) unittest.mock.MagicMock now supports the __round__() magic method. commit 96200eb2ffcda05de14099cf23f60d5091366e3e Author: Mario Corchero Date: Fri Oct 19 22:57:37 2018 +0100 unittest.mock doc: Fix references to recursive seal of Mocks (GH-9028) The docs in `library/unittest.mock` have been updated to remove confusing terms about submock and be explicit about the behavior expected. commit 6c83d9f4a72905d968418bef670bb3091d2744db Author: Max B?langer Date: Thu Oct 25 14:48:58 2018 -0700 bpo-35022: unittest.mock.MagicMock now also supports __fspath__ (GH-9960) The MagicMock class supports many magic methods, but not __fspath__. To ease testing with modules such as os.path, this function is now supported by default. commit 47d94241a383e2b8a2c40e81d12d40d5947fb170 Author: Petter Strandmark Date: Sun Oct 28 21:37:10 2018 +0100 bpo-35047, unittest.mock: Better error messages on assert_called_xxx failures (GH-10090) unittest.mock now includes mock calls in exception messages if assert_not_called, assert_called_once, or assert_called_once_with fails. commit edeca92c84a3b08902ecdfe987cde00c7e617887 Author: Xtreak Date: Sat Dec 1 15:33:54 2018 +0530 bpo-31177: Skip deleted attributes while calling reset_mock (GH-9302) Victor Le dim. 2 d?c. 2018 ? 15:45, Chris Withers a ?crit : > > Hi All, > > It's been quite a long time since I last used my python commit rights, > and it appears they've evaporated in the move to GitHub. > > I'd like to get back into helping out, particularly with unittest.mock > where I've recently started helping out as a maintainer over on > https://github.com/testing-cabal/mock. > > Please let me know what I need to do, > > Chris > > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: https://mail.python.org/mailman/options/python-dev/vstinner%40redhat.com From chris at simplistix.co.uk Mon Dec 3 13:34:42 2018 From: chris at simplistix.co.uk (Chris Withers) Date: Mon, 3 Dec 2018 18:34:42 +0000 Subject: [Python-Dev] getting merge rights back on github In-Reply-To: References: <390ddce3-9d42-d270-4495-69f6bd567fc8@simplistix.co.uk> Message-ID: Noted, I'll see what I can do... On 03/12/2018 10:26, Victor Stinner wrote: > Hi, > > unittest.mock definitely needs your help :-) I merged a few changes > last months, but I had to rely on Mario Corchero (who authored the new > seal() method) for the review, since I don't know well this module. > Michael Foord (original author) doesn't seem available for reviews. > More reviewers shouldn't hurt :-) If you search for "mock" in pull > requests: > https://github.com/python/cpython/pulls?utf8=%E2%9C%93&q=is%3Apr+is%3Aopen+mock > > I see at least 5 open PRs: > > https://github.com/python/cpython/pull/10555 > https://github.com/python/cpython/pull/9296 > https://github.com/python/cpython/pull/7491 > https://github.com/python/cpython/pull/4476 > https://github.com/python/cpython/pull/1982 > > You can also search into the bug tracker ;-) > > > unittest.mock changes that I merged last months: > > commit 552be9d7e64f91b8e4ba5b29cd5dcc442d56f92c > Author: Mario Corchero > Date: Tue Oct 17 12:35:11 2017 +0100 > > bpo-30541: Add new method to seal mocks (GH61923) > > The new method allows the developer to control when to stop the > feature of mocks that automagically creates new mocks when accessing > an attribute that was not declared before > > Signed-off-by: Mario Corchero > > commit 6c4fab0f4b95410a1a964a75dcdd953697eff089 > Author: John Reese > Date: Tue May 22 13:01:10 2018 -0700 > > bpo-33516: Add support for __round__ in MagicMock (GH-6880) > > unittest.mock.MagicMock now supports the __round__() magic method. > > commit 96200eb2ffcda05de14099cf23f60d5091366e3e > Author: Mario Corchero > Date: Fri Oct 19 22:57:37 2018 +0100 > > unittest.mock doc: Fix references to recursive seal of Mocks (GH-9028) > > The docs in `library/unittest.mock` have been updated to remove > confusing terms about submock and be explicit about the behavior > expected. > > commit 6c83d9f4a72905d968418bef670bb3091d2744db > Author: Max B?langer > Date: Thu Oct 25 14:48:58 2018 -0700 > > bpo-35022: unittest.mock.MagicMock now also supports __fspath__ (GH-9960) > > The MagicMock class supports many magic methods, but not __fspath__. To ease > testing with modules such as os.path, this function is now > supported by default. > > commit 47d94241a383e2b8a2c40e81d12d40d5947fb170 > Author: Petter Strandmark > Date: Sun Oct 28 21:37:10 2018 +0100 > > bpo-35047, unittest.mock: Better error messages on > assert_called_xxx failures (GH-10090) > > unittest.mock now includes mock calls in exception messages if > assert_not_called, assert_called_once, or assert_called_once_with > fails. > > commit edeca92c84a3b08902ecdfe987cde00c7e617887 > Author: Xtreak > Date: Sat Dec 1 15:33:54 2018 +0530 > > bpo-31177: Skip deleted attributes while calling reset_mock (GH-9302) > > Victor > Le dim. 2 d?c. 2018 ? 15:45, Chris Withers a ?crit : >> Hi All, >> >> It's been quite a long time since I last used my python commit rights, >> and it appears they've evaporated in the move to GitHub. >> >> I'd like to get back into helping out, particularly with unittest.mock >> where I've recently started helping out as a maintainer over on >> https://github.com/testing-cabal/mock. >> >> Please let me know what I need to do, >> >> Chris >> >> >> _______________________________________________ >> Python-Dev mailing list >> Python-Dev at python.org >> https://mail.python.org/mailman/listinfo/python-dev >> Unsubscribe: https://mail.python.org/mailman/options/python-dev/vstinner%40redhat.com From nad at python.org Tue Dec 4 03:42:28 2018 From: nad at python.org (Ned Deily) Date: Tue, 4 Dec 2018 03:42:28 -0500 Subject: [Python-Dev] 3.7.2rc1 and 3.6.8rc1 cutoffs ahead, last 3.6.x bugfix release! Message-ID: <95B9F244-38A8-4E47-8E9D-6DAD289C84A2@python.org> https://discuss.python.org/t/3-7-2rc1-and-3-6-8rc1-cutoffs-ahead-last-3-6-x-bugfix-release/510 We're reaching the end of the year and it's time for another pair of Python 3 maintenance releases, 3.7.2 and 3.6.8, before we ring out 2018. Since there are still some open release blocker issues and I haven't been bugging you about them, I've moved the code cutoff for the release candidates to this coming Friday, 12-07, by the end of the day (AOE). That gives us all another 4 days to review open issues and PRs. Please give highest attention to any release blockers you have been shepherding or reviewing. Thanks! A reminder: as previously announced, 3.6.8 is planned to be the last bugfix release of the 3.6 series. Python 3.6.0 was released on 2016-12-23, so by the time 3.6.8 is released, 3.6.x will have been in bugfix mode almost exactly 2 years. When a new feature release is made and enters "bugfix" mode, our policy has long been to continue to maintain the previous bugfix branch for at least one more release and then move that branch to "security fix only" mode. 3.7.0 (and 3.6.6) was released nearly six months ago and, with the release of 3.6.8, there will have been two additional 3.6.x bugfix releases since then. So, barring any showstopper issues that might arise, the upcoming 3.6.8rc1 is your last chance to make bugfix changes for 3.6.x. Following the successful release of 3.6.8, only security fixes will be accepted for the 3.6 branch and future 3.6.x releases will be source-only and scheduled as needed; no further binary installers will be produced for 3.6. Refer to the Dev Guide sections and release PEPs linked below for more information. https://devguide.python.org/devcycle/ https://devguide.python.org/#branchstatus https://www.python.org/dev/peps/pep-0494/ https://www.python.org/dev/peps/pep-0537/ -- Ned Deily nad at python.org -- [] From vstinner at redhat.com Tue Dec 4 10:19:46 2018 From: vstinner at redhat.com (Victor Stinner) Date: Tue, 4 Dec 2018 16:19:46 +0100 Subject: [Python-Dev] Internal header files (Include/internal/*.h) are now installed Message-ID: Hi, Since Python 3.7, "internal" C API (only declared if Py_BUILD_CORE is defined) are moving from Include/*.h to Include/internal/*.h. These API must not be used outside CPython. In Python 3.7, "make install" doesn't install them for example. I would like to move more private functions (prefixed by "_Py") to the "internal" API. Since I'm not 100% sure that it's ok, I decided to modify "make install" to also install Include/internal/ headers (to $prefix/include/python3.8m/internal/). These headers might be useful for low-level debug tools like debuggers or profilers, to access directly memory without calling functions. These APIs require to use the same compiler and likely the same compiler options than CPython. It's especially true for atomic variables (Include/internal/pycore_atomic.h). Are you ok to install "internal" header files? If yes, should we modify "make install" of Python 3.7 to also install them? Victor From solipsis at pitrou.net Tue Dec 4 10:31:57 2018 From: solipsis at pitrou.net (Antoine Pitrou) Date: Tue, 4 Dec 2018 16:31:57 +0100 Subject: [Python-Dev] Internal header files (Include/internal/*.h) are now installed References: Message-ID: <20181204163157.2b15cf6a@fsol> On Tue, 4 Dec 2018 16:19:46 +0100 Victor Stinner wrote: > Hi, > > Since Python 3.7, "internal" C API (only declared if Py_BUILD_CORE is > defined) are moving from Include/*.h to Include/internal/*.h. These > API must not be used outside CPython. In Python 3.7, "make install" > doesn't install them for example. > > I would like to move more private functions (prefixed by "_Py") to the > "internal" API. Since I'm not 100% sure that it's ok, I decided to > modify "make install" to also install Include/internal/ headers (to > $prefix/include/python3.8m/internal/). > > These headers might be useful for low-level debug tools like debuggers > or profilers, to access directly memory without calling functions. > These APIs require to use the same compiler and likely the same > compiler options than CPython. It's especially true for atomic > variables (Include/internal/pycore_atomic.h). > > Are you ok to install "internal" header files? If yes, should we > modify "make install" of Python 3.7 to also install them? +1 to both. Regards Antoine. From vstinner at redhat.com Tue Dec 4 11:17:12 2018 From: vstinner at redhat.com (Victor Stinner) Date: Tue, 4 Dec 2018 17:17:12 +0100 Subject: [Python-Dev] Internal header files (Include/internal/*.h) are now installed In-Reply-To: <20181204163157.2b15cf6a@fsol> References: <20181204163157.2b15cf6a@fsol> Message-ID: Le mar. 4 d?c. 2018 ? 16:35, Antoine Pitrou a ?crit : > > Are you ok to install "internal" header files? If yes, should we > > modify "make install" of Python 3.7 to also install them? > > +1 to both. Ok, I reopened https://bugs.python.org/issue35296 and wrote a PR: https://github.com/python/cpython/pull/10897 Victor From chris at withers.org Tue Dec 4 14:13:32 2018 From: chris at withers.org (Chris Withers) Date: Tue, 4 Dec 2018 19:13:32 +0000 Subject: [Python-Dev] any way to subscribe to bugs and PRs on a particular topic? Message-ID: Hello, I'd like to see if I can help with unittest.mock, but don't have a huge amount of bandwidth and can't even parse let alone process the whole firehose of bpo and GH PRs. Is there? any way I can get bugs.python.org and github PRs to only tell me about things, preferably by email, that affect or involve unittest.mock? cheers, Chris From storchaka at gmail.com Tue Dec 4 14:42:10 2018 From: storchaka at gmail.com (Serhiy Storchaka) Date: Tue, 4 Dec 2018 21:42:10 +0200 Subject: [Python-Dev] any way to subscribe to bugs and PRs on a particular topic? In-Reply-To: References: Message-ID: 04.12.18 21:13, Chris Withers ????: > I'd like to see if I can help with unittest.mock, but don't have a huge > amount of bandwidth and can't even parse let alone process the whole > firehose of bpo and GH PRs. > > Is there? any way I can get bugs.python.org and github PRs to only tell > me about things, preferably by email, that affect or involve unittest.mock? You can add yourself into the experts list: https://github.com/python/devguide/blob/master/experts.rst. This will help to add you to nosy list in new issues. You can find existing unittest.mock related issues and PRs by using search on the bug tracker and GitHub. From mariatta at python.org Tue Dec 4 14:46:12 2018 From: mariatta at python.org (Mariatta Wijaya) Date: Tue, 4 Dec 2018 11:46:12 -0800 Subject: [Python-Dev] any way to subscribe to bugs and PRs on a particular topic? In-Reply-To: References: Message-ID: For GitHub PRs, you can add yourself to CODEOWNERS file, so you will be automatically requested review if a PR contains changes to unittest.mock. (and you'll receive review-request notification) https://github.com/python/cpython/blob/master/.github/CODEOWNERS When GitHub sends you review request notification email, it will cc review_requested at noreply.github.com, so you can create a filter based on that. ? On Tue, Dec 4, 2018 at 11:21 AM Chris Withers wrote: > Hello, > > I'd like to see if I can help with unittest.mock, but don't have a huge > amount of bandwidth and can't even parse let alone process the whole > firehose of bpo and GH PRs. > > Is there any way I can get bugs.python.org and github PRs to only tell > me about things, preferably by email, that affect or involve unittest.mock? > > cheers, > > Chris > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/mariatta%40python.org > -------------- next part -------------- An HTML attachment was scrubbed... URL: From tjreedy at udel.edu Tue Dec 4 15:01:45 2018 From: tjreedy at udel.edu (Terry Reedy) Date: Tue, 4 Dec 2018 15:01:45 -0500 Subject: [Python-Dev] any way to subscribe to bugs and PRs on a particular topic? In-Reply-To: References: Message-ID: On 12/4/2018 2:13 PM, Chris Withers wrote: > Hello, Welcome back. unittest.mock is important for everyone writing non-trivial tests. > I'd like to see if I can help with unittest.mock, but don't have a huge > amount of bandwidth and can't even parse let alone process the whole > firehose of bpo and GH PRs. > > Is there? any way I can get bugs.python.org and github PRs to only tell > me about things, preferably by email, that affect or involve unittest.mock? Tracker: Edit https://devguide.python.org/experts/ to add yourself as expert for unittest.mock and you will see such issues if the OP or triager types 'unittest.mock' in the nosy field and clicks the list. Note: consider adding yourself also as unittest and Misc - testing expert. Certain people, such as Zach Ware, can make nosy listing automatic upon a selection of a component, such as 'testing'. Every Friday, a list of new issues for the last week is posted here (pydev). It does not take too long to scan +-50 titles. PRs. In the repository, add a line to .github/CODEOWNERS (which starts with instructions) so you are notified whenever a PR touches Lib/unittest/mock.py. From other example, it seem that the following might work (but I am not an expert on this). @@/*unittest*/*mock* -- Terry Jan Reedy From vstinner at redhat.com Wed Dec 5 05:51:04 2018 From: vstinner at redhat.com (Victor Stinner) Date: Wed, 5 Dec 2018 11:51:04 +0100 Subject: [Python-Dev] test_urllib2net fixed to repair Travis CI Message-ID: Hi, It seems like Travis CI changed its security or network configuration: some FTP tests of test_urllib2net started to fail with "425 Security: Bad IP connecting" yesterday: https://bugs.python.org/issue35411 The failure seems to be reproducible and so prevents to merge any change in any branch. It blocked our whole workflow :-( Good news: I quickly fixed the CI! (2.7, 3.6, 3.7 and master branches) I skipped the test on Travis CI: AppVeyor runs these tests in pre-commit and the buildbot farm run them in post-commit. Bad news: You may have to update your pull requests (ex: merge master into your branch) to retrieve this fix if the test_urllib2net failed on Travis CI with "425 Security: Bad IP connecting" on your PR. Victor From njs at pobox.com Wed Dec 5 10:18:00 2018 From: njs at pobox.com (Nathaniel Smith) Date: Wed, 5 Dec 2018 07:18:00 -0800 Subject: [Python-Dev] test_urllib2net fixed to repair Travis CI In-Reply-To: References: Message-ID: Travis is in the middle of moving everything from AWS to GCE, which is probably related: https://blog.travis-ci.com/2018-11-19-required-linux-infrastructure-migration As noted there, GCE has different IP addresses. But I suspect it's not the new IP address that's the problem, but rather the fact that the GCE setup is known to break outgoing ftp: https://blog.travis-ci.com/2018-07-23-the-tale-of-ftp-at-travis-ci -n On Wed, Dec 5, 2018, 02:53 Victor Stinner Hi, > > It seems like Travis CI changed its security or network configuration: > some FTP tests of test_urllib2net started to fail with "425 Security: > Bad IP connecting" yesterday: > > https://bugs.python.org/issue35411 > > The failure seems to be reproducible and so prevents to merge any > change in any branch. It blocked our whole workflow :-( > > Good news: I quickly fixed the CI! (2.7, 3.6, 3.7 and master branches) > I skipped the test on Travis CI: AppVeyor runs these tests in > pre-commit and the buildbot farm run them in post-commit. > > Bad news: You may have to update your pull requests (ex: merge master > into your branch) to retrieve this fix if the test_urllib2net failed > on Travis CI with "425 Security: Bad IP connecting" on your PR. > > Victor > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/njs%40pobox.com > -------------- next part -------------- An HTML attachment was scrubbed... URL: From noreply at pypi.org Wed Dec 5 15:01:53 2018 From: noreply at pypi.org (PyPI) Date: Wed, 5 Dec 2018 20:01:53 +0000 Subject: [Python-Dev] [PyPI] Email verification Message-ID: <010101677ff66451-5cf87aad-b3f5-49f6-aa10-e42c74fb9d74-000000@us-west-2.amazonses.com> Someone, perhaps you, has added this email address to their PyPI account: Python-dev at python.org If you wish to proceed with this request, follow the link below to verify your email address: https://pypi.org/account/verify-email/?token=eyJhY3Rpb24iOiJlbWFpbC12ZXJpZnkiLCJlbWFpbC5pZCI6IjMyODU0NyJ9.XAgusA.38y3N4T1GonyEeWWpwxm6bE7cn-JDJe0P6LD0tom2vV5WFoS-DDNrUs22weH1UK8kWsqaIhneX9J7tof-my0Jw This link will expire in 72 hours. If you did not make this request, you can safely ignore this email. -------------- next part -------------- An HTML attachment was scrubbed... URL: From brett at python.org Wed Dec 5 19:28:43 2018 From: brett at python.org (Brett Cannon) Date: Wed, 5 Dec 2018 16:28:43 -0800 Subject: [Python-Dev] [PyPI] Email verification In-Reply-To: <010101677ff66451-5cf87aad-b3f5-49f6-aa10-e42c74fb9d74-000000@us-west-2.amazonses.com> References: <010101677ff66451-5cf87aad-b3f5-49f6-aa10-e42c74fb9d74-000000@us-west-2.amazonses.com> Message-ID: I've reported this to infrastructure at . On Wed., Dec. 5, 2018, 12:05 PyPI Someone, perhaps you, has added this email address (Python-dev at python.org) > to their PyPI account. > > If you wish to proceed with this request, click this link to verify your > email address > > > This link will expire in 72 hours. > > If you did not make this request, you can safely ignore this email. > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/brett%40python.org > -------------- next part -------------- An HTML attachment was scrubbed... URL: From vstinner at redhat.com Fri Dec 7 07:35:17 2018 From: vstinner at redhat.com (Victor Stinner) Date: Fri, 7 Dec 2018 13:35:17 +0100 Subject: [Python-Dev] I reverted "Add Windows App Store package" change Message-ID: Hi, I had to revert a change since it broke buildbots. Sadly, I don't have the bandwidth to investigate the failures and try to fix the change :-( A large change has been pushed into the 3.7 and master branches to "Add Windows App Store package": "Release Windows Store app containing Python" https://bugs.python.org/issue34977 These changes broke all Windows buildbots: "Almost all Windows buildbots are failing to compile" https://bugs.python.org/issue35437 So I reverted these two changes. It's a large change which mostly add new files, but also make changes in different files: --- commit 468a15aaf9206448a744fc5eab3fc21f51966aad Author: Steve Dower Date: Thu Dec 6 21:09:20 2018 -0800 bpo-34977: Add Windows App Store package (GH-10245) .azure-pipelines/windows-appx-test.yml | 65 +++ .gitattributes | 1 + Doc/make.bat | 2 + Lib/test/test_pathlib.py | 2 +- Lib/test/test_venv.py | 1 + Lib/venv/__init__.py | 49 +- .../2018-10-30-13-39-17.bpo-34977.0l7_QV.rst | 1 + PC/classicAppCompat.can.xml | Bin 0 -> 3978 bytes PC/classicAppCompat.cat | Bin 0 -> 10984 bytes PC/classicAppCompat.sccd | Bin 0 -> 18503 bytes PC/getpathp.c | 8 +- PC/icons/pythonwx150.png | Bin 0 -> 8187 bytes PC/icons/pythonwx44.png | Bin 0 -> 2232 bytes PC/icons/pythonx150.png | Bin 0 -> 8271 bytes PC/icons/pythonx44.png | Bin 0 -> 2178 bytes PC/icons/pythonx50.png | Bin 0 -> 2190 bytes PC/launcher.c | 220 +++++++- PC/layout/__init__.py | 0 PC/layout/__main__.py | 14 + PC/layout/main.py | 612 +++++++++++++++++++++ PC/layout/support/__init__.py | 0 PC/layout/support/appxmanifest.py | 487 ++++++++++++++++ PC/layout/support/catalog.py | 44 ++ PC/layout/support/constants.py | 28 + .../support/distutils.command.bdist_wininst.py | 25 + PC/layout/support/filesets.py | 100 ++++ PC/layout/support/logging.py | 93 ++++ PC/layout/support/options.py | 122 ++++ PC/layout/support/pip.py | 79 +++ PC/layout/support/props.py | 110 ++++ {Tools/nuget => PC/layout/support}/python.props | 0 PC/pylauncher.rc | 6 + PC/python_uwp.cpp | 226 ++++++++ PC/store_info.txt | 146 +++++ PCbuild/_tkinter.vcxproj | 6 + PCbuild/find_msbuild.bat | 10 + PCbuild/pcbuild.proj | 3 + PCbuild/pcbuild.sln | 72 +++ PCbuild/python.props | 3 +- PCbuild/python_uwp.vcxproj | 86 +++ PCbuild/pythoncore.vcxproj | 15 + PCbuild/pythonw_uwp.vcxproj | 86 +++ PCbuild/venvlauncher.vcxproj | 85 +++ PCbuild/venvwlauncher.vcxproj | 85 +++ Tools/msi/buildrelease.bat | 13 +- Tools/msi/make_appx.ps1 | 71 +++ Tools/msi/make_cat.ps1 | 34 ++ Tools/msi/make_zip.proj | 9 +- Tools/msi/make_zip.py | 250 --------- Tools/msi/sdktools.psm1 | 43 ++ Tools/msi/sign_build.ps1 | 34 ++ Tools/nuget/make_pkg.proj | 38 +- 52 files changed, 3053 insertions(+), 331 deletions(-) --- Note: the commit message is a single line. Now I have questions :-) It seems like the change is "experimental": https://bugs.python.org/issue34977#msg331267 """ Making the *release experimental* as part of the next 3.7 update was approved by Ned (over email), so I merged the build. As soon as we snap for the RC I'll kick off an update and make the store page public, and hopefully can promote it enough to get eyes on it. Unfortunately, I discovered as part of a test submission that the minimum Windows version matters, and it's a version that hasn't been rolled out fully yet *because of some bugs*, so there may not be that many people who can use it this first time. But that will *improve over time*, and I'm sure I can find enough people to flush out issues before the next release (anyone on the Windows Insiders program should be fine). """ If it's experimental, why pushing it *right now* to the 3.7 branch? Can't we wait until it's stable enough? Even if it's stable, I'm not sure sure that it should go into 3.7 which is a stable branch. I guess that Steve would like to get this feature into 3.7.2. Ned Deily (3.7 release changer) approved this change. The pull request was merged one week after it has been created, I don't see any review. https://github.com/python/cpython/pull/10245 In general, I'm fine with merging changes without any kind of review, when the change is small. I'm doing it *frequently* when I'm confident that the change is small and safe enough. But here, the change is quite large. Sadly, I know the problem: the lack of available developers for review. There are very few developers who know Windows and are available to provide a review. Sometimes, Zachary Ware or Eryk Sun are around and can help. I'm fine with iterating in the master branch, since the change seems to be separated from CPython core: it mostly add new files. My concern is more about changes on "Python itself": the venv and tests changes. In the middle of the large change, there is small change on the venv module: diff --git a/Lib/venv/__init__.py b/Lib/venv/__init__.py index 043420897e..5438b0d4e5 100644 --- a/Lib/venv/__init__.py +++ b/Lib/venv/__init__.py @@ -64,10 +64,11 @@ class EnvBuilder: self.system_site_packages = False self.create_configuration(context) self.setup_python(context) + if not self.upgrade: + self.setup_scripts(context) if self.with_pip: self._setup_pip(context) if not self.upgrade: - self.setup_scripts(context) self.post_setup(context) if true_system_site_packages: Moreover, the commit also changes tests: * Lib/test/test_pathlib.py * Lib/test/test_venv.py The commit message is just "bpo-34977: Add Windows App Store package (GH-10245)", it doesn't explain these changes. I would prefer to see these changes extracted into separated commits. Sadly, right now on the cpython project on GitHub, we are only allowed to squash all commits into a single commit (note: the individual commit in the PR doesn't explain these venv/tests changes neither). So I suggest to create multiple PRs (at least one for venv+pathlib changes and a second for the Windows AppStore). I guess that these changes are bugfixes or enhancement. Having a separated change should also ease to backport these changes up to 3.6 ;-) -- Jeremy Kloth just created: "Correctly detect installed SDK versions" https://bugs.python.org/issue35433 I'm not sure if it's related to the AppStore change? Victor -- Night gathers, and now my watch begins. It shall not end until my death. From steve.dower at python.org Fri Dec 7 10:16:03 2018 From: steve.dower at python.org (Steve Dower) Date: Fri, 7 Dec 2018 07:16:03 -0800 Subject: [Python-Dev] I reverted "Add Windows App Store package" change In-Reply-To: References: Message-ID: <4937db92-3a2e-5a6a-6e3b-a4e502277d16@python.org> Thanks for fixing up the buildbots, but please be a little more thorough before making publicly incorrect statements. The change is 99% adding new files that are not part of Python, but participate in the build for Windows (and are available and incredibly useful for everyone). These are essentially zero-impact. In the PR I pointed out exactly where to look for interesting changes and it didn't help get any traction :) The other changes are either in Windows-only files or tests. The one exception is venv, where they are in "if os.name=='nt'" blocks. I also pinged our venv expert a few times with no response. The PR was put up two *months* ago, not one week. In that time, it's been heavily tested by myself and a number of people who I *am* able to share the output of this with. I've also been chatting with the release manager for 3.7 about it and he's been on board (the words may have been "on your own head", but that's close enough to "on board") It didn't break all Windows buildbots. That said, it's totally my fault for merging and then not watching. Also for not submitting custom builds to all the buildbots (Can we still do that? I'm not seeing any UI right now... I did run a number of test release builds on the release machine, so I knew that was going to be okay.) Now, to answer your questions: releasing in a new package with slightly different semantics *has* to be somewhat experimental, because nobody can test it until it's been released. This isn't about iterating on this change in master, it's about getting broad public feedback on a release mechanism that currently does not exist. Historically, when changes to the part you point out have been extracted out from their context, they've been rejected on the basis that they aren't necessary. But now the original PR is broken (because the tests don't pass), and so there will never be a need. This time I decided to specifically point out numerous times where the interesting changes were and was not able to get reviews from bpo/github (I did get many reviews and some testing from others). As it happens, I split out many changes to do with pathconfig that came from this. You rejected them because they weren't necessary :) Most of the code is a Python script to do "make install" on Windows. A very common request is "how do I copy built files into the right place", and the answer has always been "it's complicated". Now we can measure how complicated it is in terms of lines of Python code, but at least the answer is "run this script". Yes, it could go into its own PR, but that runs into the context problem again. If I'm going to be forced to bypass review on a dependency just to make it possible to merge a real change, then they may as well go together. (The new script is Black formatted, so probably I could get Lukasz to approve it on its own? :) ) I hope that explains a bit better. People wait two months and more for simpler reviews, but part of me being a core developer is to accelerate that for Windows-targeted work. That's all I did here, and I'm happy with my reasoning. I've reposted the PRs at https://github.com/python/cpython/pull/11027 and https://github.com/python/cpython/pull/11028 with fixes for the one issue that Victor couldn't investigate. If someone can get a Windows buildbot to run against them that would be great (not you Zach - your buildbots were fine :) ). Cheers, Steve On 07Dec2018 0435, Victor Stinner wrote: > Hi, > > I had to revert a change since it broke buildbots. Sadly, I don't have > the bandwidth to investigate the failures and try to fix the change > :-( > > > A large change has been pushed into the 3.7 and master branches to > "Add Windows App Store package": > > "Release Windows Store app containing Python" > https://bugs.python.org/issue34977 > > These changes broke all Windows buildbots: > > "Almost all Windows buildbots are failing to compile" > https://bugs.python.org/issue35437 > > So I reverted these two changes. > > It's a large change which mostly add new files, but also make changes > in different files: > --- > commit 468a15aaf9206448a744fc5eab3fc21f51966aad > Author: Steve Dower > Date: Thu Dec 6 21:09:20 2018 -0800 > > bpo-34977: Add Windows App Store package (GH-10245) > > .azure-pipelines/windows-appx-test.yml | 65 +++ > .gitattributes | 1 + > Doc/make.bat | 2 + > Lib/test/test_pathlib.py | 2 +- > Lib/test/test_venv.py | 1 + > Lib/venv/__init__.py | 49 +- > .../2018-10-30-13-39-17.bpo-34977.0l7_QV.rst | 1 + > PC/classicAppCompat.can.xml | Bin 0 -> 3978 bytes > PC/classicAppCompat.cat | Bin 0 -> 10984 bytes > PC/classicAppCompat.sccd | Bin 0 -> 18503 bytes > PC/getpathp.c | 8 +- > PC/icons/pythonwx150.png | Bin 0 -> 8187 bytes > PC/icons/pythonwx44.png | Bin 0 -> 2232 bytes > PC/icons/pythonx150.png | Bin 0 -> 8271 bytes > PC/icons/pythonx44.png | Bin 0 -> 2178 bytes > PC/icons/pythonx50.png | Bin 0 -> 2190 bytes > PC/launcher.c | 220 +++++++- > PC/layout/__init__.py | 0 > PC/layout/__main__.py | 14 + > PC/layout/main.py | 612 +++++++++++++++++++++ > PC/layout/support/__init__.py | 0 > PC/layout/support/appxmanifest.py | 487 ++++++++++++++++ > PC/layout/support/catalog.py | 44 ++ > PC/layout/support/constants.py | 28 + > .../support/distutils.command.bdist_wininst.py | 25 + > PC/layout/support/filesets.py | 100 ++++ > PC/layout/support/logging.py | 93 ++++ > PC/layout/support/options.py | 122 ++++ > PC/layout/support/pip.py | 79 +++ > PC/layout/support/props.py | 110 ++++ > {Tools/nuget => PC/layout/support}/python.props | 0 > PC/pylauncher.rc | 6 + > PC/python_uwp.cpp | 226 ++++++++ > PC/store_info.txt | 146 +++++ > PCbuild/_tkinter.vcxproj | 6 + > PCbuild/find_msbuild.bat | 10 + > PCbuild/pcbuild.proj | 3 + > PCbuild/pcbuild.sln | 72 +++ > PCbuild/python.props | 3 +- > PCbuild/python_uwp.vcxproj | 86 +++ > PCbuild/pythoncore.vcxproj | 15 + > PCbuild/pythonw_uwp.vcxproj | 86 +++ > PCbuild/venvlauncher.vcxproj | 85 +++ > PCbuild/venvwlauncher.vcxproj | 85 +++ > Tools/msi/buildrelease.bat | 13 +- > Tools/msi/make_appx.ps1 | 71 +++ > Tools/msi/make_cat.ps1 | 34 ++ > Tools/msi/make_zip.proj | 9 +- > Tools/msi/make_zip.py | 250 --------- > Tools/msi/sdktools.psm1 | 43 ++ > Tools/msi/sign_build.ps1 | 34 ++ > Tools/nuget/make_pkg.proj | 38 +- > 52 files changed, 3053 insertions(+), 331 deletions(-) > --- > > Note: the commit message is a single line. > > > Now I have questions :-) > > > It seems like the change is "experimental": > https://bugs.python.org/issue34977#msg331267 > > """ > Making the *release experimental* as part of the next 3.7 update was > approved by Ned (over email), so I merged the build. As soon as we > snap for the RC I'll kick off an update and make the store page > public, and hopefully can promote it enough to get eyes on it. > > Unfortunately, I discovered as part of a test submission that the > minimum Windows version matters, and it's a version that hasn't been > rolled out fully yet *because of some bugs*, so there may not be that > many people who can use it this first time. But that will *improve > over time*, and I'm sure I can find enough people to flush out issues > before the next release (anyone on the Windows Insiders program should > be fine). > """ > > If it's experimental, why pushing it *right now* to the 3.7 branch? > Can't we wait until it's stable enough? Even if it's stable, I'm not > sure sure that it should go into 3.7 which is a stable branch. > > I guess that Steve would like to get this feature into 3.7.2. Ned > Deily (3.7 release changer) approved this change. > > The pull request was merged one week after it has been created, I > don't see any review. > https://github.com/python/cpython/pull/10245 > > In general, I'm fine with merging changes without any kind of review, > when the change is small. I'm doing it *frequently* when I'm confident > that the change is small and safe enough. But here, the change is > quite large. Sadly, I know the problem: the lack of available > developers for review. There are very few developers who know Windows > and are available to provide a review. Sometimes, Zachary Ware or Eryk > Sun are around and can help. > > I'm fine with iterating in the master branch, since the change seems > to be separated from CPython core: it mostly add new files. My concern > is more about changes on "Python itself": the venv and tests changes. > > In the middle of the large change, there is small change on the venv module: > > diff --git a/Lib/venv/__init__.py b/Lib/venv/__init__.py > index 043420897e..5438b0d4e5 100644 > --- a/Lib/venv/__init__.py > +++ b/Lib/venv/__init__.py > @@ -64,10 +64,11 @@ class EnvBuilder: > self.system_site_packages = False > self.create_configuration(context) > self.setup_python(context) > + if not self.upgrade: > + self.setup_scripts(context) > if self.with_pip: > self._setup_pip(context) > if not self.upgrade: > - self.setup_scripts(context) > self.post_setup(context) > if true_system_site_packages: > > Moreover, the commit also changes tests: > > * Lib/test/test_pathlib.py > * Lib/test/test_venv.py > > The commit message is just "bpo-34977: Add Windows App Store package > (GH-10245)", it doesn't explain these changes. > > I would prefer to see these changes extracted into separated commits. > Sadly, right now on the cpython project on GitHub, we are only allowed > to squash all commits into a single commit (note: the individual > commit in the PR doesn't explain these venv/tests changes neither). So > I suggest to create multiple PRs (at least one for venv+pathlib > changes and a second for the Windows AppStore). > > I guess that these changes are bugfixes or enhancement. Having a > separated change should also ease to backport these changes up to 3.6 > ;-) > > -- > > Jeremy Kloth just created: > "Correctly detect installed SDK versions" > https://bugs.python.org/issue35433 > > I'm not sure if it's related to the AppStore change? > > Victor > From steve.dower at python.org Fri Dec 7 10:41:38 2018 From: steve.dower at python.org (Steve Dower) Date: Fri, 7 Dec 2018 07:41:38 -0800 Subject: [Python-Dev] I reverted "Add Windows App Store package" change In-Reply-To: <4937db92-3a2e-5a6a-6e3b-a4e502277d16@python.org> References: <4937db92-3a2e-5a6a-6e3b-a4e502277d16@python.org> Message-ID: <1bff9c35-2284-27ee-11b3-bab6072bf9aa@python.org> As a slight aside, 8 out of 8 buildbot messages on the PR look like false positives, and none of the true positives sent a message. What happened there? On 07Dec2018 0716, Steve Dower wrote: > Thanks for fixing up the buildbots, but please be a little more thorough > before making publicly incorrect statements. > > The change is 99% adding new files that are not part of Python, but > participate in the build for Windows (and are available and incredibly > useful for everyone). These are essentially zero-impact. In the PR I > pointed out exactly where to look for interesting changes and it didn't > help get any traction :) > > The other changes are either in Windows-only files or tests. The one > exception is venv, where they are in "if os.name=='nt'" blocks. I also > pinged our venv expert a few times with no response. > > The PR was put up two *months* ago, not one week. In that time, it's > been heavily tested by myself and a number of people who I *am* able to > share the output of this with. I've also been chatting with the release > manager for 3.7 about it and he's been on board (the words may have been > "on your own head", but that's close enough to "on board") > > It didn't break all Windows buildbots. > > That said, it's totally my fault for merging and then not watching. Also > for not submitting custom builds to all the buildbots (Can we still do > that? I'm not seeing any UI right now... I did run a number of test > release builds on the release machine, so I knew that was going to be okay.) > > Now, to answer your questions: releasing in a new package with slightly > different semantics *has* to be somewhat experimental, because nobody > can test it until it's been released. This isn't about iterating on this > change in master, it's about getting broad public feedback on a release > mechanism that currently does not exist. > > Historically, when changes to the part you point out have been extracted > out from their context, they've been rejected on the basis that they > aren't necessary. But now the original PR is broken (because the tests > don't pass), and so there will never be a need. This time I decided to > specifically point out numerous times where the interesting changes were > and was not able to get reviews from bpo/github (I did get many reviews > and some testing from others). > > As it happens, I split out many changes to do with pathconfig that came > from this. You rejected them because they weren't necessary :) > > Most of the code is a Python script to do "make install" on Windows. A > very common request is "how do I copy built files into the right place", > and the answer has always been "it's complicated". Now we can measure > how complicated it is in terms of lines of Python code, but at least the > answer is "run this script". Yes, it could go into its own PR, but that > runs into the context problem again. If I'm going to be forced to bypass > review on a dependency just to make it possible to merge a real change, > then they may as well go together. > > (The new script is Black formatted, so probably I could get Lukasz to > approve it on its own? :) ) > > I hope that explains a bit better. People wait two months and more for > simpler reviews, but part of me being a core developer is to accelerate > that for Windows-targeted work. That's all I did here, and I'm happy > with my reasoning. > > I've reposted the PRs at https://github.com/python/cpython/pull/11027 > and https://github.com/python/cpython/pull/11028 with fixes for the one > issue that Victor couldn't investigate. If someone can get a Windows > buildbot to run against them that would be great (not you Zach - your > buildbots were fine :) ). > > Cheers, > Steve > > On 07Dec2018 0435, Victor Stinner wrote: >> Hi, >> >> I had to revert a change since it broke buildbots. Sadly, I don't have >> the bandwidth to investigate the failures and try to fix the change >> :-( >> >> >> A large change has been pushed into the 3.7 and master branches to >> "Add Windows App Store package": >> >> "Release Windows Store app containing Python" >> https://bugs.python.org/issue34977 >> >> These changes broke all Windows buildbots: >> >> "Almost all Windows buildbots are failing to compile" >> https://bugs.python.org/issue35437 >> >> So I reverted these two changes. >> >> It's a large change which mostly add new files, but also make changes >> in different files: >> --- >> commit 468a15aaf9206448a744fc5eab3fc21f51966aad >> Author: Steve Dower >> Date: Thu Dec 6 21:09:20 2018 -0800 >> >> bpo-34977: Add Windows App Store package (GH-10245) >> >> .azure-pipelines/windows-appx-test.yml | 65 +++ >> .gitattributes | 1 + >> Doc/make.bat | 2 + >> Lib/test/test_pathlib.py | 2 +- >> Lib/test/test_venv.py | 1 + >> Lib/venv/__init__.py | 49 +- >> .../2018-10-30-13-39-17.bpo-34977.0l7_QV.rst | 1 + >> PC/classicAppCompat.can.xml | Bin 0 -> 3978 bytes >> PC/classicAppCompat.cat | Bin 0 -> 10984 bytes >> PC/classicAppCompat.sccd | Bin 0 -> 18503 bytes >> PC/getpathp.c | 8 +- >> PC/icons/pythonwx150.png | Bin 0 -> 8187 bytes >> PC/icons/pythonwx44.png | Bin 0 -> 2232 bytes >> PC/icons/pythonx150.png | Bin 0 -> 8271 bytes >> PC/icons/pythonx44.png | Bin 0 -> 2178 bytes >> PC/icons/pythonx50.png | Bin 0 -> 2190 bytes >> PC/launcher.c | 220 +++++++- >> PC/layout/__init__.py | 0 >> PC/layout/__main__.py | 14 + >> PC/layout/main.py | 612 +++++++++++++++++++++ >> PC/layout/support/__init__.py | 0 >> PC/layout/support/appxmanifest.py | 487 ++++++++++++++++ >> PC/layout/support/catalog.py | 44 ++ >> PC/layout/support/constants.py | 28 + >> .../support/distutils.command.bdist_wininst.py | 25 + >> PC/layout/support/filesets.py | 100 ++++ >> PC/layout/support/logging.py | 93 ++++ >> PC/layout/support/options.py | 122 ++++ >> PC/layout/support/pip.py | 79 +++ >> PC/layout/support/props.py | 110 ++++ >> {Tools/nuget => PC/layout/support}/python.props | 0 >> PC/pylauncher.rc | 6 + >> PC/python_uwp.cpp | 226 ++++++++ >> PC/store_info.txt | 146 +++++ >> PCbuild/_tkinter.vcxproj | 6 + >> PCbuild/find_msbuild.bat | 10 + >> PCbuild/pcbuild.proj | 3 + >> PCbuild/pcbuild.sln | 72 +++ >> PCbuild/python.props | 3 +- >> PCbuild/python_uwp.vcxproj | 86 +++ >> PCbuild/pythoncore.vcxproj | 15 + >> PCbuild/pythonw_uwp.vcxproj | 86 +++ >> PCbuild/venvlauncher.vcxproj | 85 +++ >> PCbuild/venvwlauncher.vcxproj | 85 +++ >> Tools/msi/buildrelease.bat | 13 +- >> Tools/msi/make_appx.ps1 | 71 +++ >> Tools/msi/make_cat.ps1 | 34 ++ >> Tools/msi/make_zip.proj | 9 +- >> Tools/msi/make_zip.py | 250 --------- >> Tools/msi/sdktools.psm1 | 43 ++ >> Tools/msi/sign_build.ps1 | 34 ++ >> Tools/nuget/make_pkg.proj | 38 +- >> 52 files changed, 3053 insertions(+), 331 deletions(-) >> --- >> >> Note: the commit message is a single line. >> >> >> Now I have questions :-) >> >> >> It seems like the change is "experimental": >> https://bugs.python.org/issue34977#msg331267 >> >> """ >> Making the *release experimental* as part of the next 3.7 update was >> approved by Ned (over email), so I merged the build. As soon as we >> snap for the RC I'll kick off an update and make the store page >> public, and hopefully can promote it enough to get eyes on it. >> >> Unfortunately, I discovered as part of a test submission that the >> minimum Windows version matters, and it's a version that hasn't been >> rolled out fully yet *because of some bugs*, so there may not be that >> many people who can use it this first time. But that will *improve >> over time*, and I'm sure I can find enough people to flush out issues >> before the next release (anyone on the Windows Insiders program should >> be fine). >> """ >> >> If it's experimental, why pushing it *right now* to the 3.7 branch? >> Can't we wait until it's stable enough? Even if it's stable, I'm not >> sure sure that it should go into 3.7 which is a stable branch. >> >> I guess that Steve would like to get this feature into 3.7.2. Ned >> Deily (3.7 release changer) approved this change. >> >> The pull request was merged one week after it has been created, I >> don't see any review. >> https://github.com/python/cpython/pull/10245 >> >> In general, I'm fine with merging changes without any kind of review, >> when the change is small. I'm doing it *frequently* when I'm confident >> that the change is small and safe enough. But here, the change is >> quite large. Sadly, I know the problem: the lack of available >> developers for review. There are very few developers who know Windows >> and are available to provide a review. Sometimes, Zachary Ware or Eryk >> Sun are around and can help. >> >> I'm fine with iterating in the master branch, since the change seems >> to be separated from CPython core: it mostly add new files. My concern >> is more about changes on "Python itself": the venv and tests changes. >> >> In the middle of the large change, there is small change on the venv module: >> >> diff --git a/Lib/venv/__init__.py b/Lib/venv/__init__.py >> index 043420897e..5438b0d4e5 100644 >> --- a/Lib/venv/__init__.py >> +++ b/Lib/venv/__init__.py >> @@ -64,10 +64,11 @@ class EnvBuilder: >> self.system_site_packages = False >> self.create_configuration(context) >> self.setup_python(context) >> + if not self.upgrade: >> + self.setup_scripts(context) >> if self.with_pip: >> self._setup_pip(context) >> if not self.upgrade: >> - self.setup_scripts(context) >> self.post_setup(context) >> if true_system_site_packages: >> >> Moreover, the commit also changes tests: >> >> * Lib/test/test_pathlib.py >> * Lib/test/test_venv.py >> >> The commit message is just "bpo-34977: Add Windows App Store package >> (GH-10245)", it doesn't explain these changes. >> >> I would prefer to see these changes extracted into separated commits. >> Sadly, right now on the cpython project on GitHub, we are only allowed >> to squash all commits into a single commit (note: the individual >> commit in the PR doesn't explain these venv/tests changes neither). So >> I suggest to create multiple PRs (at least one for venv+pathlib >> changes and a second for the Windows AppStore). >> >> I guess that these changes are bugfixes or enhancement. Having a >> separated change should also ease to backport these changes up to 3.6 >> ;-) >> >> -- >> >> Jeremy Kloth just created: >> "Correctly detect installed SDK versions" >> https://bugs.python.org/issue35433 >> >> I'm not sure if it's related to the AppStore change? >> >> Victor From jeremy.kloth at gmail.com Fri Dec 7 11:13:35 2018 From: jeremy.kloth at gmail.com (Jeremy Kloth) Date: Fri, 7 Dec 2018 09:13:35 -0700 Subject: [Python-Dev] I reverted "Add Windows App Store package" change In-Reply-To: <1bff9c35-2284-27ee-11b3-bab6072bf9aa@python.org> References: <4937db92-3a2e-5a6a-6e3b-a4e502277d16@python.org> <1bff9c35-2284-27ee-11b3-bab6072bf9aa@python.org> Message-ID: > On 07Dec2018 0716, Steve Dower wrote: > > It didn't break all Windows buildbots. > > > > That said, it's totally my fault for merging and then not watching. Also > > for not submitting custom builds to all the buildbots (Can we still do > > that? I'm not seeing any UI right now... I did run a number of test > > release builds on the release machine, so I knew that was going to be okay.) As to the breakage on my buildbot (https://buildbot.python.org/all/#/workers/12), it seems to be caused by not having VS2017 (my guess due to missing C++ headers). Unless something has changed recently, the guides still state the building with VS2015 is OK and I would like to keep my buildbot testing at the *minimum* required components for building (hence the VS9.0 builder) to keep us honest with any changes. I have no problem adding VS2017 for newer Python versions, but I think is build scripts always chose the latest version of the build tools thus making testing with older toolsets impossible. -- Jeremy Kloth From zachary.ware+pydev at gmail.com Fri Dec 7 11:17:10 2018 From: zachary.ware+pydev at gmail.com (Zachary Ware) Date: Fri, 7 Dec 2018 10:17:10 -0600 Subject: [Python-Dev] I reverted "Add Windows App Store package" change In-Reply-To: <4937db92-3a2e-5a6a-6e3b-a4e502277d16@python.org> References: <4937db92-3a2e-5a6a-6e3b-a4e502277d16@python.org> Message-ID: On Fri, Dec 7, 2018 at 9:17 AM Steve Dower wrote: > Also for not submitting custom builds to all the buildbots (Can we still do > that? I'm not seeing any UI right now... I did run a number of test > release builds on the release machine, so I knew that was going to be okay.) The UX is not great, but you can trigger a custom builder build by pushing to the `buildbot-custom` branch of python/cpython. Eventually, time will materialize to make it possible for core devs to trigger builds from branches on their personal fork since we do have proper auth now. -- Zach From vstinner at redhat.com Fri Dec 7 11:17:58 2018 From: vstinner at redhat.com (Victor Stinner) Date: Fri, 7 Dec 2018 17:17:58 +0100 Subject: [Python-Dev] I reverted "Add Windows App Store package" change In-Reply-To: <1bff9c35-2284-27ee-11b3-bab6072bf9aa@python.org> References: <4937db92-3a2e-5a6a-6e3b-a4e502277d16@python.org> <1bff9c35-2284-27ee-11b3-bab6072bf9aa@python.org> Message-ID: Le ven. 7 d?c. 2018 ? 16:42, Steve Dower a ?crit : > As a slight aside, 8 out of 8 buildbot messages on the PR look like > false positives, and none of the true positives sent a message. What > happened there? I don't know why the PR didn't get notifications about the regression. I got something more than 20 emails in my buildbot mail box :-) Let me have a look: https://github.com/python/cpython/pull/10245#issuecomment-445184041 There are multiple messages related to the 'custom' build: Pablo's created a custom build ("buildbot-custom" Git branch) which reverted your PR in master. It seems like Pablo's buildbot which sends notifications to PRs decided that his commit is attached to your PR. You can ignore all notifications from the "custom" build. The GitHub notifications are still experimental. If someone is interested, see how you can enhance these notifications ;-) s390x Debian 3.7 failed on "git clone". Well, that's a random network issue. We are working on trying to make this specific issue quiet: https://github.com/python/buildmaster-config/pull/69 Anyone is welcome to help to enhance the Python CIs, there are plenty of things to enhance ;-) Buildbots are complex beast, but the situation should be better nowadays than 2 years ago. The number of random failure is quite low. Maybe random failures are more visible today because notifications are sent on state change (basically when the color changes). Two years ago, all buildbots were always red (fail), so we never get any notification :-) I'm still fixing random failures every week: test_eintr, test_socket, etc. Victor From vstinner at redhat.com Fri Dec 7 11:31:47 2018 From: vstinner at redhat.com (Victor Stinner) Date: Fri, 7 Dec 2018 17:31:47 +0100 Subject: [Python-Dev] I reverted "Add Windows App Store package" change In-Reply-To: <4937db92-3a2e-5a6a-6e3b-a4e502277d16@python.org> References: <4937db92-3a2e-5a6a-6e3b-a4e502277d16@python.org> Message-ID: Le ven. 7 d?c. 2018 ? 16:16, Steve Dower a ?crit : > The other changes are either in Windows-only files or tests. The one > exception is venv, where they are in "if os.name=='nt'" blocks. I also > pinged our venv expert a few times with no response. Yeah, the lack of review is an issue in Python, I'm well aware of that... But I guess that it would be easier to get a review on a change modifying a single file (venv) than one PR modifying 52 files? Usually, I'm scared by giant PRs and I just skip them :-) > The PR was put up two *months* ago, not one week. Oh, wait... I read Oct 30 and I counted 7 days... sorry :-) November becomes December, not October. Sorry about that. > That said, it's totally my fault for merging and then not watching. That's fine. A revert doesn't mean that your change is wrong. The only intent is to repair the CI as soon as possible to spot other regressions, and give us more time to design the proper fix: https://pythondev.readthedocs.io/ci.html#revert-on-fail > (Can we still do that? I'm not seeing any UI right now... I did run a number of test > release builds on the release machine, so I knew that was going to be okay.) The current process is described at: https://devguide.python.org/buildbots/#custom-builders > Historically, when changes to the part you point out have been extracted > out from their context, they've been rejected on the basis that they > aren't necessary. I don't think that there is a general rule. It's usually decided on a case by case basis, and I know that each core dev has a different opinion on this question :-) I do prefer multiple small commits. I would be simpler if it would be possible to have a "patch serie": list of pull requests, or a single PR with multiple commits but don't squash them. > As it happens, I split out many changes to do with pathconfig that came > from this. You rejected them because they weren't necessary :) I'm sorry, I don't recall, which changes? > I hope that explains a bit better. It does, thanks :-) Victor From status at bugs.python.org Fri Dec 7 12:10:08 2018 From: status at bugs.python.org (Python tracker) Date: Fri, 7 Dec 2018 18:10:08 +0100 (CET) Subject: [Python-Dev] Summary of Python tracker Issues Message-ID: <20181207171008.6E36A116856@psf.upfronthosting.co.za> ACTIVITY SUMMARY (2018-11-30 - 2018-12-07) Python tracker at https://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue. Do NOT respond to this message. Issues counts and deltas: open 6904 (+24) closed 40284 (+50) total 47188 (+74) Open issues with patches: 2743 Issues opened (49) ================== #35364: Datetime ???fromtimestamp()??? ignores inheritance if timezone https://bugs.python.org/issue35364 opened by Delgan #35366: Monkey Patching class derived from ctypes.Union doesn't work https://bugs.python.org/issue35366 opened by rvijayc #35368: [2.7] Make PyMem_Malloc() thread-safe in debug mode https://bugs.python.org/issue35368 opened by vstinner #35370: Provide API to set the tracing function to be used for running https://bugs.python.org/issue35370 opened by fabioz #35374: Windows doc build does not find autodetected hhc.exe https://bugs.python.org/issue35374 opened by chrullrich #35376: modulefinder skips nested modules with same name as top-level https://bugs.python.org/issue35376 opened by rdb #35377: urlparse doesn't validate the scheme https://bugs.python.org/issue35377 opened by devkral #35378: multiprocessing.Pool.imaps iterators do not maintain alive the https://bugs.python.org/issue35378 opened by pablogsal #35379: IDLE's close fails when io.filename set to None https://bugs.python.org/issue35379 opened by rhettinger #35381: Heap-allocated posixmodule types https://bugs.python.org/issue35381 opened by eelizondo #35383: lib2to3 raises ParseError on argument called "print" https://bugs.python.org/issue35383 opened by n_rosenstein #35385: time module: why not using tzname from the glibc? https://bugs.python.org/issue35385 opened by vstinner #35387: Dialogs on IDLE are accompanied by a small black window https://bugs.python.org/issue35387 opened by wordtech #35388: _PyRuntime_Initialize() called after Py_Finalize() does nothin https://bugs.python.org/issue35388 opened by vstinner #35391: threading.RLock exception handling while waiting https://bugs.python.org/issue35391 opened by Omer Bartal #35392: Create asyncio/sockutils.py https://bugs.python.org/issue35392 opened by asvetlov #35393: Typo in documentation https://bugs.python.org/issue35393 opened by autom #35394: Add __slots__ = () to asyncio protocols https://bugs.python.org/issue35394 opened by asvetlov #35397: Undeprecate and document urllib.parse.unwrap https://bugs.python.org/issue35397 opened by steven.daprano #35398: SQLite incorrect row count for UPDATE https://bugs.python.org/issue35398 opened by Montana Low #35399: Sysconfig bug https://bugs.python.org/issue35399 opened by neyuru #35400: PGOMGR : warning PG0188: https://bugs.python.org/issue35400 opened by neyuru #35401: Upgrade Windows and macOS installers to use OpenSSL 1.1.0j / 1 https://bugs.python.org/issue35401 opened by ned.deily #35402: Upgrade macOS (and Windows?) installer to Tcl/Tk 8.6.9.1 https://bugs.python.org/issue35402 opened by ned.deily #35403: support application/wasm in mimetypes and http.server https://bugs.python.org/issue35403 opened by pmpp #35404: Document how to import _structure in email.message https://bugs.python.org/issue35404 opened by charlax #35409: Async generator might re-throw GeneratorExit on aclose() https://bugs.python.org/issue35409 opened by vxgmichel #35410: copy.deepcopy does not respect metaclasses with __deepcopy__ i https://bugs.python.org/issue35410 opened by elibixby #35412: test_future4 ran no test https://bugs.python.org/issue35412 opened by vstinner #35413: test_multiprocessing_fork: test_del_pool() leaks dangling thre https://bugs.python.org/issue35413 opened by vstinner #35414: A reference counting bug in PyState_RemoveModule() https://bugs.python.org/issue35414 opened by ZackerySpytz #35415: fileno argument to socket.socket is not validated https://bugs.python.org/issue35415 opened by Dima.Tisnek #35416: Fix potential resource warnings in distutils https://bugs.python.org/issue35416 opened by Tiger-222 #35417: Double parenthesis in print function running 2to3 in already c https://bugs.python.org/issue35417 opened by jondaa #35419: Thread.is_alive while running Process.is_alive causes either p https://bugs.python.org/issue35419 opened by Hexorg #35420: how to migrate a c-extension module to one that supports subin https://bugs.python.org/issue35420 opened by mattip #35422: misleading error message from ssl.get_server_certificate() whe https://bugs.python.org/issue35422 opened by cedricvanrompay #35423: Signal handling machinery still relies on "pending calls". https://bugs.python.org/issue35423 opened by eric.snow #35424: multiprocessing.Pool: emit ResourceWarning https://bugs.python.org/issue35424 opened by vstinner #35425: test_eintr fails randomly on AMD64 FreeBSD 10-STABLE Non-Debug https://bugs.python.org/issue35425 opened by vstinner #35426: test_signal.test_interprocess_signal() race condition https://bugs.python.org/issue35426 opened by vstinner #35427: logging UnicodeDecodeError from undecodable strftime output https://bugs.python.org/issue35427 opened by mark.dickinson #35428: xml.etree.ElementTree.tostring violates W3 standards allowing https://bugs.python.org/issue35428 opened by Zim #35429: Incorrect use of raise NotImplemented https://bugs.python.org/issue35429 opened by rth #35431: The math module should provide a function for computing binomi https://bugs.python.org/issue35431 opened by kellerfuchs #35432: str.format and string.Formatter bug with French (and other) lo https://bugs.python.org/issue35432 opened by canuck7 #35433: Correctly detect installed SDK versions https://bugs.python.org/issue35433 opened by jkloth #35435: Documentation of 3.3 is available https://bugs.python.org/issue35435 opened by matrixise #35436: Add missing PyErr_NoMemory() calls https://bugs.python.org/issue35436 opened by ZackerySpytz Most recent 15 issues with no replies (15) ========================================== #35425: test_eintr fails randomly on AMD64 FreeBSD 10-STABLE Non-Debug https://bugs.python.org/issue35425 #35420: how to migrate a c-extension module to one that supports subin https://bugs.python.org/issue35420 #35419: Thread.is_alive while running Process.is_alive causes either p https://bugs.python.org/issue35419 #35415: fileno argument to socket.socket is not validated https://bugs.python.org/issue35415 #35413: test_multiprocessing_fork: test_del_pool() leaks dangling thre https://bugs.python.org/issue35413 #35410: copy.deepcopy does not respect metaclasses with __deepcopy__ i https://bugs.python.org/issue35410 #35409: Async generator might re-throw GeneratorExit on aclose() https://bugs.python.org/issue35409 #35404: Document how to import _structure in email.message https://bugs.python.org/issue35404 #35402: Upgrade macOS (and Windows?) installer to Tcl/Tk 8.6.9.1 https://bugs.python.org/issue35402 #35401: Upgrade Windows and macOS installers to use OpenSSL 1.1.0j / 1 https://bugs.python.org/issue35401 #35400: PGOMGR : warning PG0188: https://bugs.python.org/issue35400 #35399: Sysconfig bug https://bugs.python.org/issue35399 #35397: Undeprecate and document urllib.parse.unwrap https://bugs.python.org/issue35397 #35394: Add __slots__ = () to asyncio protocols https://bugs.python.org/issue35394 #35392: Create asyncio/sockutils.py https://bugs.python.org/issue35392 Most recent 15 issues waiting for review (15) ============================================= #35436: Add missing PyErr_NoMemory() calls https://bugs.python.org/issue35436 #35433: Correctly detect installed SDK versions https://bugs.python.org/issue35433 #35431: The math module should provide a function for computing binomi https://bugs.python.org/issue35431 #35429: Incorrect use of raise NotImplemented https://bugs.python.org/issue35429 #35424: multiprocessing.Pool: emit ResourceWarning https://bugs.python.org/issue35424 #35423: Signal handling machinery still relies on "pending calls". https://bugs.python.org/issue35423 #35416: Fix potential resource warnings in distutils https://bugs.python.org/issue35416 #35415: fileno argument to socket.socket is not validated https://bugs.python.org/issue35415 #35414: A reference counting bug in PyState_RemoveModule() https://bugs.python.org/issue35414 #35409: Async generator might re-throw GeneratorExit on aclose() https://bugs.python.org/issue35409 #35404: Document how to import _structure in email.message https://bugs.python.org/issue35404 #35398: SQLite incorrect row count for UPDATE https://bugs.python.org/issue35398 #35394: Add __slots__ = () to asyncio protocols https://bugs.python.org/issue35394 #35381: Heap-allocated posixmodule types https://bugs.python.org/issue35381 #35378: multiprocessing.Pool.imaps iterators do not maintain alive the https://bugs.python.org/issue35378 Top 10 most discussed issues (10) ================================= #34172: multiprocessing.Pool and ThreadPool leak resources after being https://bugs.python.org/issue34172 25 msgs #35431: The math module should provide a function for computing binomi https://bugs.python.org/issue35431 18 msgs #10496: Python startup should not require passwd entry https://bugs.python.org/issue10496 13 msgs #34977: Release Windows Store app containing Python https://bugs.python.org/issue34977 12 msgs #35378: multiprocessing.Pool.imaps iterators do not maintain alive the https://bugs.python.org/issue35378 11 msgs #22005: datetime.__setstate__ fails decoding python2 pickle https://bugs.python.org/issue22005 9 msgs #34850: Emit a syntax warning for "is" with a literal https://bugs.python.org/issue34850 7 msgs #35377: urlparse doesn't validate the scheme https://bugs.python.org/issue35377 7 msgs #35433: Correctly detect installed SDK versions https://bugs.python.org/issue35433 7 msgs #35364: Datetime ???fromtimestamp()??? ignores inheritance if timezone https://bugs.python.org/issue35364 6 msgs Issues closed (50) ================== #16865: ctypes arrays >=2GB in length causes exception https://bugs.python.org/issue16865 closed by serhiy.storchaka #20371: datetime.datetime.replace bypasses a subclass's __new__ https://bugs.python.org/issue20371 closed by belopolsky #22496: urllib2 fails against IIS (urllib2 can't parse 401 reply www-a https://bugs.python.org/issue22496 closed by deronnax #25862: TextIOWrapper assertion failure after read() and SEEK_CUR https://bugs.python.org/issue25862 closed by serhiy.storchaka #29564: ResourceWarning: suggest to enable tracemalloc in the message https://bugs.python.org/issue29564 closed by vstinner #31177: unittest mock's reset_mock throws an error when an attribute h https://bugs.python.org/issue31177 closed by xtreak #32787: Better error handling in ctypes https://bugs.python.org/issue32787 closed by serhiy.storchaka #33709: test.support.FS_NONASCII returns incorrect result in Windows w https://bugs.python.org/issue33709 closed by serhiy.storchaka #34052: sqlite's create_function() raises exception on unhashable call https://bugs.python.org/issue34052 closed by serhiy.storchaka #34185: Lib/test/test_bdb.py failed when ran as a script https://bugs.python.org/issue34185 closed by serhiy.storchaka #34604: Possible mojibake in pwd.getpwnam and grp.getgrnam https://bugs.python.org/issue34604 closed by serhiy.storchaka #34738: Distutils: ZIP files don't include directory entries https://bugs.python.org/issue34738 closed by serhiy.storchaka #34784: Heap-allocated StructSequences https://bugs.python.org/issue34784 closed by petr.viktorin #34987: A possible null pointer dereference in _pickle.c's save_reduce https://bugs.python.org/issue34987 closed by ZackerySpytz #35226: mock.call equality surprisingly broken https://bugs.python.org/issue35226 closed by cjw296 #35250: Minor parameter documentation mismatch for turtle https://bugs.python.org/issue35250 closed by serhiy.storchaka #35305: subprocess.Popen(['/sbin/ldconfig', '-p'], stdin=PIPE) itself https://bugs.python.org/issue35305 closed by vstinner #35310: select which was interrupted by EINTR isn't re-run if the time https://bugs.python.org/issue35310 closed by vstinner #35316: test_eintr fails randomly on macOS https://bugs.python.org/issue35316 closed by vstinner #35341: Add generic version of OrderedDict to typing module https://bugs.python.org/issue35341 closed by levkivskyi #35344: platform: get macOS version rather than darwin version https://bugs.python.org/issue35344 closed by vstinner #35352: test_asyncio fails on RHEL8, or on Fedora using NEXT security https://bugs.python.org/issue35352 closed by vstinner #35357: unittest.mock.call can't represent calls to a method called 'p https://bugs.python.org/issue35357 closed by cjw296 #35359: [2.7][Windows] Define _CRT_SECURE_NO_WARNINGS to build Modules https://bugs.python.org/issue35359 closed by vstinner #35363: test_eintr: test_open() hangs randomly on x86-64 El Capitan 3. https://bugs.python.org/issue35363 closed by vstinner #35365: Use wchar_t* buffer instead of Unicode object in code page dec https://bugs.python.org/issue35365 closed by serhiy.storchaka #35367: FileExistsError During os.symlink() Displays Arrow in the Wron https://bugs.python.org/issue35367 closed by eryksun #35369: List sorting makes duplicate comparisons https://bugs.python.org/issue35369 closed by rhettinger #35371: Fix undefined behavior in os.utime() on Windows https://bugs.python.org/issue35371 closed by serhiy.storchaka #35372: Code page decoder incorrectly handles input >2GiB https://bugs.python.org/issue35372 closed by serhiy.storchaka #35373: PyInit_timezone() must return a value https://bugs.python.org/issue35373 closed by vstinner #35375: name shadowing while a module tries to import another https://bugs.python.org/issue35375 closed by ksriram #35380: Enable TCP_NODELAY for proactor event loop https://bugs.python.org/issue35380 closed by asvetlov #35382: Something wrong with pymysql https://bugs.python.org/issue35382 closed by mark.dickinson #35384: The repr of ctypes.CArgObject fails for non-ascii character https://bugs.python.org/issue35384 closed by serhiy.storchaka #35386: ftp://www.pythontest.net/ returns error 500 https://bugs.python.org/issue35386 closed by vstinner #35389: Use gnu_get_libc_version() in platform.libc_ver()? https://bugs.python.org/issue35389 closed by vstinner #35390: ctypes not possible to pass NULL c_void_p in structure by refe https://bugs.python.org/issue35390 closed by dtamayo #35395: Typo in asyncio eventloop documentation https://bugs.python.org/issue35395 closed by asvetlov #35396: Add support for __fspath__ to fnmatch.fnmatchase and filter https://bugs.python.org/issue35396 closed by adelfino #35405: Wrong traceback for AssertionError while running under pdb https://bugs.python.org/issue35405 closed by xtreak #35406: calendar.nextmonth and calendar.prevmonth functions doesn't ch https://bugs.python.org/issue35406 closed by asocia #35407: Datetime function with selenium https://bugs.python.org/issue35407 closed by steven.daprano #35408: Python3.7 crash in PyCFunction_New due to broken _PyObject_GC_ https://bugs.python.org/issue35408 closed by vstinner #35411: FTP tests of test_urllib2net fail on Travis CI: 425 Security: https://bugs.python.org/issue35411 closed by vstinner #35418: python hung or stuck somtimes randomly on windows server 2008R https://bugs.python.org/issue35418 closed by Cao Hongfu #35421: Expected result is not clear in case of list.append(list) https://bugs.python.org/issue35421 closed by eric.smith #35430: Lib/argparse.py uses `is` for string comparison https://bugs.python.org/issue35430 closed by serhiy.storchaka #35434: Wrong bpo linked in What's New in 3.8 https://bugs.python.org/issue35434 closed by Mariatta #35437: Almost all Windows buildbots are failing to compile https://bugs.python.org/issue35437 closed by vstinner From tjreedy at udel.edu Fri Dec 7 16:40:03 2018 From: tjreedy at udel.edu (Terry Reedy) Date: Fri, 7 Dec 2018 16:40:03 -0500 Subject: [Python-Dev] I reverted "Add Windows App Store package" change In-Reply-To: References: <4937db92-3a2e-5a6a-6e3b-a4e502277d16@python.org> Message-ID: On 12/7/2018 11:31 AM, Victor Stinner wrote: > I would be simpler if it would be possible to have a "patch serie": > list of pull requests, One can make an 'index issue' with multiple dependencies, each with a PR. I do this for multiple independent changes to a module or related modules. > or a single PR with multiple commits but don't squash them. Already possible on Github. If one clicks [View Changes], multiple commits in a PR are combined for display purposes into a net PR diff. But the individual commits can be examined and reviewed individually and are not squashed into one by github (and should not be by authors) until the merge into cpython. Typically, authors effectively 'squash' all initial changes into one commit, make a PR, and only add additional commits in review. But authors *can* split the initial editing into multiple commits with an eye to easing review -- and record information for future maintainers. Simple bugfix example: Add test to test_mod that fails with TwinkleError. Posted to issue by Joe Blow. Make new test pass using the 'underhand' strategy. The split above is not really necessary, but PR 10245 squashed changes to 52 files of 15 file types into one initial commit. https://github.com/python/cpython/pull/10245/commits/17ba155a7b794926ce705ee0e2af787fbd2996e6 View Files displays them alphabetically. Jump to ... lists them in the same order, but includes the functions changed, so it is hundreds of lines. I think this PR would have benefited from having perhaps 10 or more initial commits, each with a comment about the group of files included. In icon commit could have said something about the source and purpose of the added icon files. One commit could have included the venv and test changes that you want to review. An added message, as long as needed, could have explained these particular changes. -- Terry Jan Reedy From steve.dower at python.org Fri Dec 7 18:06:25 2018 From: steve.dower at python.org (Steve Dower) Date: Fri, 7 Dec 2018 15:06:25 -0800 Subject: [Python-Dev] I reverted "Add Windows App Store package" change In-Reply-To: References: <4937db92-3a2e-5a6a-6e3b-a4e502277d16@python.org> Message-ID: <94dcd7b1-6566-bf8b-0fc0-40b264858290@python.org> On 07Dec2018 1340, Terry Reedy wrote: > Simple bugfix example: > Add test to test_mod that fails with TwinkleError. > Posted to issue by Joe Blow. > Make new test pass using the 'underhand' strategy. > > The split above is not really necessary, but PR 10245 squashed changes > to 52 files of 15 file types into one initial commit. > > https://github.com/python/cpython/pull/10245/commits/17ba155a7b794926ce705ee0e2af787fbd2996e6 > > > View Files displays them alphabetically.? Jump to ... lists them in the > same order, but includes the functions changed, so it is hundreds of lines. > > I think this PR would have benefited from having perhaps 10 or more > initial commits, each with a comment about the group of files included. > In icon commit could have said something about the source and purpose of > the added icon files.? One commit could have included the venv and test > changes that you want to review.? An added message, as long as needed, > could have explained these particular changes. > This is great in theory, but if you look back at the original PR it would have been 100+ commits (I was occasionally squashing and force pushing). What you're really proposing is: * do all the work using the git workflow (stream-of-consiousness commits) * squash all the commits at the end * re-expand the single commit into logical groupings of files and re-commit them So it's easy to sit back and imagine that I had all the perfect changes ready to go and deliberately chose to make it harder to review, but that's not the case at all. Making it sound like this is how development works is really disparaging to people who find themselves not producing perfect commit histories. And let's be honest, there's no good tooling for turning a series of interdependent commits into a smaller set of sensible ones. Squashing at least gets rid of the changes that were reverted as part of the entire PR, and if you then just want to split by file you're really just asking for extra work. If someone had bothered to say "I'll review the parts of it that are relevant to me if you can split them out" then I would have, but nobody (in public) showed any interest in reviewing the changes at all - I had some private reviews done by colleagues at work, who weren't so demanding about it. I review as many PRs as I send these days (maybe more? I'm not counting), and I always try to make an effort to have mercy towards people to save them having to work extra hard just to make my life a little bit easier. It grates to have had an incremental change visible in public for over two months, have it be totally ignored the entire time, and then have to defend something that I already did. Luckily I get paid work time for doing this; really can't see why I'd want to suffer through this for free... From ncoghlan at gmail.com Sat Dec 8 11:32:53 2018 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sun, 9 Dec 2018 02:32:53 +1000 Subject: [Python-Dev] I reverted "Add Windows App Store package" change In-Reply-To: <94dcd7b1-6566-bf8b-0fc0-40b264858290@python.org> References: <4937db92-3a2e-5a6a-6e3b-a4e502277d16@python.org> <94dcd7b1-6566-bf8b-0fc0-40b264858290@python.org> Message-ID: On Sat, 8 Dec 2018 at 09:10, Steve Dower wrote: > And let's be honest, there's no good tooling for turning a series of > interdependent commits into a smaller set of sensible ones. Squashing at > least gets rid of the changes that were reverted as part of the entire > PR, and if you then just want to split by file you're really just asking > for extra work. Whether the UX counts as "good" or not is open to debate (I consider it pretty good for the complexity of the task it handles), but if you ever want to revise the history of a complex patch series to make it easier for reviewers to follow: 1. Use "git rebase --interactive" to squash all the ad hoc commits into a single commit 2. Use "git reset HEAD^" to unstage that squashed monolithic commit 3. Use "git add --patch" to compose a new commit series that takes a reviewer through a logical set of changes, rather than the messy reality of what actually happened (I have no idea if there are any GUI tools which expose this level of commit series editing power, but it exists on the command line, so presumably there are graphical equivalents) Most of the time this isn't worth the hassle, with either reviewing the PR as a whole being good enough, or else there being subsets of the changes which can be made into separate PRs that can be reviewed and accepted independently. However, on those occassions where you're needing to tell the reviewers a story not only about what you changed, but also why you changed it, then an hour or two spent up front creating a more coherent commit history may save multiple hours of discussion later. Cheers, Nick. P.S. Gerrit's entire review model is based around this idea of dependent patch series, but code review UX turned out to be a situation where optimising for the simple, common case (i.e. GitHub's approach) proved to be a much better idea than focusing on the less common complex cases. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From tjreedy at udel.edu Sat Dec 8 22:56:12 2018 From: tjreedy at udel.edu (Terry Reedy) Date: Sat, 8 Dec 2018 22:56:12 -0500 Subject: [Python-Dev] I reverted "Add Windows App Store package" change In-Reply-To: References: <4937db92-3a2e-5a6a-6e3b-a4e502277d16@python.org> <94dcd7b1-6566-bf8b-0fc0-40b264858290@python.org> Message-ID: On 12/8/2018 11:32 AM, Nick Coghlan wrote: > Whether the UX counts as "good" or not is open to debate (I consider > it pretty good for the complexity of the task it handles), but if you > ever want to revise the history of a complex patch series to make it > easier for reviewers to follow: > > 1. Use "git rebase --interactive" to squash all the ad hoc commits > into a single commit > 2. Use "git reset HEAD^" to unstage that squashed monolithic commit > 3. Use "git add --patch" to compose a new commit series that takes a > reviewer through a logical set of changes, rather than the messy > reality of what actually happened Thank you for the information. I am sure I will use it. > (I have no idea if there are any GUI tools which expose this level of > commit series editing power, but it exists on the command line, so > presumably there are graphical equivalents) On Windows, I use git gui routinely for making commits. It lists files with workspace changes in Unstaged and Staged boxes. ^T (sTage) and ^U (Unstage) moves a hightlighted file to the other one. A third box shows the diff for a highlighted file. One can also revise the 'last' commit (never done this yet). A fourth box is for entering and editing commit messages. -- Terry Jan Reedy From barry at barrys-emacs.org Sun Dec 9 09:32:54 2018 From: barry at barrys-emacs.org (Barry Scott) Date: Sun, 9 Dec 2018 14:32:54 +0000 Subject: [Python-Dev] windows compiler list missing 3.7 details on wiki In-Reply-To: References: <16BCAA90-23DF-4B6A-8A78-DEB397109133@barrys-emacs.org> Message-ID: <873A4CF1-7651-4144-8A4E-D598740A6100@barrys-emacs.org> > On 3 Nov 2018, at 17:57, Steve Dower wrote: > > Yes. Visual Studio 2015 or later can be used (and as this is the only way to get the compiler right now, I think it's fine to list that as the requirement - note that the "Visual Studio Build Tools" installer doesn't include the IDE itself). > > Feel free to update the wiki. Just got around to doing this. That page is marked as Immutable Page for me. Do I need to apply for edit permissions? I'm user BarryScott on the wiki. Barry > > Cheers, > Steve > > On 03Nov2018 0208, Barry Scott wrote: >> On https://wiki.python.org/moin/WindowsCompilers details for 3.7 are missing. >> I'm assuming its still VC V14 >> Barry > From nad at python.org Mon Dec 10 02:16:46 2018 From: nad at python.org (Ned Deily) Date: Mon, 10 Dec 2018 02:16:46 -0500 Subject: [Python-Dev] 3.7.2rc1 and 3.6.8rc1 cutoffs ahead, last 3.6.x bugfix release! In-Reply-To: <95B9F244-38A8-4E47-8E9D-6DAD289C84A2@python.org> References: <95B9F244-38A8-4E47-8E9D-6DAD289C84A2@python.org> Message-ID: <58C8B7D8-80EF-46D3-BCBE-4ACC95382D0A@python.org> On Dec 4, 2018, at 03:42, Ned Deily wrote: > https://discuss.python.org/t/3-7-2rc1-and-3-6-8rc1-cutoffs-ahead-last-3-6-x-bugfix-release/510 An update: as of the planned Friday cutoff, we still had a few open issues. Since 3.6.8 is the last planned bugfix for the 3.6 branch, I would like to make sure we leave it in as good a state as possible before it moves to security-fix-only mode. Also, the Windows App Store support for 3.7.x that Steve D has been working on is in final review and it would be great to have that in the release candidate. So I'm going to extend the cutoff for 3.7.2rc1 and 3.6.8rc1 until **Monday, 12-10, end of day (AOE**), in other words **about 30 hours from now**. Thanks for all your efforts so far! > We're reaching the end of the year and it's time for another pair of Python 3 maintenance releases, 3.7.2 and 3.6.8, before we ring out 2018. Since there are still some open release blocker issues and I haven't been bugging you about them, I've moved the code cutoff for the release candidates to this coming Friday, 12-07, by the end of the day (AOE). That gives us all another 4 days to review open issues and PRs. Please give highest attention to any release blockers you have been shepherding or reviewing. Thanks! > > A reminder: as previously announced, 3.6.8 is planned to be the last bugfix release of the 3.6 series. Python 3.6.0 was released on 2016-12-23, so by the time 3.6.8 is released, 3.6.x will have been in bugfix mode almost exactly 2 years. When a new feature release is made and enters "bugfix" mode, our policy has long been to continue to maintain the previous bugfix branch for at least one more release and then move that branch to "security fix only" mode. 3.7.0 (and 3.6.6) was released nearly six months ago and, with the release of 3.6.8, there will have been two additional 3.6.x bugfix releases since then. So, barring any showstopper issues that might arise, the upcoming 3.6.8rc1 is your last chance to make bugfix changes for 3.6.x. Following the successful release of 3.6.8, only security fixes will be accepted for the 3.6 branch and future 3.6.x releases will be source-only and scheduled as needed; no further binary installers will be produced for 3.6. Refer to the Dev Guide > sections and release PEPs linked below for more information. > > > https://devguide.python.org/devcycle/ > https://devguide.python.org/#branchstatus > https://www.python.org/dev/peps/pep-0494/ > https://www.python.org/dev/peps/pep-0537/ > > -- > Ned Deily > nad at python.org -- [] > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: https://mail.python.org/mailman/options/python-dev/nad%40python.org -- Ned Deily nad at python.org -- [] From vstinner at redhat.com Tue Dec 11 09:21:31 2018 From: vstinner at redhat.com (Victor Stinner) Date: Tue, 11 Dec 2018 15:21:31 +0100 Subject: [Python-Dev] Usage of the multiprocessing API and object lifetime Message-ID: Hi, tzickel reported a reference cycle bug in multiprocessing which keeps threads and processes alive: https://bugs.python.org/issue34172 He wrote a fix which has been merged in 3.6, 3.7 and master branches. But Pablo Galindo noticed that the fix breaks the following code (he added "I found the weird code in the example in several projects."): import multiprocessing def the_test(): print("Begin") for x in multiprocessing.Pool().imap(int, ["4", "3"]): print(x) print("End") the_test() Pablo proposed to add a strong reference to the Pool from multiprocessing iterators: https://bugs.python.org/issue35378 I blocked his pull request because I see this change as a risk of new reference cycles. Since we are close to 3.6 and 3.7 releases, I decided to revert the multiprocessing fix instead. Pablo's issue35378 evolved to add a weak reference in iterators to try to detect when the Pool is destroyed: raise an exception from the iterator, if possible. Then a discussion started on how the multiprocessing API is supposed to be used and about the lifetime of multiprocessing objects. I would prefer to make the multiprocessing API stricter: Python shouldn't try to guess how long an object is going to live. The API user has to *explicitly* release resources. tzickel noted that the documentations says: "When the pool object is garbage collected terminate() will be called immediately." And that multiprocessing rely on the garbage collector to release resources, especially using multiprocessing.util.Finalize tool: class Finalize(object): ''' Class which supports object finalization using weakrefs ''' def __init__(self, obj, callback, ...): ... if obj is not None: self._weakref = weakref.ref(obj, self) else: assert exitpriority is not None ... _finalizer_registry[self._key] = self I propose to start to emit ResourceWarning in Python 3.8 when objects are not released explicitly. I wrote a first change to emit ResourceWarning in the Pool object: https://bugs.python.org/issue35424 https://github.com/python/cpython/pull/10974 By the way, I'm surprised that "with pool:" doesn't release all resources. An additional "pool.join()" is needed to ensure that all resources are released. It's a little bit surprising to have to emit a ResourceWarning if join() has not been called, even if the code uses "with pool:". I don't know well the multiprocessing API, so I'm not sure in which directions we should go: best effort to support strange legacy code with "implicit object lifetime", or become stricter in Python 3.8? >From a technical point of view, I would prefer to become stricter. Relying on the garbage collector means that the code is going to behave badly on PyPy which uses a different garbage collector implementation :-( Victor -- Night gathers, and now my watch begins. It shall not end until my death. From solipsis at pitrou.net Tue Dec 11 10:10:51 2018 From: solipsis at pitrou.net (Antoine Pitrou) Date: Tue, 11 Dec 2018 16:10:51 +0100 Subject: [Python-Dev] Usage of the multiprocessing API and object lifetime References: Message-ID: <20181211161051.457a6afa@fsol> Hi, On Tue, 11 Dec 2018 15:21:31 +0100 Victor Stinner wrote: > > Pablo's issue35378 evolved to add a weak reference in iterators to try > to detect when the Pool is destroyed: raise an exception from the > iterator, if possible. That's an ok fix for me. > By the way, I'm surprised that "with pool:" doesn't release all > resources. That's not a problem, as long as the destructor _does_ release resources. > From a technical point of view, I would prefer to become stricter. Using "with pool:" is fine, we shouldn't start raising a warning for it. What you are proposing here starts to smell like an anti-pattern to me. Python _is_ a garbage-collected language, so by definition, there _are_ going to be resources that are automatically collected when an object disappears. If I'm allocating a 2GB bytes object, then PyPy may delay the deallocation much longer than CPython. Do you propose we add a release() method to bytes objects to avoid this issue (and emit a warning for people who don't call release() on bytes objects)? You can't change the language's philosophy. We warn about open files because those have user-visible consequences (such as unflushed buffers, or not being able to delete the file on Windows). If there is no user-visible consequence to not calling join() on a Pool, then we shouldn't warn about it. Regards Antoine. From vstinner at redhat.com Tue Dec 11 10:33:54 2018 From: vstinner at redhat.com (Victor Stinner) Date: Tue, 11 Dec 2018 16:33:54 +0100 Subject: [Python-Dev] Usage of the multiprocessing API and object lifetime In-Reply-To: <20181211161051.457a6afa@fsol> References: <20181211161051.457a6afa@fsol> Message-ID: Le mar. 11 d?c. 2018 ? 16:14, Antoine Pitrou a ?crit : > What you are proposing here starts to smell like an anti-pattern to > me. Python _is_ a garbage-collected language, so by definition, there > _are_ going to be resources that are automatically collected when an > object disappears. If I'm allocating a 2GB bytes object, then PyPy may > delay the deallocation much longer than CPython. Do you propose we add > a release() method to bytes objects to avoid this issue (and emit a > warning for people who don't call release() on bytes objects)? We are not talking about simple strings, but processes and threads. > You can't change the language's philosophy. We warn about open files > because those have user-visible consequences (such as unflushed > buffers, or not being able to delete the file on Windows). If there is > no user-visible consequence to not calling join() on a Pool, then we > shouldn't warn about it. "user-visible consequences" are that resources are kept alive longer than I would expect. When I use a context manager, I expect that Python will magically releases everything for me. For example, "with subprocess.Popen() as popen: ..." ensures that all pipes are closed and the process completes, before we exit the block. Another example, "with open() as fp: ..." ensures that the file descriptor is closed before we exit the block. I modified subprocess.Popen.__del__() in Python 3.6 to emit a ResourceWarning if the subprocess is still running, to suggest the developer to explicitly manage the resource (ex: call .wait()). I prefer to explicitly manager resources like processes and threads since they can exit with error: killed by a signal, waitpid() failure (exit status already read by a different function), etc. I prefer to control where the error occurs. I hate when Python logs strange error during shutdown. Logging errors during shutdown is too late: for example, the log triggers a new error because a stdlib module has been cleared. That's why we need hacks like "_warn=warnings.warn" below: class Popen(object): ... def __del__(self, _maxsize=sys.maxsize, _warn=warnings.warn): ... if self.returncode is None: _warn("subprocess %s is still running" % self.pid, ResourceWarning, source=self) ... Victor From p.f.moore at gmail.com Tue Dec 11 10:36:42 2018 From: p.f.moore at gmail.com (Paul Moore) Date: Tue, 11 Dec 2018 15:36:42 +0000 Subject: [Python-Dev] Usage of the multiprocessing API and object lifetime In-Reply-To: <20181211161051.457a6afa@fsol> References: <20181211161051.457a6afa@fsol> Message-ID: On Tue, 11 Dec 2018 at 15:13, Antoine Pitrou wrote: > On Tue, 11 Dec 2018 15:21:31 +0100 > Victor Stinner wrote: > > > > Pablo's issue35378 evolved to add a weak reference in iterators to try > > to detect when the Pool is destroyed: raise an exception from the > > iterator, if possible. > > That's an ok fix for me. > > > By the way, I'm surprised that "with pool:" doesn't release all > > resources. > > That's not a problem, as long as the destructor _does_ release > resources. > > > From a technical point of view, I would prefer to become stricter. > > Using "with pool:" is fine, we shouldn't start raising a warning for it. > > What you are proposing here starts to smell like an anti-pattern to > me. Python _is_ a garbage-collected language, so by definition, there > _are_ going to be resources that are automatically collected when an > object disappears. If I'm allocating a 2GB bytes object, then PyPy may > delay the deallocation much longer than CPython. Do you propose we add > a release() method to bytes objects to avoid this issue (and emit a > warning for people who don't call release() on bytes objects)? > > You can't change the language's philosophy. We warn about open files > because those have user-visible consequences (such as unflushed > buffers, or not being able to delete the file on Windows). If there is > no user-visible consequence to not calling join() on a Pool, then we > shouldn't warn about it. I agree with Antoine here. On Tue, 11 Dec 2018 15:21:31 +0100 Victor Stinner wrote: > The API user has to *explicitly* release resources. That's definitely not Python's philosophy. In Python, users should not have to worry about resource management themselves, that's the job of the language runtime. We provide the "with" construct so that when users *want* to manage resources explicitly (because there is an impact outside of the Python runtime's control, for example) then they can do so. But leaving resource management to the runtime is completely fine. > I propose to start to emit ResourceWarning in Python 3.8 when objects are not released explicitly. Strong -1 on this. > I don't know well the multiprocessing API Nor do I, but I'm against making fundamental design changes such as you propose *without* a deep understanding of the multiprocessing API. If you feel sufficiently strongly that the current design is wrong, then you should understand the principles and reasons behind that design before trying to change it. Paul From solipsis at pitrou.net Tue Dec 11 10:46:35 2018 From: solipsis at pitrou.net (Antoine Pitrou) Date: Tue, 11 Dec 2018 16:46:35 +0100 Subject: [Python-Dev] Usage of the multiprocessing API and object lifetime In-Reply-To: References: <20181211161051.457a6afa@fsol> Message-ID: <20181211164635.7cc0609c@fsol> On Tue, 11 Dec 2018 16:33:54 +0100 Victor Stinner wrote: > Le mar. 11 d?c. 2018 ? 16:14, Antoine Pitrou a ?crit : > > What you are proposing here starts to smell like an anti-pattern to > > me. Python _is_ a garbage-collected language, so by definition, there > > _are_ going to be resources that are automatically collected when an > > object disappears. If I'm allocating a 2GB bytes object, then PyPy may > > delay the deallocation much longer than CPython. Do you propose we add > > a release() method to bytes objects to avoid this issue (and emit a > > warning for people who don't call release() on bytes objects)? > > We are not talking about simple strings, but processes and threads. Right, but do those have an impact on the program's correctness, or simply on its performance (or memory consumption)? > "user-visible consequences" are that resources are kept alive longer > than I would expect. When I use a context manager, I expect that > Python will magically releases everything for me. I think there's a balancing act here: between "with pool" releasing everything, and not taking too much time to execute the __exit__ method. Currently, threads and processes may finish quietly between __exit__ and __del__, without adding significant latencies to your program's execution. > I prefer to explicitly manager resources like processes and threads > since they can exit with error: killed by a signal, waitpid() failure > (exit status already read by a different function), etc. But multiprocessing.Pool manages them implicitly _by design_. People who want to manage processes explicitly can use the Process class directly ;-) Regards Antoine. From vstinner at redhat.com Tue Dec 11 11:22:49 2018 From: vstinner at redhat.com (Victor Stinner) Date: Tue, 11 Dec 2018 17:22:49 +0100 Subject: [Python-Dev] Usage of the multiprocessing API and object lifetime In-Reply-To: <20181211164635.7cc0609c@fsol> References: <20181211161051.457a6afa@fsol> <20181211164635.7cc0609c@fsol> Message-ID: Le mar. 11 d?c. 2018 ? 17:06, Antoine Pitrou a ?crit : > > We are not talking about simple strings, but processes and threads. > > Right, but do those have an impact on the program's correctness, or > simply on its performance (or memory consumption)? Performance. I made a similar change in the socketserver: server_close() now waits until child processes (ForkingMixIn) and threads (ThreadingMixIn) complete: * https://bugs.python.org/issue31233 * https://bugs.python.org/issue31151 I added an opt-in option "block_on_close" to get Python 3.6 behavior on server_close(): https://bugs.python.org/issue33540 I don't know if these changes are similar to my questions about multiprocessing API, since socketserver didn't expose the list of processes/threads and doesn't provide a method to wait until they complete. Well... ForkingMixIn has an *undocumented* collect_children() which is non-blocking by default (I added a keyword-only 'blocking' parameter). Another example: the close() method of an asyncio event loop doesn't wait until threads/processes complete: "asyncio: BaseEventLoop.close() shutdowns the executor without waiting causing leak of dangling threads" https://bugs.python.org/issue34037 It's unclear to me how this issue should be fixed. Victor From solipsis at pitrou.net Tue Dec 11 11:37:36 2018 From: solipsis at pitrou.net (Antoine Pitrou) Date: Tue, 11 Dec 2018 17:37:36 +0100 Subject: [Python-Dev] Usage of the multiprocessing API and object lifetime In-Reply-To: References: <20181211161051.457a6afa@fsol> <20181211164635.7cc0609c@fsol> Message-ID: <20181211173736.266ff61f@fsol> On Tue, 11 Dec 2018 17:22:49 +0100 Victor Stinner wrote: > Le mar. 11 d?c. 2018 ? 17:06, Antoine Pitrou a ?crit : > > > We are not talking about simple strings, but processes and threads. > > > > Right, but do those have an impact on the program's correctness, or > > simply on its performance (or memory consumption)? > > Performance. Well, at least we shouldn't emit ResourceWarning for performance issues. So if someone used "with pool:", they shouldn't get a ResourceWarning afterwards, even if some threads are still not finished running. > I don't know if these changes are similar to my questions about > multiprocessing API, since socketserver didn't expose the list of > processes/threads and doesn't provide a method to wait until they > complete. Well... ForkingMixIn has an *undocumented* > collect_children() which is non-blocking by default (I added a > keyword-only 'blocking' parameter). socketserver is not the same as a multiprocessing Pool *at all*. The usage will be vastly different between the two. Just because we did this change for socketserver is not enough of a reason to do it for multiprocessing too. > Another example: the close() method of an asyncio event loop doesn't > wait until threads/processes complete: > > "asyncio: BaseEventLoop.close() shutdowns the executor without waiting > causing leak of dangling threads" > https://bugs.python.org/issue34037 It's not a leak though, it's a resource that's collected a bit later. It may annoy the test machinery (and this could be a reason to add private or public APIs to help finish the threads), but it shouldn't be an actual issue for user code. Here as well, I think you should be careful not to introduce annoyances (unwanted latencies) in user code. Regards Antoine. From pablogsal at gmail.com Tue Dec 11 12:47:35 2018 From: pablogsal at gmail.com (Pablo Galindo Salgado) Date: Tue, 11 Dec 2018 17:47:35 +0000 Subject: [Python-Dev] Usage of the multiprocessing API and object lifetime Message-ID: > > Pablo's issue35378 evolved to add a weak reference in iterators to try > > to detect when the Pool is destroyed: raise an exception from the > > iterator, if possible. > That's an ok fix for me. I am playing with weakreferences inside the iterator and result objects, but this may not be enough/a complete solution. For example, take the code of ApplyResult.get: def get(self, timeout=None): if self._pool() is None: # self._pool is a weakref raise RuntimeError("The pool is dead. Aborting") <--- new code self.wait(timeout) It can be that the pool is alive when we check for it (self._pool() is None) but while the code is waiting with no timeout (timeout=None) the pool dies, effectively leaving the program deadlocked with no error. -- I agree that misusage of the pool should not be encouraged but in this situation the fact that this code hangs: import multiprocessing for x in multiprocessing.Pool().imap(int, ["4", "3"]): print(x) is a bit worriying because although is incorrect and an abuse of the API, users can do this easily with no error message other than a misterious hang. I have found this on several places and people were very confused because usually the interpreter throws some kind of error indication. In my humble opinion, we should try to avoid hanging as a consequence of the misusage, whatever we do. Pablo -------------- next part -------------- An HTML attachment was scrubbed... URL: From p.f.moore at gmail.com Tue Dec 11 13:36:49 2018 From: p.f.moore at gmail.com (Paul Moore) Date: Tue, 11 Dec 2018 18:36:49 +0000 Subject: [Python-Dev] Usage of the multiprocessing API and object lifetime In-Reply-To: References: Message-ID: On Tue, 11 Dec 2018 at 17:50, Pablo Galindo Salgado wrote: > I agree that misusage of the pool should not be encouraged but in this situation the fact that > this code hangs: > > import multiprocessing > > for x in multiprocessing.Pool().imap(int, ["4", "3"]): > print(x) > > > is a bit worriying because although is incorrect and an abuse of the API, users can do this easily with > no error message other than a misterious hang. OK, so the first problem here (to me, at least) is that it's not obvious *why* this code is incorrect and an abuse of the API. It takes a reasonable amount of thinking about the problem to notice that the Pool object isn't retained, but the iterator returned from imap is. And when the pool is collected, the worker processes are terminated, causing the hang, as the worker never sends a result back to the main process. But it's not obvious to me why the pool is collected before the imap method has completed. As I understand it, originally the code worked because the pool *didn't* call terminate() when collected. Now it does, and we have a problem. I'm not *entirely* sure why, if the pool is terminated, the wait in the iterator doesn't terminate immediately with some sort of "process being waited on died" error, but let's assume there are good reasons for that (as I mentioned before, I'm not an expert in multiprocessing, so I'm OK with assuming that the original design, done by people who *are* experts, is sound :-)) Your original solution would have added a strong reference back to the pool from the iterator. At first glance, that seems like a reasonable solution to me. Victor is worried about the "risk of new reference cycles". But reference cycles are not a problem - we have the cyclic GC precisely to deal with them. So I'd like to see a better justification for rejecting that solution than "there might be reference cycles". But in response to that, you made the iterator have a weak reference back to the pool. That's flawed because it doesn't prevent the pool being terminated - as you say, the deadlock is still present. > I have found this on several places and people were > very confused because usually the interpreter throws some kind of error indication. In my humble opinion, > we should try to avoid hanging as a consequence of the misusage, whatever we do. I agree with this. But that implies to me that we should be holding a strong reference to the pool, As a (somewhat weak) analogy, consider for n in map(int, ["1", "2"]): print(n) That won't fail if the list gets collected, because map keeps a reference to the list. My intuition would be that the Pool().imap example would hold a reference to the pool on essentially the same basis. The more I think about this, the more I struggle to see Victor's logic for rejecting your original solution. And I *certainly* don't see why this issue should justify changing the whole API to require users to explicitly manage pool lifetimes. Paul From pablogsal at gmail.com Tue Dec 11 14:18:22 2018 From: pablogsal at gmail.com (Pablo Galindo Salgado) Date: Tue, 11 Dec 2018 19:18:22 +0000 Subject: [Python-Dev] Usage of the multiprocessing API and object lifetime In-Reply-To: References: Message-ID: > > Your original solution would have added a strong reference back to the > pool from the iterator. At first glance, that seems like a reasonable > solution to me. Victor is worried about the "risk of new reference > cycles". But reference cycles are not a problem - we have the cyclic > GC precisely to deal with them. So I'd like to see a better > justification for rejecting that solution than "there might be > reference cycles". But in response to that, you made the iterator have > a weak reference back to the pool. That's flawed because it doesn't > prevent the pool being terminated - as you say, the deadlock is still > present. Just to be clear: I am in favour of the strong reference, but I also understand the "danger" (leaking the pool until the pool's generation reaches the threshold and the gc runs) and that is the reason I was experimenting with the weakreference. On Tue, 11 Dec 2018 at 18:37, Paul Moore wrote: > On Tue, 11 Dec 2018 at 17:50, Pablo Galindo Salgado > wrote: > > I agree that misusage of the pool should not be encouraged but in this > situation the fact that > > this code hangs: > > > > import multiprocessing > > > > for x in multiprocessing.Pool().imap(int, ["4", "3"]): > > print(x) > > > > > > is a bit worriying because although is incorrect and an abuse of the > API, users can do this easily with > > no error message other than a misterious hang. > > OK, so the first problem here (to me, at least) is that it's not > obvious *why* this code is incorrect and an abuse of the API. It takes > a reasonable amount of thinking about the problem to notice that the > Pool object isn't retained, but the iterator returned from imap is. > And when the pool is collected, the worker processes are terminated, > causing the hang, as the worker never sends a result back to the main > process. But it's not obvious to me why the pool is collected before > the imap method has completed. > > As I understand it, originally the code worked because the pool > *didn't* call terminate() when collected. Now it does, and we have a > problem. I'm not *entirely* sure why, if the pool is terminated, the > wait in the iterator doesn't terminate immediately with some sort of > "process being waited on died" error, but let's assume there are good > reasons for that (as I mentioned before, I'm not an expert in > multiprocessing, so I'm OK with assuming that the original design, > done by people who *are* experts, is sound :-)) > > Your original solution would have added a strong reference back to the > pool from the iterator. At first glance, that seems like a reasonable > solution to me. Victor is worried about the "risk of new reference > cycles". But reference cycles are not a problem - we have the cyclic > GC precisely to deal with them. So I'd like to see a better > justification for rejecting that solution than "there might be > reference cycles". But in response to that, you made the iterator have > a weak reference back to the pool. That's flawed because it doesn't > prevent the pool being terminated - as you say, the deadlock is still > present. > > > I have found this on several places and people were > > very confused because usually the interpreter throws some kind of error > indication. In my humble opinion, > > we should try to avoid hanging as a consequence of the misusage, > whatever we do. > > I agree with this. But that implies to me that we should be holding a > strong reference to the pool, > > As a (somewhat weak) analogy, consider > > for n in map(int, ["1", "2"]): > print(n) > > That won't fail if the list gets collected, because map keeps a > reference to the list. My intuition would be that the Pool().imap > example would hold a reference to the pool on essentially the same > basis. > > The more I think about this, the more I struggle to see Victor's logic > for rejecting your original solution. And I *certainly* don't see why > this issue should justify changing the whole API to require users to > explicitly manage pool lifetimes. > > Paul > -------------- next part -------------- An HTML attachment was scrubbed... URL: From njs at pobox.com Tue Dec 11 14:48:24 2018 From: njs at pobox.com (Nathaniel Smith) Date: Tue, 11 Dec 2018 11:48:24 -0800 Subject: [Python-Dev] Usage of the multiprocessing API and object lifetime In-Reply-To: <20181211161051.457a6afa@fsol> References: <20181211161051.457a6afa@fsol> Message-ID: On Tue, Dec 11, 2018, 07:13 Antoine Pitrou > What you are proposing here starts to smell like an anti-pattern to > me. Python _is_ a garbage-collected language, so by definition, there > _are_ going to be resources that are automatically collected when an > object disappears. If I'm allocating a 2GB bytes object, then PyPy may > delay the deallocation much longer than CPython. Do you propose we add > a release() method to bytes objects to avoid this issue (and emit a > warning for people who don't call release() on bytes objects)? > I know this question is rhetorical, but it does actually have a principled answer. Memory is a special resource, because the GC has a complete picture of memory use in the program, and if there's a danger of running out of memory then the GC will detect that and quickly run a collection before anyone has a chance to notice. But it doesn't know about other resources like descriptors, threads, processes, etc., so it can't detect or prevent unbounded leaks of these resources. Therefore, in a classic GC-ed language, bytes() doesn't need to be explicitly released, but all other kinds of resources do. And according to the language spec, Python is a classic GC-ed language. But things are complicated, because CPython isn't a classic GC-ed language, exactly. In practice it's a sort of hybrid RAII/GC language. People regularly write programs that on the refcount quick-release semantics for correctness. A quick way to check: the only thing a reference cycle does is make CPython start acting like an ordinary GC-ed language, so if you're worrying about reference cycles, that's a strong sign that you're writing CPython, not Python. This puts libraries like multiprocessing in a tricky position, because some users are writing CPython, and some are writing Python, and the two groups have contradictory expectations for how resource management should be handled, yet somehow we have to make both groups happy. I don't know what multiprocessing should do here, but I certainly admire the problem :-). -n > -------------- next part -------------- An HTML attachment was scrubbed... URL: From nad at python.org Tue Dec 11 15:07:48 2018 From: nad at python.org (Ned Deily) Date: Tue, 11 Dec 2018 15:07:48 -0500 Subject: [Python-Dev] 3.7.2rc1 and 3.6.8rc1 cutoffs ahead, last 3.6.x bugfix release! In-Reply-To: <58C8B7D8-80EF-46D3-BCBE-4ACC95382D0A@python.org> References: <95B9F244-38A8-4E47-8E9D-6DAD289C84A2@python.org> <58C8B7D8-80EF-46D3-BCBE-4ACC95382D0A@python.org> Message-ID: https://discuss.python.org/t/3-7-2rc1-and-3-6-8rc1-cutoffs-ahead-last-3-6-x-bugfix-release/510/3?u=nad OK, thanks to a lot of hard work by many of you, I think we are ready to start the manufacturing of **3.7.2rc1** and **3.6.8rc1**. For the **3.7 branch**, as usual feel free to continue to merge the usual changes appropriate for a `bugfix` branch; unless otherwise marked and agreed on as a **release blocker** for 3.7.2 final, any new 3.7 merges will be released in 3.7.3. For the **3.6 branch**, as announced 3.6.8 is planned to be **the last bugfix release** for the 3.6 series; future 3.6.x releases will be as needed and contain **only security fixes** and source only. Of course, if any release blocker regressions show up after 3.6.8rc1, we will consider merging fixes for them. This means that, **as of now, the 3.6 branch is no longer open to normal bugfixes**, only security fixes and release blocker regressions fixes and only with the approval of the release manager. Therefore, as we have done with previous branches moving to security-fix mode, merges to the 3.6 branch on the `cpython` GitHub repo are now restricted to the release managers. If you feel a change to 3.6 is needed either because it is a **release blocker regression** in 3.6.8 or because it is a **security issue**, please ensure there is a bpo issue describing the problem, mark it as **release blocker** priority, and submit the necessary PR. At some point, on or after the 3.6.8 release, we will be going through the open 3.6 PRs, open PRs with the `needs backport to 3.6` label, and bpo issues marked for 3.6 and updating or closing them as needed. **Please do not mark new PRs with the** `needs backport to 3.6` **label** unless you feel the proposed change meets one of the criteria above. Thanks for your help! -- Ned Deily nad at python.org -- [] From solipsis at pitrou.net Tue Dec 11 17:39:48 2018 From: solipsis at pitrou.net (Antoine Pitrou) Date: Tue, 11 Dec 2018 23:39:48 +0100 Subject: [Python-Dev] Usage of the multiprocessing API and object lifetime In-Reply-To: References: <20181211161051.457a6afa@fsol> Message-ID: <20181211233948.4da9ed19@fsol> On Tue, 11 Dec 2018 11:48:24 -0800 Nathaniel Smith wrote: > > I know this question is rhetorical, but it does actually have a principled > answer. Memory is a special resource, because the GC has a complete picture > of memory use in the program, and if there's a danger of running out of > memory then the GC will detect that and quickly run a collection before > anyone has a chance to notice. But it doesn't know about other resources > like descriptors, threads, processes, etc., so it can't detect or prevent > unbounded leaks of these resources. > > Therefore, in a classic GC-ed language, bytes() doesn't need to be > explicitly released, but all other kinds of resources do. I would disagree here. You may /like/ to release other kinds of resources explicitly, but you don't /need/ to. It is actually obvious for things such as mutexes which, while the GC doesn't know about them, are small system resources. And nobody's asking Python to add a method to deterministically destroy the mutex that's inside a Lock object (*). (also, calling Lock.release() already does something else :-)) Arguably, things are more complicated for things like threads and processes. But here we are talking not about threads and processes themselves, but about an abstraction (the Pool object) that is /designed/ to hide threads and processes in favour of higher level semantics organized around the idea of task submission. One important characteristic here is that, when the pool is idle, those threads and processes aren't holding important resources (user-allocated resources) alive (**). The idle pool just has a bookkeeping overhead. Usually, people don't really care how exactly a Pool manages its helper threads and worker processes (it has both at the same time), and they are fine with the internal bookkeeping overhead. For the people who care (***), the Pool.join() method is there to be called. (*) actually, it's not a mutex, it's a semaphore (**) unless the user sets a global variable from an executing task, which I think of as an anti-pattern :-) (***) for example because they used the anti-pattern above :-) Regards Antoine. From nad at python.org Tue Dec 11 22:14:04 2018 From: nad at python.org (Ned Deily) Date: Tue, 11 Dec 2018 22:14:04 -0500 Subject: [Python-Dev] [RELEASE] Python 3.7.2rc1 and 3.6.8rc1 now available for testing Message-ID: <091E715F-EC84-4796-B193-D96414F4D62E@python.org> https://blog.python.org/2018/12/python-372rc1-and-368rc1-now-available.html Python 3.7.2rc1 and 3.6.8rc1 are now available. 3.7.2rc1 is the release preview of the next maintenance release of Python 3.7, the latest feature release of Python. 3.6.8rc1 is the release preview of the next and last maintenance release of Python 3.6, the previous feature release of Python. Assuming no critical problems are found prior to 2018-12-20, no code changes are planned between these release candidates and the final releases. These release candidates are intended to give you the opportunity to test the new security and bug fixes in 3.7.2 and 3.6.8. We strongly encourage you to test your projects and report issues found to bugs.python.org as soon as possible. Please keep in mind that these are preview releases and, thus, their use is not recommended for production environments. You can find these releases and more information here: https://www.python.org/downloads/release/python-372rc1/ https://www.python.org/downloads/release/python-368rc1/ -- Ned Deily nad at python.org -- [] From vstinner at redhat.com Wed Dec 12 11:46:36 2018 From: vstinner at redhat.com (Victor Stinner) Date: Wed, 12 Dec 2018 17:46:36 +0100 Subject: [Python-Dev] [RELEASE] Python 3.7.2rc1 and 3.6.8rc1 now available for testing In-Reply-To: <091E715F-EC84-4796-B193-D96414F4D62E@python.org> References: <091E715F-EC84-4796-B193-D96414F4D62E@python.org> Message-ID: It's not easy to find the changelog, so here you have the direct links: * 3.7: https://docs.python.org/3.7/whatsnew/changelog.html#changelog * 3.6: https://docs.python.org/3.6/whatsnew/changelog.html#changelog Victor Le mer. 12 d?c. 2018 ? 04:43, Ned Deily a ?crit : > > https://blog.python.org/2018/12/python-372rc1-and-368rc1-now-available.html > > > Python 3.7.2rc1 and 3.6.8rc1 are now available. 3.7.2rc1 is the release > preview of the next maintenance release of Python 3.7, the latest > feature release of Python. 3.6.8rc1 is the release preview of the next > and last maintenance release of Python 3.6, the previous feature > release of Python. Assuming no critical problems are found prior to > 2018-12-20, no code changes are planned between these release > candidates and the final releases. These release candidates are > intended to give you the opportunity to test the new security and bug > fixes in 3.7.2 and 3.6.8. We strongly encourage you to test your > projects and report issues found to bugs.python.org as soon as > possible. Please keep in mind that these are preview releases and, > thus, their use is not recommended for production environments. > > You can find these releases and more information here: > https://www.python.org/downloads/release/python-372rc1/ > https://www.python.org/downloads/release/python-368rc1/ > > > -- > Ned Deily > nad at python.org -- [] > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: https://mail.python.org/mailman/options/python-dev/vstinner%40redhat.com -- Night gathers, and now my watch begins. It shall not end until my death. From nad at python.org Wed Dec 12 12:06:10 2018 From: nad at python.org (Ned Deily) Date: Wed, 12 Dec 2018 12:06:10 -0500 Subject: [Python-Dev] [RELEASE] Python 3.7.2rc1 and 3.6.8rc1 now available for testing In-Reply-To: References: <091E715F-EC84-4796-B193-D96414F4D62E@python.org> Message-ID: On Dec 12, 2018, at 11:46, Victor Stinner wrote: > It's not easy to find the changelog, so here you have the direct links: > > * 3.7: https://docs.python.org/3.7/whatsnew/changelog.html#changelog > * 3.6: https://docs.python.org/3.6/whatsnew/changelog.html#changelog ?? It's easy to find the changelog! There is a link to a release's changelog on the release page included in the announcement: >> You can find these releases and more information here: >> https://www.python.org/downloads/release/python-372rc1/ >> https://www.python.org/downloads/release/python-368rc1/ Click on the "Full Changelog" link just before the Files section. More information! Or you can always go to the top-level page of the documentation set for a branch or release, click on the "What's new in ... " link, then click on the "changelog" link near the top. -- Ned Deily nad at python.org -- [] From vstinner at redhat.com Wed Dec 12 12:26:42 2018 From: vstinner at redhat.com (Victor Stinner) Date: Wed, 12 Dec 2018 18:26:42 +0100 Subject: [Python-Dev] [RELEASE] Python 3.7.2rc1 and 3.6.8rc1 now available for testing In-Reply-To: References: <091E715F-EC84-4796-B193-D96414F4D62E@python.org> Message-ID: Le mer. 12 d?c. 2018 ? 18:09, Ned Deily a ?crit : > ?? It's easy to find the changelog! There is a link to a release's changelog on the release page included in the announcement: (...) Oh. It's at the end, right. I missed it :-) > Or you can always go to the top-level page of the documentation set for a branch or release, click on the "What's new in ... " link, then click on the "changelog" link near the top. Maybe we should add the link to https://docs.python.org/dev/whatsnew/ as well. Anyway, thanks for the releases :-) Victor -- Night gathers, and now my watch begins. It shall not end until my death. From status at bugs.python.org Fri Dec 14 12:10:03 2018 From: status at bugs.python.org (Python tracker) Date: Fri, 14 Dec 2018 18:10:03 +0100 (CET) Subject: [Python-Dev] Summary of Python tracker Issues Message-ID: <20181214171003.8AFEE133887@psf.upfronthosting.co.za> ACTIVITY SUMMARY (2018-12-07 - 2018-12-14) Python tracker at https://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue. Do NOT respond to this message. Issues counts and deltas: open 6908 ( +4) closed 40341 (+57) total 47249 (+61) Open issues with patches: 2751 Issues opened (41) ================== #35440: Setup failed 0x80072f7d - Unspecified error https://bugs.python.org/issue35440 opened by DesignEngineer #35441: Dead (and buggy) code due to mishandling of PyList_SetItem() e https://bugs.python.org/issue35441 opened by ZackerySpytz #35442: Chain of several subcommands in argparse https://bugs.python.org/issue35442 opened by porton #35448: ConfigParser .read() - handling of nonexistent files https://bugs.python.org/issue35448 opened by Adrian Wielgosik #35449: documenting objects https://bugs.python.org/issue35449 opened by stefan #35450: venv module doesn't create a copy of python binary by default https://bugs.python.org/issue35450 opened by mkkot #35453: pathlib.Path: glob and rglob should accept PathLike patterns https://bugs.python.org/issue35453 opened by ciupicri #35455: Solaris thread_time doesn't work with current implementation https://bugs.python.org/issue35455 opened by kulikjak #35457: robotparser reads empty robots.txt file as "all denied" https://bugs.python.org/issue35457 opened by larsfuse #35459: Use PyDict_GetItemWithError() instead of PyDict_GetItem() https://bugs.python.org/issue35459 opened by serhiy.storchaka #35460: Add PyDict_GetItemStringWithError https://bugs.python.org/issue35460 opened by ronaldoussoren #35461: Document C API functions which swallow exceptions https://bugs.python.org/issue35461 opened by serhiy.storchaka #35462: test_imaplib.test_enable_UTF8_True_append() failed on AMD64 Fr https://bugs.python.org/issue35462 opened by vstinner #35463: mock uses incorrect signature for partial and partialmethod wi https://bugs.python.org/issue35463 opened by xtreak #35465: [asyncio] Document loop.add_signal_handler https://bugs.python.org/issue35465 opened by hniksic #35466: Use a linked list for the ceval pending calls. https://bugs.python.org/issue35466 opened by eric.snow #35467: IDLE: unrequested pasting into Shell after restart https://bugs.python.org/issue35467 opened by terry.reedy #35469: [2.7] time.asctime() regression https://bugs.python.org/issue35469 opened by vstinner #35470: A deadly decref in _PyImport_FindExtensionObjectEx() https://bugs.python.org/issue35470 opened by ZackerySpytz #35472: python 3.7.2 rc1 bumped the build requirements for no reason https://bugs.python.org/issue35472 opened by doko #35473: Intel compiler (icc) does not fully support C11 Features, incl https://bugs.python.org/issue35473 opened by jamie schnaitter #35474: mimetypes.guess_all_extensions potentially mutates list https://bugs.python.org/issue35474 opened by rmccampbell7 #35475: Docs do not show PyImport_AddModuleObject() returns a borrowed https://bugs.python.org/issue35475 opened by eric.snow #35476: _imp_create_dynamic_impl() does not clear error. https://bugs.python.org/issue35476 opened by eric.snow #35478: multiprocessing: ApplyResult.get() hangs if the pool is termin https://bugs.python.org/issue35478 opened by vstinner #35479: multiprocessing.Pool.join() always takes at least 100 ms https://bugs.python.org/issue35479 opened by vstinner #35480: argparse: add a full fledged parser as a subparser https://bugs.python.org/issue35480 opened by porton #35482: python372rc1.chm is ill https://bugs.python.org/issue35482 opened by Ma Lin #35483: tarfile.extractall on existing symlink in Ubuntu overwrites ta https://bugs.python.org/issue35483 opened by michael.brandl at aid-driving.eu #35484: Segmentation fault due to faulthandler on Solaris https://bugs.python.org/issue35484 opened by kulikjak #35485: Mac: tkinter windows turn black while resized https://bugs.python.org/issue35485 opened by terry.reedy #35486: subprocess module breaks backwards compatibility with import h https://bugs.python.org/issue35486 opened by dw #35488: pathlib Path.match does not behave as described https://bugs.python.org/issue35488 opened by anthony shaw #35490: Remove the DecodeFSDefault return converter in Argument Clinic https://bugs.python.org/issue35490 opened by serhiy.storchaka #35492: Missing colon on func statement in library/sys doc https://bugs.python.org/issue35492 opened by seluj78 #35493: multiprocessing.Pool._worker_handler(): use SIGCHLD to be noti https://bugs.python.org/issue35493 opened by vstinner #35494: Inaccurate error message for f-string https://bugs.python.org/issue35494 opened by seblin #35495: argparse does not honor default argument for nargs=argparse.RE https://bugs.python.org/issue35495 opened by rgov #35496: left-to-right violation in match order https://bugs.python.org/issue35496 opened by steve.newcomb #35497: Libary select docs enhance https://bugs.python.org/issue35497 opened by Manjusaka #35498: Parents objects in pathlib.Path don't support slices as __geti https://bugs.python.org/issue35498 opened by thejcannon Most recent 15 issues with no replies (15) ========================================== #35498: Parents objects in pathlib.Path don't support slices as __geti https://bugs.python.org/issue35498 #35497: Libary select docs enhance https://bugs.python.org/issue35497 #35496: left-to-right violation in match order https://bugs.python.org/issue35496 #35495: argparse does not honor default argument for nargs=argparse.RE https://bugs.python.org/issue35495 #35490: Remove the DecodeFSDefault return converter in Argument Clinic https://bugs.python.org/issue35490 #35483: tarfile.extractall on existing symlink in Ubuntu overwrites ta https://bugs.python.org/issue35483 #35482: python372rc1.chm is ill https://bugs.python.org/issue35482 #35476: _imp_create_dynamic_impl() does not clear error. https://bugs.python.org/issue35476 #35474: mimetypes.guess_all_extensions potentially mutates list https://bugs.python.org/issue35474 #35473: Intel compiler (icc) does not fully support C11 Features, incl https://bugs.python.org/issue35473 #35463: mock uses incorrect signature for partial and partialmethod wi https://bugs.python.org/issue35463 #35461: Document C API functions which swallow exceptions https://bugs.python.org/issue35461 #35457: robotparser reads empty robots.txt file as "all denied" https://bugs.python.org/issue35457 #35455: Solaris thread_time doesn't work with current implementation https://bugs.python.org/issue35455 #35448: ConfigParser .read() - handling of nonexistent files https://bugs.python.org/issue35448 Most recent 15 issues waiting for review (15) ============================================= #35497: Libary select docs enhance https://bugs.python.org/issue35497 #35494: Inaccurate error message for f-string https://bugs.python.org/issue35494 #35492: Missing colon on func statement in library/sys doc https://bugs.python.org/issue35492 #35490: Remove the DecodeFSDefault return converter in Argument Clinic https://bugs.python.org/issue35490 #35479: multiprocessing.Pool.join() always takes at least 100 ms https://bugs.python.org/issue35479 #35478: multiprocessing: ApplyResult.get() hangs if the pool is termin https://bugs.python.org/issue35478 #35475: Docs do not show PyImport_AddModuleObject() returns a borrowed https://bugs.python.org/issue35475 #35470: A deadly decref in _PyImport_FindExtensionObjectEx() https://bugs.python.org/issue35470 #35466: Use a linked list for the ceval pending calls. https://bugs.python.org/issue35466 #35465: [asyncio] Document loop.add_signal_handler https://bugs.python.org/issue35465 #35461: Document C API functions which swallow exceptions https://bugs.python.org/issue35461 #35459: Use PyDict_GetItemWithError() instead of PyDict_GetItem() https://bugs.python.org/issue35459 #35455: Solaris thread_time doesn't work with current implementation https://bugs.python.org/issue35455 #35450: venv module doesn't create a copy of python binary by default https://bugs.python.org/issue35450 #35441: Dead (and buggy) code due to mishandling of PyList_SetItem() e https://bugs.python.org/issue35441 Top 10 most discussed issues (10) ================================= #35348: Problems with handling the file command output in platform.arc https://bugs.python.org/issue35348 11 msgs #35402: Upgrade macOS and Windows installers to Tcl 8.6.9 and Tk 8.6.9 https://bugs.python.org/issue35402 11 msgs #33725: Python crashes on macOS after fork with no exec https://bugs.python.org/issue33725 10 msgs #35292: Make SimpleHTTPRequestHandler load mimetypes lazily https://bugs.python.org/issue35292 10 msgs #35208: IDLE: Squeezed lines count ignores window width https://bugs.python.org/issue35208 8 msgs #35449: documenting objects https://bugs.python.org/issue35449 8 msgs #32417: fromutc does not respect datetime subclasses https://bugs.python.org/issue32417 7 msgs #34616: implement "Async exec" https://bugs.python.org/issue34616 6 msgs #35493: multiprocessing.Pool._worker_handler(): use SIGCHLD to be noti https://bugs.python.org/issue35493 6 msgs #34313: Tkinter crashes with Tk-related error on macOS with ActiveTcl https://bugs.python.org/issue34313 5 msgs Issues closed (55) ================== #6835: doctest problem with decorated function when decorator is defi https://bugs.python.org/issue6835 closed by serhiy.storchaka #9960: test_structmembers fails on s390x (bigendian 64-bit): int/Py_s https://bugs.python.org/issue9960 closed by serhiy.storchaka #9999: test_shutil cross-file-system tests are fragile (may not test https://bugs.python.org/issue9999 closed by serhiy.storchaka #15095: test_imaplib problem - intermittent skips and LOGINDISABLED no https://bugs.python.org/issue15095 closed by vstinner #17185: unittest mock create_autospec doesn't correctly replace mocksi https://bugs.python.org/issue17185 closed by cjw296 #20118: test_imaplib test_linetoolong fails on 2.7 in SSL test on some https://bugs.python.org/issue20118 closed by vstinner #20800: Cannot run gui tests twice. https://bugs.python.org/issue20800 closed by terry.reedy #22005: datetime.__setstate__ fails decoding python2 pickle https://bugs.python.org/issue22005 closed by serhiy.storchaka #23874: Encrypted MSI fails to install with code 2755 https://bugs.python.org/issue23874 closed by terry.reedy #25219: Update doc for Idle command line options. https://bugs.python.org/issue25219 closed by terry.reedy #26704: unittest.mock.patch: Double patching instance method: Attribut https://bugs.python.org/issue26704 closed by cjw296 #31374: expat: warning: "_POSIX_C_SOURCE" redefined https://bugs.python.org/issue31374 closed by vstinner #31446: _winapi.CreateProcess (used by subprocess) is not thread-safe https://bugs.python.org/issue31446 closed by serhiy.storchaka #31823: Opaque default value for close_fds argument in Popen.__init__ https://bugs.python.org/issue31823 closed by gregory.p.smith #32153: mock.create_autospec fails if an attribute is a partial functi https://bugs.python.org/issue32153 closed by cjw296 #32788: Better error handling in sqlite3 https://bugs.python.org/issue32788 closed by serhiy.storchaka #33106: Deleting a key in a read-only gdbm results in KeyError, not gd https://bugs.python.org/issue33106 closed by xiang.zhang #33747: Failed separate test_patch_propogrates_exc_on_exit in test_uni https://bugs.python.org/issue33747 closed by serhiy.storchaka #34245: Python library should be installed writable https://bugs.python.org/issue34245 closed by ned.deily #34977: Release Windows Store app containing Python https://bugs.python.org/issue34977 closed by steve.dower #35050: Off-by-one bug in AF_ALG https://bugs.python.org/issue35050 closed by vstinner #35052: Coverity scan: copy/paste error in Lib/xml/dom/minidom.py https://bugs.python.org/issue35052 closed by vstinner #35097: IDLE add doc subsection for editor windows https://bugs.python.org/issue35097 closed by terry.reedy #35122: Process not exiting on unhandled exception when using multipro https://bugs.python.org/issue35122 closed by pablogsal #35213: IDLE: use 'macOS' where appropriate. https://bugs.python.org/issue35213 closed by terry.reedy #35330: When using mock to wrap an existing object, side_effect requir https://bugs.python.org/issue35330 closed by cjw296 #35338: set union/intersection/difference could accept zero arguments https://bugs.python.org/issue35338 closed by josh.r #35346: Modernize Lib/platform.py code https://bugs.python.org/issue35346 closed by vstinner #35351: LTOFLAGS are passed to BASECFLAGS when using LTO https://bugs.python.org/issue35351 closed by ned.deily #35383: lib2to3 raises ParseError on argument called "print" https://bugs.python.org/issue35383 closed by martin.panter #35401: Upgrade Windows and macOS installers to use OpenSSL 1.1.0j / 1 https://bugs.python.org/issue35401 closed by ned.deily #35413: test_multiprocessing_fork: test_del_pool() leaks dangling thre https://bugs.python.org/issue35413 closed by vstinner #35426: test_signal.test_interprocess_signal() race condition https://bugs.python.org/issue35426 closed by vstinner #35429: Incorrect use of raise NotImplemented https://bugs.python.org/issue35429 closed by brett.cannon #35433: Correctly detect installed SDK versions https://bugs.python.org/issue35433 closed by steve.dower #35438: Cleanup extension functions using _PyObject_LookupSpecial https://bugs.python.org/issue35438 closed by josh.r #35439: New class instance not initializing variables of type list https://bugs.python.org/issue35439 closed by ammar2 #35443: Add Tail Call Optimization https://bugs.python.org/issue35443 closed by midnio #35444: Unify and optimize the helper for getting a builtin object https://bugs.python.org/issue35444 closed by serhiy.storchaka #35445: Do not ignore errors when create posix.environ https://bugs.python.org/issue35445 closed by serhiy.storchaka #35446: incorrect example https://bugs.python.org/issue35446 closed by serhiy.storchaka #35447: Redundant try-except block in urllib https://bugs.python.org/issue35447 closed by serhiy.storchaka #35451: Incorrect reference counting for sys.warnoptions and sys._xopt https://bugs.python.org/issue35451 closed by serhiy.storchaka #35452: Make PySys_HasWarnOptions() to never raise an exception https://bugs.python.org/issue35452 closed by serhiy.storchaka #35454: Fix miscellaneous issues in error handling https://bugs.python.org/issue35454 closed by serhiy.storchaka #35456: asyncio.Task.set_result() and set_exception() missing docstrin https://bugs.python.org/issue35456 closed by yselivanov #35458: test_shutil.test_disk_usage() randomly fails when tests are ru https://bugs.python.org/issue35458 closed by vstinner #35464: json.dumps very unclear exception https://bugs.python.org/issue35464 closed by serhiy.storchaka #35468: [3.6/3.7] idlelib/help.html mentions 3.8alpha0 docs https://bugs.python.org/issue35468 closed by terry.reedy #35471: Remove macpath module https://bugs.python.org/issue35471 closed by vstinner #35477: multiprocessing.Pool.__enter__() should raise an exception if https://bugs.python.org/issue35477 closed by vstinner #35481: Run Tasks cannot Concurrent https://bugs.python.org/issue35481 closed by asvetlov #35487: setup()'s package_data not following directory symlink https://bugs.python.org/issue35487 closed by F. Eugene Aumson #35489: Argument Clinic should use "const Py_UNICODE *" for the Py_UNI https://bugs.python.org/issue35489 closed by serhiy.storchaka #35491: multiprocessing: enhance repr() to ease debugging https://bugs.python.org/issue35491 closed by vstinner From arj.python at gmail.com Sun Dec 16 04:17:47 2018 From: arj.python at gmail.com (Abdur-Rahmaan Janhangeer) Date: Sun, 16 Dec 2018 13:17:47 +0400 Subject: [Python-Dev] Julien Palard joins the Python Release Team as Documentation Expert In-Reply-To: References: Message-ID: Great, very well versed in docs management ^^ On Tue, Oct 30, 2018 at 7:04 AM Ned Deily wrote: > > https://discuss.python.org/t/julien-palard-joins-the-python-release-team-as-documentation-expert/313 > -- Abdur-Rahmaan Janhangeer Mauritius Garanti sans virus. www.avast.com <#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2> -------------- next part -------------- An HTML attachment was scrubbed... URL: From solipsis at pitrou.net Mon Dec 17 15:39:03 2018 From: solipsis at pitrou.net (Antoine Pitrou) Date: Mon, 17 Dec 2018 21:39:03 +0100 Subject: [Python-Dev] Governance vote results Message-ID: <20181217213903.5c59e9c7@fsol> Hello, I think readers of this mailing-list could be interested in the vote's results, which have been published just today: https://discuss.python.org/t/python-governance-vote-december-2018-results/546/13 In short, PEP 8016 ("The Steering Council Model", by Nathaniel and Donald) will define how Python development operates going forward. I encourage people to read the long-form results at the URL above, since the detailed rankings provide valuable information about the committers' overall preference as well. The next step will probably be to elect the first Steering Council: https://discuss.python.org/t/organizing-the-council-elections/549 That election again will be reserved to Python committers. Regards Antoine. From storchaka at gmail.com Wed Dec 19 04:14:41 2018 From: storchaka at gmail.com (Serhiy Storchaka) Date: Wed, 19 Dec 2018 11:14:41 +0200 Subject: [Python-Dev] 3.7.2rc1 and 3.6.8rc1 cutoffs ahead, last 3.6.x bugfix release! In-Reply-To: <95B9F244-38A8-4E47-8E9D-6DAD289C84A2@python.org> References: <95B9F244-38A8-4E47-8E9D-6DAD289C84A2@python.org> Message-ID: <033423e6-5918-24e1-c5a5-daf829a3e1ef@gmail.com> 04.12.18 10:42, Ned Deily ????: > A reminder: as previously announced, 3.6.8 is planned to be the last bugfix release of the 3.6 series. Python 3.6.0 was released on 2016-12-23, so by the time 3.6.8 is released, 3.6.x will have been in bugfix mode almost exactly 2 years. When a new feature release is made and enters "bugfix" mode, our policy has long been to continue to maintain the previous bugfix branch for at least one more release and then move that branch to "security fix only" mode. 3.7.0 (and 3.6.6) was released nearly six months ago and, with the release of 3.6.8, there will have been two additional 3.6.x bugfix releases since then. So, barring any showstopper issues that might arise, the upcoming 3.6.8rc1 is your last chance to make bugfix changes for 3.6.x. Following the successful release of 3.6.8, only sec > urity fixes will be accepted for the 3.6 branch and future 3.6.x releases will be source-only and scheduled as needed; no further binary installers will be produced for 3.6. Refer to the Dev Guide > sections and release PEPs linked below for more information. Can we revise our policy and prolong the bug fixing mode for 3.6? 3.6 is the default Python in Ubuntu 18.04 LTS and RHEL 8. And due to several important syntax features it can be a minimal required version for long time. I merged several PRs before releasing 3.6.8rc1, but there are still less trivial bugfix PRs which need more time for reviewing, and there are bugs for which no PR is created yet. There is also a number of documentation PRs. I propose to allow backporting bugfixes to 3.6 if they do not need excessive work, but stop to fix 3.6 only bugs. After migrating to GitHab, backporting became less painful, most of backports to 3.6 can be done automatically from master or from 3.7. From cstratak at redhat.com Wed Dec 19 05:28:01 2018 From: cstratak at redhat.com (Charalampos Stratakis) Date: Wed, 19 Dec 2018 05:28:01 -0500 (EST) Subject: [Python-Dev] 3.7.2rc1 and 3.6.8rc1 cutoffs ahead, last 3.6.x bugfix release! In-Reply-To: <033423e6-5918-24e1-c5a5-daf829a3e1ef@gmail.com> References: <95B9F244-38A8-4E47-8E9D-6DAD289C84A2@python.org> <033423e6-5918-24e1-c5a5-daf829a3e1ef@gmail.com> Message-ID: <1109428671.99278808.1545215281839.JavaMail.zimbra@redhat.com> ----- Original Message ----- > From: "Serhiy Storchaka" > To: python-dev at python.org > Cc: python-committers at python.org > Sent: Wednesday, December 19, 2018 10:14:41 AM > Subject: Re: [Python-Dev] 3.7.2rc1 and 3.6.8rc1 cutoffs ahead, last 3.6.x bugfix release! > > 04.12.18 10:42, Ned Deily ????: > > A reminder: as previously announced, 3.6.8 is planned to be the last bugfix > > release of the 3.6 series. Python 3.6.0 was released on 2016-12-23, so by > > the time 3.6.8 is released, 3.6.x will have been in bugfix mode almost > > exactly 2 years. When a new feature release is made and enters "bugfix" > > mode, our policy has long been to continue to maintain the previous bugfix > > branch for at least one more release and then move that branch to > > "security fix only" mode. 3.7.0 (and 3.6.6) was released nearly six > > months ago and, with the release of 3.6.8, there will have been two > > additional 3.6.x bugfix releases since then. So, barring any showstopper > > issues that might arise, the upcoming 3.6.8rc1 is your last chance to make > > bugfix changes for 3.6.x. Following the successful release of 3.6.8, only > > sec > > urity fixes will be accepted for the 3.6 branch and future 3.6.x releases > > will be source-only and scheduled as needed; no further binary > > installers will be produced for 3.6. Refer to the Dev Guide > > sections and release PEPs linked below for more information. > > Can we revise our policy and prolong the bug fixing mode for 3.6? 3.6 is > the default Python in Ubuntu 18.04 LTS and RHEL 8. And due to several > important syntax features it can be a minimal required version for long > time. > That would actually be a great move, even if it doesn't happen, thanks for considering it. > I merged several PRs before releasing 3.6.8rc1, but there are still less > trivial bugfix PRs which need more time for reviewing, and there are > bugs for which no PR is created yet. There is also a number of > documentation PRs. I propose to allow backporting bugfixes to 3.6 if > they do not need excessive work, but stop to fix 3.6 only bugs. After > migrating to GitHab, backporting became less painful, most of backports > to 3.6 can be done automatically from master or from 3.7. > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/cstratak%40redhat.com > -- Regards, Charalampos Stratakis Software Engineer Python Maintenance Team, Red Hat From vstinner at redhat.com Wed Dec 19 09:01:08 2018 From: vstinner at redhat.com (Victor Stinner) Date: Wed, 19 Dec 2018 15:01:08 +0100 Subject: [Python-Dev] 3.7.2rc1 and 3.6.8rc1 cutoffs ahead, last 3.6.x bugfix release! In-Reply-To: <1109428671.99278808.1545215281839.JavaMail.zimbra@redhat.com> References: <95B9F244-38A8-4E47-8E9D-6DAD289C84A2@python.org> <033423e6-5918-24e1-c5a5-daf829a3e1ef@gmail.com> <1109428671.99278808.1545215281839.JavaMail.zimbra@redhat.com> Message-ID: Hi, I am working in the Red Hat "Python-maint" team which is maintaining Python 3.6 as the main Python interpreter in RHEL 8, which will likely be supported for at least 10 years. And we have been supporting Python 2.7 in RHEL 7. So obviously, being able to benefit of the upstream effort and infra to have a Python 3.6 Long Time Support (LTS) would help us :-) The question is more who else would benefit from that and is it worth it? I don't want Python upstream to pay the price of the maintenance burden of RHEL 8 lifecycle. For example, supporting a version means to have a working CI (Travis CI, AppVeyor, VSTS, buildbots). I would suggest to only support a very few platforms for the LTS. I propose to restrict to Linux. It doesn't mean to break other platforms on purpose, just to restrict CI to the bare minimum. If Microsoft is interested, we can also support Windows as well. RHEL 7 is based on Python 2.7.5 which has been released in 2013 (5 years ago) and there are 150 patches on top of it: it means that around 30 patches are added per year. I would suggest to have a very strict policy on which changes are backported into 3.6: only the most critical bugfixes, but all security fixes obviously. If we extend Python 3.6 lifetime, do we need a new release manager when the initial lifetime (usually 5 years) ends? Benjamin Peterson accepted to be the Python 2.7 release manager for 10 years (instead of 5 years initially). We could ask Ned Deily about Python 3.6 LTS :-) We would need a group of people reviewing individual 3.6 pull requests to decide to pick them or not. I would volunteer to review these PRs and merge them. The idea isn't new, Nick Coghlan proposed a "ELPython" last year: https://github.com/elpython/elpython-meta The Linux kernel also have multiple LTS kernel which are supported longer than usual releases: they are now supported for 6 years. See "Longterm" at: https://www.kernel.org/category/releases.html Victor -- Night gathers, and now my watch begins. It shall not end until my death. From chris.barker at noaa.gov Wed Dec 19 12:01:15 2018 From: chris.barker at noaa.gov (Chris Barker - NOAA Federal) Date: Wed, 19 Dec 2018 09:01:15 -0800 Subject: [Python-Dev] 3.7.2rc1 and 3.6.8rc1 cutoffs ahead, last 3.6.x bugfix release! In-Reply-To: <033423e6-5918-24e1-c5a5-daf829a3e1ef@gmail.com> References: <95B9F244-38A8-4E47-8E9D-6DAD289C84A2@python.org> <033423e6-5918-24e1-c5a5-daf829a3e1ef@gmail.com> Message-ID: I?m all for extending the life of 3.6, it sure feels recent to me! > 3.6 is the default Python in Ubuntu 18.04 LTS and RHEL 8. And due to several important syntax features it can be a minimal required version for long time. Which is a good argument for why we may not need longer term support for 3.5, but minimum requirement doesn?t mean we need to support it longer term, as the alternative is to upgrade to 3.7 ? and most code that works on 3.6 will work on 3.7, yes? That is ? if you want recent bug fixes, upgrade your Python. If you have a working 3.6 based system, you only need security fixes, yes? I fully understand the need to keep a working older system up to date with security fixes, but I?ve always been confused by the desire to run an older base system, while still requiring newer subsystems, whether that?s an older Linux with a newer python, or and older python with a newer package. -CHB > > I merged several PRs before releasing 3.6.8rc1, but there are still less trivial bugfix PRs which need more time for reviewing, and there are bugs for which no PR is created yet. There is also a number of documentation PRs. I propose to allow backporting bugfixes to 3.6 if they do not need excessive work, but stop to fix 3.6 only bugs. After migrating to GitHab, backporting became less painful, most of backports to 3.6 can be done automatically from master or from 3.7. > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: https://mail.python.org/mailman/options/python-dev/chris.barker%40noaa.gov From storchaka at gmail.com Wed Dec 19 12:42:29 2018 From: storchaka at gmail.com (Serhiy Storchaka) Date: Wed, 19 Dec 2018 19:42:29 +0200 Subject: [Python-Dev] 3.7.2rc1 and 3.6.8rc1 cutoffs ahead, last 3.6.x bugfix release! In-Reply-To: References: <95B9F244-38A8-4E47-8E9D-6DAD289C84A2@python.org> <033423e6-5918-24e1-c5a5-daf829a3e1ef@gmail.com> Message-ID: 19.12.18 19:01, Chris Barker - NOAA Federal via Python-Dev ????: >> 3.6 is the default Python in Ubuntu 18.04 LTS and RHEL 8. And due to several important syntax features it can be a minimal required version for long time. > > Which is a good argument for why we may not need longer term support > for 3.5, but minimum requirement doesn?t mean we need to support it > longer term, as the alternative is to upgrade to 3.7 ? and most code > that works on 3.6 will work on 3.7, yes? No. There were several minor potentially breaking changes. The most famous one is making "async" a keyword. Different libraries were broken for different causes, and the end user code can be broken too. AFAIK the popular package tensorflow is not 3.7 compatible for today. From nad at python.org Wed Dec 19 15:10:02 2018 From: nad at python.org (Ned Deily) Date: Wed, 19 Dec 2018 15:10:02 -0500 Subject: [Python-Dev] 3.7.2rc1 and 3.6.8rc1 cutoffs ahead, last 3.6.x bugfix release! In-Reply-To: References: <95B9F244-38A8-4E47-8E9D-6DAD289C84A2@python.org> <033423e6-5918-24e1-c5a5-daf829a3e1ef@gmail.com> Message-ID: On Dec 19, 2018, at 12:42, Serhiy Storchaka wrote: > 19.12.18 19:01, Chris Barker - NOAA Federal via Python-Dev ????: >>> [ extending 3.6 bugfix release phase? ] >> [...] > [...] FYI, the discussion of this topic has proceeded to a conclusion on the python-committers mailing list. TL;DR we are not going to change our plans now. Thanks for all the thoughtful comments! https://mail.python.org/pipermail/python-committers/2018-December/006487.html -- Ned Deily nad at python.org -- [] From tjreedy at udel.edu Wed Dec 19 17:30:06 2018 From: tjreedy at udel.edu (Terry Reedy) Date: Wed, 19 Dec 2018 17:30:06 -0500 Subject: [Python-Dev] 3.7.2rc1 and 3.6.8rc1 cutoffs ahead, last 3.6.x bugfix release! In-Reply-To: <033423e6-5918-24e1-c5a5-daf829a3e1ef@gmail.com> References: <95B9F244-38A8-4E47-8E9D-6DAD289C84A2@python.org> <033423e6-5918-24e1-c5a5-daf829a3e1ef@gmail.com> Message-ID: On 12/19/2018 4:14 AM, Serhiy Storchaka wrote: I propose to allow backporting bugfixes to 3.6 if > they do not need excessive work, but stop to fix 3.6 only bugs. I think this would need a PEP. > After migrating to GitHab, backporting became less painful, Before GitHub, we forward ported. For me, for idlelib, that was faster, though the current automation makes it about equal effort. The main pain point for non-IDLE patches was NEWS conflicts, which have been eliminated. > most of backports to 3.6 can be done automatically from master or from 3.7. That is true in part because all bugfixes are backported, so that most of the code in most modules is the same in 3.7 and 3.6. As soon as some bugfixes are not backported, nearby bug fixes start getting conflicts they would not have gotten otherwise. For idlelib, I know that if some PRs are not backported (which currently *is* trivial), then it will cease being trivial for every patch. As it is, 2 PRs that touch the same module can conflict, so that one requires editing when the other is merged. I often delay writing a followup PR until the first is merged. -- Terry Jan Reedy From J.Demeyer at UGent.be Fri Dec 21 06:26:22 2018 From: J.Demeyer at UGent.be (Jeroen Demeyer) Date: Fri, 21 Dec 2018 12:26:22 +0100 Subject: [Python-Dev] Windows porting: request to review PR #880 Message-ID: <5C1CCDDE.3030309@UGent.be> Can somebody please review https://github.com/python/cpython/pull/880 That addresses a severe problem on Windows making it impossible to build any C++ extension module with some compilers. From tjreedy at udel.edu Fri Dec 21 11:29:30 2018 From: tjreedy at udel.edu (Terry Reedy) Date: Fri, 21 Dec 2018 11:29:30 -0500 Subject: [Python-Dev] Windows porting: request to review PR #880 In-Reply-To: <5C1CCDDE.3030309@UGent.be> References: <5C1CCDDE.3030309@UGent.be> Message-ID: On 12/21/2018 6:26 AM, Jeroen Demeyer wrote: > Can somebody please review https://github.com/python/cpython/pull/880 > > That addresses a severe problem on Windows making it impossible to build > any C++ extension module with some compilers. Issue is https://bugs.python.org/issue11566. Someone needs to check whether it applies to 3.7 and/or 3.8. -- Terry Jan Reedy From status at bugs.python.org Fri Dec 21 12:11:05 2018 From: status at bugs.python.org (Python tracker) Date: Fri, 21 Dec 2018 18:11:05 +0100 (CET) Subject: [Python-Dev] Summary of Python tracker Issues Message-ID: <20181221171105.976F21532F6@psf.upfronthosting.co.za> ACTIVITY SUMMARY (2018-12-14 - 2018-12-21) Python tracker at https://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue. Do NOT respond to this message. Issues counts and deltas: open 6896 (-13) closed 40407 (+67) total 47303 (+54) Open issues with patches: 2737 Issues opened (31) ================== #35500: Align expected and actual calls on mock.assert_called_with err https://bugs.python.org/issue35500 opened by xtreak #35501: "make coverage" should not leak coverage compiler flags to thi https://bugs.python.org/issue35501 opened by vstinner #35502: Memory leak in xml.etree.ElementTree.iterparse https://bugs.python.org/issue35502 opened by jess.j #35503: os.path.islink() works with cygwin installation but not python https://bugs.python.org/issue35503 opened by bward #35505: Test test_imaplib fail in test_imap4_host_default_value https://bugs.python.org/issue35505 opened by Petr Stupka #35507: multiprocessing: seg fault when creating RawArray from numpy c https://bugs.python.org/issue35507 opened by sh37211 #35512: patch.dict resolves in_dict eagerly (should be late resolved) https://bugs.python.org/issue35512 opened by jason.coombs #35514: Docs on reference count detail. enhancement. https://bugs.python.org/issue35514 opened by bombs-kim #35517: selector.EpollSelector: add new parameter to support EPOLLEXCL https://bugs.python.org/issue35517 opened by Manjusaka #35518: test_timeout uses blackhole.snakebite.net domain which doesn't https://bugs.python.org/issue35518 opened by vstinner #35520: Python won't build with dtrace enabled on some systems. https://bugs.python.org/issue35520 opened by kulikjak #35524: using/windows launcher image might be deprecated https://bugs.python.org/issue35524 opened by seluj78 #35525: Incorrect keyword name in NNTP.starttls() documentation https://bugs.python.org/issue35525 opened by cmcp22 #35527: Make fields selectively immutable in dataclasses https://bugs.python.org/issue35527 opened by rhettinger #35528: [DOC] [LaTeX] Sphinx 2.0 uses GNU FreeFont as default for xela https://bugs.python.org/issue35528 opened by jfbu #35530: Counter-intuitive logging API https://bugs.python.org/issue35530 opened by porton #35533: argparse standard error usage for exit / error https://bugs.python.org/issue35533 opened by philiprowlands #35534: SIGSEGV in stackdepth_walk https://bugs.python.org/issue35534 opened by gozdal #35535: time.strptime() unexpectedly gives the same result for %U and https://bugs.python.org/issue35535 opened by Paul Keating #35536: Calling built-in locals() and globals() in C++ leads to System https://bugs.python.org/issue35536 opened by ???????????? ?????????????????? #35537: use os.posix_spawn in subprocess https://bugs.python.org/issue35537 opened by nanjekyejoannah #35539: Cannot properly close terminated process https://bugs.python.org/issue35539 opened by hniksic #35540: dataclasses.asdict breaks with defaultdict fields https://bugs.python.org/issue35540 opened by wrmsr #35545: asyncio.base_events.create_connection doesn't handle scoped IP https://bugs.python.org/issue35545 opened by maxifree #35546: String formatting produces incorrect result with left-aligned https://bugs.python.org/issue35546 opened by serhiy.storchaka #35547: email.parser / email.policy does not correctly handle multiple https://bugs.python.org/issue35547 opened by mjpieters #35548: memoryview needlessly (?) requires represented object to be ha https://bugs.python.org/issue35548 opened by Kentzo #35549: Add partial_match: bool = False argument to unicodedata.lookup https://bugs.python.org/issue35549 opened by rominf #35550: Some define guards for Solaris are wrong https://bugs.python.org/issue35550 opened by kulikjak #35551: Encoding and alias issues https://bugs.python.org/issue35551 opened by blkserene #35552: Do not read memory past the specified limit in PyUnicode_FromF https://bugs.python.org/issue35552 opened by serhiy.storchaka Most recent 15 issues with no replies (15) ========================================== #35552: Do not read memory past the specified limit in PyUnicode_FromF https://bugs.python.org/issue35552 #35551: Encoding and alias issues https://bugs.python.org/issue35551 #35550: Some define guards for Solaris are wrong https://bugs.python.org/issue35550 #35549: Add partial_match: bool = False argument to unicodedata.lookup https://bugs.python.org/issue35549 #35548: memoryview needlessly (?) requires represented object to be ha https://bugs.python.org/issue35548 #35540: dataclasses.asdict breaks with defaultdict fields https://bugs.python.org/issue35540 #35539: Cannot properly close terminated process https://bugs.python.org/issue35539 #35533: argparse standard error usage for exit / error https://bugs.python.org/issue35533 #35528: [DOC] [LaTeX] Sphinx 2.0 uses GNU FreeFont as default for xela https://bugs.python.org/issue35528 #35525: Incorrect keyword name in NNTP.starttls() documentation https://bugs.python.org/issue35525 #35520: Python won't build with dtrace enabled on some systems. https://bugs.python.org/issue35520 #35514: Docs on reference count detail. enhancement. https://bugs.python.org/issue35514 #35512: patch.dict resolves in_dict eagerly (should be late resolved) https://bugs.python.org/issue35512 #35505: Test test_imaplib fail in test_imap4_host_default_value https://bugs.python.org/issue35505 #35501: "make coverage" should not leak coverage compiler flags to thi https://bugs.python.org/issue35501 Most recent 15 issues waiting for review (15) ============================================= #35552: Do not read memory past the specified limit in PyUnicode_FromF https://bugs.python.org/issue35552 #35550: Some define guards for Solaris are wrong https://bugs.python.org/issue35550 #35545: asyncio.base_events.create_connection doesn't handle scoped IP https://bugs.python.org/issue35545 #35520: Python won't build with dtrace enabled on some systems. https://bugs.python.org/issue35520 #35517: selector.EpollSelector: add new parameter to support EPOLLEXCL https://bugs.python.org/issue35517 #35502: Memory leak in xml.etree.ElementTree.iterparse https://bugs.python.org/issue35502 #35498: Parents objects in pathlib.Path don't support slices as __geti https://bugs.python.org/issue35498 #35494: Inaccurate error message for f-string https://bugs.python.org/issue35494 #35488: pathlib Path.match does not behave as described https://bugs.python.org/issue35488 #35479: multiprocessing.Pool.join() always takes at least 100 ms https://bugs.python.org/issue35479 #35478: multiprocessing: ApplyResult.get() hangs if the pool is termin https://bugs.python.org/issue35478 #35470: A deadly decref in _PyImport_FindExtensionObjectEx() https://bugs.python.org/issue35470 #35466: Use a linked list for the ceval pending calls. https://bugs.python.org/issue35466 #35459: Use PyDict_GetItemWithError() instead of PyDict_GetItem() https://bugs.python.org/issue35459 #35455: Solaris thread_time doesn't work with current implementation https://bugs.python.org/issue35455 Top 10 most discussed issues (10) ================================= #35517: selector.EpollSelector: add new parameter to support EPOLLEXCL https://bugs.python.org/issue35517 18 msgs #35537: use os.posix_spawn in subprocess https://bugs.python.org/issue35537 17 msgs #35530: Counter-intuitive logging API https://bugs.python.org/issue35530 9 msgs #35502: Memory leak in xml.etree.ElementTree.iterparse https://bugs.python.org/issue35502 7 msgs #35485: Mac: tkinter windows turn black while resized https://bugs.python.org/issue35485 5 msgs #35524: using/windows launcher image might be deprecated https://bugs.python.org/issue35524 5 msgs #29707: os.path.ismount() always returns false for mount --bind on sam https://bugs.python.org/issue29707 4 msgs #35518: test_timeout uses blackhole.snakebite.net domain which doesn't https://bugs.python.org/issue35518 4 msgs #35547: email.parser / email.policy does not correctly handle multiple https://bugs.python.org/issue35547 4 msgs #19974: tarfile doesn't overwrite symlink by directory https://bugs.python.org/issue19974 3 msgs Issues closed (63) ================== #10320: printf %qd is nonstandard https://bugs.python.org/issue10320 closed by serhiy.storchaka #10496: Python startup should not require passwd entry https://bugs.python.org/issue10496 closed by vstinner #16516: argparse types (and actions) must be hashable https://bugs.python.org/issue16516 closed by gvanrossum #18085: Verifying refcounts.dat https://bugs.python.org/issue18085 closed by serhiy.storchaka #18799: Resurrect and fix test_404 in Lib/test/test_xmlrpc.py https://bugs.python.org/issue18799 closed by serhiy.storchaka #20164: Undocumented KeyError from os.path.expanduser https://bugs.python.org/issue20164 closed by serhiy.storchaka #23057: [Windows] asyncio: support signal handlers on Windows (feature https://bugs.python.org/issue23057 closed by asvetlov #32077: Documentation: Some Unicode object functions don't indicate wh https://bugs.python.org/issue32077 closed by serhiy.storchaka #32706: test_check_hostname() of test_ftplib started to fail randomly https://bugs.python.org/issue32706 closed by vstinner #33306: Improving SyntaxError for unmatched parentheses https://bugs.python.org/issue33306 closed by serhiy.storchaka #34238: When BROWSER is set on Mac webbrowser.register_standard_browse https://bugs.python.org/issue34238 closed by serhiy.storchaka #34589: Py_Initialize() and Py_Main() should not enable C locale coerc https://bugs.python.org/issue34589 closed by ncoghlan #34686: Add `-r`, as opposed to `-R` to Python core interpreter https://bugs.python.org/issue34686 closed by lepaperwan #34833: [CI] Azure Pipeline: Initialize Agent failed https://bugs.python.org/issue34833 closed by vstinner #35012: [3.7] test_multiprocessing_spawn hangs randomly on AppVeyor https://bugs.python.org/issue35012 closed by vstinner #35238: Alleviate memory reservation of fork_exec in subprocess.Popen https://bugs.python.org/issue35238 closed by oesteban #35257: Avoid leaking linker flags into distutils: add PY_LDFLAGS_NODI https://bugs.python.org/issue35257 closed by ned.deily #35259: Py_FinalizeEx unconditionally exists in Py_LIMITED_API https://bugs.python.org/issue35259 closed by serhiy.storchaka #35266: Add _PyPreConfig and rework _PyCoreConfig and _PyMainInterpret https://bugs.python.org/issue35266 closed by vstinner #35268: Windows 10 asyncio reading continously stdin and stdout Stockf https://bugs.python.org/issue35268 closed by cheryl.sabella #35337: Check index in PyTuple_GET_ITEM/PyTuple_SET_ITEM in debug mode https://bugs.python.org/issue35337 closed by vstinner #35348: Problems with handling the file command output in platform.arc https://bugs.python.org/issue35348 closed by vstinner #35368: [2.7] Make PyMem_Malloc() thread-safe in debug mode https://bugs.python.org/issue35368 closed by vstinner #35394: Add __slots__ = () to asyncio protocols https://bugs.python.org/issue35394 closed by asvetlov #35412: test_future4 ran no test https://bugs.python.org/issue35412 closed by vstinner #35415: fileno argument to socket.socket is not validated https://bugs.python.org/issue35415 closed by asvetlov #35424: multiprocessing.Pool: emit ResourceWarning https://bugs.python.org/issue35424 closed by vstinner #35441: Dead (and buggy) code due to mishandling of PyList_SetItem() e https://bugs.python.org/issue35441 closed by serhiy.storchaka #35450: venv module doesn't create a copy of python binary by default https://bugs.python.org/issue35450 closed by cheryl.sabella #35461: Document C API functions which swallow exceptions https://bugs.python.org/issue35461 closed by serhiy.storchaka #35465: [asyncio] Document loop.add_signal_handler https://bugs.python.org/issue35465 closed by yselivanov #35469: [2.7] time.asctime() regression https://bugs.python.org/issue35469 closed by vstinner #35472: python 3.7.2 rc1 bumped sphinx requirements a bit too much https://bugs.python.org/issue35472 closed by mdk #35475: Docs do not show PyImport_AddModuleObject() returns a borrowed https://bugs.python.org/issue35475 closed by serhiy.storchaka #35480: argparse: add a full fledged parser as a subparser https://bugs.python.org/issue35480 closed by terry.reedy #35482: can't open python368rc1.chm and python372rc1.chm https://bugs.python.org/issue35482 closed by steve.dower #35490: Remove the DecodeFSDefault return converter in Argument Clinic https://bugs.python.org/issue35490 closed by serhiy.storchaka #35492: Missing colon on func statement in library/sys doc https://bugs.python.org/issue35492 closed by terry.reedy #35496: left-to-right violation in match order https://bugs.python.org/issue35496 closed by steve.newcomb #35497: Libary select docs enhance https://bugs.python.org/issue35497 closed by xiang.zhang #35499: "make profile-opt" overrides CFLAGS_NODIST https://bugs.python.org/issue35499 closed by vstinner #35504: `del OSError().characters_written` raises `SystemError` https://bugs.python.org/issue35504 closed by serhiy.storchaka #35506: Doc: fix keyword `as` link from `import` and `try` https://bugs.python.org/issue35506 closed by serhiy.storchaka #35508: array.index should take optional start and stop indices like f https://bugs.python.org/issue35508 closed by kyuupichan #35509: Unable to inherit from logging.Formatter https://bugs.python.org/issue35509 closed by yan12125 #35510: pickling derived dataclasses https://bugs.python.org/issue35510 closed by satra #35511: Some methods of profile.Profile are not supported but the docs https://bugs.python.org/issue35511 closed by asvetlov #35513: Lib/test/lock_tests.py should not use time.time(), but time.mo https://bugs.python.org/issue35513 closed by vstinner #35515: Matrix operator star creates a false matrix https://bugs.python.org/issue35515 closed by christian.heimes #35516: platform.system_alias(): add macOS support https://bugs.python.org/issue35516 closed by vstinner #35519: Can not run test without test module for tests which import ra https://bugs.python.org/issue35519 closed by vstinner #35521: IDLE: Add doc section for Code Context and ref links. https://bugs.python.org/issue35521 closed by terry.reedy #35522: os.stat().st_ctime and time.time() mismatch https://bugs.python.org/issue35522 closed by rbiswas143 #35523: Remove old ctypes callback workaround: creating the first inst https://bugs.python.org/issue35523 closed by vstinner #35526: __future__.barry_as_FLUFL documented as mandatory for Python 3 https://bugs.python.org/issue35526 closed by barry #35529: A reference counting bug in ctypes https://bugs.python.org/issue35529 closed by serhiy.storchaka #35531: xml.etree.ElementTree Elment.find bug, fails to find tag https://bugs.python.org/issue35531 closed by spacether #35532: numpy-stl library problem, class stl.base.BaseMesh lacks funct https://bugs.python.org/issue35532 closed by serhiy.storchaka #35538: splitext does not seems to handle filepath ending in . https://bugs.python.org/issue35538 closed by mrabarnett #35541: CLI error when .python_history contains unicode characters https://bugs.python.org/issue35541 closed by eryksun #35542: stack exhaustion in 3.6.7 https://bugs.python.org/issue35542 closed by shuoz #35543: re.sub is only replacing max. of 2 string found by regexp. https://bugs.python.org/issue35543 closed by serhiy.storchaka #35544: unicode.encode docstring says return value can be unicode https://bugs.python.org/issue35544 closed by serhiy.storchaka From facundobatista at gmail.com Sun Dec 23 07:29:57 2018 From: facundobatista at gmail.com (Facundo Batista) Date: Sun, 23 Dec 2018 09:29:57 -0300 Subject: [Python-Dev] About "python-porting" mail list Message-ID: Hello! This list (which I co-admin, with Georg) is getting less and less traffic as months pass by. See: https://mail.python.org/pipermail/python-porting/ The interwebs has been collecting ton of resources about porting py2 to 3 during these years. Any not-yet-answered question surely can be done in a list with more participants. Can we kill this list? Thanks! Regards, -- . Facundo Blog: http://www.taniquetil.com.ar/plog/ PyAr: http://www.python.org.ar/ Twitter: @facundobatista From skip.montanaro at gmail.com Sun Dec 23 08:13:26 2018 From: skip.montanaro at gmail.com (Skip Montanaro) Date: Sun, 23 Dec 2018 07:13:26 -0600 Subject: [Python-Dev] About "python-porting" mail list In-Reply-To: References: Message-ID: > The interwebs has been collecting ton of resources about porting py2 > to 3 during these years. Any not-yet-answered question surely can be > done in a list with more participants. > > Can we kill this list? Would it perhaps make sense to replace the list with an auto-reply about the list's demise, then auto-forward their message to python-list? Skip From barry at python.org Sun Dec 23 12:20:02 2018 From: barry at python.org (Barry Warsaw) Date: Sun, 23 Dec 2018 12:20:02 -0500 Subject: [Python-Dev] About "python-porting" mail list In-Reply-To: References: Message-ID: <8D64E8B2-BCE1-4CEE-90E8-1EE7F57A9E0C@python.org> On Dec 23, 2018, at 07:29, Facundo Batista wrote: > > This list (which I co-admin, with Georg) is getting less and less > traffic as months pass by. See: > > https://mail.python.org/pipermail/python-porting/ > > The interwebs has been collecting ton of resources about porting py2 > to 3 during these years. Any not-yet-answered question surely can be > done in a list with more participants. > > Can we kill this list? Is it a maintenance burden? If not, then maybe we should wait until 2.7 is EOL?d? There might be an uptick in traffic as reality sets in for more people. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: Message signed with OpenPGP URL: From facundobatista at gmail.com Mon Dec 24 06:29:22 2018 From: facundobatista at gmail.com (Facundo Batista) Date: Mon, 24 Dec 2018 08:29:22 -0300 Subject: [Python-Dev] About "python-porting" mail list In-Reply-To: <8D64E8B2-BCE1-4CEE-90E8-1EE7F57A9E0C@python.org> References: <8D64E8B2-BCE1-4CEE-90E8-1EE7F57A9E0C@python.org> Message-ID: El dom., 23 de dic. de 2018 a la(s) 14:20, Barry Warsaw (barry at python.org) escribi?: > > Can we kill this list? > > Is it a maintenance burden? If not, then maybe we should wait until 2.7 is EOL?d? There might be an uptick in traffic as reality sets in for more people. > It's a good idea to wait until next year. Thanks! -- . Facundo Blog: http://www.taniquetil.com.ar/plog/ PyAr: http://www.python.org.ar/ Twitter: @facundobatista From nad at python.org Mon Dec 24 06:57:53 2018 From: nad at python.org (Ned Deily) Date: Mon, 24 Dec 2018 06:57:53 -0500 Subject: [Python-Dev] [RELEASE] Python 3.7.2 and 3.6.8 are now available Message-ID: <3041C039-C5F6-4A01-BEB8-783238B07970@python.org> https://blog.python.org/2018/12/python-372-and-368-are-now-available.html Python 3.7.2 and 3.6.8 are now available. Python 3.7.2 is the next maintenance release of Python 3.7, the latest feature release of Python. You can find Python 3.7.2 here: https://www.python.org/downloads/release/python-372/ See the What?s New In Python 3.7 document for more information about the many new features and optimizations included in the 3.7 series. Detailed information about the changes made in 3.7.2 can be found in its change log. We are also happy to announce the availability of Python 3.6.8: https://www.python.org/downloads/release/python-368/ Python 3.6.8 is planned to be the last bugfix release of Python 3.6. Per our support policy, we plan to provide security fixes for Python 3.6 as needed through 2021, five years following its initial release. Thanks to all of the many volunteers who help make Python Development and these releases possible! Please consider supporting our efforts by volunteering yourself or through organization contributions to the Python Software Foundation. https://docs.python.org/3.7/whatsnew/3.7.html https://docs.python.org/3.7/whatsnew/changelog.html#python-3-7-2-final https://docs.python.org/3.6/whatsnew/changelog.html#python-3-6-8-final https://www.python.org/psf/ -- Ned Deily nad at python.org -- [] From status at bugs.python.org Fri Dec 28 13:07:52 2018 From: status at bugs.python.org (Python tracker) Date: Fri, 28 Dec 2018 18:07:52 +0000 Subject: [Python-Dev] Summary of Python tracker Issues Message-ID: <20181228180752.1.0694503AAD01DD49@roundup.psfhosted.org> ACTIVITY SUMMARY (2018-12-21 - 2018-12-28) Python tracker at https://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue. Do NOT respond to this message. Issues counts and deltas: open 6902 ( +6) closed 40454 (+47) total 47356 (+53) Open issues with patches: 2739 Issues opened (33) ================== #35556: See if frozen modules can use relative imports https://bugs.python.org/issue35556 opened by brett.cannon #35557: Allow lowercase hexadecimal characters in base64.b16decode() https://bugs.python.org/issue35557 opened by djhoulihan #35560: format(float(123), "00") causes segfault in debug builds https://bugs.python.org/issue35560 opened by xtreak #35561: Valgrind reports Syscall param epoll_ctl(event) points to unin https://bugs.python.org/issue35561 opened by nikratio #35563: Doc: warnings.rst - add links to references https://bugs.python.org/issue35563 opened by cheryl.sabella #35568: Expose the C raise() function in the signal module, for use on https://bugs.python.org/issue35568 opened by njs #35569: OSX: Enable IPV6_RECVPKTINFO https://bugs.python.org/issue35569 opened by chrysn #35570: 2to3 creates code using deprecated imp module https://bugs.python.org/issue35570 opened by hanno #35571: Parallel Timeout Class https://bugs.python.org/issue35571 opened by Stefan Volz #35573: is_HDN is returns false positive and false negative value for https://bugs.python.org/issue35573 opened by Divya Rani #35577: side_effect mocked method lose reference to instance https://bugs.python.org/issue35577 opened by Adnan Umer #35578: Add test for Argument Clinic converters https://bugs.python.org/issue35578 opened by serhiy.storchaka #35580: Windows IocpProactor: CreateIoCompletionPort 4th arg 0xfffffff https://bugs.python.org/issue35580 opened by jeffr at livedata.com #35581: Document @typing.type_check_only https://bugs.python.org/issue35581 opened by srittau #35582: Argument Clinic: inline parsing code for functions with only p https://bugs.python.org/issue35582 opened by serhiy.storchaka #35583: (glibc2.28/MIPS32EL) python 3.7.x interpreter segmentation fau https://bugs.python.org/issue35583 opened by broly #35584: Wrong statement about ^ in howto/regex.rst https://bugs.python.org/issue35584 opened by mdk #35586: Open pyexpat compilation, Make shows error???missing separator https://bugs.python.org/issue35586 opened by kbengine #35588: Speed up mod/divmod/floordiv for Fraction type https://bugs.python.org/issue35588 opened by scoder #35589: BaseSelectorEventLoop.sock_sendall() performance regression: e https://bugs.python.org/issue35589 opened by Huazuo Gao #35590: logging.handlers.SysLogHandler with STREAM connects in constru https://bugs.python.org/issue35590 opened by jso2460 #35592: Not able to use Python 3.7.2 due to SSL issue https://bugs.python.org/issue35592 opened by Gunasekar Rajendran #35593: Register standard browser: Chrome https://bugs.python.org/issue35593 opened by eamanu #35594: Python script generating Segmentation Fault https://bugs.python.org/issue35594 opened by Daugeras #35595: Add sys flag to always show full paths in stack traces (instea https://bugs.python.org/issue35595 opened by Scott Arciszewski #35596: Fatal Python error: initfsencoding: unable to load the file sy https://bugs.python.org/issue35596 opened by hyu #35598: IDLE: Modernize config_key module https://bugs.python.org/issue35598 opened by cheryl.sabella #35599: asyncio windows_events.py IocpProactor bug https://bugs.python.org/issue35599 opened by jeffr at livedata.com #35600: Expose siphash https://bugs.python.org/issue35600 opened by Dima.Tisnek #35601: Race condition in test_signal_handling_args x86-64 High Sierra https://bugs.python.org/issue35601 opened by pablogsal #35602: cleanup code may fail in test_asyncio.test_unix_events.Selecto https://bugs.python.org/issue35602 opened by pablogsal #35603: table header in output of difflib.HtmlDiff.make_table is not e https://bugs.python.org/issue35603 opened by xtreak #35605: backported patch requires new sphinx, minimum sphinx version w https://bugs.python.org/issue35605 opened by Anthony Sottile Most recent 15 issues with no replies (15) ========================================== #35603: table header in output of difflib.HtmlDiff.make_table is not e https://bugs.python.org/issue35603 #35602: cleanup code may fail in test_asyncio.test_unix_events.Selecto https://bugs.python.org/issue35602 #35601: Race condition in test_signal_handling_args x86-64 High Sierra https://bugs.python.org/issue35601 #35599: asyncio windows_events.py IocpProactor bug https://bugs.python.org/issue35599 #35593: Register standard browser: Chrome https://bugs.python.org/issue35593 #35586: Open pyexpat compilation, Make shows error???missing separator https://bugs.python.org/issue35586 #35582: Argument Clinic: inline parsing code for functions with only p https://bugs.python.org/issue35582 #35581: Document @typing.type_check_only https://bugs.python.org/issue35581 #35577: side_effect mocked method lose reference to instance https://bugs.python.org/issue35577 #35571: Parallel Timeout Class https://bugs.python.org/issue35571 #35556: See if frozen modules can use relative imports https://bugs.python.org/issue35556 #35552: Do not read memory past the specified limit in PyUnicode_FromF https://bugs.python.org/issue35552 #35551: Encoding and alias issues https://bugs.python.org/issue35551 #35550: Some define guards for Solaris are wrong https://bugs.python.org/issue35550 #35540: dataclasses.asdict breaks with defaultdict fields https://bugs.python.org/issue35540 Most recent 15 issues waiting for review (15) ============================================= #35603: table header in output of difflib.HtmlDiff.make_table is not e https://bugs.python.org/issue35603 #35602: cleanup code may fail in test_asyncio.test_unix_events.Selecto https://bugs.python.org/issue35602 #35601: Race condition in test_signal_handling_args x86-64 High Sierra https://bugs.python.org/issue35601 #35598: IDLE: Modernize config_key module https://bugs.python.org/issue35598 #35596: Fatal Python error: initfsencoding: unable to load the file sy https://bugs.python.org/issue35596 #35588: Speed up mod/divmod/floordiv for Fraction type https://bugs.python.org/issue35588 #35582: Argument Clinic: inline parsing code for functions with only p https://bugs.python.org/issue35582 #35581: Document @typing.type_check_only https://bugs.python.org/issue35581 #35578: Add test for Argument Clinic converters https://bugs.python.org/issue35578 #35568: Expose the C raise() function in the signal module, for use on https://bugs.python.org/issue35568 #35563: Doc: warnings.rst - add links to references https://bugs.python.org/issue35563 #35560: format(float(123), "00") causes segfault in debug builds https://bugs.python.org/issue35560 #35557: Allow lowercase hexadecimal characters in base64.b16decode() https://bugs.python.org/issue35557 #35552: Do not read memory past the specified limit in PyUnicode_FromF https://bugs.python.org/issue35552 #35550: Some define guards for Solaris are wrong https://bugs.python.org/issue35550 Top 10 most discussed issues (10) ================================= #33830: Error in the output of one example in the httplib docs https://bugs.python.org/issue33830 11 msgs #35596: Fatal Python error: initfsencoding: unable to load the file sy https://bugs.python.org/issue35596 9 msgs #35588: Speed up mod/divmod/floordiv for Fraction type https://bugs.python.org/issue35588 8 msgs #35548: memoryview needlessly (?) requires represented object to be ha https://bugs.python.org/issue35548 7 msgs #35560: format(float(123), "00") causes segfault in debug builds https://bugs.python.org/issue35560 7 msgs #35485: Mac: tkinter windows turn black while resized https://bugs.python.org/issue35485 6 msgs #35563: Doc: warnings.rst - add links to references https://bugs.python.org/issue35563 5 msgs #35568: Expose the C raise() function in the signal module, for use on https://bugs.python.org/issue35568 5 msgs #35573: is_HDN is returns false positive and false negative value for https://bugs.python.org/issue35573 5 msgs #35594: Python script generating Segmentation Fault https://bugs.python.org/issue35594 5 msgs Issues closed (44) ================== #11191: test_search_cpp error on AIX (with xlc) https://bugs.python.org/issue11191 closed by ncoghlan #11192: test_socket error on AIX https://bugs.python.org/issue11192 closed by ncoghlan #21524: Allowing to pass pathlib.Path object in mimetypes.guess_type f https://bugs.python.org/issue21524 closed by cheryl.sabella #22703: Idle Code Context menu entrie(s) https://bugs.python.org/issue22703 closed by terry.reedy #23867: Argument Clinic: inline parsing code for 1-argument functions https://bugs.python.org/issue23867 closed by serhiy.storchaka #24307: [Python 2] pip error on windows whose current user name contai https://bugs.python.org/issue24307 closed by serhiy.storchaka #24390: Python 3.4.3 64 bits is not "high dpi aware" https://bugs.python.org/issue24390 closed by cheryl.sabella #27643: test_ctypes fails on AIX with xlc https://bugs.python.org/issue27643 closed by ncoghlan #29886: links between binascii.{un,}hexlify / bytes.{,to}hex https://bugs.python.org/issue29886 closed by rhettinger #30455: Generate all tokens related code and docs from Grammar/Tokens https://bugs.python.org/issue30455 closed by serhiy.storchaka #30561: sync-up gammavariate and expovariate code https://bugs.python.org/issue30561 closed by rhettinger #31440: wrong default module search path in help message https://bugs.python.org/issue31440 closed by cheryl.sabella #31503: Enhance dir(module) to be informed by __all__ by updating modu https://bugs.python.org/issue31503 closed by cheryl.sabella #32067: Deprecate accepting unrecognized braces in regular expressions https://bugs.python.org/issue32067 closed by serhiy.storchaka #32070: Clarify the behavior of the staticmethod builtin https://bugs.python.org/issue32070 closed by rhettinger #34193: Fix pluralization in TypeError messages in getargs.c https://bugs.python.org/issue34193 closed by serhiy.storchaka #34373: test_time errors on AIX https://bugs.python.org/issue34373 closed by ncoghlan #34711: Lib/http/server.py: Return HTTPStatus.NOT_FOUND if path.endswi https://bugs.python.org/issue34711 closed by ncoghlan #34764: Improve documentation example for using iter() with sentinel v https://bugs.python.org/issue34764 closed by rhettinger #34897: distutils test errors when CXX is not set https://bugs.python.org/issue34897 closed by ncoghlan #35009: argparse throws UnicodeEncodeError for printing help with unic https://bugs.python.org/issue35009 closed by xtreak #35208: IDLE: Squeezed line count ignores wrapping before newline https://bugs.python.org/issue35208 closed by taleinat #35387: Dialogs on IDLE are accompanied by a small black window https://bugs.python.org/issue35387 closed by ned.deily #35402: Upgrade macOS and Windows installers to Tcl 8.6.9 and Tk 8.6.9 https://bugs.python.org/issue35402 closed by ned.deily #35553: Test https://bugs.python.org/issue35553 closed by EWDurbin #35554: Test https://bugs.python.org/issue35554 closed by EWDurbin #35555: IDLE: Gray out Code Context on non-editor windows https://bugs.python.org/issue35555 closed by terry.reedy #35558: venv: running activate.bat gives ' parameter format not correc https://bugs.python.org/issue35558 closed by Nils-Hero #35559: Optimize base64.b16decode to use compiled regex https://bugs.python.org/issue35559 closed by xtreak #35562: Issue in sizeof() function https://bugs.python.org/issue35562 closed by ammar2 #35564: [DOC] Sphinx 2.0 will require master_doc variable set in conf. https://bugs.python.org/issue35564 closed by mdk #35565: Add detail to an assertion failure message in wsgiref https://bugs.python.org/issue35565 closed by rhettinger #35566: DOC: Add links to annotation glossary term https://bugs.python.org/issue35566 closed by rhettinger #35567: Convert membership test from dict-of-constants to a set https://bugs.python.org/issue35567 closed by rhettinger #35572: Logging module cleanup https://bugs.python.org/issue35572 closed by vinay.sajip #35574: Coconut support https://bugs.python.org/issue35574 closed by steven.daprano #35575: Improved range syntax https://bugs.python.org/issue35575 closed by rhettinger #35576: function splitextTest does not return expected value https://bugs.python.org/issue35576 closed by serhiy.storchaka #35579: Typo in in asyncio-task documentation https://bugs.python.org/issue35579 closed by asvetlov #35585: Speedup Enum lookup https://bugs.python.org/issue35585 closed by asvetlov #35587: Python 3.7.2 Embed - zipimport.ZipImportError (Cannot load pyt https://bugs.python.org/issue35587 closed by steve.dower #35591: IDLE: Traceback on Find Selection https://bugs.python.org/issue35591 closed by terry.reedy #35597: Bug in Python's compiler https://bugs.python.org/issue35597 closed by tim.peters #35604: Is python used more than Java Nowadays? https://bugs.python.org/issue35604 closed by steven.daprano