, argc=4, argv=0x7ffce435b2d8, init=, fini=, rtld_fini=, stack_end=0x7ffce435b2c8) at libc-start.c:287 #26 0x000000000057c581 in _start () ==== Google shows similar trace has been reported back in 2009 for python 2.6 https://bugs.launchpad.net/ubuntu/+source/system-config-printer/+bug/478071 ---------- components: Interpreter Core messages: 280640 nosy: pwp333 priority: normal severity: normal status: open title: When using pyro4 with more than 15 threads, python 2.7.12 cores frequently type: crash versions: Python 2.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sat Nov 12 11:29:25 2016 From: report at bugs.python.org (Damian Kotlar) Date: Sat, 12 Nov 2016 16:29:25 +0000 Subject: [New-bugs-announce] [issue28674] fosa.zd.djkps@gmail.com Message-ID: <1478968165.19.0.779355016378.issue28674@psf.upfronthosting.co.za> Changes by Damian Kotlar : ---------- assignee: docs at python components: 2to3 (2.x to 3.x conversion tool), Cross-Build, Demos and Tools, Documentation, Extension Modules, Installation, Interpreter Core, Library (Lib), SSL, Tests, XML, email files: samsungapps.html nosy: Alex.Willmer, barry, docs at python, fosa.zd.djkps at gmail.com, r.david.murray priority: normal severity: normal status: open title: fosa.zd.djkps at gmail.com versions: Python 2.7, Python 3.3, Python 3.4, Python 3.5, Python 3.6, Python 3.7 Added file: http://bugs.python.org/file45461/samsungapps.html _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sat Nov 12 14:05:03 2016 From: report at bugs.python.org (Big Stone) Date: Sat, 12 Nov 2016 19:05:03 +0000 Subject: [New-bugs-announce] [issue28675] about PEP 528 / PEP 529 Message-ID: <1478977503.51.0.187349952636.issue28675@psf.upfronthosting.co.za> New submission from Big Stone: Hi, if possible a few words of explanation to implement it may be helpfull, as I don't see the difference yet, in the typical case where it was not good before: https://cloud.githubusercontent.com/assets/4312421/20240181/5bf917fe-a912-11e6-95e8-e65fcb9a8845.PNG (more an after PEP sales service request than a bug, I presume) ---------- components: Windows messages: 280667 nosy: Big Stone, paul.moore, steve.dower, tim.golden, zach.ware priority: normal severity: normal status: open title: about PEP 528 / PEP 529 versions: Python 3.6 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sat Nov 12 14:30:20 2016 From: report at bugs.python.org (Gareth Rees) Date: Sat, 12 Nov 2016 19:30:20 +0000 Subject: [New-bugs-announce] [issue28676] On macOS Sierra, warning: implicit declaration of function 'getentropy' Message-ID: <1478979020.94.0.0997412321445.issue28676@psf.upfronthosting.co.za> New submission from Gareth Rees: On macOS Sierra (OSX 10.12.1): $ ./configure --with-pydebug && make [... lots of output omitted ...] gcc -c -Wno-unused-result -Wsign-compare -g -O0 -Wall -Wstrict-prototypes -std=c99 -Wextra -Wno-unused-result -Wno-unused-parameter -Wno-missing-field-initializers -I. -I./Include -DPy_BUILD_CORE -o Python/random.o Python/random.c Python/random.c:97:19: warning: implicit declaration of function 'getentropy' is invalid in C99 [-Wimplicit-function-declaration] res = getentropy(buffer, len); ^ 1 warning generated. This is because OSX 10.12.1 has getentropy() but does not have getrandom(). You can see this in pyconfig.h: /* Define to 1 if you have the `getentropy' function. */ #define HAVE_GETENTROPY 1 /* Define to 1 if the getrandom() function is available */ /* #undef HAVE_GETRANDOM */ and this means that in Python/random.c the header is not included: # ifdef HAVE_GETRANDOM # include # elif defined(HAVE_GETRANDOM_SYSCALL) # include # endif It's necessary include if either HAVE_GETRANDOM or HAVE_GETENTROPY is defined. ---------- components: Build, macOS files: getentropy.patch keywords: patch messages: 280669 nosy: Gareth.Rees, ned.deily, ronaldoussoren priority: normal severity: normal status: open title: On macOS Sierra, warning: implicit declaration of function 'getentropy' type: compile error versions: Python 3.7 Added file: http://bugs.python.org/file45465/getentropy.patch _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sat Nov 12 16:32:17 2016 From: report at bugs.python.org (Viorel Tabara) Date: Sat, 12 Nov 2016 21:32:17 +0000 Subject: [New-bugs-announce] [issue28677] difficult to parse sentence structure in "When an instance attribute is referenced that isn't a data attribute" Message-ID: <1478986337.93.0.398450030776.issue28677@psf.upfronthosting.co.za> New submission from Viorel Tabara: Method objects are explained at: https://docs.python.org/3/tutorial/classes.html#method-objects I'd like to suggest a change from: When an instance attribute is referenced that isn?t a data attribute, its class is searched. to something along this line: When an instance attribute --- that isn?t a data attribute --- is referenced, the instance parent class will be searched. ---------- assignee: docs at python components: Documentation messages: 280671 nosy: docs at python, viorel priority: normal severity: normal status: open title: difficult to parse sentence structure in "When an instance attribute is referenced that isn't a data attribute" type: enhancement _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sat Nov 12 16:43:17 2016 From: report at bugs.python.org (Lewis McCarthy) Date: Sat, 12 Nov 2016 21:43:17 +0000 Subject: [New-bugs-announce] [issue28678] tarfile extract docs inconsistent naming of parameter numeric_owner or numeric_only Message-ID: <1478986997.97.0.266901565228.issue28678@psf.upfronthosting.co.za> New submission from Lewis McCarthy: https://docs.python.org/3/library/tarfile.html#tarfile-objects Most of the documentation for the extract and extractall methods refers to a parameter named numeric_owner. However, the notes on what changed in v3.5 refer instead to numeric_only. One of these names is incorrect and should be changed to match the other. Same issue in the 3.6 and 3.7 docs. https://docs.python.org/3.6/library/tarfile.html#tarfile-objects https://docs.python.org/3.7/library/tarfile.html#tarfile-objects ---------- assignee: docs at python components: Documentation messages: 280673 nosy: docs at python, pseudonym priority: normal severity: normal status: open title: tarfile extract docs inconsistent naming of parameter numeric_owner or numeric_only type: behavior versions: Python 3.5, Python 3.6, Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sat Nov 12 21:32:16 2016 From: report at bugs.python.org (Yudai Fujiwara) Date: Sun, 13 Nov 2016 02:32:16 +0000 Subject: [New-bugs-announce] [issue28679] CGIHTTPServer displays raw python code when the url contains '/' after '?' Message-ID: <1479004336.61.0.664243507807.issue28679@psf.upfronthosting.co.za> Changes by Yudai Fujiwara : ---------- components: Library (Lib) nosy: Yudai Fujiwara priority: normal severity: normal status: open title: CGIHTTPServer displays raw python code when the url contains '/' after '?' type: security versions: Python 2.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sat Nov 12 21:39:12 2016 From: report at bugs.python.org (Timothy Widrick) Date: Sun, 13 Nov 2016 02:39:12 +0000 Subject: [New-bugs-announce] [issue28680] bdist_wininst generated 64-bit executable looks for 3.5-32 Message-ID: <1479004752.08.0.80929351166.issue28680@psf.upfronthosting.co.za> New submission from Timothy Widrick: Short version: wininst-14.0-amd64.exe has "-32" in it. It shouldn't, should it? Longer version: Running "python setup.py bdist_wininst" gives me a file named like: project-version.win-amd64-py3.5.exe When I run that, I get: Python version 3.5-32 required, which was not found in the registry. My "fix" was to edit the wininst-14.0-amd64.exe and replace the "-32" with 3 null bytes. ---------- components: Distutils messages: 280682 nosy: Timothy Widrick, dstufft, eric.araujo priority: normal severity: normal status: open title: bdist_wininst generated 64-bit executable looks for 3.5-32 type: behavior versions: Python 3.5 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sat Nov 12 22:36:36 2016 From: report at bugs.python.org (Xue Fuqiao) Date: Sun, 13 Nov 2016 03:36:36 +0000 Subject: [New-bugs-announce] [issue28681] About function renaming in the tutorial Message-ID: <1479008196.92.0.673244295473.issue28681@psf.upfronthosting.co.za> New submission from Xue Fuqiao: In https://hg.python.org/cpython/file/6fbb7c9d77c6/Doc/tutorial/controlflow.rst#l295 : A function definition introduces the function name in the current symbol table. The value of the function name has a type that is recognized by the interpreter as a user-defined function. This value can be assigned to another name which can then also be used as a function. This serves as a general renaming mechanism Maybe "aliasing" is a better term than "renaming" here, since the original function name can still be used after the "renaming". ---------- assignee: docs at python components: Documentation files: renaming.patch keywords: patch messages: 280683 nosy: docs at python, xfq priority: normal severity: normal status: open title: About function renaming in the tutorial type: enhancement versions: Python 3.3, Python 3.4, Python 3.5, Python 3.6, Python 3.7 Added file: http://bugs.python.org/file45468/renaming.patch _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun Nov 13 04:24:10 2016 From: report at bugs.python.org (Serhiy Storchaka) Date: Sun, 13 Nov 2016 09:24:10 +0000 Subject: [New-bugs-announce] [issue28682] Bytes support in os.fwalk() Message-ID: <1479029050.5.0.426622691368.issue28682@psf.upfronthosting.co.za> New submission from Serhiy Storchaka: os.walk() supports string and bytes paths, but os.fwalk() always supported only string paths. Proposed patch adds support of bytes paths in os.fwalk(). ---------- components: Library (Lib) files: fwalk-bytes.patch keywords: patch messages: 280690 nosy: larry, loewis, serhiy.storchaka priority: normal severity: normal stage: patch review status: open title: Bytes support in os.fwalk() type: enhancement versions: Python 3.7 Added file: http://bugs.python.org/file45470/fwalk-bytes.patch _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun Nov 13 14:32:04 2016 From: report at bugs.python.org (Xavier de Gaye) Date: Sun, 13 Nov 2016 19:32:04 +0000 Subject: [New-bugs-announce] [issue28683] bind() on a unix socket raises PermissionError on Android for a non-root user Message-ID: <1479065524.08.0.12999801833.issue28683@psf.upfronthosting.co.za> New submission from Xavier de Gaye: Tests in test_socket and test_asyncore fail on Android when bind() on a unix socket raises PermissionError for a non-root user. This occurs also in test_asyncio and this is logged in a separate issue as asyncio has its own project. Patch attached. ---------- assignee: xdegaye components: Tests files: bind_unix_socket.patch keywords: patch messages: 280709 nosy: xdegaye priority: normal severity: normal stage: patch review status: open title: bind() on a unix socket raises PermissionError on Android for a non-root user type: behavior versions: Python 3.6, Python 3.7 Added file: http://bugs.python.org/file45475/bind_unix_socket.patch _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun Nov 13 15:39:34 2016 From: report at bugs.python.org (Xavier de Gaye) Date: Sun, 13 Nov 2016 20:39:34 +0000 Subject: [New-bugs-announce] [issue28684] [asyncio] bind() on a unix socket raises PermissionError on Android for a non-root user Message-ID: <1479069574.19.0.731580166174.issue28684@psf.upfronthosting.co.za> New submission from Xavier de Gaye: Tests in test_asyncio fail on Android when bind() on a unix socket raises PermissionError for a non-root user. See the attached test_asyncio.log. See also issue 28683 and its patch(es). ---------- components: Tests, asyncio files: test_asyncio.log messages: 280714 nosy: giampaolo.rodola, gvanrossum, haypo, xdegaye, yselivanov priority: normal severity: normal stage: needs patch status: open title: [asyncio] bind() on a unix socket raises PermissionError on Android for a non-root user type: behavior versions: Python 3.6, Python 3.7 Added file: http://bugs.python.org/file45476/test_asyncio.log _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun Nov 13 16:19:40 2016 From: report at bugs.python.org (Elliot Gorokhovsky) Date: Sun, 13 Nov 2016 21:19:40 +0000 Subject: [New-bugs-announce] [issue28685] Optimizing list.sort() by performing safety checks in advance Message-ID: <1479071980.11.0.232140584241.issue28685@psf.upfronthosting.co.za> New submission from Elliot Gorokhovsky: When python compares two objects, there are many safety checks that have to be performed before the actual comparison can take place. These include checking the types of the two objects, as well as checking various type-specific properties, like character width for strings or the number of machine-words that represent a long. Obviously, the vast majority of the work done during a list.sort() is spent in comparisons, so even small optimizations can be important. What I noticed is that since we have n objects, but we're doing O(nlogn) comparisons, it pays off to do all the safety checks in a single pass and then select an optimized comparison function that leverages the information gained to avoid as many sort-time checks as possible. I made the following assumptions: 1. In practice, lists to be sorted are almost always type-homogeneous. 2. Most lists to be sorted consist of homogeneous elements of the following types: a. Latin strings (i.e. strings whose characters are one machine-word wide) b. Longs that fit in a single machine-word c. Floats d. Tuples of the above 3. The above assumptions also hold for lists constructed by taking a list of tuples to be sorted and extracting the first elements. 4. The vast majority of tuple comparisons never get past the first element (i.e. the first elements of tuples are rarely equal). My patch adds the following routine to list.sort(): 1. Go through the list and see which of the assumptions, if any, hold (except (4), which can't be checked in advance). 2. Replace PyObject_RichCompareBool() with an optimized compare function that is able to ignore some safety checks by applying the assumption we've already verified to be true. There are then two questions: when none of the assumptions hold, how expensive is (1)? And when we do get a "hit", how much time do we save by applying (2)? Those questions, of course, can only be answered by benchmarks. So here they are, computed on an isolated CPU core, on two python interpreters, both compiled with ./configure --with-optimizations: # This is a runnable script. Just build the reference interpreter in python-ref and the patched interpreter in python-dev. # Pathological cases (where we pay for (1) but don't get any savings from (2)): what kind of losses do we suffer? # How expensive is (1) for empty lists? python-dev/python -m perf timeit -s "l=[]" "sorted(l)" --rigorous python-ref/python -m perf timeit -s "l=[]" "sorted(l)" --rigorous # Median +- std dev: 212 ns +- 9 ns # Median +- std dev: 216 ns +- 10 ns # (difference is within std dev) # How expensive is (1) for singleton lists? python-dev/python -m perf timeit -s "l=[None]" "sorted(l)" --rigorous python-ref/python -m perf timeit -s "l=[None]" "sorted(l)" --rigorous # Median +- std dev: 235 ns +- 16 ns # Median +- std dev: 238 ns +- 15 ns # (difference is within std dev) # How expensive is (1) for non-type-homogeneous lists? # (we use small lists because as n gets large, the fraction time spent in (1) gets smaller) python-dev/python -m perf timeit -s "import random; l=[random.uniform(-1,1) for _ in range(0,10)]+[1]" "sorted(l)" --rigorous python-ref/python -m perf timeit -s "import random; l=[random.uniform(-1,1) for _ in range(0,10)]+[1]" "sorted(l)" --rigorous # Median +- std dev: 784 ns +- 35 ns # Median +- std dev: 707 ns +- 51 ns # 10% slower. While this is unfortunate, this case almost never occurs in practice, and the losses are certainly offset by the gains we'll see below. # Also, note that for large lists, the difference would be *much* smaller, since the time spent in (1) gets smaller like 1/log(n). # So basically, we only pay a penalty for non-type-homogeneous small lists. # OK, now for the cases that actually occur in practice: # What kind of gains do we get for floats? python-dev/python -m perf timeit -s "import random; l=[random.uniform(-1,1) for _ in range(0,100)]" "sorted(l)" --rigorous python-ref/python -m perf timeit -s "import random; l=[random.uniform(-1,1) for _ in range(0,100)]" "sorted(l)" --rigorous # Median +- std dev: 3.63 us +- 0.20 us # Median +- std dev: 8.81 us +- 0.37 us # Wow! 59% faster! And if we used a large list, we would've gotten even better numbers. # What kind of gains do we get for latin strings? python-dev/python -m perf timeit -s "import random; l=[str(random.uniform(-1,1)) for _ in range(0,100)]" "sorted(l)" --rigorous python-ref/python -m perf timeit -s "import random; l=[str(random.uniform(-1,1)) for _ in range(0,100)]" "sorted(l)" --rigorous # Median +- std dev: 9.51 us +- 0.28 us # Median +- std dev: 15.9 us +- 0.7 us # 40% faster! # What kind of gains do we get for non-latin strings (which I imagine aren't that common to sort in practice anyway) python-dev/python -m perf timeit -s "import random; l=[str(random.uniform(-1,1))+'\uffff' for _ in range(0,100)]" "sorted(l)" --rigorous python-ref/python -m perf timeit -s "import random; l=[str(random.uniform(-1,1))+'\uffff' for _ in range(0,100)]" "sorted(l)" --rigorous # Median +- std dev: 12.2 us +- 0.4 us # Median +- std dev: 14.2 us +- 0.6 us # 14%. Not as impressive, but again, this is a bit of a pathological case. # What kind of gains do we get for ints that fit in a machine word? (I'll keep them in (-2^15,2^15) to be safe) python-dev/python -m perf timeit -s "import random; l=[int(random.uniform(-1,1)*2**15) for _ in range(0,100)]" "sorted(l)" --rigorous python-ref/python -m perf timeit -s "import random; l=[int(random.uniform(-1,1)*2**15) for _ in range(0,100)]" "sorted(l)" --rigorous # Median +- std dev: 4.92 us +- 0.35 us # Median +- std dev: 9.59 us +- 0.75 us # 49% faster. Pretty cool stuff! # What kind of gains do we get for pathologically large ints? python-dev/python -m perf timeit -s "import random; l=[int(random.uniform(-1,1)*2**100) for _ in range(0,100)]" "sorted(l)" --rigorous python-ref/python -m perf timeit -s "import random; l=[int(random.uniform(-1,1)*2**100) for _ in range(0,100)]" "sorted(l)" --rigorous # Median +- std dev: 6.93 us +- 0.26 us # Median +- std dev: 8.93 us +- 0.23 us # 22% faster. The same comparison function is used as in the pathological string case, but the numbers are different because comparing strings # is more expensive than comparing ints, so we save less time from skipping the type checks. Regardless, 22% is still pretty cool. And I can't # imagine it's that common to sort huge ints anyway, unless you're doing some sort of math conjecture testing or something. # What kind of gains do we get for tuples whose first elements are floats? python-dev/python -m perf timeit -s "import random; l=[(random.uniform(-1,1), 'whatever') for _ in range(0,100)]" "sorted(l)" --rigorous python-ref/python -m perf timeit -s "import random; l=[(random.uniform(-1,1), 'whatever') for _ in range(0,100)]" "sorted(l)" --rigorous # Median +- std dev: 5.83 us +- 0.50 us # Median +- std dev: 21.5 us +- 0.5 us # WOW! 73% faster! # A note: I know of at least one application where this is relevant. The DEAP evolutionary algorithms library, which is very popular, has to sort # tuples of floats when it's selecting individuals for the next generation. It spends a non-trivial amount of time sorting those tuples of floats, # and this patch cuts that non-trivial time by 73%! Pretty cool! Now we can evolve more individuals more better! # What kind of gains do we get for tuples whose first elements are latin strings? python-dev/python -m perf timeit -s "import random; l=[(str(random.uniform(-1,1)), 42) for _ in range(0,100)]" "sorted(l)" --rigorous python-ref/python -m perf timeit -s "import random; l=[(str(random.uniform(-1,1)), 42) for _ in range(0,100)]" "sorted(l)" --rigorous # Median +- std dev: 14.9 us +- 0.8 us # Median +- std dev: 31.2 us +- 1.3 us # 52%. Not too shabby! # What kind of gains do we get for tuples of other stuff? # Obviously there are lots of combinations possible, I won't waste your time documenting all of them. # Suffice it to say that the gains for lists tuples of x is always greater than the gains for lists of x, since we bypass # two layers of safety checks: checks on the tuples, and then checks on the x. # What kind of gains do we for arbitrary lists of objects of the same type? # See the benchmarks for large ints and non-latin strings. It will be different for each object, since it depends on the ratio between # the cost of type-check and the cost of comparison. Regardless, it is always non-trivial, unless the cost of comparison is huge. # End of script # TL;DR: This patch makes common list.sort() cases 40-75% faster, and makes very uncommon pathological cases at worst 15% slower. It accomplishes this by performing the endless, but necessary, safety checks involved in comparison up front, and then using optimized compares that ignore the already-performed checks during the actual sort. ---------- components: Interpreter Core files: fastsort.patch keywords: patch messages: 280718 nosy: elliot.gorokhovsky priority: normal severity: normal status: open title: Optimizing list.sort() by performing safety checks in advance type: performance versions: Python 3.7 Added file: http://bugs.python.org/file45477/fastsort.patch _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Nov 14 03:07:24 2016 From: report at bugs.python.org (wdhwg001) Date: Mon, 14 Nov 2016 08:07:24 +0000 Subject: [New-bugs-announce] [issue28686] py.exe ignored PATH when using python3 shebang Message-ID: <1479110844.15.0.911795795866.issue28686@psf.upfronthosting.co.za> New submission from wdhwg001: https://github.com/pypa/virtualenv/issues/979 TL;DR py.exe does not search PATH when using `#! /usr/bin/env python3`. And the right steps I think should be: 1. Search ./ 2. Search PATH 3. Fallback to system installed Python 3 But currently py.exe seems only do Step 3 once the shebang has substring of `python3`. ---------- components: Windows messages: 280740 nosy: paul.moore, steve.dower, tim.golden, wdhwg001, zach.ware priority: normal severity: normal status: open title: py.exe ignored PATH when using python3 shebang type: behavior versions: Python 3.5 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Nov 14 05:38:47 2016 From: report at bugs.python.org (Igor Skochinsky) Date: Mon, 14 Nov 2016 10:38:47 +0000 Subject: [New-bugs-announce] [issue28687] Python 2.7.12 windows x64 installer fails after previous bad uninstall Message-ID: <1479119927.24.0.974377093409.issue28687@psf.upfronthosting.co.za> New submission from Igor Skochinsky: It's somewhat my fault but I think it's not such an uncommon scenario. I had installed 2.7.10 x64 some time ago into C:\Python27_64 but then had to delete it to save space. Rather foolishly instead of using the uninstaller, I just trashed the directory and now I have a problem: neither the Control Panel uninstall item, nor the fresh 2.7.12 installer work. The latter fails when it tries to use the nonexisting Python to remove pip: -------------------------------- MSI (s) (04:B0) [11:00:09:881]: Product: Python 2.7.10 (64-bit) -- Error 1721. There is a problem with this Windows Installer package. A program required for this install to complete could not be run. Contact your support personnel or package vendor. Action: RemovePip, location: C:\Python27_64\python.exe, command: -B -m ensurepip._uninstall -------------------------------- In the UI just the first message is shown, without the failed commandline, so it's not at all clear what went wrong. I think at the very least the user should be informed about the failed command so they can clean up the old installer's remains, or maybe the installation should be allowed to proceed anyway, replacing the bad paths in the registry. In the end, I had to use "Msizap TWA {E2B51919-207A-43EB-AE78-733F9C6797C3}" after which I could install 2.7.12 into a separate directory ---------- components: Installation messages: 280747 nosy: Igor.Skochinsky priority: normal severity: normal status: open title: Python 2.7.12 windows x64 installer fails after previous bad uninstall type: behavior versions: Python 2.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Nov 14 07:56:46 2016 From: report at bugs.python.org (STINNER Victor) Date: Mon, 14 Nov 2016 12:56:46 +0000 Subject: [New-bugs-announce] [issue28688] Warning -- warnings.filters was modified by test_warnings Message-ID: <1479128206.47.0.155651436814.issue28688@psf.upfronthosting.co.za> New submission from STINNER Victor: The issue #23839 modified the test runner to always clear caches before running tests. As a side effect, test_warnings started to complain with: Warning -- warnings.filters was modified by test_warnings The issue comes from the following function of test_warnings/__init__.py: def setUpModule(): py_warnings.onceregistry.clear() c_warnings.onceregistry.clear() I suggest to rewrite this function as a setUp/tearDown method in BaseTest and *restores* the old value of these dictionaries. I guess that the bug affects all Python versions, even if only Python 3.7 logs a warning. ---------- components: Tests messages: 280767 nosy: haypo, martin.panter, serhiy.storchaka priority: normal severity: normal status: open title: Warning -- warnings.filters was modified by test_warnings versions: Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Nov 14 09:10:47 2016 From: report at bugs.python.org (Christian Heimes) Date: Mon, 14 Nov 2016 14:10:47 +0000 Subject: [New-bugs-announce] [issue28689] OpenSSL 1.1.0c test failures Message-ID: <1479132647.76.0.450425015981.issue28689@psf.upfronthosting.co.za> New submission from Christian Heimes: OpenSSL 1.1.0c broke a bunch of tests. The same tests are passing fine with OpenSSL 1.1.0 to 1.1.0b. It looks like a problem with EOF / connection close error. I'm seeing similar problems in MIT KRB5's OpenSSL plugin, too. ====================================================================== ERROR: test_ciphers (test.test_ssl.SimpleBackgroundTests) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/heimes/dev/python/cpython/Lib/test/test_ssl.py", line 1658, in test_ciphers s.connect(self.server_addr) File "/home/heimes/dev/python/cpython/Lib/ssl.py", line 1093, in connect self._real_connect(addr, False) File "/home/heimes/dev/python/cpython/Lib/ssl.py", line 1084, in _real_connect self.do_handshake() File "/home/heimes/dev/python/cpython/Lib/ssl.py", line 1061, in do_handshake self._sslobj.do_handshake() File "/home/heimes/dev/python/cpython/Lib/ssl.py", line 683, in do_handshake self._sslobj.do_handshake() ConnectionResetError: [Errno 104] Connection reset by peer ====================================================================== ERROR: test_connect (test.test_ssl.SimpleBackgroundTests) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/heimes/dev/python/cpython/Lib/test/test_ssl.py", line 1483, in test_connect s.connect(self.server_addr) File "/home/heimes/dev/python/cpython/Lib/ssl.py", line 1093, in connect self._real_connect(addr, False) File "/home/heimes/dev/python/cpython/Lib/ssl.py", line 1084, in _real_connect self.do_handshake() File "/home/heimes/dev/python/cpython/Lib/ssl.py", line 1061, in do_handshake self._sslobj.do_handshake() File "/home/heimes/dev/python/cpython/Lib/ssl.py", line 683, in do_handshake self._sslobj.do_handshake() ConnectionResetError: [Errno 104] Connection reset by peer ====================================================================== ERROR: test_connect_cadata (test.test_ssl.SimpleBackgroundTests) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/heimes/dev/python/cpython/Lib/test/test_ssl.py", line 1600, in test_connect_cadata s.connect(self.server_addr) File "/home/heimes/dev/python/cpython/Lib/ssl.py", line 1093, in connect self._real_connect(addr, False) File "/home/heimes/dev/python/cpython/Lib/ssl.py", line 1084, in _real_connect self.do_handshake() File "/home/heimes/dev/python/cpython/Lib/ssl.py", line 1061, in do_handshake self._sslobj.do_handshake() File "/home/heimes/dev/python/cpython/Lib/ssl.py", line 683, in do_handshake self._sslobj.do_handshake() ConnectionResetError: [Errno 104] Connection reset by peer ====================================================================== ERROR: test_connect_capath (test.test_ssl.SimpleBackgroundTests) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/heimes/dev/python/cpython/Lib/test/test_ssl.py", line 1579, in test_connect_capath s.connect(self.server_addr) File "/home/heimes/dev/python/cpython/Lib/ssl.py", line 1093, in connect self._real_connect(addr, False) File "/home/heimes/dev/python/cpython/Lib/ssl.py", line 1080, in _real_connect socket.connect(self, addr) ConnectionRefusedError: [Errno 111] Connection refused ====================================================================== ERROR: test_connect_with_context (test.test_ssl.SimpleBackgroundTests) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/heimes/dev/python/cpython/Lib/test/test_ssl.py", line 1541, in test_connect_with_context s.connect(self.server_addr) File "/home/heimes/dev/python/cpython/Lib/ssl.py", line 1093, in connect self._real_connect(addr, False) File "/home/heimes/dev/python/cpython/Lib/ssl.py", line 1084, in _real_connect self.do_handshake() File "/home/heimes/dev/python/cpython/Lib/ssl.py", line 1061, in do_handshake self._sslobj.do_handshake() File "/home/heimes/dev/python/cpython/Lib/ssl.py", line 683, in do_handshake self._sslobj.do_handshake() ConnectionResetError: [Errno 104] Connection reset by peer ====================================================================== ERROR: test_get_server_certificate (test.test_ssl.SimpleBackgroundTests) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/heimes/dev/python/cpython/Lib/test/test_ssl.py", line 1645, in test_get_server_certificate _test_get_server_certificate(self, *self.server_addr, cert=SIGNING_CA) File "/home/heimes/dev/python/cpython/Lib/test/test_ssl.py", line 1830, in _test_get_server_certificate pem = ssl.get_server_certificate((host, port), ca_certs=cert) File "/home/heimes/dev/python/cpython/Lib/ssl.py", line 1215, in get_server_certificate with create_connection(addr) as sock: File "/home/heimes/dev/python/cpython/Lib/socket.py", line 722, in create_connection raise err File "/home/heimes/dev/python/cpython/Lib/socket.py", line 713, in create_connection sock.connect(sa) ConnectionRefusedError: [Errno 111] Connection refused ====================================================================== ERROR: test_session_handling (test.test_ssl.ThreadedTests) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/heimes/dev/python/cpython/Lib/test/test_ssl.py", line 3552, in test_session_handling s.connect((HOST, server.port)) File "/home/heimes/dev/python/cpython/Lib/ssl.py", line 1093, in connect self._real_connect(addr, False) File "/home/heimes/dev/python/cpython/Lib/ssl.py", line 1080, in _real_connect socket.connect(self, addr) ConnectionRefusedError: [Errno 111] Connection refused ====================================================================== ERROR: test_tls_unique_channel_binding (test.test_ssl.ThreadedTests) Test tls-unique channel binding. ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/heimes/dev/python/cpython/Lib/test/test_ssl.py", line 3167, in test_tls_unique_channel_binding s.connect((HOST, server.port)) File "/home/heimes/dev/python/cpython/Lib/ssl.py", line 1093, in connect self._real_connect(addr, False) File "/home/heimes/dev/python/cpython/Lib/ssl.py", line 1080, in _real_connect socket.connect(self, addr) ConnectionRefusedError: [Errno 111] Connection refused ---------- assignee: christian.heimes components: SSL, Tests messages: 280776 nosy: christian.heimes priority: high severity: normal status: open title: OpenSSL 1.1.0c test failures type: behavior versions: Python 3.6, Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Nov 14 11:24:47 2016 From: report at bugs.python.org (Walter Farrell) Date: Mon, 14 Nov 2016 16:24:47 +0000 Subject: [New-bugs-announce] [issue28690] Loop in re (regular expression) processing Message-ID: <1479140687.65.0.411208896351.issue28690@psf.upfronthosting.co.za> New submission from Walter Farrell: Given: pattern = r"(^|[^\\])<(pm [^ ]+( +|'[^']*'|\"[^\"]*\"|[^>]+)+)>" s = "Bain, F. W. " added to the end of s, it returns quickly with a match. Without the ">" it should fail, but instead seems to loop. (If I use the regex module instead of re, it fails properly and quickly.) Python 3.5.1, Windows 10: Python 3.5.1 (v3.5.1:37a07cee5969, Dec 6 2015, 01:54:25) [MSC v.1900 64 bit (AMD64)] on win32 ---------- components: Library (Lib), Regular Expressions messages: 280790 nosy: Walter Farrell, ezio.melotti, mrabarnett priority: normal severity: normal status: open title: Loop in re (regular expression) processing type: resource usage versions: Python 3.5 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Nov 14 11:53:06 2016 From: report at bugs.python.org (STINNER Victor) Date: Mon, 14 Nov 2016 16:53:06 +0000 Subject: [New-bugs-announce] [issue28691] python3.6 -Werror -c '"\c"' fails with an assertion error Message-ID: <1479142386.33.0.817704504707.issue28691@psf.upfronthosting.co.za> New submission from STINNER Victor: The issue #28128 (changeset 259745f9a1e4) introduced a DeprecationWarning on invalid Unicode sequence like "\c". When Python is run with -Werror, Python crashs with an assertion error: $ ./python -Werror -c '"\c"' python: Objects/abstract.c:2232: PyObject_Call: Assertion `!PyErr_Occurred()' failed. Aborted (core dumped) Gdb traceback: (gdb) where #0 0x00007ffff711f6f5 in __GI_raise (sig=sig at entry=6) at ../sysdeps/unix/sysv/linux/raise.c:54 #1 0x00007ffff71212fa in __GI_abort () at abort.c:89 #2 0x00007ffff7117f97 in __assert_fail_base (fmt=, assertion=assertion at entry=0x69b577 "!PyErr_Occurred()", file=file at entry=0x69a922 "Objects/abstract.c", line=line at entry=2232, function=function at entry=0x69b9c8 <__PRETTY_FUNCTION__.11084> "PyObject_Call") at assert.c:92 #3 0x00007ffff7118042 in __GI___assert_fail (assertion=0x69b577 "!PyErr_Occurred()", file=0x69a922 "Objects/abstract.c", line=2232, function=0x69b9c8 <__PRETTY_FUNCTION__.11084> "PyObject_Call") at assert.c:101 #4 0x000000000045afe8 in PyObject_Call (func=, args=(2, 'No such file or directory', ''), kwargs=0x0) at Objects/abstract.c:2232 #5 0x00000000005e77de in PyErr_SetFromErrnoWithFilenameObjects (exc=, filenameObject='', filenameObject2=0x0) at Python/errors.c:559 #6 0x00000000005e7631 in PyErr_SetFromErrnoWithFilenameObject (exc=, filenameObject='') at Python/errors.c:471 #7 0x000000000043e839 in _Py_fopen_obj (path='', mode=0x6c86e7 "rb") at Python/fileutils.c:1138 #8 0x00000000005e8e54 in PyErr_ProgramTextObject (filename='', lineno=1) at Python/errors.c:1179 #9 0x0000000000598e90 in ast_error (c=0x7fffffffd8b0, n=0x7ffff02d31f0, errmsg=0x7ffff02c1a80 "invalid escape sequence \\c") at Python/ast.c:673 #10 0x00000000005a3ae8 in warn_invalid_escape_sequence (c=0x7fffffffd8b0, n=0x7ffff02d31f0, first_invalid_escape_char=99 'c') at Python/ast.c:4134 #11 0x00000000005a4123 in decode_unicode_with_escapes (c=0x7fffffffd8b0, n=0x7ffff02d31f0, s=0x7ffff03157f0 "\\c\313\313\313\313\313\313\313\313\313", , len=2) at Python/ast.c:4202 #12 0x00000000005a63b8 in parsestr (c=0x7fffffffd8b0, n=0x7ffff02d31f0, bytesmode=0x7fffffffd29c, rawmode=0x7fffffffd298, result=0x7fffffffd290, fstr=0x7fffffffd288, fstrlen=0x7fffffffd280) at Python/ast.c:5114 #13 0x00000000005a64cb in parsestrplus (c=0x7fffffffd8b0, n=0x7ffff02d31a8) at Python/ast.c:5145 #14 0x000000000059d17b in ast_for_atom (c=0x7fffffffd8b0, n=0x7ffff02d31a8) at Python/ast.c:2112 #15 0x000000000059e4d9 in ast_for_atom_expr (c=0x7fffffffd8b0, n=0x7ffff02d3160) at Python/ast.c:2467 #16 0x000000000059e649 in ast_for_power (c=0x7fffffffd8b0, n=0x7ffff02d3118) at Python/ast.c:2504 #17 0x000000000059ef95 in ast_for_expr (c=0x7fffffffd8b0, n=0x7ffff02d3118) at Python/ast.c:2692 #18 0x000000000059f8d2 in ast_for_testlist (c=0x7fffffffd8b0, n=0x7ffff0351d30) at Python/ast.c:2883 #19 0x000000000059f992 in ast_for_expr_stmt (c=0x7fffffffd8b0, n=0x7ffff0351ce8) at Python/ast.c:2906 #20 0x00000000005a3475 in ast_for_stmt (c=0x7fffffffd8b0, n=0x7ffff0351ce8) at Python/ast.c:3983 #21 0x00000000005997e8 in PyAST_FromNodeObject (n=0x7ffff0351c10, flags=0x7fffffffdcf0, filename='', arena=0x7ffff02ec040) at Python/ast.c:839 #22 0x000000000042a146 in PyParser_ASTFromFileObject (fp=0x7ffff74a78c0 <_IO_2_1_stdin_>, filename='', enc=0x7ffff7e574f8 "UTF-8", start=256, ps1=0x7ffff02cdd18 ">>> ", ps2=0x7ffff02cdde8 "... ", flags=0x7fffffffdcf0, errcode=0x7fffffffda4c, arena=0x7ffff02ec040) at Python/pythonrun.c:1169 #23 0x0000000000426b00 in PyRun_InteractiveOneObject (fp=0x7ffff74a78c0 <_IO_2_1_stdin_>, filename='', flags=0x7fffffffdcf0) at Python/pythonrun.c:212 #24 0x0000000000426698 in PyRun_InteractiveLoopFlags (fp=0x7ffff74a78c0 <_IO_2_1_stdin_>, filename_str=0x697276 "", flags=0x7fffffffdcf0) at Python/pythonrun.c:112 #25 0x0000000000426491 in PyRun_AnyFileExFlags (fp=0x7ffff74a78c0 <_IO_2_1_stdin_>, filename=0x697276 "", closeit=0, flags=0x7fffffffdcf0) at Python/pythonrun.c:74 #26 0x00000000004421cd in run_file (fp=0x7ffff74a78c0 <_IO_2_1_stdin_>, filename=0x0, p_cf=0x7fffffffdcf0) at Modules/main.c:319 #27 0x0000000000443086 in Py_Main (argc=2, argv=0x9d2010) at Modules/main.c:779 #28 0x000000000041d2c9 in main (argc=2, argv=0x7fffffffdf28) at ./Programs/python.c:69 ---------- messages: 280793 nosy: ebarry, eric.smith, haypo, ned.deily priority: release blocker severity: normal status: open title: python3.6 -Werror -c '"\c"' fails with an assertion error versions: Python 3.6, Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Nov 14 13:37:59 2016 From: report at bugs.python.org (Serhiy Storchaka) Date: Mon, 14 Nov 2016 18:37:59 +0000 Subject: [New-bugs-announce] [issue28692] gettext: deprecate selecting plural form by fractional numbers Message-ID: <1479148679.54.0.229445108959.issue28692@psf.upfronthosting.co.za> New submission from Serhiy Storchaka: GNU gettext library accepts only integer value for selecting plural form. Python gettext accepts arbitrary numbers. But gettext formulas are not purposed to support non-integer values and can return incorrect result. For example (in Ukrainian): "1 ?????", but "1.5 ?????", not "1.5 ????". "2 ???????", but "2.75 ???????", not "2.75 ????????". "5 ????", but "5.7 ?????", not "5.7 ????". Separate plural form should be used for fractional numbers. Even if fractional part happens to be zero, it is acceptable (e.g. "Time elapsed: 1.000 seconds" in English). Proposed patch deprecates fractional numbers for selecting plural form in gettext. ---------- components: Library (Lib) messages: 280804 nosy: Tim.Graham, loewis, serhiy.storchaka priority: normal severity: normal stage: patch review status: open title: gettext: deprecate selecting plural form by fractional numbers type: enhancement versions: Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Nov 14 15:09:08 2016 From: report at bugs.python.org (Mingye Wang) Date: Mon, 14 Nov 2016 20:09:08 +0000 Subject: [New-bugs-announce] [issue28693] No HKSCS support in Windows cp950 Message-ID: <1479154148.6.0.43343383532.issue28693@psf.upfronthosting.co.za> New submission from Mingye Wang: Python's cp950 implementation lacks support for HKSCS ('big5hkscs'). This support, which maps HKSCS Big5-EUDC code points to Unicode PUA code points algorithmically, is found in Windows Vista+ as well as an update for XP. An experiment session is shown below. I will use '2>>>' to denote a Win32 build of Python 2.7.10 running under a console window set to cp950 (via chcp), and '3>>>' to denote a Python 3.4.3 build running under Cygwin's UTF-8 mintty. HKSCS-2008's table is used http://www.ogcio.gov.hk/en/business/tech_promotion/ccli/terms/doc/hkscs-2008-big5-iso.txt for a list of HKSCS characters; note though, its non-PUA mappings are not found in Windows. Let's start with the first character in that list. 3>>> u'\u43F0' '?' 3>>> print(u'\uF266') # provisional PUA ? 3>>> u'\u43F0'.encode('cp950') # FAIL 3>>> u'\uF266'.encode('cp950') # FAIL 3>>> u'\u43F0'.encode('hkscs') b'\x87@' 3>>> u'\uF266'.encode('hkscs') # FAIL` These experiments above show how Python 3 handles HKSCS characters, and how U+43F0 should normally be encoded. Now let's switch to Windows console, which would be using Windows' decode-to-Unicode routine for cp950. 2>>> print b'\x87@' ? Let's try to identify this character: 3>>> u'?' '\uf266' So indeed there is some sort of HKSCS going on. But note what Windows has is really not any kind of new HKSCS: > Big5 ucs93 ucs00 ucs03 + 1-6 > 876B 9734 9734 9734 > 876C F292 F292 27BEF > 876D 5BDB 5BDB 5BDB 2>>> print b'\x87\x6b,\x87\x6c,\x87\x6d' ?,?,? 3>>> u'?,?,?' '\uf291,\uf292,\uf293' Just as for all other code pages, you can always find Microsoft's mapping at ftp://ftp.unicode.org/Public/MAPPINGS/VENDORS/MICSFT/WindowsBestFit/bestfit950.txt. If you are uncomfortable with adding a whole new table and wasting space (this is done for hkscs btw), use the algorithmic mapping at https://en.wikipedia.org/wiki/Code_page_950. ---------- components: Unicode messages: 280811 nosy: Artoria2e5, ezio.melotti, haypo priority: normal severity: normal status: open title: No HKSCS support in Windows cp950 type: behavior versions: Python 2.7, Python 3.3, Python 3.4, Python 3.5, Python 3.6, Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Nov 14 20:53:20 2016 From: report at bugs.python.org (Lance Ware) Date: Tue, 15 Nov 2016 01:53:20 +0000 Subject: [New-bugs-announce] [issue28694] tkinter interface to fontchooser Message-ID: <1479174800.7.0.200377827869.issue28694@psf.upfronthosting.co.za> New submission from Lance Ware: Tcl/Tk 8.6 now has a fontchooser command. I have developed a tkinter interface to it, similar to colorchooser/askcolor. How does one go about contributing this or proposing for future enhancement to the tkinter module suite? ---------- components: Tkinter messages: 280820 nosy: Lance Ware priority: normal severity: normal status: open title: tkinter interface to fontchooser type: enhancement versions: Python 3.5 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Nov 15 06:01:28 2016 From: report at bugs.python.org (Christian Heimes) Date: Tue, 15 Nov 2016 11:01:28 +0000 Subject: [New-bugs-announce] [issue28695] Add SSL_CTX_set_client_cert_engine Message-ID: <1479207688.81.0.066994858547.issue28695@psf.upfronthosting.co.za> New submission from Christian Heimes: Python's ssl module does not support smartcard authentication of clients. In order to use an external engine like OpenSC's engine_pkcs11, SSLContext must be configured to use a loaded engine for client cert auth. It's really simple. Pseudo code without error reporting, engine_id is a char*: ENGINE *e = ENGINE_by_id(engine_id); SSL_CTX_set_client_cert_engine(ctx, e); ---------- assignee: christian.heimes components: SSL messages: 280830 nosy: christian.heimes priority: normal severity: normal stage: needs patch status: open title: Add SSL_CTX_set_client_cert_engine type: behavior _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Nov 15 07:52:50 2016 From: report at bugs.python.org (Lev Veshnyakov) Date: Tue, 15 Nov 2016 12:52:50 +0000 Subject: [New-bugs-announce] [issue28696] imap from ThreadPool hangs by an exception in a generator function Message-ID: <1479214370.8.0.751710549169.issue28696@psf.upfronthosting.co.za> New submission from Lev Veshnyakov: It's only in imap, in map it's ok. The following code explains the issue: from multiprocessing.pool import ThreadPool pool = ThreadPool(10) def gen(): yield 1 + '1' # here is an error try: next((pool.imap(str, gen()))) except: # Will be catched using pool.map normally, but using pool.imap will be not. # Instead it hangs. This is the same for ThreadPool and Pool. print('this will not be printed because thread is hanging') ---------- components: Library (Lib) files: test.py messages: 280833 nosy: lev-veshnyakov priority: normal severity: normal status: open title: imap from ThreadPool hangs by an exception in a generator function type: behavior versions: Python 3.4, Python 3.5 Added file: http://bugs.python.org/file45486/test.py _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Nov 15 11:03:31 2016 From: report at bugs.python.org (Ulrich Petri) Date: Tue, 15 Nov 2016 16:03:31 +0000 Subject: [New-bugs-announce] [issue28697] asyncio.Lock, Condition, Semaphore docs don't mention `async with` syntax Message-ID: <1479225811.34.0.0222578393475.issue28697@psf.upfronthosting.co.za> New submission from Ulrich Petri: The docs for asyncio's Lock, Condition and Semaphore should use the new clean `async with lock:` syntax instead of the older (and IMO rather ugly) `with (yield from lock):` version. ---------- assignee: docs at python components: Documentation, asyncio messages: 280861 nosy: docs at python, gvanrossum, ulope, yselivanov priority: normal severity: normal status: open title: asyncio.Lock, Condition, Semaphore docs don't mention `async with` syntax versions: Python 3.5, Python 3.6, Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Nov 15 12:23:56 2016 From: report at bugs.python.org (Alex Wang) Date: Tue, 15 Nov 2016 17:23:56 +0000 Subject: [New-bugs-announce] [issue28698] Python 3.x ctypes c_wchar_p return is different from official documentation example Message-ID: <1479230636.43.0.863548306007.issue28698@psf.upfronthosting.co.za> New submission from Alex Wang: - Python Version Python 3.5.2 - Issue I found that the c_wchar_p and c_char_p return results behaves different from what it is based on ctypes doc. From the ctypes doc of Python 3.5: >>> c_wchar_p("Hello, World") c_wchar_p('Hello, World') It return the ctypes string. But my results of execute the same cmd in Python3.5 console: >>> from ctypes import * >>> c_wchar_p("Hello, World") c_wchar_p(1374004842736) >>> c_wchar_p("Hello, World") c_wchar_p(1374004841680) >>> c_wchar_p("Hello, World") c_wchar_p(1374004842736) So seems like the orignial string part replaced by memory address. Digged in more, and found out if it is Python 2.x, then the return shows the string like the Python 3.5 ctypes doc shows. But in Python 3.x, it always return these numbers. Checked on multiple PCs, all seen the same thing. And understood the part that, we can use .value to return the original string. Same thing observed on create_string_buffer() and create_unicode_buffer(). Meanwhile, for other data types like c_int(), c_bool, c_void_p etc., not see this. - Question Can anyone provide a explaination about this behavior of ctypes? And is there any way to fix the Python3.x return resuls as the same as what is doc wrote? (It seems that when it behave like this, I had issue with passing Python string to C function, when I interact with a C DLL.) - Repro This can be reproduce in python.org/shell easily. Thanks a lot in advance~ ---------- components: ctypes messages: 280869 nosy: Alex Wang priority: normal severity: normal status: open title: Python 3.x ctypes c_wchar_p return is different from official documentation example type: behavior versions: Python 3.5 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Nov 15 13:22:19 2016 From: report at bugs.python.org (Lev Veshnyakov) Date: Tue, 15 Nov 2016 18:22:19 +0000 Subject: [New-bugs-announce] [issue28699] Imap from ThreadPool behaves unexpectedly Message-ID: <1479234139.82.0.483433491582.issue28699@psf.upfronthosting.co.za> New submission from Lev Veshnyakov: Consider the following code: from multiprocessing.pool import ThreadPool pool = ThreadPool(10) def gen(): yield 1 + '1' # here is an error print(list(pool.imap(str, gen()))) # prints [] print(list(pool.map(str, gen()))) # raises TypeError The difference is, that the line with imap prints an empty list, while the line with map raises an exception, as expected. Change the above snippet of code, adding additional yield statement: from multiprocessing.pool import ThreadPool pool = ThreadPool(10) def gen(): yield 1 yield 1 + '1' # here is an error print(list(pool.imap(str, gen()))) # raises TypeError print(list(pool.map(str, gen()))) # also would raise TypeError So now both map and imap will raise the exception, as expected. Therefore I suppose the behavior of imap showed in the first case is wrong. ---------- components: Library (Lib) messages: 280872 nosy: lev-veshnyakov priority: normal severity: normal status: open title: Imap from ThreadPool behaves unexpectedly type: behavior versions: Python 3.5 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Nov 15 14:27:18 2016 From: report at bugs.python.org (Greg Ward) Date: Tue, 15 Nov 2016 19:27:18 +0000 Subject: [New-bugs-announce] [issue28700] test_dbm failure: KeyError: b'0' (regression in 3.6) Message-ID: <1479238038.8.0.290390662143.issue28700@psf.upfronthosting.co.za> New submission from Greg Ward: test_dbm.py fails reliably for me in 3.6, but in 3.5 it passes ~80% of the time. The failure in both cases is KeyError: b'0', which has come up previously in http://bugs.python.org/issue20094 and http://bugs.python.org/issue14120. But since we've switched from 20% failure rate to 100% failure rate, I figured something must have changed. I used "hg bisect" to track it down to a recent commit: changeset: 103360:0bd618fe0639 user: Victor Stinner date: Wed Sep 07 17:40:12 2016 -0700 summary: Implement compact dict Here is how it fails: $ ./python -m test -v test_dbm == CPython 3.6.0a4+ (default:0bd618fe0639, Nov 15 2016, 14:07:07) [GCC 5.4.0 20160609] == Linux-4.4.0-47-generic-x86_64-with-debian-stretch-sid little-endian == hash algorithm: siphash24 64bit == /home/data/src/cpython/3.6/build/test_python_10093 Testing with flags: sys.flags(debug=0, inspect=0, interactive=0, optimize=0, dont_write_bytecode=0, no_user_site=0, no_site=0, ignore_environment=0, verbose=0, bytes_warning=0, quiet=0, hash_randomization=1, isolated=0) Run tests sequentially 0:00:00 [1/1] test_dbm test_keys (test.test_dbm.WhichDBTestCase) ... ok test_whichdb (test.test_dbm.WhichDBTestCase) ... ok test_whichdb_ndbm (test.test_dbm.WhichDBTestCase) ... BDB0004 fop_read_meta: @test_10093_tmp_ndbm.db: unexpected file type or format ok test_anydbm_access (test.test_dbm.TestCase-dbm.ndbm) ... ok test_anydbm_creation (test.test_dbm.TestCase-dbm.ndbm) ... ERROR BDB3028 @test_10093_tmp.db: unable to flush: No such file or directory test_anydbm_creation_n_file_exists_with_invalid_contents (test.test_dbm.TestCase-dbm.ndbm) ... ok test_anydbm_keys (test.test_dbm.TestCase-dbm.ndbm) ... ok test_anydbm_modification (test.test_dbm.TestCase-dbm.ndbm) ... ERROR BDB3028 @test_10093_tmp.db: unable to flush: No such file or directory test_anydbm_not_existing (test.test_dbm.TestCase-dbm.ndbm) ... ok test_anydbm_read (test.test_dbm.TestCase-dbm.ndbm) ... ERROR test_error (test.test_dbm.TestCase-dbm.ndbm) ... ok test_anydbm_access (test.test_dbm.TestCase-dbm.dumb) ... ok test_anydbm_creation (test.test_dbm.TestCase-dbm.dumb) ... ok test_anydbm_creation_n_file_exists_with_invalid_contents (test.test_dbm.TestCase-dbm.dumb) ... ok test_anydbm_keys (test.test_dbm.TestCase-dbm.dumb) ... ok test_anydbm_modification (test.test_dbm.TestCase-dbm.dumb) ... ok test_anydbm_not_existing (test.test_dbm.TestCase-dbm.dumb) ... ok test_anydbm_read (test.test_dbm.TestCase-dbm.dumb) ... ok test_error (test.test_dbm.TestCase-dbm.dumb) ... ok ====================================================================== ERROR: test_anydbm_creation (test.test_dbm.TestCase-dbm.ndbm) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/data/src/cpython/3.6/Lib/test/test_dbm.py", line 73, in test_anydbm_creation self.read_helper(f) File "/home/data/src/cpython/3.6/Lib/test/test_dbm.py", line 114, in read_helper self.assertEqual(self._dict[key], f[key.encode("ascii")]) KeyError: b'0' ====================================================================== ERROR: test_anydbm_modification (test.test_dbm.TestCase-dbm.ndbm) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/data/src/cpython/3.6/Lib/test/test_dbm.py", line 88, in test_anydbm_modification self.read_helper(f) File "/home/data/src/cpython/3.6/Lib/test/test_dbm.py", line 114, in read_helper self.assertEqual(self._dict[key], f[key.encode("ascii")]) KeyError: b'0' ====================================================================== ERROR: test_anydbm_read (test.test_dbm.TestCase-dbm.ndbm) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/data/src/cpython/3.6/Lib/test/test_dbm.py", line 94, in test_anydbm_read self.read_helper(f) File "/home/data/src/cpython/3.6/Lib/test/test_dbm.py", line 114, in read_helper self.assertEqual(self._dict[key], f[key.encode("ascii")]) KeyError: b'0' ---------------------------------------------------------------------- Ran 19 tests in 0.052s FAILED (errors=3) test test_dbm failed test_dbm failed 1 test failed: test_dbm Total duration: 77 ms Tests result: FAILURE ---------- components: Tests messages: 280877 nosy: gward priority: normal severity: normal status: open title: test_dbm failure: KeyError: b'0' (regression in 3.6) type: behavior versions: Python 3.6 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Nov 15 14:50:05 2016 From: report at bugs.python.org (Serhiy Storchaka) Date: Tue, 15 Nov 2016 19:50:05 +0000 Subject: [New-bugs-announce] [issue28701] Replace PyUnicode_CompareWithASCIIString with _PyUnicode_EqualToASCIIString Message-ID: <1479239405.55.0.31899621759.issue28701@psf.upfronthosting.co.za> New submission from Serhiy Storchaka: Proposed patch replaces calls of public function PyUnicode_CompareWithASCIIString() with new private function _PyUnicode_EqualToASCIIString(). The problem with PyUnicode_CompareWithASCIIString() is that it returns -1 for the result "less than" and error, but the error case is never checked. The patch is purposed for following purposes: 1. Readability. ``_PyUnicode_EqualToASCIIString(...)`` looks more readable than ``PyUnicode_CompareWithASCIIString(...) == 0`` or ``!PyUnicode_CompareWithASCIIString(...)``, especially in large expression. I always have to make an effort to understand correctly the meaning of the latter expression. 2. Efficiency. If the strings are not equal, _PyUnicode_EqualToASCIIString() can quickly return false, but PyUnicode_CompareWithASCIIString() needs to check whether what string is larger. 3. Correctness. Since no caller checks the error of PyUnicode_CompareWithASCIIString(), it is incorrectly interpreted as "less then". Exception set by PyUnicode_CompareWithASCIIString() can be leaked in following code causing mystical error or crash in debug build. There are too many callers to add error checking for them all. These would be non-trivial error-prone changes that add new lines of the code, new variables and new returns or gotos. On other hand replacing PyUnicode_CompareWithASCIIString() with _PyUnicode_EqualToASCIIString() is done by simple script. _PyUnicode_EqualToASCIIString() returns true value (1) if strings are equal, false value (0) if they are different, and doesn't raise exceptions. Unlike to PyUnicode_CompareWithASCIIString() it works only with ASCII characters and returns false if any string contains non-ASCII characters. The patch also documents the return value of PyUnicode_CompareWithASCIIString() in case of error. See issue21449 for similar issue with _PyUnicode_CompareWithId(). ---------- components: Interpreter Core, Unicode files: _PyUnicode_EqualToASCIIString.patch keywords: patch messages: 280882 nosy: ezio.melotti, haypo, josh.r, serhiy.storchaka, xiang.zhang priority: normal severity: normal stage: patch review status: open title: Replace PyUnicode_CompareWithASCIIString with _PyUnicode_EqualToASCIIString type: behavior versions: Python 2.7, Python 3.5, Python 3.6, Python 3.7 Added file: http://bugs.python.org/file45490/_PyUnicode_EqualToASCIIString.patch _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Nov 15 15:04:43 2016 From: report at bugs.python.org (Greg Ward) Date: Tue, 15 Nov 2016 20:04:43 +0000 Subject: [New-bugs-announce] [issue28702] Confusing error message when None used in expressions, eg. "'NoneType' object has no attribute 'foo'" Message-ID: <1479240283.12.0.750407106609.issue28702@psf.upfronthosting.co.za> New submission from Greg Ward: Python's error message when you let None accidentally sneak into an expression where it doesn't belong could be better. The canonical example is attribute lookup: >>> a = None >>> a.foo Traceback (most recent call last): File "", line 1, in AttributeError: 'NoneType' object has no attribute 'foo' This assumes that the programmer knows there is only one object of type NoneType, and it is None. That's a lot to assume of a beginner, whether they are coming from another programming language ("null has a type? that's crazy talk!") or are completely new to programming ("null? none? nil? wat...??"). There are plenty of other places this use of NoneType in error messages comes up: >>> a + 1 Traceback (most recent call last): File "", line 1, in TypeError: unsupported operand type(s) for +: 'NoneType' and 'int' >>> 1 + a Traceback (most recent call last): File "", line 1, in TypeError: unsupported operand type(s) for +: 'int' and 'NoneType' >>> len(a) Traceback (most recent call last): File "", line 1, in TypeError: object of type 'NoneType' has no len() >>> a < 1 Traceback (most recent call last): File "", line 1, in TypeError: '<' not supported between instances of 'NoneType' and 'int' I think we can do better than this. For example, here is an proposed improvement to user experience for getting and setting attributes on None: >>> a.foo Traceback (most recent call last): File "", line 1, in AttributeError: attempt to access attribute 'foo' of None, but None has no attributes >>> a.foo = 42 Traceback (most recent call last): File "", line 1, in AttributeError: attempt to set attribute 'foo' on None, but None is read-only Let the bikeshedding commence. I have a working patch, but need to write tests. Will upload it here when that is done. ---------- assignee: gward components: Interpreter Core messages: 280884 nosy: gward priority: normal severity: normal status: open title: Confusing error message when None used in expressions, eg. "'NoneType' object has no attribute 'foo'" type: enhancement versions: Python 3.6 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Nov 15 15:18:49 2016 From: report at bugs.python.org (Yury Selivanov) Date: Tue, 15 Nov 2016 20:18:49 +0000 Subject: [New-bugs-announce] [issue28703] Fix asyncio.iscoroutinefunction to handle Mock objects Message-ID: <1479241129.21.0.885185313052.issue28703@psf.upfronthosting.co.za> New submission from Yury Selivanov: Proxy for https://github.com/python/asyncio/pull/459 ---------- assignee: yselivanov components: asyncio messages: 280886 nosy: gvanrossum, yselivanov priority: normal severity: normal stage: resolved status: open title: Fix asyncio.iscoroutinefunction to handle Mock objects type: behavior versions: Python 3.5, Python 3.6, Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Nov 15 15:25:21 2016 From: report at bugs.python.org (Yury Selivanov) Date: Tue, 15 Nov 2016 20:25:21 +0000 Subject: [New-bugs-announce] [issue28704] Fix create_unix_server to support Path-like objects (PEP 519) Message-ID: <1479241521.58.0.821851179075.issue28704@psf.upfronthosting.co.za> Changes by Yury Selivanov : ---------- assignee: yselivanov components: asyncio nosy: gvanrossum, yselivanov priority: normal severity: normal stage: resolved status: open title: Fix create_unix_server to support Path-like objects (PEP 519) type: behavior versions: Python 3.6, Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Nov 15 15:40:24 2016 From: report at bugs.python.org (Brett Cannon) Date: Tue, 15 Nov 2016 20:40:24 +0000 Subject: [New-bugs-announce] [issue28705] Clean up design FAQ question about compiling to C Message-ID: <1479242424.69.0.486701583989.issue28705@psf.upfronthosting.co.za> New submission from Brett Cannon: E.g. drop reference to Pyrex and Weave (especially since the latter has a stale link to https://scipy.github.io/devdocs/tutorial/weave.html and is Python 2.7-only and not recommended for new code). ---------- assignee: brett.cannon components: Documentation messages: 280891 nosy: brett.cannon priority: normal severity: normal status: open title: Clean up design FAQ question about compiling to C _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Nov 15 16:08:30 2016 From: report at bugs.python.org (=?utf-8?b?SmnFmcOtIEhvZm1hbg==?=) Date: Tue, 15 Nov 2016 21:08:30 +0000 Subject: [New-bugs-announce] [issue28706] msvc9compiler does not find a vcvarsall.bat of Visual C++ for Python 9.0 Message-ID: <1479244110.83.0.177633464347.issue28706@psf.upfronthosting.co.za> New submission from Ji?? Hofman: When running "c:\Program Files\Python27\Scripts\pip2.7.exe" install wx following error occurs: Complete output from command "c:\program files\python27\python.exe" -u -c "import setuptools, tokenize;__file__='c:\\users\\jiri\\appdata\\local\\temp\\pip-build-lqokio\\wx\\setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record c:\users\jiri\appdata\local\temp\pip-51m45m-record\install-record.txt --single-version-externally-managed --compile: running install running build WARNING: Building this way assumes that all generated files have been generated already. If that is not the case then use build.py directly to generate the source and perform the build stage. You can use --skip-build with the bdist_* or install commands to avoid this message and the wxWidgets and Phoenix build steps in the future. "c:\program files\python27\python.exe" -u build.py build Will build using: "c:\program files\python27\python.exe" 2.7.12 (v2.7.12:d33e0cf91556, Jun 27 2016, 15:24:40) [MSC v.1500 64 bit (AMD64)] Python's architecture is 64bit cfg.VERSION: 3.0.3 Running command: build Running command: build_wx Command '"c:\program files\python27\python.exe" -c "import distutils.msvc9compiler as msvc; mc = msvc.MSVCCompiler(); mc.initialize(); print(mc.cc)"' failed with exit code 1. Traceback (most recent call last): File "", line 1, in File "c:\program files\python27\lib\distutils\msvc9compiler.py", line 385, in initialize vc_env = query_vcvarsall(VERSION, plat_spec) File "c:\program files\python27\lib\distutils\msvc9compiler.py", line 273, in query_vcvarsall raise DistutilsPlatformError("Unable to find vcvarsall.bat") distutils.errors.DistutilsPlatformError: Unable to find vcvarsall.bat Finished command: build_wx (0.138s) Finished command: build (0.139s) Command '"c:\program files\python27\python.exe" -u build.py build' failed with exit code 1. I found out that the problem is in msvc9compiler. The dir where vcvarsall.bat is installed is not found. It is installed (by default) in C:\Users\Jiri\AppData\Local\Programs\Common\Microsoft\Visual C++ for Python\9.0 Needed registry keys do not exist after installation of the Visual C++ for Python 9.0. My OS is Windows 10 Home (64bit). ---------- components: Distutils messages: 280894 nosy: dstufft, eric.araujo, jyrkih priority: normal severity: normal status: open title: msvc9compiler does not find a vcvarsall.bat of Visual C++ for Python 9.0 versions: Python 2.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Nov 15 17:31:49 2016 From: report at bugs.python.org (=?utf-8?q?St=C3=A9phane_Wirtel?=) Date: Tue, 15 Nov 2016 22:31:49 +0000 Subject: [New-bugs-announce] [issue28707] add 'directory' option to the http.server module Message-ID: <1479249109.08.0.530385659795.issue28707@psf.upfronthosting.co.za> New submission from St?phane Wirtel: When we execute the http.server module, the tool will use the current directory (os.getcwd()) but sometimes we would like to specify a directory on the command line. With the next patch, I try to fix this missing feature ;-) Just with python -m http.server -d /tmp by default the system will use the current directory. if necessary, I will show an error if the directory does not exist. ---------- files: chdir-httpserver.diff keywords: patch messages: 280898 nosy: matrixise priority: normal severity: normal status: open title: add 'directory' option to the http.server module versions: Python 3.6, Python 3.7 Added file: http://bugs.python.org/file45493/chdir-httpserver.diff _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Nov 15 17:52:38 2016 From: report at bugs.python.org (David Hirschfeld) Date: Tue, 15 Nov 2016 22:52:38 +0000 Subject: [New-bugs-announce] [issue28708] Low FD_SETSIZE limit on Windows Message-ID: <1479250358.07.0.768722440644.issue28708@psf.upfronthosting.co.za> New submission from David Hirschfeld: Back in 1999 the compile-time constant FD_SETSIZE was raised from (the default on Windows) 64 to 512 open sockets to support *serious async servers* http://bugs.python.org/issue210843 https://github.com/python/cpython/blame/master/Modules/selectmodule.c#L29-L36 The world has moved on and serious async servers require far more than 512 sockets available. This is one of the key reasons why tornado explicitly states: > Tornado will also run on Windows, although this configuration is not officially supported and is recommended only for development use. Yes, using select on Windows is the wrong thing to do, but it's far preferable to be able to use a library which makes use of select, putting up with the poor performance than it is to be told to use linux which often isn't an option. Since there's no alternative other than recompiling Python it would be good if this constant could be increased to a more useful (these days) value of say 16384. As a data point ZMQ have recently increased the value of this constant to 16k themselves https://github.com/zeromq/libzmq/pull/2035 ---------- messages: 280900 nosy: David Hirschfeld priority: normal severity: normal status: open title: Low FD_SETSIZE limit on Windows type: resource usage versions: Python 3.6, Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Nov 15 19:00:14 2016 From: report at bugs.python.org (Josh Rosenberg) Date: Wed, 16 Nov 2016 00:00:14 +0000 Subject: [New-bugs-announce] [issue28709] PyStructSequence_NewType is broken Message-ID: <1479254414.47.0.784230878249.issue28709@psf.upfronthosting.co.za> New submission from Josh Rosenberg: I could be missing something, but it looks like PyStructSequence_NewType is guaranteed broken. Specifically, it allocates the memory with PyType_GenericAlloc (use PyType_Type as the base), and PyType declares itself to have garbage collected instances, so the memory is allocated with _PyObject_GC_Malloc and added to GC tracking just before PyType_GenericAlloc returns. Problem is, PyStructSequence_Init2 copies from a template struct which sets Py_TPFLAGS_DEFAULT. So even though the new struct sequence is GC allocated and managed, it doesn't set Py_TPFLAGS_HEAPTYPE, which means when GC tries to traverse it, type's type_traverse errors out with: Fatal Python error: type_traverse() called for non-heap type 'NameOfStructSequence' It's possible I'm missing something here, so I've attached simple test code for others to confirm (it omits most error checking for simplicity/readability). Just compile the extension module, then run (with the module in the working directory): python -c "import testnewtype; Foo = testnewtype.makeseq('Foo', ['x', 'y'])" There is a commented out line in the test code that explicitly sets the HEAPTYPE flag after type construction (no idea if that's supposed to be supported), and uncommenting it seems to fix the crash (though again, if retroactively flagging as HEAPTYPE is unsupported, something else may break here). I can't find any actual use of PyStructSequence_NewType in the CPython code base, which probably explains why this hasn't been seen; odds are, most extensions using struct sequences are using Init, not NewType, and I only ran into this because I was experimenting with a struct sequence based replacement for collections.namedtuple (to end the start up time objections to using namedtuple in the built-in modules, e.g. #28638). ---------- components: Interpreter Core files: testnewtype.c messages: 280904 nosy: josh.r priority: normal severity: normal status: open title: PyStructSequence_NewType is broken versions: Python 3.4, Python 3.5, Python 3.6, Python 3.7 Added file: http://bugs.python.org/file45495/testnewtype.c _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Nov 15 19:59:42 2016 From: report at bugs.python.org (Patrick Lehmann) Date: Wed, 16 Nov 2016 00:59:42 +0000 Subject: [New-bugs-announce] [issue28710] Sphinx incompatible markup in configparser.ConfigParser. Message-ID: <1479257982.94.0.253343336813.issue28710@psf.upfronthosting.co.za> New submission from Patrick Lehmann: Why does e.g. configparser.ConfigParser contain doc strings with Sphinx incompatible markup? The markup starts with back-tick, but ends with a single quote. Example: https://github.com/python/cpython/blob/master/Lib/configparser.py?ts=2#L26 Sphinx writes these messages: D:\git\PoC\py\lib\ExtendedConfigParser\__init__.py:docstring of lib.ExtendedConfigParser.ExtendedConfigParser.read_file:3: WARNING: Inline interpreted text or phrase reference start-str ing without end-string. Note: ExtendedConfigParser is class derived from configparser.ConfigParser. Inherited methods get documented too. ------------------------ Btw. I have some improvements for this class, how can I contribute them? Who is the maintainer for this class? Please contact me: Patrick.Lehmann at tu-dresden.de The improved version is available at GitHub: https://github.com/Paebbels/ExtendedConfigParser?ts=2 ---------- assignee: docs at python components: Documentation messages: 280907 nosy: Patrick Lehmann, docs at python priority: normal severity: normal status: open title: Sphinx incompatible markup in configparser.ConfigParser. versions: Python 3.4, Python 3.5 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Nov 15 23:25:25 2016 From: report at bugs.python.org (Bryan) Date: Wed, 16 Nov 2016 04:25:25 +0000 Subject: [New-bugs-announce] [issue28711] IDLE doesn't open Message-ID: <1479270325.09.0.272646595028.issue28711@psf.upfronthosting.co.za> New submission from Bryan: Hello there I am using python 2.7 on windows 10 because my college class requires it, I am having issues when trying to open the IDLE. When i click on it the blue ring loads and then noting happens. I started to have to issue when i was changing the key settings so i can right- click or at least trying to figure out how to do it. If anyone can help it will greatly appreciated. ---------- messages: 280910 nosy: bg90299 priority: normal severity: normal status: open title: IDLE doesn't open type: behavior versions: Python 2.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed Nov 16 00:40:28 2016 From: report at bugs.python.org (Mingye Wang) Date: Wed, 16 Nov 2016 05:40:28 +0000 Subject: [New-bugs-announce] [issue28712] Non-Windows mappings for a couple of Windows code pages Message-ID: <1479274828.26.0.726447169651.issue28712@psf.upfronthosting.co.za> New submission from Mingye Wang: Mappings for 0x81 and 0x8D in multiple Windows code pages diverge from what Windows does. Attached is a script that tests for this behavior. (These two bytes are not necessary the only problems, but for sure they are the most widespread and famous ones. Again, refer to Unicode best fit for something that works.) This problem is seen in Python 2.7.10 on Windows 10b14959, but apparently it is known since long ago[1]. Python 3.4.3 on Cygwin also fails ``b'\x81\x8d'.encode('cp1252')``. [1]: https://ftfy.readthedocs.io/en/latest/#module-ftfy.bad_codecs.sloppy ---------- components: Unicode files: pycp.py messages: 280914 nosy: Artoria2e5, ezio.melotti, haypo priority: normal severity: normal status: open title: Non-Windows mappings for a couple of Windows code pages type: behavior versions: Python 2.7, Python 3.3, Python 3.4, Python 3.5, Python 3.6, Python 3.7 Added file: http://bugs.python.org/file45497/pycp.py _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed Nov 16 02:37:22 2016 From: report at bugs.python.org (Daisuke Miyakawa) Date: Wed, 16 Nov 2016 07:37:22 +0000 Subject: [New-bugs-announce] [issue28713] Recent tutorial for recent Python3 still uses IOError. Message-ID: <1479281842.16.0.362729785805.issue28713@psf.upfronthosting.co.za> New submission from Daisuke Miyakawa: https://docs.python.org/3.5/tutorial/errors.html for arg in sys.argv[1:]: try: f = open(arg, 'r') except IOError: print('cannot open', arg) else: print(arg, 'has', len(f.readlines()), 'lines') f.close() Although IOError is still available as an alias to OSError, it should not be used in the tutorial, I believe. ---------- assignee: docs at python components: Documentation messages: 280924 nosy: dmiyakawa, docs at python priority: normal severity: normal status: open title: Recent tutorial for recent Python3 still uses IOError. versions: Python 3.5 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed Nov 16 10:00:35 2016 From: report at bugs.python.org (George Fischhof) Date: Wed, 16 Nov 2016 15:00:35 +0000 Subject: [New-bugs-announce] [issue28714] Addition to Documentation of configparser.ConfigParser.write() documentaion Message-ID: <1479308435.4.0.112292626323.issue28714@psf.upfronthosting.co.za> New submission from George Fischhof: Hi There, I used configparser.ConfigParser.write() to update a config file. And I found that my config wa duplicated. The file was opened with mode 'r+' I figured out that write (I mean the write method of configparser) writes at actual file position. I issued a file.seek(0) command before write and the result was good. So I think documentaion should advice to user to reopen the file with mode 'w' or to issue a file.seek(0) command before using the ConfigParser.write() I used the following python version on windows: Python 3.5.1 (v3.5.1:37a07cee5969, Dec 6 2015, 01:38:48) [MSC v.1900 32 bit (Intel)] on win32 affected documentation: https://docs.python.org/3.5/library/configparser.html#configparser.ConfigParser.write Best Regards, George Fischhof ---------- assignee: docs at python components: Documentation messages: 280960 nosy: docs at python, georgefischhof priority: normal severity: normal status: open title: Addition to Documentation of configparser.ConfigParser.write() documentaion versions: Python 3.5 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed Nov 16 12:46:27 2016 From: report at bugs.python.org (Serhiy Storchaka) Date: Wed, 16 Nov 2016 17:46:27 +0000 Subject: [New-bugs-announce] [issue28715] Check result of PyUnicode_AsUTF8 Message-ID: <1479318387.76.0.271221021143.issue28715@psf.upfronthosting.co.za> New submission from Serhiy Storchaka: The function PyUnicode_AsUTF8() can fail, but not all callers check for error. Proposed patch adds missed checks. In some places the code is rewritten to not use PyUnicode_AsUTF8() at all. ---------- components: Extension Modules, Interpreter Core files: check-PyUnicode_AsUTF8.patch keywords: patch messages: 280972 nosy: serhiy.storchaka priority: normal severity: normal stage: patch review status: open title: Check result of PyUnicode_AsUTF8 type: crash versions: Python 3.5, Python 3.6, Python 3.7 Added file: http://bugs.python.org/file45506/check-PyUnicode_AsUTF8.patch _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed Nov 16 13:06:27 2016 From: report at bugs.python.org (Wolfgang Maier) Date: Wed, 16 Nov 2016 18:06:27 +0000 Subject: [New-bugs-announce] [issue28716] Fractions instantiation revisited Message-ID: <1479319587.68.0.845103759053.issue28716@psf.upfronthosting.co.za> New submission from Wolfgang Maier: I've spent a bit of time lately trying to optimize the instantiation of Fractions. This is related to Issue22464, but instead of focusing on constructing Fractions from ints, my attempts revolve around improving instantiation from strings, floats and decimals. I'm attaching a patch with all my changes for discussion, but here's an overview of what's in it: CHANGES TO INSTANTIATION FROM STRINGS: - isinstance checking for str is performed before the more expensive check for numbers.Rational (which has to go through abc) - instantiation from strings doesn't use a regex pattern anymore for parsing; this is faster in many cases (and never slower) than the current version - while reimplementing string parsing I added PEP 515 support (this is the only behavior change in the patch) combined this gives me the following performance changes for instantiation of Fractions from different arguments (profiled with 1,000,000 random inputs each): 'x/y'-type of strings: ---------------------- ncalls tottime percall cumtime percall filename:lineno(function) old: 1000000 12.162 0.000 27.778 0.000 fractions.py:84(__new__) new: 1000000 9.619 0.000 12.428 0.000 frc.py:68(__new__) scientific notation (e.g., '1.234E-7'): --------------------------------------- ncalls tottime percall cumtime percall filename:lineno(function) old: 1000000 18.209 0.000 37.341 0.000 fractions.py:84(__new__) new: 1000000 15.509 0.000 21.252 0.000 frc.py:68(__new__) integer strings (e.g. '123456'): -------------------------------- ncalls tottime percall cumtime percall filename:lineno(function) old: 1000000 11.272 0.000 26.201 0.000 fractions.py:84(__new__) new: 1000000 9.347 0.000 12.425 0.000 frc.py:68(__new__) from other Fractions (to test effect on instantiation of numbers.Rational): ncalls tottime percall cumtime percall filename:lineno(function) old: 1000000 4.708 0.000 10.093 0.000 fractions.py:84(__new__) new: 1000000 4.835 0.000 10.572 0.000 frc.py:68(__new__) from int subclass (as another numbers.Rational): ncalls tottime percall cumtime percall filename:lineno(function) old: 1000000 3.446 0.000 8.044 0.000 fractions.py:84(__new__) new: 1000000 3.795 0.000 8.836 0.000 frc.py:68(__new__) SUMMARY of this part ==================== + very significant speedup for instatiation from strings of different forms with (near) negligible effects on instantiation from numbers.Rational. + correct parsing of PEP 515-like number strings - possibly slower error bubbling with invalid input (untested) CHANGES TO INSTANTIATION FROM FLOAT AND DECIMAL: - no explicit check for float and decimal in standard constructor; instead, simply try to call as_integer_ratio as last resort (even after checking for numbers.Rational) - speed up alternate from_float and from_decimal constructor class methods by rearranging type checks and making use of _normalize flag - import decimal only on demand (when using from_decimal) Resulting performance changes: standard constructor with float: -------------------------------- ncalls tottime percall cumtime percall filename:lineno(function) old: 1000000 4.362 0.000 12.452 0.000 fractions.py:84(__new__) new: 1000000 6.693 0.000 16.020 0.000 frc.py:68(__new__) from_float: ----------- ncalls tottime percall cumtime percall filename:lineno(function) old: 1000000 3.146 0.000 17.769 0.000 fractions.py:193(from_float) new: 1000000 2.544 0.000 7.591 0.000 frc.py:205(from_float) standard constructor with decimal: -------------------------------- ncalls tottime percall cumtime percall filename:lineno(function) old: 1000000 4.672 0.000 20.795 0.000 fractions.py:84(__new__) new: 1000000 7.097 0.000 24.526 0.000 frc.py:68(__new__) from_decimal: ------------- ncalls tottime percall cumtime percall filename:lineno(function) old: 1000000 5.054 0.000 34.473 0.000 fractions.py:207(from_decimal) new: 1000000 2.942 0.000 16.013 0.000 frc.py:220(from_decimal) SUMMARY of this part: - standard constructor becomes a bit slower for floats and Decimals specialized class methods become a lot faster (for Decimal the benchmarks are based on _pydecimal - with the C implementation the percent differences would be even larger) - eliminates decimal from regularly imported modules - allows Fraction instantiation from duck-typing classes that provide as_integer_ratio I hope at least some of this is interesting. ---------- components: Library (Lib) files: fractions.patch keywords: patch messages: 280977 nosy: wolma priority: normal severity: normal status: open title: Fractions instantiation revisited type: behavior versions: Python 3.7 Added file: http://bugs.python.org/file45507/fractions.patch _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed Nov 16 15:10:59 2016 From: report at bugs.python.org (William McIlhagga) Date: Wed, 16 Nov 2016 20:10:59 +0000 Subject: [New-bugs-announce] [issue28717] rounding error in % operator Message-ID: <1479327059.34.0.959568530383.issue28717@psf.upfronthosting.co.za> New submission from William McIlhagga: '%.1f' % 0.25 yields the string '0.2' '%.1f' % 0.75 yields the string '0.8' This is wrong. ---------- messages: 280984 nosy: William McIlhagga priority: normal severity: normal status: open title: rounding error in % operator type: behavior versions: Python 2.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed Nov 16 15:12:07 2016 From: report at bugs.python.org (Jim Nasby) Date: Wed, 16 Nov 2016 20:12:07 +0000 Subject: [New-bugs-announce] [issue28718] '*' matches entire path in fnmatch.translate Message-ID: <1479327127.15.0.931047394142.issue28718@psf.upfronthosting.co.za> New submission from Jim Nasby: A '*' in fnmatch.translate is converted into '.*', which will greedily match directory separators. This doesn't match shell behavior, which is that * will only match file names: decibel at decina:[14:07]~$ls ~/tmp/*/1|head ls: /Users/decibel/tmp/*/1: No such file or directory decibel at decina:[14:07]~$ls ~/tmp/d*/base/1|head 112 >From a posix standpoint, this would easily be fixed by using '[^/]*' instead of '.*'. I'm not sure how to make this work cross-platform though. It's worth noting that some programs (rsync, git) support **, which would correctly translate to '.*'. ---------- components: Library (Lib) messages: 280985 nosy: Jim Nasby priority: normal severity: normal status: open title: '*' matches entire path in fnmatch.translate _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed Nov 16 16:16:06 2016 From: report at bugs.python.org (Bert JW Regeer) Date: Wed, 16 Nov 2016 21:16:06 +0000 Subject: [New-bugs-announce] [issue28719] zipfile increase in size Message-ID: <1479330966.86.0.764348227568.issue28719@psf.upfronthosting.co.za> New submission from Bert JW Regeer: I am the current maintainer of WebOb, and noticed that on Python 3.6 and 3.7 I noticed that a test started failing. Granted, the test is checking the size of the file created and it is not the brightest idea in a test, but it's been stable since Python 2.5... https://travis-ci.org/Pylons/webob/jobs/176505096#L224 shows the failure. _________________________ test_response_file_body_tell _________________________ def test_response_file_body_tell(): import zipfile from webob.response import ResponseBodyFile rbo = ResponseBodyFile(Response()) assert rbo.tell() == 0 writer = zipfile.ZipFile(rbo, 'w') writer.writestr('zinfo_or_arcname', b'foo') writer.close() > assert rbo.tell() == 133 E assert 145 == 133 E + where 145 = >>() E + where >> = >.tell tests/test_response.py:608: AssertionError I am not sure that this is necessarily a bug, but it would be good to know why files are no longer generated the same way. ---------- messages: 280992 nosy: X-Istence priority: normal severity: normal status: open title: zipfile increase in size type: resource usage versions: Python 3.6, Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed Nov 16 16:30:05 2016 From: report at bugs.python.org (Yury Selivanov) Date: Wed, 16 Nov 2016 21:30:05 +0000 Subject: [New-bugs-announce] [issue28720] Add collections.abc.AsyncGenerator Message-ID: <1479331805.85.0.746323979181.issue28720@psf.upfronthosting.co.za> New submission from Yury Selivanov: This patch adds collections.abc.AsyncGenerator (closely modelled after collections.abc.Generator). Ned, is it OK if this goes into 3.6? This is something I completely forgot to do as part of PEP 525. This ABC is needed for asynchronous generators compiled with Cython and async-generator-like objects/wrappers. I'll update the PEP if we can push this to 3.6. ---------- assignee: yselivanov components: Library (Lib) files: agen_abc.patch keywords: patch messages: 280993 nosy: gvanrossum, ned.deily, yselivanov priority: release blocker severity: normal stage: patch review status: open title: Add collections.abc.AsyncGenerator type: enhancement versions: Python 3.6, Python 3.7 Added file: http://bugs.python.org/file45512/agen_abc.patch _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed Nov 16 18:12:03 2016 From: report at bugs.python.org (Yury Selivanov) Date: Wed, 16 Nov 2016 23:12:03 +0000 Subject: [New-bugs-announce] [issue28721] Fix asynchronous generators athrow() and aclose() to handle StopAsyncIteration Message-ID: <1479337923.66.0.886081078513.issue28721@psf.upfronthosting.co.za> New submission from Yury Selivanov: 1. aclose() should not propagate StopAsyncIteration: async def agen(): try: yield except: pass gen = agen() await agen.asend(None) await agen.aclose() # <- this shouldn't raise StopAsyncIteration 2. athrow() should propagate StopAsyncIteration when the exception was swallowed by the generator and it returned: async def agen(): try: yield except: pass gen = agen() await agen.asend(None) await agen.athrow(ValueError) # <- this should raise StopAsyncIteration ---------- assignee: yselivanov components: Interpreter Core files: agen.patch keywords: patch messages: 281008 nosy: ned.deily, yselivanov priority: release blocker severity: normal stage: commit review status: open title: Fix asynchronous generators athrow() and aclose() to handle StopAsyncIteration type: behavior versions: Python 3.6, Python 3.7 Added file: http://bugs.python.org/file45514/agen.patch _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed Nov 16 20:19:40 2016 From: report at bugs.python.org (Rajiv Bakulesh Shah) Date: Thu, 17 Nov 2016 01:19:40 +0000 Subject: [New-bugs-announce] [issue28722] doctest example exit status Message-ID: <1479345580.78.0.277825169146.issue28722@psf.upfronthosting.co.za> New submission from Rajiv Bakulesh Shah: It might be nice if the doctest example set the appropriate exit status. Apologies if this is beyond the scope of the example, but I thought it might be good practice. Here is the file: https://github.com/python/cpython/blob/master/Doc/library/doctest.rst Here is the example as written: if __name__ == "__main__": import doctest doctest.testmod() Here is my proposal: if __name__ == '__main__': import doctest import sys results = doctest.testmod() sys.exit(bool(results.failed)) I'm happy to fork the repo and submit a PR, if that makes things easier. I'm not familiar with the protocol here. Thanks for the great work! ---------- assignee: docs at python components: Documentation messages: 281015 nosy: Brainix, docs at python priority: normal severity: normal status: open title: doctest example exit status versions: Python 3.6 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed Nov 16 23:14:18 2016 From: report at bugs.python.org (Simon Holland) Date: Thu, 17 Nov 2016 04:14:18 +0000 Subject: [New-bugs-announce] [issue28723] tkinter, radiobutton, indicatoron=0 has no effect. Message-ID: <1479356058.22.0.131810563362.issue28723@psf.upfronthosting.co.za> New submission from Simon Holland: tkinters radiobutton's have an option 'indicatoron=0' which should display Radio Buttons as actual labelled buttons. button = tk.Radiobutton(self, text=option, variable = var, value = answer, indicatoron=0) Screenshots of expected and actual results ... http://imgur.com/a/2fI02 taken from ... http://stackoverflow.com/questions/34459221/tkinter-radiobutton-indicatoron-value-doesnt-effect-anything Kkinter 8.5 descrbes the functionality as so : Normally a checkbutton displays as its indicator a box that shows whether the checkbutton is set or not. You can get this behavior by setting indic- atoron=1. However, if you set indicatoron=0, the indicator disappears, and the entire widget becomes a push-push button that looks raised when it is cleared and sunken when it is set. You may want to increase the bor- derwidth value to make it easier to see the state of such a control. ---------- components: Tkinter files: u1mdKuC.png messages: 281022 nosy: Inyoka priority: normal severity: normal status: open title: tkinter, radiobutton, indicatoron=0 has no effect. type: behavior versions: Python 3.4, Python 3.5 Added file: http://bugs.python.org/file45517/u1mdKuC.png _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu Nov 17 10:05:33 2016 From: report at bugs.python.org (Shinya Okano) Date: Thu, 17 Nov 2016 15:05:33 +0000 Subject: [New-bugs-announce] [issue28724] Add method send_io, recv_io to the socket module. Message-ID: <1479395133.65.0.0720889900419.issue28724@psf.upfronthosting.co.za> New submission from Shinya Okano: This patch makes it easy to pass file descriptor in using AF_UNIX. Ruby language libraries have such methods. see slso: - https://docs.ruby-lang.org/en/2.3.0/UNIXSocket.html#method-i-send_io ---------- components: Library (Lib) files: socket_send_io_recv_io.patch keywords: patch messages: 281041 nosy: tokibito priority: normal severity: normal status: open title: Add method send_io, recv_io to the socket module. type: enhancement versions: Python 3.7 Added file: http://bugs.python.org/file45519/socket_send_io_recv_io.patch _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu Nov 17 10:43:29 2016 From: report at bugs.python.org (Gavin Panella) Date: Thu, 17 Nov 2016 15:43:29 +0000 Subject: [New-bugs-announce] [issue28725] Awaiting an awaitable returned from an async function does nothing Message-ID: <1479397409.08.0.1550595702.issue28725@psf.upfronthosting.co.za> New submission from Gavin Panella: The following will sleep: async def one(): await asyncio.sleep(10) async def two(): await one() loop.run_until_complete(two()) but the following will not: async def one(): return asyncio.sleep(10) async def two(): await one() loop.run_until_complete(two()) I would expect run_until_complete to keep bouncing awaitable results back into the event-loop until a non-awaitable is returned. In my code I work around this with: result = loop.run_until_complete(...) while inspect.isawaitable(result): result = loop.run_until_complete(result) I would also expect that the await in `two` would have DTRT with the returned generator/coroutine. ---------- components: asyncio messages: 281043 nosy: allenap, gvanrossum, yselivanov priority: normal severity: normal status: open title: Awaiting an awaitable returned from an async function does nothing type: behavior versions: Python 3.5 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu Nov 17 10:56:15 2016 From: report at bugs.python.org (Pravic Ehysta) Date: Thu, 17 Nov 2016 15:56:15 +0000 Subject: [New-bugs-announce] [issue28726] py.exe launches wrong version Message-ID: <1479398175.28.0.0714056726862.issue28726@psf.upfronthosting.co.za> New submission from Pravic Ehysta: d:\>py -3.5 Requested Python version (3.5) not installed d:\>py -3.3 Requested Python version (3.3) not installed d:\>py -3.4 Python 3.5.2 (v3.5.2:4def2a2901a5, Jun 25 2016, 22:18:55) [MSC v.1900 64 bit (AMD64)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> ^Z ---------- components: Windows messages: 281045 nosy: Pravic Ehysta, paul.moore, steve.dower, tim.golden, zach.ware priority: normal severity: normal status: open title: py.exe launches wrong version type: behavior versions: Python 3.5 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu Nov 17 11:02:48 2016 From: report at bugs.python.org (STINNER Victor) Date: Thu, 17 Nov 2016 16:02:48 +0000 Subject: [New-bugs-announce] [issue28727] Implement comparison (x==y and x!=y) for _sre.SRE_Pattern Message-ID: <1479398568.09.0.867806263276.issue28727@psf.upfronthosting.co.za> New submission from STINNER Victor: Attached patch implements rich comparison for _sre.SRE_Pattern objects created by re.compile(). Comparison between patterns is used in the warnings module to not add duplicated filters, see issue #18383: New changeset f57f4e33ba5e by Martin Panter in branch '3.5': Issue #18383: Avoid adding duplicate filters when warnings is reloaded https://hg.python.org/cpython/rev/f57f4e33ba5e For the warnings module, it became a problem in test_warnings since the Python test runner started to clear all caches. When re.purge() is called, re.compile() creates a new object, whereas with the cache it returns the same object and so the two patterns are equal since it's the same object. => see issue #28688 ---------- files: pattern_compare.patch keywords: patch messages: 281046 nosy: haypo priority: normal severity: normal status: open title: Implement comparison (x==y and x!=y) for _sre.SRE_Pattern type: enhancement versions: Python 3.7 Added file: http://bugs.python.org/file45520/pattern_compare.patch _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu Nov 17 13:35:43 2016 From: report at bugs.python.org (SilentGhost) Date: Thu, 17 Nov 2016 18:35:43 +0000 Subject: [New-bugs-announce] [issue28728] test_host_resolution in test_socket fails on duplicate assert Message-ID: <1479407743.41.0.909577209709.issue28728@psf.upfronthosting.co.za> New submission from SilentGhost: Commit 540a9c69c2ea introduced double assertRaises which now is failing on ubuntu 16.10 If it is necessary, then it's not obvious why and there is no comment, but here is the one-line patch that removes the duplicated line and makes the test pass for me. ---------- components: Tests files: test_socket.diff keywords: patch messages: 281061 nosy: SilentGhost, neologix priority: normal severity: normal stage: patch review status: open title: test_host_resolution in test_socket fails on duplicate assert type: behavior versions: Python 3.6, Python 3.7 Added file: http://bugs.python.org/file45524/test_socket.diff _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu Nov 17 18:25:40 2016 From: report at bugs.python.org (Peter Inglesby) Date: Thu, 17 Nov 2016 23:25:40 +0000 Subject: [New-bugs-announce] [issue28729] Exception from sqlite3 adapter masked by sqlite3.InterfaceError Message-ID: <1479425140.19.0.0824888445668.issue28729@psf.upfronthosting.co.za> New submission from Peter Inglesby: The following code raises `sqlite3.InterfaceError: Error binding parameter 0 - probably unsupported type.` when I would expect it to raise `AssertionError: Problem in adapter`. import sqlite3 class Point: def __init__(self, x, y): self.x, self.y = x, y def adapt_point(point): assert False, 'Problem in adapter' sqlite3.register_adapter(Point, adapt_point) con = sqlite3.connect(":memory:") cur = con.cursor() p = Point(4.0, -3.2) cur.execute("select ?", (p,)) ---------- messages: 281066 nosy: inglesp priority: normal severity: normal status: open title: Exception from sqlite3 adapter masked by sqlite3.InterfaceError type: behavior versions: Python 2.7, Python 3.5, Python 3.6, Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu Nov 17 20:31:03 2016 From: report at bugs.python.org (Jeff Reback) Date: Fri, 18 Nov 2016 01:31:03 +0000 Subject: [New-bugs-announce] [issue28730] __reduce__ not being called in dervied extension class from datetime.datetime Message-ID: <1479432663.49.0.198725524634.issue28730@psf.upfronthosting.co.za> New submission from Jeff Reback: xref to https://github.com/pandas-dev/pandas/issues/14679. pandas has had a cython extension class to datetime.datetime for quite some time. A simple __reduce__ is defined. def __reduce__(self): object_state = self.value, self.freq, self.tzinfo print(object_state) return (Timestamp, object_state) In 3.5.2: Python 3.5.2 |Continuum Analytics, Inc.| (default, Jul 2 2016, 17:52:12) [GCC 4.2.1 Compatible Apple LLVM 4.2 (clang-425.0.28)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> import pandas as pd >>> import pickle >>> pickle.dumps(pd.Timestamp('20130101')) (1356998400000000000, None, None) b'\x80\x03cpandas.tslib\nTimestamp\nq\x00\x8a\x08\x00\x00\xc6\xe8\xda\x06\xd5\x12NN\x87q\x01Rq\x02.' But in 3.6.03b Python 3.6.0b3 | packaged by conda-forge | (default, Nov 2 2016, 03:28:12) [GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.54)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> import pandas as pd >>> import pickle >>> pickle.dumps(pd.Timestamp('20130101')) b'\x80\x03cpandas.tslib\nTimestamp\nq\x00C\n\x07\xdd\x01\x01\x00\x00\x00\x00\x00\x00q\x01\x85q\x02Rq\x03.' So it appears __reduce__ is no longer called at all (I tried defining __getstate__, __getnewargs__ as well, but to no avail). Instead it looks like datetime.datetime.__reduce__ (well a c function is actually called). Link to the codebase. https://github.com/pandas-dev/pandas/blob/master/pandas/tslib.pyx#L490 ---------- components: IO messages: 281070 nosy: Jeff Reback priority: normal severity: normal status: open title: __reduce__ not being called in dervied extension class from datetime.datetime type: behavior versions: Python 3.6 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Nov 18 06:03:05 2016 From: report at bugs.python.org (INADA Naoki) Date: Fri, 18 Nov 2016 11:03:05 +0000 Subject: [New-bugs-announce] [issue28731] _PyDict_NewPresized() creates too small dict Message-ID: <1479466985.37.0.124038643632.issue28731@psf.upfronthosting.co.za> New submission from INADA Naoki: _PyDict_NewPresized(6) creates dict which keysize is 8. But it's capacity is 5 (8 * 2 // 3 = 5). This internal API is used by BUILD_MAP and BUILD_CONST_KEYMAP. ---------- assignee: inada.naoki files: PyDict_NewPresized-too-small.patch keywords: patch messages: 281092 nosy: haypo, inada.naoki priority: high severity: normal stage: patch review status: open title: _PyDict_NewPresized() creates too small dict type: performance versions: Python 3.6, Python 3.7 Added file: http://bugs.python.org/file45531/PyDict_NewPresized-too-small.patch _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Nov 18 06:22:35 2016 From: report at bugs.python.org (srinivasaraoMaddukuri) Date: Fri, 18 Nov 2016 11:22:35 +0000 Subject: [New-bugs-announce] [issue28732] spawnl crash on windows7 Message-ID: <1479468155.36.0.0223780421373.issue28732@psf.upfronthosting.co.za> New submission from srinivasaraoMaddukuri: in window7 (using python 3.4.3,3.5.2) following script crashes import os os.spawnl( os.P_NOWAIT, 'C:/Tcl/bin/tclsh.exe' )" Note: similar issue is already exist https://bugs.python.org/issue8036 ---------- components: Interpreter Core messages: 281095 nosy: srinim priority: normal severity: normal status: open title: spawnl crash on windows7 type: crash versions: Python 3.4, Python 3.5 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Nov 18 08:17:52 2016 From: report at bugs.python.org (=?utf-8?q?Micha=C5=82_Bultrowicz?=) Date: Fri, 18 Nov 2016 13:17:52 +0000 Subject: [New-bugs-announce] [issue28733] Show how to use mock_open in modules other that __main__ Message-ID: <1479475072.8.0.338397997846.issue28733@psf.upfronthosting.co.za> New submission from Micha? Bultrowicz: Documentation of mock_open doesn't say how to use it in real-life test situations (when you're probably not mocking in __main__). I've spent some time scratching my head and googling for the way to mock-out the "open" function, want to spare other people the hassle. The thing is that "open" needs to be mocked out from the magical "builtins" module, and not from the place of usage (like when mocking everything that's not built-in). So it's not obvious how to do that, especially that the example with __main__ makes it look like the normal mocking approach should work. I still don't fully understand why mocking "__main__.open" can work from interpreter, but that's a different thing... ---------- assignee: docs at python components: Documentation files: mock_open_doc.patch keywords: patch messages: 281114 nosy: butla, docs at python priority: normal severity: normal status: open title: Show how to use mock_open in modules other that __main__ type: enhancement versions: Python 3.3, Python 3.4, Python 3.5, Python 3.6, Python 3.7 Added file: http://bugs.python.org/file45533/mock_open_doc.patch _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Nov 18 10:41:15 2016 From: report at bugs.python.org (Adam Stewart) Date: Fri, 18 Nov 2016 15:41:15 +0000 Subject: [New-bugs-announce] [issue28734] argparse: successive parsing wipes out nargs=? values Message-ID: <1479483675.86.0.774570236197.issue28734@psf.upfronthosting.co.za> New submission from Adam Stewart: I'm writing a wrapper that optionally accepts a file and reads more options from that file. The wrapper then needs to pass all of these options and the file to another program (qsub). Here is a minimal example to reproduce the behavior I'm seeing: >>> import argparse >>> parser = argparse.ArgumentParser() >>> parser.add_argument('-a') >>> parser.add_argument('file', nargs='?') >>> args = parser.parse_args(['-a', '3', 'myFile']) >>> print(args) Namespace(file='myFile', a='3') >>> parser.parse_args(['-a', '4'], namespace=args) >>> print(args) Namespace(file=None, a='4') The behavior I expect is that the file should remain as 'myFile', but it is being wiped out. Is there any way to prevent this, or is this actually a bug? I can recreate this problem in Python 2.7 and 3.5. ---------- components: Library (Lib) messages: 281128 nosy: ajstewart priority: normal severity: normal status: open title: argparse: successive parsing wipes out nargs=? values type: behavior versions: Python 3.5 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Nov 18 11:07:14 2016 From: report at bugs.python.org (Rafael Jacinto Caricio da Fonseca) Date: Fri, 18 Nov 2016 16:07:14 +0000 Subject: [New-bugs-announce] [issue28735] Mock is equal to ANY but MagicMock is not Message-ID: <1479485234.58.0.222215885157.issue28735@psf.upfronthosting.co.za> New submission from Rafael Jacinto Caricio da Fonseca: On Python 3.5.2 mock.Mock() is equal to mock.ANY, but mock.MagicMock() is not. Minimal example: In Python 3.5.2: >>> from unittest import mock >>> mock.Mock() == mock.ANY True >>> mock.ANY == mock.Mock() True >>> mock.MagicMock() == mock.ANY False >>> mock.ANY == mock.MagicMock() True ---------- components: Library (Lib) messages: 281135 nosy: rafael.fonseca priority: normal severity: normal status: open title: Mock is equal to ANY but MagicMock is not type: behavior versions: Python 3.5 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Nov 18 11:32:34 2016 From: report at bugs.python.org (Eric Leadbetter) Date: Fri, 18 Nov 2016 16:32:34 +0000 Subject: [New-bugs-announce] [issue28736] multiprocessing.Lock() no longer has .acquire() Message-ID: <1479486754.06.0.767072130944.issue28736@psf.upfronthosting.co.za> New submission from Eric Leadbetter: The documentation on the multiprocessing library in Python 3 uses Lock.acquire()/Lock.release() in the example for primitive synchronization (https://docs.python.org/3/library/multiprocessing.html#synchronization-between-processes). Lock() has been changed in Python 3 to use coroutines and so the documentation should replace the call to Lock.acquire() with an appropriate yield statement. ---------- assignee: docs at python components: Documentation messages: 281141 nosy: Eric Leadbetter, docs at python priority: normal severity: normal status: open title: multiprocessing.Lock() no longer has .acquire() versions: Python 3.3, Python 3.4, Python 3.5, Python 3.6, Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Nov 18 11:47:36 2016 From: report at bugs.python.org (Sam Gross) Date: Fri, 18 Nov 2016 16:47:36 +0000 Subject: [New-bugs-announce] [issue28737] Document that tp_dealloc handler must call PyObject_GC_UnTrack if Py_TPFLAGS_HAVE_GC is set Message-ID: <1479487656.43.0.616591816842.issue28737@psf.upfronthosting.co.za> New submission from Sam Gross: In general, an a PyTypeObject that has Py_TPFLAGS_HAVE_GC set must call PyObject_GC_UnTrack() before it frees any PyObject* references it owns. The only reference to this requirement I found is in https://docs.python.org/3/c-api/gcsupport.html#c._PyObject_GC_TRACK. This requirement should be documented in: 1. https://docs.python.org/3/c-api/typeobj.html#c.PyTypeObject.tp_dealloc 2. https://docs.python.org/3/extending/newtypes.html A call to PyObject_GC_UnTrack() should also be added to he official "noddy4" example. Currently, the example is incorrect and can crash if a referred-to object triggers a GC from it's destructor. See the following example which segfaults: https://github.com/colesbury/noddy It may be worthwhile to have _Py_Dealloc call PyObject_GC_UnTrack() if the PyTypeObject has Py_TPFLAGS_HAVE_GC set. Considering that the official Python extension example is missing the call, it seems likely that extension writers often forget to include it. ---------- assignee: docs at python components: Documentation, Extension Modules messages: 281146 nosy: colesbury, docs at python priority: normal severity: normal status: open title: Document that tp_dealloc handler must call PyObject_GC_UnTrack if Py_TPFLAGS_HAVE_GC is set versions: Python 3.5 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Nov 18 14:38:36 2016 From: report at bugs.python.org (Wojtek Ruszczewski) Date: Fri, 18 Nov 2016 19:38:36 +0000 Subject: [New-bugs-announce] [issue28738] Document SIGBREAK as argument for signal() under Windows. Message-ID: <1479497916.69.0.534558560438.issue28738@psf.upfronthosting.co.za> New submission from Wojtek Ruszczewski: SIGBREAK should be listed as acceptable for signal.signal() under Windows. Some context. Registering a handler for SIGBREAK may be useful as this is the signal that generating CTRL_BREAK_EVENT results in (and the latter combined with the CREATE_NEW_PROCESS_GROUP flag might be the closest that one can get to terminating a process group). Some pointers: * The changed documentation fragment: https://docs.python.org/3/library/signal.html#signal.signal. * MSDN doesn't say so much about SIGBREAK: https://msdn.microsoft.com/en-us/library/windows/desktop/ms682541(v=vs.85).aspx. * SIGBREAK was added to the signal module with #466877. * The signal number check looks as follows: https://github.com/python/cpython/blob/3.6/Modules/signalmodule.c#L402. ---------- assignee: docs at python components: Documentation, Windows files: sigbreak.patch keywords: patch messages: 281159 nosy: docs at python, paul.moore, steve.dower, tim.golden, wrwrwr, zach.ware priority: normal severity: normal status: open title: Document SIGBREAK as argument for signal() under Windows. type: enhancement versions: Python 3.7 Added file: http://bugs.python.org/file45538/sigbreak.patch _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Nov 18 15:43:51 2016 From: report at bugs.python.org (Yury Selivanov) Date: Fri, 18 Nov 2016 20:43:51 +0000 Subject: [New-bugs-announce] [issue28739] PEP 498: docstrings as f-strings Message-ID: <1479501831.35.0.531195928426.issue28739@psf.upfronthosting.co.za> New submission from Yury Selivanov: Can f-strings be used as docstrings? Right now: class Foo: f'spam' Foo.__doc__ is None I couldn't find that f-strings cannot be used as docstrings in neither PEP 498 not in the 3.6 documentation, so I suppose this is a bug. ---------- components: Interpreter Core messages: 281166 nosy: eric.smith, gvanrossum, martin.panter, yselivanov priority: normal severity: normal status: open title: PEP 498: docstrings as f-strings versions: Python 3.6, Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Nov 18 16:26:35 2016 From: report at bugs.python.org (STINNER Victor) Date: Fri, 18 Nov 2016 21:26:35 +0000 Subject: [New-bugs-announce] [issue28740] Add sys.getandroidapilevel() Message-ID: <1479504395.69.0.761590307394.issue28740@psf.upfronthosting.co.za> New submission from STINNER Victor: Android slowly becomes a first-citizen class platform in CPython thanks to Xavier de Gaye and other motivated developers, thanks to all of them :-) To fix the issue #28596, we need a function to check if we are running Android. Chi Hsuan Yen proposed to use sysconfig to get the new ANDROID_API_LEVEL configuration option: + if sysconfig.get_config_var('ANDROID_API_LEVEL'): But I asked to avoid sysconfig to reduce imports at Python startup (especially when the site module is not loaded). I proposed to add a new function to the sys module: sys.getandroidapilevel(). sys.getandroidapilevel() would only be available on Android, as sys.getwindowsversion() is only available on Windows, and would return ANDROID_API_LEVEL as an integer. I'm not sure about the type: should we use a string? A tuple of integers like sys.version_info? sys.getwindowsversion() returns a named tuple: https://docs.python.org/dev/library/sys.html#sys.getwindowsversion I'm sorry, I don't have access to an Android development platform, so I let someone else implement it :-) ---------- messages: 281169 nosy: Chi Hsuan Yen, haypo, xdegaye priority: normal severity: normal status: open title: Add sys.getandroidapilevel() type: enhancement versions: Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Nov 18 16:51:56 2016 From: report at bugs.python.org (Yury Selivanov) Date: Fri, 18 Nov 2016 21:51:56 +0000 Subject: [New-bugs-announce] [issue28741] PEP 498: single '}' is not allowed Message-ID: <1479505916.83.0.442816788413.issue28741@psf.upfronthosting.co.za> New submission from Yury Selivanov: The PEP should document the "single '}' is not allowed" error. ---------- assignee: docs at python components: Documentation messages: 281173 nosy: docs at python, eric.smith, yselivanov priority: normal severity: normal status: open title: PEP 498: single '}' is not allowed versions: Python 3.6, Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Nov 18 19:17:09 2016 From: report at bugs.python.org (Peter Eckersley) Date: Sat, 19 Nov 2016 00:17:09 +0000 Subject: [New-bugs-announce] [issue28742] argparse.ArgumentDefaultsHelpFormatter sometimes provides inaccurate documentation of defaults, so they should be overrideable Message-ID: <1479514629.23.0.218870455687.issue28742@psf.upfronthosting.co.za> New submission from Peter Eckersley: When argparse lists the default values for cli flags and arguments, it shows argparse's view of the internal Pythonic default, not the default behaviour of the program as a whole. This can be wrong in many cases, as documented at https://bugs.python.org/issue27153 and https://github.com/certbot/certbot/issues/3734. To mitigate this, we should allow the caller to set arbitrary strings that argparse will display to the user as the "default" in the CLI help. I wrote a first patch to do this via a new "help_default" argument that can be added to argparse actions: https://gist.github.com/pde/daec08cadc21bca0d62852020887ee13 The patch was written against argparse.py from Python2.7, but seems to apply and run okay against the Python 3.x version. ---------- messages: 281183 nosy: pde priority: normal severity: normal status: open title: argparse.ArgumentDefaultsHelpFormatter sometimes provides inaccurate documentation of defaults, so they should be overrideable versions: Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Nov 18 19:47:44 2016 From: report at bugs.python.org (Martin Panter) Date: Sat, 19 Nov 2016 00:47:44 +0000 Subject: [New-bugs-announce] [issue28743] test_choices_algorithms() in test_random uses lots of memory Message-ID: <1479516464.04.0.874513077096.issue28743@psf.upfronthosting.co.za> New submission from Martin Panter: Revision 32bfc81111b6 added test.test_random.MersenneTwister_TestBasicOps.test_choices_algorithms(), which runs the following code: n = 13132817 # 13 million self.gen.choices(range(n), [1]*n, k=10000) When I build Python with the ?--with-pydebug? configure option on x86-64 Linux, this call uses over 1.2 GB of memory. My computer only has 2 GiB, so it tends to slow down the whole operating system and/or trigger Linux?s out-of-memory killler. Especially if other tests are run concurrently. Is it practical to reduce the magnitude of the test parameters, or optimize the implementation to use less memory? If not, perhaps we could hook into the mechanism that other tests use when the allocate large blocks of memory, to cause them to be skipped in low-memory situations. ---------- components: Tests messages: 281185 nosy: martin.panter, rhettinger priority: normal severity: normal status: open title: test_choices_algorithms() in test_random uses lots of memory type: resource usage versions: Python 3.6, Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Nov 18 20:52:48 2016 From: report at bugs.python.org (Renner) Date: Sat, 19 Nov 2016 01:52:48 +0000 Subject: [New-bugs-announce] [issue28744] Basic precision calc error Message-ID: <1479520368.98.0.185264707762.issue28744@psf.upfronthosting.co.za> New submission from Renner: Simple code: print('%.55f' %(1.1 + 2.2 - 3.3)) print('%.55f' %(1.1 + 2.2)) is supposed to produce 0.0000000000000000000000000000000000000000000000000000000 3.3000000000000000000000000000000000000000000000000000000 But when I run it, Actually it produces 0.0000000000000004440892098500626161694526672363281250000 3.3000000000000002664535259100375697016716003417968750000 Found by chance... python 3.5.2 sysname:'Darwin' release:'15.6.0 version:'Darwin Kernel Version 15.6.0: Thu Jun 23 18:25:34 PDT 2016; root:xnu-3248.60.10~1/RELEASE_X86_64') ---------- components: Interpreter Core messages: 281191 nosy: Renner priority: normal severity: normal status: open title: Basic precision calc error type: behavior versions: Python 3.5 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sat Nov 19 00:56:25 2016 From: report at bugs.python.org (woo yoo) Date: Sat, 19 Nov 2016 05:56:25 +0000 Subject: [New-bugs-announce] [issue28745] Python 3.5.2 "from ... import" statement is different from official documentation Message-ID: <1479534985.73.0.593618849184.issue28745@psf.upfronthosting.co.za> New submission from woo yoo: I've experiment with the statement,however the result did not match the official description. I created a namespace package named l007, within which submodule named l009 was placed.I typed "from l007 import l009" in the interpreter, the execution was ok, while in which case a ImportError should have been raised. Is my understanding wrong? ---------- assignee: docs at python components: Documentation files: ??.PNG messages: 281202 nosy: docs at python, woo yoo priority: normal severity: normal status: open title: Python 3.5.2 "from ... import" statement is different from official documentation type: behavior versions: Python 3.5 Added file: http://bugs.python.org/file45544/??.PNG _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sat Nov 19 09:19:33 2016 From: report at bugs.python.org (Xavier de Gaye) Date: Sat, 19 Nov 2016 14:19:33 +0000 Subject: [New-bugs-announce] [issue28746] cannot set_inheritable() for a file descriptor on Android Message-ID: <1479565173.32.0.268004385274.issue28746@psf.upfronthosting.co.za> New submission from Xavier de Gaye: test_socket on Android fails with: FAIL: test_set_inheritable (test.test_socket.InheritanceTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "/sdcard/org.bitbucket.pyona/lib/python3.7/test/test_socket.py", line 4936, in test_set_inheritable self.assertEqual(sock.get_inheritable(), True) AssertionError: False != True ====================================================================== FAIL: test_set_inheritable_cloexec (test.test_socket.InheritanceTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "/sdcard/org.bitbucket.pyona/lib/python3.7/test/test_socket.py", line 4965, in test_set_inheritable_cloexec 0) AssertionError: 1 != 0 Setting a file descriptor with os.set_inheritable() also fails. Setting directly the file descriptor with fcntl.fcntl() succeeds. ---------- assignee: xdegaye components: Interpreter Core messages: 281221 nosy: xdegaye priority: normal severity: normal stage: needs patch status: open title: cannot set_inheritable() for a file descriptor on Android type: behavior versions: Python 3.6, Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sat Nov 19 16:36:12 2016 From: report at bugs.python.org (Steve Dower) Date: Sat, 19 Nov 2016 21:36:12 +0000 Subject: [New-bugs-announce] [issue28747] Expose SSL_CTX_set_cert_verify_callback Message-ID: <1479591372.9.0.198427018061.issue28747@psf.upfronthosting.co.za> New submission from Steve Dower: As a prerequisite for fixing issues such as issue20916 (dynamic download/update of CAs and CRLs), we really need to be able to plug into the certificate verification function for OpenSSL. This patch adds SSLContext._set_cert_verify_callback, which will allow Python code to inject its own verification function. No other functionality is added, but I have proof-of-concept code that uses this patch to delegate all certificate handling to Windows and it works beautifully (better than I expected :) ). If possible, I'd like to get this into Python 3.6. I intend to turn that proof-of-concept into an actual released library and would like to be able to do it sooner rather than later. Targeting 3.6 is the main reason I named the function with an underscore, but I'd be happy to drop it. ---------- assignee: christian.heimes components: SSL messages: 281230 nosy: christian.heimes, ned.deily, steve.dower priority: normal severity: normal stage: patch review status: open title: Expose SSL_CTX_set_cert_verify_callback type: security versions: Python 3.6, Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun Nov 20 04:32:00 2016 From: report at bugs.python.org (Serhiy Storchaka) Date: Sun, 20 Nov 2016 09:32:00 +0000 Subject: [New-bugs-announce] [issue28748] Make _Py_PackageContext of type "const char *" Message-ID: <1479634320.59.0.400146468836.issue28748@psf.upfronthosting.co.za> New submission from Serhiy Storchaka: Currently _Py_PackageContext has type "char *". But it is either NULL or a pointer to internal readonly UTF-8 representation of Unicode object. Adding the const qualifier makes it clear that the data is immutable. I don't think this change will break third-party code. ---------- components: Interpreter Core files: _Py_PackageContext-const.patch keywords: patch messages: 281256 nosy: brett.cannon, eric.snow, ncoghlan, serhiy.storchaka priority: normal severity: normal stage: patch review status: open title: Make _Py_PackageContext of type "const char *" type: enhancement versions: Python 3.7 Added file: http://bugs.python.org/file45556/_Py_PackageContext-const.patch _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun Nov 20 05:29:58 2016 From: report at bugs.python.org (Serhiy Storchaka) Date: Sun, 20 Nov 2016 10:29:58 +0000 Subject: [New-bugs-announce] [issue28749] Fixed the documentation of the mapping codec APIs Message-ID: <1479637798.25.0.994184606714.issue28749@psf.upfronthosting.co.za> New submission from Serhiy Storchaka: Proposed patch adds the documentation of PyUnicode_Translate() and fixes the documentation of other mapping codec APIs (it is incorrect in Python 3): PyUnicode_DecodeCharmap(), PyUnicode_AsCharmapString(), PyUnicode_EncodeCharmap(), and PyUnicode_TranslateCharmap(). ---------- assignee: docs at python components: Documentation, Unicode files: docs-PyUnicode_Translate.patch keywords: patch messages: 281259 nosy: docs at python, ezio.melotti, haypo, serhiy.storchaka priority: normal severity: normal stage: patch review status: open title: Fixed the documentation of the mapping codec APIs type: behavior versions: Python 3.5, Python 3.6, Python 3.7 Added file: http://bugs.python.org/file45557/docs-PyUnicode_Translate.patch _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun Nov 20 08:58:32 2016 From: report at bugs.python.org (Xiang Zhang) Date: Sun, 20 Nov 2016 13:58:32 +0000 Subject: [New-bugs-announce] [issue28750] Replace string with bytes in doc of unicode-escape an raw-unicode-escape Message-ID: <1479650312.85.0.102292696312.issue28750@psf.upfronthosting.co.za> New submission from Xiang Zhang: The docs of the encoders of unicode-escape and raw-unicode-escape still tell the result of the encoding is Python string object. It should be Python bytes object. ---------- assignee: docs at python components: Documentation files: unicode-escape-doc.patch keywords: patch messages: 281263 nosy: docs at python, xiang.zhang priority: normal severity: normal stage: patch review status: open title: Replace string with bytes in doc of unicode-escape an raw-unicode-escape versions: Python 3.5, Python 3.6, Python 3.7 Added file: http://bugs.python.org/file45558/unicode-escape-doc.patch _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun Nov 20 09:29:55 2016 From: report at bugs.python.org (Ned Batchelder) Date: Sun, 20 Nov 2016 14:29:55 +0000 Subject: [New-bugs-announce] [issue28751] Fix comments in code.h Message-ID: <1479652195.59.0.115870067836.issue28751@psf.upfronthosting.co.za> New submission from Ned Batchelder: A field moved in PyCodeObject, but comments mentioning it were not updated. Also, there's a stray word? ---------- messages: 281268 nosy: brett.cannon, nedbat priority: normal severity: normal status: open title: Fix comments in code.h versions: Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun Nov 20 10:55:23 2016 From: report at bugs.python.org (Jason R. Coombs) Date: Sun, 20 Nov 2016 15:55:23 +0000 Subject: [New-bugs-announce] [issue28752] datetime object fails to restore from reduction Message-ID: <1479657323.42.0.875075153657.issue28752@psf.upfronthosting.co.za> New submission from Jason R. Coombs: On Python 3.5, the datetime would reduce and restore cleanly. $ python3.5 -c "import datetime; func, params = datetime.datetime.now().__reduce__(); func(*params)" With Python 3.6.0b3, it now fails with a TypeError. $ python3.6 -c "import datetime; func, params = datetime.datetime.now().__reduce__(); func(*params)" Traceback (most recent call last): File "", line 1, in TypeError: an integer is required (got type bytes) ---------- messages: 281278 nosy: jason.coombs priority: normal severity: normal status: open title: datetime object fails to restore from reduction versions: Python 3.6, Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun Nov 20 11:44:04 2016 From: report at bugs.python.org (Julien Palard) Date: Sun, 20 Nov 2016 16:44:04 +0000 Subject: [New-bugs-announce] [issue28753] Clinic: Converting Your First Function is not up to date Message-ID: <1479660244.5.0.379774731624.issue28753@psf.upfronthosting.co.za> New submission from Julien Palard: It looks like the "Converting Your First Function" has been written with clinic-generated C code interspersed with user C code. But it looks like nowadays a `clinic/{}.c.h` file is generated, so the "Converting Your First Function" should tell us to add the `#include "clinic/?`. Also, the documentation states: > If any of these items differ in any way, adjust your Argument > Clinic function specification and rerun Tools/clinic/clinic.py > until they are the same. But since `METH_FASTCALL` it may be wrong (Clinic generated `METH_FASTCALL` but it was `METH_VARARGS|METH_KEYWORDS`). ---------- messages: 281281 nosy: mdk priority: normal severity: normal status: open title: Clinic: Converting Your First Function is not up to date _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun Nov 20 11:51:59 2016 From: report at bugs.python.org (Julien Palard) Date: Sun, 20 Nov 2016 16:51:59 +0000 Subject: [New-bugs-announce] [issue28754] Argument Clinic for bisect.bisect_left Message-ID: <1479660719.6.0.85429244405.issue28754@psf.upfronthosting.co.za> New submission from Julien Palard: Today I read https://docs.python.org/3.6/howto/clinic.html so I tried one: bisect.bisect_left. I was unable to do `bisect_right`, as it's an "alias" for `bisect`, and there's a unit-test checking `self.assertEqual(self.module.bisect, self.module.bisect_right)`. Any hint welcome. ---------- components: Argument Clinic messages: 281282 nosy: larry, mdk priority: normal severity: normal status: open title: Argument Clinic for bisect.bisect_left _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun Nov 20 17:01:57 2016 From: report at bugs.python.org (Julien Palard) Date: Sun, 20 Nov 2016 22:01:57 +0000 Subject: [New-bugs-announce] [issue28755] Rework syntax highlighing in howto/clinic.rst Message-ID: <1479679317.44.0.842687607942.issue28755@psf.upfronthosting.co.za> New submission from Julien Palard: I was reading `howto/clinic.html` and though I'll fix syntax highlighting. ---------- assignee: docs at python components: Argument Clinic, Documentation messages: 281304 nosy: docs at python, larry, mdk priority: normal severity: normal status: open title: Rework syntax highlighing in howto/clinic.rst versions: Python 2.7, Python 3.6, Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun Nov 20 20:23:46 2016 From: report at bugs.python.org (John Nagle) Date: Mon, 21 Nov 2016 01:23:46 +0000 Subject: [New-bugs-announce] [issue28756] robotfileparser always uses default Python user-agent Message-ID: <1479691426.59.0.686880429063.issue28756@psf.upfronthosting.co.za> New submission from John Nagle: urllib.robotparser.RobotFileParser always uses the default Python user agent. This agent is now blacklisted by many sites, and it's not possible to read the robots.txt file at all. ---------- components: Library (Lib) messages: 281314 nosy: nagle priority: normal severity: normal status: open title: robotfileparser always uses default Python user-agent type: enhancement versions: Python 2.7, Python 3.3, Python 3.4, Python 3.5, Python 3.6, Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun Nov 20 21:49:37 2016 From: report at bugs.python.org (Sophia I. Salinas) Date: Mon, 21 Nov 2016 02:49:37 +0000 Subject: [New-bugs-announce] [issue28757] Installation Failure Message-ID: <1479696577.66.0.987824550114.issue28757@psf.upfronthosting.co.za> New submission from Sophia I. Salinas: I downloaded Python 3.2 for Mac but when I tried to install it, I got an error that says: "The installation failed. The installer could not install the software because there was no software found to install." ---------- messages: 281319 nosy: sophiaisa priority: normal severity: normal status: open title: Installation Failure type: behavior _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Nov 21 00:17:28 2016 From: report at bugs.python.org (dontbugme) Date: Mon, 21 Nov 2016 05:17:28 +0000 Subject: [New-bugs-announce] [issue28758] UnicodeDecodeError: 'gbk' codec can't decode byte 0xab in position 74: illegal multibyte sequence Message-ID: <1479705448.56.0.0763380057233.issue28758@psf.upfronthosting.co.za> New submission from dontbugme: you can see https://github.com/mintty/mintty/issues/609 os.popen('chcp 65001 && ' + JAVA + ' -jar ' + CHECKSTYLE_JAR + ' -c ' + CHECKSTYLE_XML + ' "%s/%s"' % (COMMIT_TEMP_DIT, changed)).read() ---------- messages: 281322 nosy: dontbugme priority: normal severity: normal status: open title: UnicodeDecodeError: 'gbk' codec can't decode byte 0xab in position 74: illegal multibyte sequence versions: Python 3.6 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Nov 21 03:12:55 2016 From: report at bugs.python.org (Xavier de Gaye) Date: Mon, 21 Nov 2016 08:12:55 +0000 Subject: [New-bugs-announce] [issue28759] access to mkfifo, mknod and hard links is controled by SELinux MAC on Android Message-ID: <1479715975.06.0.930254771903.issue28759@psf.upfronthosting.co.za> New submission from Xavier de Gaye: List of the tests that fail with PermissionError when run as a non-root user on an Android emulator (API 24) and fixed by this patch: test_posix test_shutil test_stat test_genericpath test_ntpath test_posixpath test_macpath test_eintr test_pathlib Android uses SELinux mandatory access control (MAC), see: https://source.android.com/security/selinux/ Android commit message that does not grant hard link capabilities by default: https://android.googlesource.com/platform/external/sepolicy/+/85ce2c7 ---------- assignee: xdegaye components: Tests files: android_not_root.patch keywords: patch messages: 281329 nosy: xdegaye priority: normal severity: normal stage: patch review status: open title: access to mkfifo, mknod and hard links is controled by SELinux MAC on Android type: behavior versions: Python 3.6, Python 3.7 Added file: http://bugs.python.org/file45577/android_not_root.patch _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Nov 21 04:24:14 2016 From: report at bugs.python.org (Xiang Zhang) Date: Mon, 21 Nov 2016 09:24:14 +0000 Subject: [New-bugs-announce] [issue28760] Cleanup PyUnicode_AsUnicodeEscapeString Message-ID: <1479720254.49.0.534389822333.issue28760@psf.upfronthosting.co.za> New submission from Xiang Zhang: PyUnicode_AsUnicodeEscapeString now still has some old comments and codes about the original "quotes" parameter of unicodeescape_string. Current implementation could get rid of them. ---------- files: PyUnicode_AsUnicodeEscapeString.patch keywords: patch messages: 281337 nosy: xiang.zhang priority: normal severity: normal stage: patch review status: open title: Cleanup PyUnicode_AsUnicodeEscapeString type: enhancement versions: Python 3.5, Python 3.6, Python 3.7 Added file: http://bugs.python.org/file45579/PyUnicode_AsUnicodeEscapeString.patch _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Nov 21 04:40:15 2016 From: report at bugs.python.org (Serhiy Storchaka) Date: Mon, 21 Nov 2016 09:40:15 +0000 Subject: [New-bugs-announce] [issue28761] Add the const qualifier to fields name and doc of public structs Message-ID: <1479721215.94.0.926847146167.issue28761@psf.upfronthosting.co.za> New submission from Serhiy Storchaka: Proposed patch makes the fields name and doc of structures PyMemberDef, PyGetSetDef, PyStructSequence_Field, PyStructSequence_Desc, and wrapperbase being of type "const char *" rather of "char *". These structures often are initialized as static variables and name and doc fields are initialized from literal strings. These fields are always refer to constant string literal, NULL, or readonly UTF8 representation of a Unicode object. They are never refer to mutable data. Changing the data referred by these pointers is an error. Adding the const qualifier makes clear that this is an immutable data. This change may need some changes in third-party code. But it needs to change only one line in CPython sources, and I believe that most third-party projects don't need any changes at all. ---------- components: Interpreter Core files: const-name-and-doc-fields.patch keywords: patch messages: 281340 nosy: ncoghlan, serhiy.storchaka priority: normal severity: normal stage: patch review status: open title: Add the const qualifier to fields name and doc of public structs type: enhancement versions: Python 3.7 Added file: http://bugs.python.org/file45581/const-name-and-doc-fields.patch _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Nov 21 05:19:49 2016 From: report at bugs.python.org (Xavier de Gaye) Date: Mon, 21 Nov 2016 10:19:49 +0000 Subject: [New-bugs-announce] [issue28762] configure links with lockf and F_LOCK is not declared in Android API 24 Message-ID: <1479723589.5.0.525775136475.issue28762@psf.upfronthosting.co.za> New submission from Xavier de Gaye: The patch fixes the following error: ====================================================================== ERROR: test_lockf (test.test_posix.PosixTester) ---------------------------------------------------------------------- Traceback (most recent call last): File "/sdcard/org.bitbucket.pyona/lib/python3.7/test/test_posix.py", line 195, in test_lockf posix.lockf(fd, posix.F_LOCK, 4) AttributeError: module 'posix' has no attribute 'F_LOCK' ---------- assignee: xdegaye components: Interpreter Core files: test_posix_flock.patch keywords: patch messages: 281344 nosy: xdegaye priority: normal severity: normal stage: patch review status: open title: configure links with lockf and F_LOCK is not declared in Android API 24 type: behavior versions: Python 3.6, Python 3.7 Added file: http://bugs.python.org/file45582/test_posix_flock.patch _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Nov 21 06:34:55 2016 From: report at bugs.python.org (Serhiy Storchaka) Date: Mon, 21 Nov 2016 11:34:55 +0000 Subject: [New-bugs-announce] [issue28763] Use en-dashes for ranges in docs Message-ID: <1479728095.62.0.206474714084.issue28763@psf.upfronthosting.co.za> New submission from Serhiy Storchaka: It is recommended to use an em-dash instead of a hyphen for numerical ranges. It looks better in the rendered document. Python documentation already use it, but not always. Proposed patch adds more en-dashes. ---------- assignee: docs at python components: Documentation files: docs-en-dash.patch keywords: patch messages: 281347 nosy: docs at python, martin.panter, serhiy.storchaka priority: normal severity: normal status: open title: Use en-dashes for ranges in docs type: enhancement Added file: http://bugs.python.org/file45584/docs-en-dash.patch _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Nov 21 10:22:02 2016 From: report at bugs.python.org (Xavier de Gaye) Date: Mon, 21 Nov 2016 15:22:02 +0000 Subject: [New-bugs-announce] [issue28764] test_mailbox fails when run as a non-root user on Android API 24 Message-ID: <1479741722.73.0.770267227494.issue28764@psf.upfronthosting.co.za> New submission from Xavier de Gaye: The patch fixes the problem that on Android API 24, a non-root user is not allowed to create the hard links used by the mailbox module. Related to issue 28759. ---------- assignee: xdegaye components: Library (Lib) files: test_mailbox_link.patch keywords: patch messages: 281364 nosy: xdegaye priority: normal severity: normal stage: patch review status: open title: test_mailbox fails when run as a non-root user on Android API 24 type: behavior versions: Python 3.6, Python 3.7 Added file: http://bugs.python.org/file45588/test_mailbox_link.patch _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Nov 21 10:57:37 2016 From: report at bugs.python.org (STINNER Victor) Date: Mon, 21 Nov 2016 15:57:37 +0000 Subject: [New-bugs-announce] [issue28765] _sre.compile(): be more strict on types of indexgroup and groupindex Message-ID: <1479743857.98.0.418828964214.issue28765@psf.upfronthosting.co.za> New submission from STINNER Victor: Attached patch makes _sre.compile() more strict on types: groupindex must be a dictionary and indexgroup must be a tuple. Currently, indexgroup is passed as a list. I chose to convert it to a tuple to use less memory and to prevent unwanted changes. For unwanted changed, I'm not sure because groupindex remains a mutable dictionary. Do you think that it's worth it to require a tuple? Another option is to accept a list but to convert it to a list, but this change is more specific to the current implementation. ---------- files: sre_types.patch keywords: patch messages: 281373 nosy: haypo, serhiy.storchaka priority: normal severity: normal status: open title: _sre.compile(): be more strict on types of indexgroup and groupindex type: enhancement versions: Python 3.7 Added file: http://bugs.python.org/file45590/sre_types.patch _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Nov 21 11:35:39 2016 From: report at bugs.python.org (Yuwei Ba) Date: Mon, 21 Nov 2016 16:35:39 +0000 Subject: [New-bugs-announce] [issue28766] Remove the semicolon in source code Message-ID: <1479746139.91.0.549636125293.issue28766@psf.upfronthosting.co.za> New submission from Yuwei Ba: As we do not often use semicolons in Python world. I hope this historical semi would be cleaned in a future release. Source: Lib/httplib.py:1117 ---------- components: Library (Lib) files: mywork.patch hgrepos: 361 keywords: patch messages: 281378 nosy: ibigbug priority: normal severity: normal status: open title: Remove the semicolon in source code type: enhancement versions: Python 2.7 Added file: http://bugs.python.org/file45591/mywork.patch _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Nov 21 12:40:40 2016 From: report at bugs.python.org (Josh Rosenberg) Date: Mon, 21 Nov 2016 17:40:40 +0000 Subject: [New-bugs-announce] [issue28767] Readd __index__ support on ipaddress objects Message-ID: <1479750040.58.0.0255514552996.issue28767@psf.upfronthosting.co.za> New submission from Josh Rosenberg: It looks like, due to #16722, in #15559, __index__ was removed from ipaddress objects. #16722 was fixed a few months later, but the comments asking for it to be readded were put on a closed issue, so no one noticed. Can __index__ support be readded now? Should be as simple as undoing https://hg.python.org/cpython/rev/5abea8a43f19 (probably manually, assuming the subsequent history would make a direct rollback nonfeasible). Nosying the folks involved in fixing the original bugs. ---------- messages: 281384 nosy: benjamin.peterson, josh.r, ncoghlan priority: normal severity: normal status: open title: Readd __index__ support on ipaddress objects versions: Python 3.4, Python 3.5, Python 3.6, Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Nov 21 15:00:30 2016 From: report at bugs.python.org (Masayuki Yamamoto) Date: Mon, 21 Nov 2016 20:00:30 +0000 Subject: [New-bugs-announce] [issue28768] Warning: implicit declaration of function '_setmode' Message-ID: <1479758430.88.0.97682867264.issue28768@psf.upfronthosting.co.za> New submission from Masayuki Yamamoto: Platform that appeared warning is Vista Cygwin x86. Interpreter execution doesn't crash because _setmode function is supplied from cygwin1.dll that always linked. Warning reason is header io.h [*] doesn't include to source file. Therefore I wrote two patches for 3.7 and 2.7. [*] https://msdn.microsoft.com/en-us/library/tw4k6df8.aspx (Cygwin also avaliable) build log on 3.7: gcc -c -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -std=c99 -Wextra -Wno-unused-result -Wno-unused-parameter -Wno-missing-field-initializers -I. -I./Include -DPy_BUILD_CORE -o Modules/main.o Modules/main.c Modules/main.c: In function 'Py_Main': Modules/main.c:599:5: warning: implicit declaration of function '_setmode' [-Wimplicit-function-declaration] _setmode(fileno(stdin), O_BINARY); ^ gcc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -std=c99 -Wextra -Wno-unused-result -Wno-unused-parameter -Wno-missing-field-initializers -I. -I./Include -DPy_BUILD_CORE -I./Modules/_io -c ./Modules/_io/fileio.c -o Modules/fileio.o ./Modules/_io/fileio.c: In function '_io_FileIO___init___impl': ./Modules/_io/fileio.c:478:5: warning: implicit declaration of function '_setmode' [-Wimplicit-function-declaration] _setmode(self->fd, O_BINARY); ^ ---------- components: Build files: include-io.h.patch keywords: patch messages: 281389 nosy: benjamin.peterson, masamoto, stutzbach priority: normal severity: normal status: open title: Warning: implicit declaration of function '_setmode' type: compile error versions: Python 2.7, Python 3.6, Python 3.7 Added file: http://bugs.python.org/file45592/include-io.h.patch _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Nov 22 02:55:24 2016 From: report at bugs.python.org (Serhiy Storchaka) Date: Tue, 22 Nov 2016 07:55:24 +0000 Subject: [New-bugs-announce] [issue28769] Make PyUnicode_AsUTF8 returning "const char *" rather of "char *" Message-ID: <1479801324.65.0.962202188144.issue28769@psf.upfronthosting.co.za> New submission from Serhiy Storchaka: PyUnicode_AsUTF8AndSize() and PyUnicode_AsUTF8() return a reference to cached readonly UTF-8 representation of a string. Changing the content of the UTF-8 representation is an error. Proposed patch makes these functions returning "const char *" rather of "char *" to force this restriction. This is backward-incompatible change. Since PyUnicode_AsUTF8AndSize() and PyUnicode_AsUTF8() can return an error, it is more likely that the result is saved in a local variable rather than passing to other function. If the type of this variable is "char *" rather than "const char *", this would cause a compiler error. The fix is simple -- just add the const qualifier to the local variable declaration (more preferable) or cast the result of PyUnicode_AsUTF8AndSize() or PyUnicode_AsUTF8() to "char *". Both functions are not in stable API. ---------- components: Interpreter Core files: PyUnicode_AsUTF8-const.patch keywords: patch messages: 281439 nosy: ncoghlan, serhiy.storchaka priority: normal severity: normal stage: patch review status: open title: Make PyUnicode_AsUTF8 returning "const char *" rather of "char *" type: enhancement versions: Python 3.7 Added file: http://bugs.python.org/file45596/PyUnicode_AsUTF8-const.patch _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Nov 22 06:42:55 2016 From: report at bugs.python.org (STINNER Victor) Date: Tue, 22 Nov 2016 11:42:55 +0000 Subject: [New-bugs-announce] [issue28770] Update python-gdb.py for fastcalls Message-ID: <1479814975.16.0.381517171433.issue28770@psf.upfronthosting.co.za> New submission from STINNER Victor: Python 3.6 has a new C calling convention: "fast calls". python-gdb.py was disabled when compact dict was merged, see issue #27350. Sadly, I missed that fast calls also broke python-gdb.py. Attached patch fixes python-gdb.py, but I failed to fix test_gdb.py. With fast calls and patched python-gdb.py, python-gdb.py is now able to detect the Python frame of the call to the builtin id() function. It's a new feature compared to Python 3.5, but test_gdb.py should be updated to handle that. I will try to fix that later, but in the meanwhile I was interrupted because the fix for compact dict in python-gdb.py failed on buildbots, see: http://bugs.python.org/issue28023#msg281464 ---------- files: gdb_fastcall.patch keywords: patch messages: 281465 nosy: haypo priority: normal severity: normal status: open title: Update python-gdb.py for fastcalls versions: Python 3.6, Python 3.7 Added file: http://bugs.python.org/file45598/gdb_fastcall.patch _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Nov 22 06:53:08 2016 From: report at bugs.python.org (Martin Panter) Date: Tue, 22 Nov 2016 11:53:08 +0000 Subject: [New-bugs-announce] [issue28771] Update documented signatures of tp_get/setattr Message-ID: <1479815588.93.0.810601241454.issue28771@psf.upfronthosting.co.za> New submission from Martin Panter: https://docs.python.org/3.7/c-api/typeobj.html#c.PyTypeObject.tp_getattr tp_getattr and tp_setattr take a non-const char pointer in their signatures, but the documentation points to PyObject_GetAttrString() etc which were changed to take const char pointers a long time ago: revision 2f19b981ac24. This patch fixes the documentation of those two methods for Python 3. There could be more fixes needed for Python 2. ---------- assignee: docs at python components: Documentation files: typeobj-sigs.patch keywords: patch messages: 281468 nosy: docs at python, martin.panter priority: normal severity: normal stage: patch review status: open title: Update documented signatures of tp_get/setattr versions: Python 2.7, Python 3.5, Python 3.6, Python 3.7 Added file: http://bugs.python.org/file45600/typeobj-sigs.patch _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Nov 22 07:35:42 2016 From: report at bugs.python.org (Stefan Scherfke) Date: Tue, 22 Nov 2016 12:35:42 +0000 Subject: [New-bugs-announce] [issue28772] Bus error in Python 3.6.0beta Message-ID: <1479818142.38.0.0931531711664.issue28772@psf.upfronthosting.co.za> New submission from Stefan Scherfke: Hi all, I am trying to build a custom Conda installer for Python 3.6.0b4. I could successfully build an run Python. However, when I run the generated Conda installer, it dies with a "Bus error". It happens when Conda's meta-installer script tries to replace the build-prefix (e.g., /home/stefan/conda/asdf) with the prefix of the actual installation (e.g., /tmp/py36). Therefore, it opens all files in binary mode, reads their contents, replaces stuff, and writes the new contents back to the original file. This works fine with text files but it dies when it hits the first binary file. Here is a minimal example that reproduces the error: $ /tmp/py36/bin/python Python 3.6.0b4 (default, Nov 22 2016, 10:32:29) [GCC 6.2.1 20160916 (Red Hat 6.2.1-2)] on linux >>> >>> path = '/tmp/py36/lib/libpython3.6m.so.1.0' >>> f = open(path, 'wb') BusError Can anyone reproduce this? Is this a compilation error or an issue with Python itself? It works without issues with Python 3.5. Cheers, Stefan PS: If need, I can upload the Conda installer somewhere. It's ~40MiB. ---------- messages: 281475 nosy: Stefan Scherfke priority: normal severity: normal status: open title: Bus error in Python 3.6.0beta type: crash versions: Python 3.6 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Nov 22 07:43:31 2016 From: report at bugs.python.org (Manuel Krebber) Date: Tue, 22 Nov 2016 12:43:31 +0000 Subject: [New-bugs-announce] [issue28773] typing.FrozenSet missing in documentation. Message-ID: <1479818611.82.0.728843423423.issue28773@psf.upfronthosting.co.za> New submission from Manuel Krebber: The typing.FrozenSet is missing in the typing module documentation. I have attached a patch that adds it similar to the typing.Set which is already in the documentation. ---------- components: Library (Lib) files: frozenset-doc.patch keywords: patch messages: 281476 nosy: Wheerd priority: normal severity: normal status: open title: typing.FrozenSet missing in documentation. type: enhancement versions: Python 3.5, Python 3.6, Python 3.7 Added file: http://bugs.python.org/file45601/frozenset-doc.patch _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Nov 22 09:29:22 2016 From: report at bugs.python.org (Xiang Zhang) Date: Tue, 22 Nov 2016 14:29:22 +0000 Subject: [New-bugs-announce] [issue28774] Better start and end position for unicodeerror in unicode_encode_ucs1 Message-ID: <1479824962.83.0.769019580096.issue28774@psf.upfronthosting.co.za> New submission from Xiang Zhang: unicode_encode_ucs1 now recognizes as many characters as it can one time instead of one character a time. But the unicodeerror positions still only count 1(the second time). A similar problem reported in #28561. ---------- components: Interpreter Core files: unicode_encode_ucs1_error_pos.patch keywords: patch messages: 281482 nosy: haypo, serhiy.storchaka, xiang.zhang priority: normal severity: normal stage: patch review status: open title: Better start and end position for unicodeerror in unicode_encode_ucs1 type: enhancement versions: Python 3.6, Python 3.7 Added file: http://bugs.python.org/file45602/unicode_encode_ucs1_error_pos.patch _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Nov 22 12:54:12 2016 From: report at bugs.python.org (Raymond Hettinger) Date: Tue, 22 Nov 2016 17:54:12 +0000 Subject: [New-bugs-announce] [issue28775] Option to set startup directory in IDLE Message-ID: <1479837252.01.0.0581172301847.issue28775@psf.upfronthosting.co.za> New submission from Raymond Hettinger: In my Python courses, Windows users frequently ask about how to set a startup directory so they can run from their Desktop or a custom directory. The usual answer is to create a short-cut and then alter the properties on that shortcut. However, other programs they are used to will allow the startup directory to be specified from within the program (in our case, the preferences menu). Also, we should change the default directory from C:\\Python27 which is almost never the right place. A documents directory would be more appropriate. ---------- assignee: terry.reedy components: IDLE messages: 281494 nosy: rhettinger, terry.reedy priority: normal severity: normal status: open title: Option to set startup directory in IDLE versions: Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Nov 22 15:49:46 2016 From: report at bugs.python.org (Peter Inglesby) Date: Tue, 22 Nov 2016 20:49:46 +0000 Subject: [New-bugs-announce] [issue28776] Duplicate method names should be an error Message-ID: <1479847786.59.0.434760924279.issue28776@psf.upfronthosting.co.za> New submission from Peter Inglesby: It should be an error for a class to define a method twice. That is, Python should raise an exception when the following code is loaded: class C: def m(self): # do something def m(self): # do something I have just witnessed a beginner get very confused by this! (Apologies if this is a duplicate of an existing issue.) ---------- messages: 281513 nosy: inglesp priority: normal severity: normal status: open title: Duplicate method names should be an error _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Nov 22 23:58:52 2016 From: report at bugs.python.org (Georgy) Date: Wed, 23 Nov 2016 04:58:52 +0000 Subject: [New-bugs-announce] [issue28777] asinc iter queue Message-ID: <1479877132.48.0.693527153069.issue28777@psf.upfronthosting.co.za> New submission from Georgy: adding to asyncio.Queue class following methods: def __aiter__(self): return self async def __anext__(self): return await self.get() let use asyncio.Queue follow: queue = asyncio.Queue() ... async for item in queue: do_something_with(item) ---------- components: asyncio messages: 281536 nosy: RekGRpth, gvanrossum, yselivanov priority: normal severity: normal status: open title: asinc iter queue type: enhancement versions: Python 3.5 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed Nov 23 03:51:14 2016 From: report at bugs.python.org (RAUSHAN RAJ) Date: Wed, 23 Nov 2016 08:51:14 +0000 Subject: [New-bugs-announce] [issue28778] wsgiref HTTP Response Header Injection: CRLF Injection Message-ID: <1479891074.56.0.718206462594.issue28778@psf.upfronthosting.co.za> Changes by RAUSHAN RAJ : ---------- components: Library (Lib) nosy: RAUSHAN RAJ priority: normal severity: normal status: open title: wsgiref HTTP Response Header Injection: CRLF Injection type: security versions: Python 2.7, Python 3.3, Python 3.4, Python 3.5 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed Nov 23 10:26:41 2016 From: report at bugs.python.org (Antoine Pitrou) Date: Wed, 23 Nov 2016 15:26:41 +0000 Subject: [New-bugs-announce] [issue28779] set_forkserver_preload() can crash the forkserver if preloaded module instantiate multiprocessing classes Message-ID: <1479914801.47.0.114591938702.issue28779@psf.upfronthosting.co.za> New submission from Antoine Pitrou: The following script: import multiprocessing import os def f(): pass multiprocessing.Lock() if __name__ == "__main__": ctx = multiprocessing.get_context('forkserver') # modname is the script's importable name (not "__main__") modname = os.path.basename(__file__).split(".")[0] ctx.set_forkserver_preload([modname]) proc = ctx.Process(target=f) proc.start() proc.join() Fails with the following error: Traceback (most recent call last): File "/home/antoine/miniconda3/envs/dask35/lib/python3.5/multiprocessing/forkserver.py", line 178, in main _serve_one(s, listener, alive_r, handler) File "/home/antoine/miniconda3/envs/dask35/lib/python3.5/multiprocessing/forkserver.py", line 212, in _serve_one code = spawn._main(child_r) File "/home/antoine/miniconda3/envs/dask35/lib/python3.5/multiprocessing/spawn.py", line 115, in _main prepare(preparation_data) File "/home/antoine/miniconda3/envs/dask35/lib/python3.5/multiprocessing/spawn.py", line 221, in prepare set_start_method(data['start_method']) File "/home/antoine/miniconda3/envs/dask35/lib/python3.5/multiprocessing/context.py", line 231, in set_start_method raise RuntimeError('context has already been set') RuntimeError: context has already been set This makes set_forkserver_preload() quite fragile if you preload any library that may create multiprocessing resources (such as locks) at the top level. ---------- components: Library (Lib) messages: 281565 nosy: davin, pitrou, sbt priority: normal severity: normal stage: needs patch status: open title: set_forkserver_preload() can crash the forkserver if preloaded module instantiate multiprocessing classes type: behavior versions: Python 3.5, Python 3.6, Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed Nov 23 11:18:23 2016 From: report at bugs.python.org (Mark Wood) Date: Wed, 23 Nov 2016 16:18:23 +0000 Subject: [New-bugs-announce] [issue28780] netrc throws NetrcParseError for record without 'password' Message-ID: <1479917903.22.0.642970878487.issue28780@psf.upfronthosting.co.za> New submission from Mark Wood: netrc.netrc() throws a NetrcParseError if ~/.netrc contains an entry witout a 'password' field. Other users of .netrc do not do this. In my case, I have entries for sftp hosts which will use public-key authentication instead of a password. What I would suggest is that if any or all of 'login', 'account', and 'password' are omitted, simply accept that and store a 0-length string. Someone on StackOverflow says he has rewritten netrc to fix various problems but doesn't know how to contribute it. http://stackoverflow.com/questions/28754547/python-netrc-error-on-file-with-comment ---------- components: Library (Lib) messages: 281567 nosy: Mark Wood priority: normal severity: normal status: open title: netrc throws NetrcParseError for record without 'password' type: behavior versions: Python 3.4 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed Nov 23 11:25:51 2016 From: report at bugs.python.org (Mark Harris) Date: Wed, 23 Nov 2016 16:25:51 +0000 Subject: [New-bugs-announce] [issue28781] On Installation of 3.5 Python get error message Message-ID: <1479918351.94.0.34425443448.issue28781@psf.upfronthosting.co.za> New submission from Mark Harris: While I am installing python when it gets towards the end of the installation the message appears "The program can't start because python35.dll is missing from your computer. Try reinstalling the program to fix this problem". Apart from this message the installation finishes successfully. When I type python into the command prompt the message appears again. I've tried as suggested installing again, it was installing to my local file C:\Users\myusername\AppData\Local\Programs\Python\Python35-32 so I thought it might be something to do with the path not being setting up correctly and have tried several locations. The installer doesn't seem to be adding the PATH file on installation to the system as well. This all started because pip stopped working and I was trying to re-install to resolve the problem. Any suggestions on a fix? Thanks, Mark ---------- components: Installation messages: 281569 nosy: Marcopolo priority: normal severity: normal status: open title: On Installation of 3.5 Python get error message type: crash versions: Python 3.5 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed Nov 23 13:03:27 2016 From: report at bugs.python.org (Martin Richard) Date: Wed, 23 Nov 2016 18:03:27 +0000 Subject: [New-bugs-announce] [issue28782] SEGFAULT when running a given coroutine Message-ID: <1479924207.48.0.511908736136.issue28782@psf.upfronthosting.co.za> New submission from Martin Richard: Hi, I stumbled upon a SEGFAULT while trying Python 3.6.0 on a project using asyncio. I can't really figure out what's happening, so I reduced the original code triggering the bug down to a reproducible case (which looks a bit clunky, sorry). The case has been tested on two Linux systems (Archlinux and Debian), and with several versions of Python. The bug appears between 3.6.0a4 (most recent version tested not affected) and 3.60b1 (so before the C asyncio module I believe), and is not fixed in the current repository tip (changeset: 105345:3addf93f4111). I also produced a traceback using gdb (see bellow). The segfault happens around the "await" in the body of Cursor._read_data(), interestingly, if I change anything in the body of the method, the SEGFAULT disappears and the code works as expected. Also, it seems that calling it from an other coroutine (main() in the example) is required to trigger the bug. Cheers, Martin Case (also attached as test.py) : import asyncio loop = asyncio.get_event_loop() class Connection: def read_until(self, *args, **kwargs): return self async def all(self): return b"\n" class Cursor: def __init__(self): self._connection = Connection() self._max_bytes = 100 self._data = bytearray() async def _read_data(self): # XXX segfault there, if I change anything in the code, it works... while True: data = await self._connection.read_until( b'\n', max_bytes=self._max_bytes).all() self._max_bytes -= len(data) if data == b'\n': break self._data.extend(data) async def main(): await Cursor()._read_data() loop.run_until_complete(main()) Traceback extract (with Python3.6.0b4, --with-pydebug on Linux): Program received signal SIGSEGV, Segmentation fault. 0x000000000046d177 in _PyGen_yf (gen=gen at entry=0x7ffff34bdaf8) at Objects/genobject.c:361 361 Py_INCREF(yf); (gdb) bt #0 0x000000000046d177 in _PyGen_yf (gen=gen at entry=0x7ffff34bdaf8) at Objects/genobject.c:361 #1 0x000000000052f49c in _PyEval_EvalFrameDefault (f=0x7ffff67067d8, throwflag=) at Python/ceval.c:1992 #2 0x000000000052a0fc in PyEval_EvalFrameEx (f=f at entry=0x7ffff67067d8, throwflag=throwflag at entry=0) at Python/ceval.c:718 #3 0x000000000046d393 in gen_send_ex (gen=gen at entry=0x7ffff34bdc08, arg=, exc=exc at entry=0, closing=closing at entry=0) at Objects/genobject.c:189 #4 0x000000000046de8d in _PyGen_Send (gen=gen at entry=0x7ffff34bdc08, arg=) at Objects/genobject.c:308 #5 0x00007ffff384ba2c in task_step_impl (task=task at entry=0x7ffff3263bd8, exc=exc at entry=0x0) at (...)/Python-3.6.0b4/Modules/_asynciomodule.c:1963 #6 0x00007ffff384c72e in task_step (task=0x7ffff3263bd8, exc=0x0) at (...)/Python-3.6.0b4/Modules/_asynciomodule.c:2247 #7 0x00007ffff384ca79 in task_call_step (arg=, task=) at (...)/Python-3.6.0b4/Modules/_asynciomodule.c:1848 #8 TaskSendMethWrapper_call (o=, args=, kwds=) at (...)/Python-3.6.0b4/Modules/_asynciomodule.c:1167 #9 0x0000000000446702 in PyObject_Call (func=0x7ffff37d7f60, args=0x7ffff7fb8058, kwargs=0x0) at Objects/abstract.c:2246 #10 0x00000000005295c8 in do_call_core (func=func at entry=0x7ffff37d7f60, callargs=callargs at entry=0x7ffff7fb8058, kwdict=kwdict at entry=0x0) at Python/ceval.c:5054 #11 0x0000000000534c64 in _PyEval_EvalFrameDefault (f=0xb4cb48, throwflag=) at Python/ceval.c:3354 #12 0x000000000052a0fc in PyEval_EvalFrameEx (f=f at entry=0xb4cb48, throwflag=throwflag at entry=0) at Python/ceval.c:718 #13 0x000000000052a1cc in _PyFunction_FastCall (co=, args=0xb4c5b0, nargs=nargs at entry=1, globals=) at Python/ceval.c:4867 (...) ---------- components: Interpreter Core files: test.py messages: 281573 nosy: martius, yselivanov priority: normal severity: normal status: open title: SEGFAULT when running a given coroutine type: crash versions: Python 3.6, Python 3.7 Added file: http://bugs.python.org/file45612/test.py _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed Nov 23 13:29:44 2016 From: report at bugs.python.org (Steve Dower) Date: Wed, 23 Nov 2016 18:29:44 +0000 Subject: [New-bugs-announce] [issue28783] Embedded/nuget packages incorrectly reference bdist_wininst Message-ID: <1479925784.02.0.834073933599.issue28783@psf.upfronthosting.co.za> New submission from Steve Dower: For the embedded zip and nuget releases, I've omitted bdist_wininst. However, I forgot to remove the reference to it from distutils.command.__init__.__all__, and so setuptools will occasionally crash trying to import it. This patch updates Tools/msi/make_zip.py to exclude the default distutils.command.__init__ and substitute a copy that does not have the added reference in it. Doing something "smarter" in the actual Lib/distutils/command/__init__.py isn't really an option. The only reliable option is to import the module eagerly, which I don't want to do. Including bdist_wininst in these packages is also a bad option (I considered completely removing distutils from them, but it has *some* uses - can't say the same for bdist_wininst though). Ned - I'd like to apply this change to 3.6. The patch is against 3.5, and there may be some subtle differences when I merge it forward (not every change to make_zip.py has been backported), but the intent is well captured here. If this is not applied to 3.6, then the packages will not be reliably usable with pip/setuptools, which is one of the intended uses (particularly for the nuget package). ---------- assignee: steve.dower components: Windows files: no_bdist_wininst.patch keywords: patch messages: 281576 nosy: ned.deily, paul.moore, steve.dower, tim.golden, zach.ware priority: normal severity: normal stage: patch review status: open title: Embedded/nuget packages incorrectly reference bdist_wininst type: behavior versions: Python 3.5, Python 3.6, Python 3.7 Added file: http://bugs.python.org/file45613/no_bdist_wininst.patch _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu Nov 24 02:44:10 2016 From: report at bugs.python.org (Evan) Date: Thu, 24 Nov 2016 07:44:10 +0000 Subject: [New-bugs-announce] [issue28784] shlex.shlex punctuation_chars documentation should use posix=True Message-ID: <1479973450.22.0.44978748963.issue28784@psf.upfronthosting.co.za> New submission from Evan: (This discussion started on issue28595.) The new punctuation_chars keyword argument is intended to provide "compatibility with the parsing performed by common Unix shells like bash, dash, and sh", however the documentation and examples do not mention that the user should also set posix=True (which defaults to False for shlex.shlex but True for shlex.split). Longer term (over several releases), perhaps the default for posix could be changed from False to True. Alternatively, the punctuation_chars argument could also be added to shlex.split, which would avoid having to interact with shlex.shlex directly. ---------- assignee: docs at python components: Documentation messages: 281612 nosy: docs at python, evan_, r.david.murray, vinay.sajip priority: normal severity: normal status: open title: shlex.shlex punctuation_chars documentation should use posix=True type: behavior versions: Python 3.6 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu Nov 24 03:14:24 2016 From: report at bugs.python.org (Max) Date: Thu, 24 Nov 2016 08:14:24 +0000 Subject: [New-bugs-announce] [issue28785] Clarify the behavior of NotImplemented Message-ID: <1479975264.5.0.247033798803.issue28785@psf.upfronthosting.co.za> New submission from Max: Currently, there's no clear statement as to what exactly the fallback is in case `__eq__` returns `NotImplemented`. It would be good to clarify the behavior of `NotImplemented`; at least for `__eq__`, but perhaps also other rich comparison methods. For example: "When `NotImplemented` is returned from a rich comparison method, the interpreter behaves as if the rich comparison method was not defined in the first place." See http://stackoverflow.com/questions/40780004/returning-notimplemented-from-eq for more discussion. ---------- assignee: docs at python components: Documentation messages: 281616 nosy: docs at python, max priority: normal severity: normal status: open title: Clarify the behavior of NotImplemented type: enhancement versions: Python 3.6 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu Nov 24 03:42:06 2016 From: report at bugs.python.org (SilentGhost) Date: Thu, 24 Nov 2016 08:42:06 +0000 Subject: [New-bugs-announce] [issue28786] Warnings in Misc/NEWS Message-ID: <1479976926.46.0.685904317893.issue28786@psf.upfronthosting.co.za> New submission from SilentGhost: Couple of entries in Misc/NEWS are causing warnings due to use of double quotes instead of backticks for code segments. This patches fixes the formatting. ---------- assignee: docs at python components: Documentation files: escape.diff keywords: patch messages: 281620 nosy: SilentGhost, docs at python priority: normal severity: normal stage: patch review status: open title: Warnings in Misc/NEWS type: behavior versions: Python 3.7 Added file: http://bugs.python.org/file45620/escape.diff _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu Nov 24 05:10:45 2016 From: report at bugs.python.org (Charalampos Stratakis) Date: Thu, 24 Nov 2016 10:10:45 +0000 Subject: [New-bugs-announce] [issue28787] Out of tree --with--dtrace builds fail with a traceback Message-ID: <1479982245.65.0.843842109576.issue28787@psf.upfronthosting.co.za> New submission from Charalampos Stratakis: By invoking an out of tree build of python with the --with-dtrace flag enabled, make fails with an error. Create a new folder at the source directory: $ mkdir _build && cd _build $ ../configure --with-dtrace $ make /usr/bin/dtrace -o Include/pydtrace_probes.h -h -s ../Include/pydtrace.d Traceback (most recent call last): File "/usr/bin/dtrace", line 440, in sys.exit(main()) File "/usr/bin/dtrace", line 385, in main providers.probe_write(s_filename, filename + suffix) File "/usr/bin/dtrace", line 181, in probe_write hdr = open(header, mode='w') FileNotFoundError: [Errno 2] No such file or directory: 'Include/pydtrace_probes.h' Makefile:896: recipe for target 'Include/pydtrace_probes.h' failed make: *** [Include/pydtrace_probes.h] Error 1 This is because the Include directory doesn't exist. Attaching a patch to fix this. ---------- components: Build files: create-Include-dir-to-properly-generate-pydtrace_probes.h-file.patch keywords: patch messages: 281627 nosy: cstratak priority: normal severity: normal status: open title: Out of tree --with--dtrace builds fail with a traceback versions: Python 3.6, Python 3.7 Added file: http://bugs.python.org/file45622/create-Include-dir-to-properly-generate-pydtrace_probes.h-file.patch _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu Nov 24 10:01:58 2016 From: report at bugs.python.org (George Fischhof) Date: Thu, 24 Nov 2016 15:01:58 +0000 Subject: [New-bugs-announce] [issue28788] Feature request: ConfigParser should be able to write config to a given filename, not only into file object Message-ID: <1479999718.88.0.148590995726.issue28788@psf.upfronthosting.co.za> New submission from George Fischhof: Hi There, I started to use ConfigParser, and found that it has no write to file_name option, but xml paarser (ElementTree) has. This way ConfigParser works different than xml parsers, as when I use elementtree.write it will create a file with full and valid xml content. But ConfigParser is "able" ;-) to create file with invalid content (for example (this was my findings) creates duplicated sections. Because the handling of the file is in the user's hand. So it would be good for ConfigParser to handle file writing and the user!s code would became simplier: Feature request: ConfigParser should be able to write config to a given filename, not only into file object. (Like xml parser) Kind regards, George Fischhof ---------- components: Library (Lib) messages: 281636 nosy: georgefischhof priority: normal severity: normal status: open title: Feature request: ConfigParser should be able to write config to a given filename, not only into file object type: enhancement versions: Python 3.5 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu Nov 24 11:32:00 2016 From: report at bugs.python.org (Mathieu Duponchelle) Date: Thu, 24 Nov 2016 16:32:00 +0000 Subject: [New-bugs-announce] [issue28789] valgrind shows "invalid file descriptor" when calling platform.system() on my machine. Message-ID: <1480005120.89.0.855588033442.issue28789@psf.upfronthosting.co.za> New submission from Mathieu Duponchelle: I'm using Fedora 23. ?meh???~???1???valgrind python3 -c "import platform; print(platform.system())" ==10093== Memcheck, a memory error detector ==10093== Copyright (C) 2002-2015, and GNU GPL'd, by Julian Seward et al. ==10093== Using Valgrind-3.11.0 and LibVEX; rerun with -h for copyright info ==10093== Command: python3 -c import\ platform;\ print(platform.system()) ==10093== ==10094== Warning: invalid file descriptor 1024 in syscall close() ==10094== Warning: invalid file descriptor 1025 in syscall close() ==10094== Warning: invalid file descriptor 1026 in syscall close() ==10094== Warning: invalid file descriptor 1027 in syscall close() ==10094== Use --log-fd= to select an alternative log fd. ==10094== Warning: invalid file descriptor 1028 in syscall close() ==10094== Warning: invalid file descriptor 1029 in syscall close() Linux ==10093== ==10093== HEAP SUMMARY: ==10093== in use at exit: 861,318 bytes in 5,903 blocks ==10093== total heap usage: 71,738 allocs, 65,835 frees, 10,611,196 bytes allocated ==10093== ==10093== LEAK SUMMARY: ==10093== definitely lost: 0 bytes in 0 blocks ==10093== indirectly lost: 0 bytes in 0 blocks ==10093== possibly lost: 326,619 bytes in 1,420 blocks ==10093== still reachable: 534,699 bytes in 4,483 blocks ==10093== suppressed: 0 bytes in 0 blocks ==10093== Rerun with --leak-check=full to see details of leaked memory ==10093== ==10093== For counts of detected and suppressed errors, rerun with: -v ==10093== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0) ?meh???~???1???python3 --version Python 3.4.3 ?meh???~???1??? ---------- components: Library (Lib) messages: 281640 nosy: Mathieu_Du priority: normal severity: normal status: open title: valgrind shows "invalid file descriptor" when calling platform.system() on my machine. versions: Python 3.4 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu Nov 24 15:04:07 2016 From: report at bugs.python.org (Guido van Rossum) Date: Thu, 24 Nov 2016 20:04:07 +0000 Subject: [New-bugs-announce] [issue28790] Error when using Generic and __slots__ Message-ID: <1480017847.05.0.904574090231.issue28790@psf.upfronthosting.co.za> New submission from Guido van Rossum: Proxy for https://github.com/python/typing/issues/332 (issue) and https://github.com/python/typing/pull/334 (fix). """ from typing import Generic, TypeVar class C(Generic[TypeVar('T')]): __slots__ = ('potato',) # ValueError: 'potato' in __slots__ conflicts with class variable C[int] This is because bare C is created with a class member descriptor potato, and instantiating it tries to create a class with all the attributes in C. """ @ned, I'll leave it up to you whether this is of sufficient severity to put in 3.6rc1 or not. Do I need to attach the fix as a diff to this bug? ---------- assignee: gvanrossum messages: 281649 nosy: gvanrossum, ned.deily priority: normal severity: normal status: open title: Error when using Generic and __slots__ versions: Python 3.5, Python 3.6, Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu Nov 24 15:05:28 2016 From: report at bugs.python.org (Big Stone) Date: Thu, 24 Nov 2016 20:05:28 +0000 Subject: [New-bugs-announce] [issue28791] update sqlite to 3.14.2 Message-ID: <1480017928.89.0.263397799958.issue28791@psf.upfronthosting.co.za> New submission from Big Stone: I fear it's too late for Python-3.6, but sqlite-3.15.1 fixes an old annoying bug (hidden since 3.8.0), and arriving tomorrow (nov.24th) sqlite-3.15.2 should be the well ".2" stabilized version of recent 3.15 improvements towards SQL standard (support for row values) If there is no special bugs left in Python-3.6.0b4 and everybody is happy, I would suggest you upgrade from sqlite-3.14.2 to sqlite-3.15.2 ---------- messages: 281650 nosy: Big Stone priority: normal severity: normal status: open title: update sqlite to 3.14.2 versions: Python 3.6, Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu Nov 24 16:07:47 2016 From: report at bugs.python.org (STINNER Victor) Date: Thu, 24 Nov 2016 21:07:47 +0000 Subject: [New-bugs-announce] [issue28792] bisect: implement aliases in Python, remove C aliases Message-ID: <1480021667.81.0.489541330256.issue28792@psf.upfronthosting.co.za> New submission from STINNER Victor: Attached patch simplifies the _bisect module: remove _bisect.bisect and _bisect.insort aliases. Aliases are created in Lib/bisect.py. I wrote the patch to prepare the C code for Argument Clinic, see issue #28754. Note: Lib/test/test_bisect.py already contains two unit tests: def test_backcompatibility(self): self.assertEqual(self.module.bisect, self.module.bisect_right) def test_backcompatibility(self): self.assertEqual(self.module.insort, self.module.insort_right) ---------- files: bisect.patch keywords: patch messages: 281651 nosy: haypo, larry, mdk, rhettinger priority: normal severity: normal status: open title: bisect: implement aliases in Python, remove C aliases versions: Python 3.7 Added file: http://bugs.python.org/file45627/bisect.patch _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu Nov 24 17:01:06 2016 From: report at bugs.python.org (Nathaniel Smith) Date: Thu, 24 Nov 2016 22:01:06 +0000 Subject: [New-bugs-announce] [issue28793] Copy-paste error in collections.abc docs for AsyncGenerator Message-ID: <1480024866.24.0.433904664953.issue28793@psf.upfronthosting.co.za> New submission from Nathaniel Smith: There's a small copy-paste error in the docs for collections.abc.AsyncGenerator -- it's called collections.abc.Generator: "class collections.abc.Generator ABC for asynchronous generator classes that implement the protocol defined in PEP 525 and PEP 492. New in version 3.6. " I can't link directly to it, because sphinx is confused about there being multiple definitions for collections.abc.Generator, but this link is close: https://docs.python.org/3.7/library/collections.abc.html#collections.abc.AsyncIterator ---------- messages: 281661 nosy: njs, yselivanov priority: normal severity: normal status: open title: Copy-paste error in collections.abc docs for AsyncGenerator versions: Python 3.6, Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu Nov 24 17:10:04 2016 From: report at bugs.python.org (Nathaniel Smith) Date: Thu, 24 Nov 2016 22:10:04 +0000 Subject: [New-bugs-announce] [issue28794] inspect.isasyncgen and inspect.isasyncgenfunction aren't documented Message-ID: <1480025404.72.0.713291262903.issue28794@psf.upfronthosting.co.za> New submission from Nathaniel Smith: The string "isasync" does not appear to occur on this page: https://docs.python.org/3.6/library/inspect.html ---------- assignee: docs at python components: Documentation messages: 281662 nosy: docs at python, njs, yselivanov priority: normal severity: normal status: open title: inspect.isasyncgen and inspect.isasyncgenfunction aren't documented versions: Python 3.6, Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu Nov 24 17:25:57 2016 From: report at bugs.python.org (=?utf-8?q?Jan_Veleck=C3=BD?=) Date: Thu, 24 Nov 2016 22:25:57 +0000 Subject: [New-bugs-announce] [issue28795] Misleading stating, that SIGINT handler is installed by default Message-ID: <1480026357.5.0.735730414686.issue28795@psf.upfronthosting.co.za> New submission from Jan Veleck?: Hello, documentation (https://docs.python.org/2/library/signal.html) states, that Python by default installs SIGINT handler which cause KeyboardInterrupt. This is not true everytime according to implementation. "Python installs a small number of signal handlers by default: SIGPIPE ... and SIGINT is translated into a KeyboardInterrupt exception." It should also mention this "SIGINT is installed only, when default handler is set at startup.". Because user can run python script from non-interative shell as background task and SIGINT handler is not installed, regardless although user does not change Python default behaviour. Related SO: http://stackoverflow.com/questions/40775054/capturing-sigint-using-keyboardinterrupt-exception-works-in-terminal-not-in-scr/40785230?noredirect=1 ---------- assignee: docs at python components: Documentation messages: 281664 nosy: Jan Veleck?, docs at python priority: normal severity: normal status: open title: Misleading stating, that SIGINT handler is installed by default type: behavior versions: Python 2.7, Python 3.3, Python 3.4, Python 3.5, Python 3.6, Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu Nov 24 19:43:58 2016 From: report at bugs.python.org (Julien Palard) Date: Fri, 25 Nov 2016 00:43:58 +0000 Subject: [New-bugs-announce] [issue28796] FIX warnings in documentation build Message-ID: <1480034638.89.0.480412503325.issue28796@psf.upfronthosting.co.za> New submission from Julien Palard: Just a little patch to fix 4 doc build warning. ---------- messages: 281673 nosy: mdk priority: normal severity: normal status: open title: FIX warnings in documentation build _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu Nov 24 21:48:29 2016 From: report at bugs.python.org (Wombatz) Date: Fri, 25 Nov 2016 02:48:29 +0000 Subject: [New-bugs-announce] [issue28797] Modifying class __dict__ inside __set_name__ Message-ID: <1480042109.13.0.341330811366.issue28797@psf.upfronthosting.co.za> New submission from Wombatz: This behavior occurs at least in python-3.6.0b3 and python-3.6.0b4. Modifying the class __dict__ inside the __set_name__ method of a descriptor that is used inside that class can lead to a bug that possibly prevents other descriptors from being initialized (the call to their __set_name__ is prevented). That happens because internally the cpython interpreter iterates the class __dict__ when calling all the __set_name__ methods. Changing the class __dict__ while iterating it leads to undefined behavior. This is the line in the source code https://github.com/python/cpython/blob/master/Objects/typeobject.c#L7010 See the attached file for an example where the bug can be observed. It defines a "desc" class that implements the __set_name__ method where it prints the name and modifies the class __dict__ of the containing class. Later a class is defined that has multiple instances of the "desc" class as class attributes. Depending on the number of attributes not all of them are initialized. When you see the underlying C-Code the reason for this behavior is obvious but in python itself the problem might not be apparent. To fix this bug the class __dict__ could be cashed and then the __set_name__ methods could be called while iterating over that copy. ---------- components: Interpreter Core files: reproduce.py messages: 281674 nosy: Wombatz priority: normal severity: normal status: open title: Modifying class __dict__ inside __set_name__ type: behavior versions: Python 3.6 Added file: http://bugs.python.org/file45632/reproduce.py _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu Nov 24 23:29:35 2016 From: report at bugs.python.org (Liang-Bo Wang) Date: Fri, 25 Nov 2016 04:29:35 +0000 Subject: [New-bugs-announce] [issue28798] Describe character as a string of length one instead of size one in tutorial Message-ID: <1480048175.03.0.163854418884.issue28798@psf.upfronthosting.co.za> New submission from Liang-Bo Wang: In Tutorial "An Informal Introduction" section, character is introduced as a string of size one. It may be true for python 2.x, where str can be interpreted as bytes. However in Python 3, strings are unicode thus the size and the length of a string are usually not the same. Also, using size of a string can be confused with the result returned by sys.getsizeof(mystr), where sys.getsizeof('a') != 1. Therefore using "a string of length one" to describe a character may be less confusing. The patch attached changes the wording. ---------- assignee: docs at python components: Documentation files: docs_tutorial_introduction_str.diff keywords: patch messages: 281678 nosy: ccwang002, docs at python priority: normal severity: normal status: open title: Describe character as a string of length one instead of size one in tutorial versions: Python 3.5, Python 3.6, Python 3.7 Added file: http://bugs.python.org/file45634/docs_tutorial_introduction_str.diff _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Nov 25 04:02:53 2016 From: report at bugs.python.org (STINNER Victor) Date: Fri, 25 Nov 2016 09:02:53 +0000 Subject: [New-bugs-announce] [issue28799] Drop CALL_PROFILE special build? Message-ID: <1480064573.69.0.905508550534.issue28799@psf.upfronthosting.co.za> New submission from STINNER Victor: Python/ceval.c contains conditional code to compute statistics on function calls when CALL_PROFILE is defined. Extract of Misc/SpecialBuilds.txt: CALL_PROFILE ------------ Count the number of function calls executed. When this symbol is defined, the ceval mainloop and helper functions count the number of function calls made. It keeps detailed statistics about what kind of object was called and whether the call hit any of the special fast paths in the code. Statistics can later be collected by sys.callstats(). I'm unable to find any unit test on this feature. The feature was added in Python 2.3.1 by the changeset 16856c9514e0 in 2003: --- changeset: 27712:16856c9514e0 branch: legacy-trunk user: Jeremy Hylton date: Wed Feb 05 23:13:00 2003 +0000 files: Include/ceval.h Include/compile.h Misc/SpecialBuilds.txt Python/ceval.c Python/compile.c Python/sysmodule.c description: Small function call optimization and special build option for call stats. -DCALL_PROFILE: Count the number of function calls executed. When this symbol is defined, the ceval mainloop and helper functions count the number of function calls made. It keeps detailed statistics about what kind of object was called and whether the call hit any of the special fast paths in the code. Optimization: When we take the fast_function() path, which seems to be taken for most function calls, and there is minimal frame setup to do, avoid call PyEval_EvalCodeEx(). The eval code ex function does a lot of work to handle keywords args and star args, free variables, generators, etc. The inlined version simply allocates the frame and copies the arguments values into the frame. The optimization gets a little help from compile.c which adds a CO_NOFREE flag to code objects that don't have free variables or cell variables. This change allows fast_function() to get into the fast path with fewer tests. I measure a couple of percent speedup in pystone with this change, but there's surely more that can be done. --- The changeset adds an optimization using CO_NOFREE and the CALL_PROFILE feature. My problem is that with my work on FASTCALL, it became harder to track where the functions are called in practice. It maybe out of the Python/ceval.c file. I'm not sure that statistics are still computed correctly after my FASTCALL changes, and I don't know how to check it. Python has already sys.setprofile(), cProfile and profile modules. There is also sys.settrace(). Do we still need CALL_PROFILE? Attached patch removes the feature: * Calling the the untested and undocumented sys.callstats() function now emits a DeprecationWarning warning * Remove the PyEval_GetCallStats() function and its documentation PyEval_GetCallStats() seems to be part of the stable API, but I don't expect that anyone uses it outside the CPython source code since it requires to rebuild CPython with a special build flag (-D CALL_PROFILE). ---------- messages: 281687 nosy: haypo priority: normal severity: normal status: open title: Drop CALL_PROFILE special build? versions: Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Nov 25 05:55:15 2016 From: report at bugs.python.org (STINNER Victor) Date: Fri, 25 Nov 2016 10:55:15 +0000 Subject: [New-bugs-announce] [issue28800] Add RETURN_NONE Message-ID: <1480071315.87.0.448012344283.issue28800@psf.upfronthosting.co.za> New submission from STINNER Victor: Attached patch adds a new bytecode instruction: RETURN_NONE. It is similar to "LOAD_CONST; RETURN_VALUE" but it avoids the need of adding None to constants of the code object. For function with an empty body, it reduces the stack size from 1 to 0. Example on reference Python 3.7: >>> def func(): return ... >>> import dis; dis.dis(func) 1 0 LOAD_CONST 0 (None) 2 RETURN_VALUE >>> func.__code__.co_stacksize 1 Example on patched Python 3.7: >>> def func(): return ... >>> import dis; dis.dis(func) 1 0 RETURN_CONST >>> func.__code__.co_stacksize 0 If the function has a docstring, RETURN_CONST avoids adding None to code constants: >>> def func(): ... "docstring" ... return ... >>> func.__code__.co_consts ('docstring',) I will now run benchmarks on the patch. ---------- files: return_none.patch keywords: patch messages: 281696 nosy: haypo, matrixise, serhiy.storchaka priority: normal severity: normal status: open title: Add RETURN_NONE versions: Python 3.7 Added file: http://bugs.python.org/file45640/return_none.patch _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Nov 25 05:55:51 2016 From: report at bugs.python.org (Richard Prosser) Date: Fri, 25 Nov 2016 10:55:51 +0000 Subject: [New-bugs-announce] [issue28801] configparser: before_get() method of class Interpolation has positional 'parser' parameter that is not used. Message-ID: <1480071351.83.0.49150116079.issue28801@psf.upfronthosting.co.za> New submission from Richard Prosser: >From https://hg.python.org/cpython/file/3.5/Lib/configparser.py (for example): 358class Interpolation: 359 """Dummy interpolation that passes the value through with no changes.""" 360 361 def before_get(self, parser, section, option, value, defaults): 362 return value but a typical invocation misses out the 'parser' parameter: 796 return self._interpolation.before_get(self, section, option, value, 797 d) As far as I can see, this is not a keyword-only parameter, yet PyCharm seems to treat it as one. So maybe this is some new behaviour that I don't understand yet but there seems to be a fault here, IMO. I am using Python 3.5.2 on Windows 7, after using the 'futurize' tool on some legacy 2.7 code that extended self._interpolate (which no longer exists in 3+). ---------- hgrepos: 362 messages: 281697 nosy: rprosser priority: normal severity: normal status: open title: configparser: before_get() method of class Interpolation has positional 'parser' parameter that is not used. type: compile error versions: Python 3.5 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Nov 25 09:36:18 2016 From: report at bugs.python.org (Richard s. Gordon) Date: Fri, 25 Nov 2016 14:36:18 +0000 Subject: [New-bugs-announce] [issue28802] Python 3.6.0b4 Reports ncurses present in Cygwin but fails to build for cygwin Message-ID: <1480084578.87.0.616449400889.issue28802@psf.upfronthosting.co.za> Changes by Richard s. Gordon : ---------- nosy: eclectic9509 priority: normal severity: normal status: open title: Python 3.6.0b4 Reports ncurses present in Cygwin but fails to build for cygwin _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Nov 25 09:42:05 2016 From: report at bugs.python.org (Richard S. Gordon) Date: Fri, 25 Nov 2016 14:42:05 +0000 Subject: [New-bugs-announce] [issue28803] Python 3.6.0b4 Reports ncurses present in Cygwin but fails to build for Cygwin Message-ID: <1480084925.94.0.108029880914.issue28803@psf.upfronthosting.co.za> Changes by Richard S. Gordon : ---------- components: Build nosy: rigordo priority: normal severity: normal status: open title: Python 3.6.0b4 Reports ncurses present in Cygwin but fails to build for Cygwin versions: Python 3.6 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Nov 25 23:00:09 2016 From: report at bugs.python.org (liugang) Date: Sat, 26 Nov 2016 04:00:09 +0000 Subject: [New-bugs-announce] [issue28804] file tell() report incorrect file position (Windows, Linux is OK) Message-ID: <1480132809.99.0.636633604449.issue28804@psf.upfronthosting.co.za> New submission from liugang: D:\Python35\python.exe "D:\PyCharm Community Edition 2016.3\helpers\pydev\pydevconsole.py" 8871 8872 PyDev console: starting. import sys; print('Python %s on %s' % (sys.version, sys.platform)) sys.path.extend(['C:\\Users\\liugang10\\PycharmProjects\\untitled', 'C:/Users/liugang10/PycharmProjects/untitled']) Python 3.5.2 (v3.5.2:4def2a2901a5, Jun 25 2016, 22:18:55) [MSC v.1900 64 bit (AMD64)] on win32 >>> f = open('e:\\1.txt') >>> f.readline() '1\n' >>> f.tell() 340282367000166625996085689099021713410 >>> f.readline() '2\n' >>> f.tell() 340282367000166625996085689099021713412 ---------- assignee: terry.reedy components: IDLE files: 1.txt messages: 281752 nosy: liugang10, terry.reedy priority: normal severity: normal status: open title: file tell() report incorrect file position (Windows, Linux is OK) type: behavior versions: Python 3.5 Added file: http://bugs.python.org/file45648/1.txt _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sat Nov 26 06:08:54 2016 From: report at bugs.python.org (Stefan Krah) Date: Sat, 26 Nov 2016 11:08:54 +0000 Subject: [New-bugs-announce] [issue28805] Add documentation for METH_FASTCALL Message-ID: <1480158534.61.0.510309261592.issue28805@psf.upfronthosting.co.za> New submission from Stefan Krah: It looks like METH_FASTCALL gives nice speedups (#28754). Is this an internal interface or can it be documented like METH_KEYWORDS etc.? ---------- assignee: docs at python components: Documentation messages: 281766 nosy: docs at python, haypo, serhiy.storchaka, skrah priority: normal severity: normal stage: needs patch status: open title: Add documentation for METH_FASTCALL type: enhancement versions: Python 3.6 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sat Nov 26 06:13:30 2016 From: report at bugs.python.org (Xiang Zhang) Date: Sat, 26 Nov 2016 11:13:30 +0000 Subject: [New-bugs-announce] [issue28806] Improve the netrc library Message-ID: <1480158810.39.0.736994382472.issue28806@psf.upfronthosting.co.za> New submission from Xiang Zhang: netrc library now gets some problems: 1. All tokens are mandatory. (#28780) 2. Token values are limited to a limited set of characters. (#557704) 3. Does not complain about macro definition without a null line. 4. If the login name is anonymous, security check is not needed. Propose a patch to handle these. I treat it as an enhancement of the existing library instead of bug fix. ---------- components: Library (Lib) files: netrc.patch keywords: patch messages: 281767 nosy: haypo, xiang.zhang priority: normal severity: normal stage: patch review status: open title: Improve the netrc library type: enhancement versions: Python 3.7 Added file: http://bugs.python.org/file45651/netrc.patch _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sat Nov 26 08:17:32 2016 From: report at bugs.python.org (Oskar Skog) Date: Sat, 26 Nov 2016 13:17:32 +0000 Subject: [New-bugs-announce] [issue28807] [NetBSD] interpreter hangs on exit after call to subprocess.Popen() Message-ID: <1480166252.16.0.187923291691.issue28807@psf.upfronthosting.co.za> New submission from Oskar Skog: This is a platform specific bug. Only noticed this on NetBSD 6.1 x86-32 on a VirtualBox machine. To me, the patches for NetBSD do not look relevant to subprocess or core stuff. http://cvsweb.netbsd.org/bsdweb.cgi/pkgsrc/lang/python27/patches/?only_with_tag=MAIN Test: #!/usr/pkg/bin/python import subprocess subprocess.Popen(['true']) print('Program finished.') # END OF FILE On an unaffected platform it will print "Program finished." and exit, on an affected platform it will print "Program finished." and hang. ---------- components: Interpreter Core messages: 281773 nosy: oskog97 priority: normal severity: normal status: open title: [NetBSD] interpreter hangs on exit after call to subprocess.Popen() type: behavior versions: Python 2.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sat Nov 26 10:48:34 2016 From: report at bugs.python.org (Serhiy Storchaka) Date: Sat, 26 Nov 2016 15:48:34 +0000 Subject: [New-bugs-announce] [issue28808] Make PyUnicode_CompareWithASCIIString() never failing Message-ID: <1480175314.71.0.35688135657.issue28808@psf.upfronthosting.co.za> New submission from Serhiy Storchaka: PyUnicode_CompareWithASCIIString() never set an exception in 3.2 and earlier versions. Since 3.3 it sets an exception and returns -1 if the first argument is not ready Unicode object, but this was not documented until issue28701. Due to undocumenting this behavior many (if not all) callers don't check whether it returned an error. Proposed patch restores old behavior of PyUnicode_CompareWithASCIIString(). ---------- components: Interpreter Core, Unicode files: PyUnicode_CompareWithASCIIString-no-errors.patch keywords: patch messages: 281783 nosy: ezio.melotti, haypo, serhiy.storchaka priority: normal severity: normal stage: patch review status: open title: Make PyUnicode_CompareWithASCIIString() never failing type: behavior versions: Python 3.5, Python 3.6, Python 3.7 Added file: http://bugs.python.org/file45655/PyUnicode_CompareWithASCIIString-no-errors.patch _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sat Nov 26 13:05:22 2016 From: report at bugs.python.org (Soren Solari) Date: Sat, 26 Nov 2016 18:05:22 +0000 Subject: [New-bugs-announce] [issue28809] mention asyncio.gather non-deterministic task starting order Message-ID: <1480183522.09.0.687431688906.issue28809@psf.upfronthosting.co.za> New submission from Soren Solari: https://github.com/python/asyncio/issues/432 asyncio.gather documentation states: "the returned future?s result is the list of results (in the order of the original sequence, not necessarily the order of results arrival)" An additional statement like "tasks are not guaranteed to be started in a predicable order" would aid users and slove the discussion in issue 432 above. ---------- assignee: docs at python components: Documentation messages: 281787 nosy: Soren Solari, docs at python priority: normal severity: normal status: open title: mention asyncio.gather non-deterministic task starting order type: enhancement versions: Python 3.5 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sat Nov 26 13:24:04 2016 From: report at bugs.python.org (Serhiy Storchaka) Date: Sat, 26 Nov 2016 18:24:04 +0000 Subject: [New-bugs-announce] [issue28810] Document bytecode changes in 3.6 Message-ID: <1480184644.53.0.660721117392.issue28810@psf.upfronthosting.co.za> New submission from Serhiy Storchaka: There are many bytecode changes in 3.6, but seems most of them are not documented (besides short line in _bootstrap_external.py). * The bytecode now uses 16 bit units (wordcode) (issue26647). * Added FORMAT_VALUE opcode (issue25483). * Added BUILD_CONST_KEY_MAP opcode (issue27140). * Added BUILD_STRING opcode (issue27078). * Added BUILD_TUPLE_UNPACK_WITH_CALL opcode (issue28257). * Added SETUP_ANNOTATIONS and STORE_ANNOTATION opcodes (issue27985). * Changed CALL_FUNCTION, CALL_FUNCTION_KW and BUILD_MAP_UNPACK_WITH_CALL opcodes, removed CALL_FUNCTION_VAR, CALL_FUNCTION_VAR_KW opcodes, added CALL_FUNCTION_EX opcode (issue27213). * Changed MAKE_FUNCTION opcode, removed MAKE_CLOSURE opcode (issue27095). * Not related to the bytecode itself: lineno delta of code.co_lnotab now is signed (issue26107). There are third-party projects that need correct information about bytecode changes. ---------- assignee: docs at python components: Documentation messages: 281788 nosy: docs at python, serhiy.storchaka priority: high severity: normal stage: needs patch status: open title: Document bytecode changes in 3.6 type: enhancement versions: Python 3.6, Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun Nov 27 04:33:13 2016 From: report at bugs.python.org (Ram Rachum) Date: Sun, 27 Nov 2016 09:33:13 +0000 Subject: [New-bugs-announce] [issue28811] Make pathlib.PurePath.__str__ use shlex.quote Message-ID: <1480239193.34.0.752695388399.issue28811@psf.upfronthosting.co.za> New submission from Ram Rachum: I have a a PurePath object like so: path = PurePath('/home/my awesome user/file.txt') I'm SSHing into a server and I want to remove the file. So I have to do this: ssh_client.run(f'/bin/rm {shlex.quote(str(path))}') Which is really long and ugly. (I might have been able to remove the str from there if #28623 wasn't rejected.) I wish I could do this: ssh_client.run(f'/bin/rm {path}') But since my path has a space, that would only be possible if PurePath.__str__ were to use shlex.quote, and put quotes around my path (only if it includes a space). What do you think about that? ---------- components: Library (Lib) messages: 281812 nosy: cool-RR, pitrou priority: normal severity: normal status: open title: Make pathlib.PurePath.__str__ use shlex.quote type: enhancement versions: Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun Nov 27 05:13:18 2016 From: report at bugs.python.org (INADA Naoki) Date: Sun, 27 Nov 2016 10:13:18 +0000 Subject: [New-bugs-announce] [issue28812] Deadlock between GIL and pystate head_mutex. Message-ID: <1480241598.29.0.901358230906.issue28812@psf.upfronthosting.co.za> New submission from INADA Naoki: While investigating #28673, I found another deadlock relating to finalization and threading. deadlocked thread (holding gil / waiting head_mutex): #0 0x00007f35dd400a07 in futex_abstimed_wait_cancelable (private=0, abstime=0x0, expected=0, futex_word=0x5557e6d3eeb0) at ../sysdeps/unix/sysv/linux/futex-internal.h:205 #1 do_futex_wait (sem=sem at entry=0x5557e6d3eeb0, abstime=0x0) at sem_waitcommon.c:111 #2 0x00007f35dd400ab4 in __new_sem_wait_slow (sem=0x5557e6d3eeb0, abstime=0x0) at sem_waitcommon.c:181 #3 0x00007f35dd400b5a in __new_sem_wait (sem=sem at entry=0x5557e6d3eeb0) at sem_wait.c:29 #4 0x00005557e64ec01b in PyThread_acquire_lock_timed (lock=0x5557e6d3eeb0, microseconds=-1, intr_flag=intr_flag at entry=0) at Python/thread_pthread.h:352 #5 0x00005557e64ec148 in PyThread_acquire_lock (lock=, waitflag=waitflag at entry=1) at Python/thread_pthread.h:556 #6 0x00005557e64d8dbb in tstate_delete_common (tstate=tstate at entry=0x7f35cc002a80) at Python/pystate.c:426 #7 0x00005557e64d9e8d in PyThreadState_DeleteCurrent () at Python/pystate.c:462 #8 0x00005557e662d339 in t_bootstrap (boot_raw=0x7f35dc186628) at ./Modules/_threadmodule.c:1027 #9 0x00007f35dd3f870a in start_thread (arg=0x7f35c3fff700) at pthread_create.c:333 #10 0x00007f35dca220af in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:105 main thread (waiting gil / holding head_mutex): #0 pthread_cond_timedwait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_timedwait.S:225 #1 0x00005557e65f1758 in PyCOND_TIMEDWAIT (us=5000, mut=0x5557e69cc860 , cond=0x5557e69cc8a0 ) at Python/condvar.h:103 #2 take_gil (tstate=tstate at entry=0x5557e6d3f840) at Python/ceval_gil.h:224 #3 0x00005557e65f3b14 in _PyEval_EvalFrameDefault (f=Frame 0x7f35d0001878, for file finish-deadlock.py, line 7, in __del__ (self=), throwflag=) at Python/ceval.c:1140 #4 0x00005557e65f1dcf in PyEval_EvalFrameEx (f=f at entry=Frame 0x7f35d0001878, for file finish-deadlock.py, line 7, in __del__ (self=), throwflag=throwflag at entry=0) at Python/ceval.c:718 #5 0x00005557e65f1eab in _PyFunction_FastCall (co=co at entry=0x7f35dc50eac0, args=0x7fff1a1e2878, args at entry=0x7fff1a1e2870, nargs=nargs at entry=1, globals=globals at entry={'__name__': '__main__', '__doc__': None, '__package__': None, '__loader__': , '__spec__': None, '__annotations__': {}, '__builtins__': , '__cached__': None, 'sys': , 'time': , 'threading': , 'C': , 'worker': , '_': 4, 't': , _name='Thread-11', _args=(), _kwargs={}, _daemonic=True, _ident=139869249451776, _tstate_lock=<_thread.lock at remote 0x7f35dc4739e0>, _started=, acquire=, release=, _waiters=) at remote 0x7f35dc191538>,...(truncated)) at Python/ceval.c:4881 #6 0x00005557e65feb04 in _PyFunction_FastCallDict (func=func at entry=, args=args at entry=0x7fff1a1e2870, nargs=nargs at entry=1, kwargs=kwargs at entry=0x0) at Python/ceval.c:4983 #7 0x00005557e6500386 in _PyObject_FastCallDict (func=func at entry=, args=args at entry=0x7fff1a1e2870, nargs=nargs at entry=1, kwargs=kwargs at entry=0x0) at Objects/abstract.c:2301 #8 0x00005557e6500644 in _PyObject_Call_Prepend (func=, obj=obj at entry=, args=args at entry=(), kwargs=kwargs at entry=0x0) at Objects/abstract.c:2364 #9 0x00005557e651c2e4 in method_call (method=method at entry=, args=args at entry=(), kwargs=kwargs at entry=0x0) at Objects/classobject.c:317 #10 0x00005557e6500247 in _PyObject_FastCallDict (func=func at entry=, args=args at entry=0x0, nargs=nargs at entry=0, kwargs=kwargs at entry=0x0) at Objects/abstract.c:2322 #11 0x00005557e65f34a2 in PyEval_CallObjectWithKeywords (func=func at entry=, args=args at entry=0x0, kwargs=kwargs at entry=0x0) at Python/ceval.c:4705 #12 0x00005557e6577f0d in slot_tp_finalize (self=) at Objects/typeobject.c:6416 #13 0x00005557e655a07c in PyObject_CallFinalizer (self=self at entry=) at Objects/object.c:297 #14 0x00005557e655adb7 in PyObject_CallFinalizerFromDealloc (self=self at entry=) at Objects/object.c:314 #15 0x00005557e656cfde in subtype_dealloc (self=self at entry=) at Objects/typeobject.c:1151 #16 0x00005557e655ae75 in _Py_Dealloc (op=) at Objects/object.c:1786 #17 0x00005557e65312c5 in frame_dealloc (f=f at entry=Frame 0x7f35c40026a8, for file finish-deadlock.py, line 22, in worker ()) at Objects/frameobject.c:423 #18 0x00005557e655ae75 in _Py_Dealloc (op=Frame 0x7f35c40026a8, for file finish-deadlock.py, line 22, in worker ()) at Objects/object.c:1786 #19 0x00005557e64e88c1 in tb_dealloc (tb=tb at entry=0x7f35dc193f38) at Python/traceback.c:55 #20 0x00005557e655ae75 in _Py_Dealloc (op=) at Objects/object.c:1786 #21 0x00005557e64d98f3 in PyThreadState_Clear (tstate=tstate at entry=0x5557e6e307f0) at Python/pystate.c:403 #22 0x00005557e64d99cf in PyInterpreterState_Clear (interp=interp at entry=0x5557e6d3f390) at Python/pystate.c:119 #23 0x00005557e64d7b0f in Py_FinalizeEx () at Python/pylifecycle.c:672 #24 0x00005557e64eed2f in Py_Main (argc=2, argv=0x5557e6d3e010) at Modules/main.c:800 #25 0x00005557e64d3ad5 in main (argc=2, argv=0x7fff1a1e2d78) at ./Programs/python.c:69 ---------- components: Interpreter Core files: finish-deadlock.py messages: 281816 nosy: inada.naoki priority: normal severity: normal status: open title: Deadlock between GIL and pystate head_mutex. versions: Python 2.7, Python 3.5, Python 3.6, Python 3.7 Added file: http://bugs.python.org/file45660/finish-deadlock.py _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun Nov 27 06:05:46 2016 From: report at bugs.python.org (Adrian Wielgosik) Date: Sun, 27 Nov 2016 11:05:46 +0000 Subject: [New-bugs-announce] [issue28813] Remove unneeded folded consts after peephole Message-ID: <1480244746.98.0.558504847162.issue28813@psf.upfronthosting.co.za> New submission from Adrian Wielgosik: The attached patch adds new logic to peephole compiler to remove constants that are no longer needed after the main peephole pass. For example: def f(): var = 'te' + 'xt' num = -12 num = -6 * 2 return (1, (3, 4), 6) print(f.__code__.co_consts) Without the patch: (None, 'te', 'xt', 12, 6, 2, 1, 3, 4, 'text', -12, -6, -12, (3, 4), (1, (3, 4), 6)) With patch: (None, 'text', -12, -12, (1, (3, 4), 6)) (unfortunately, I couldn't get rid of None because that would make 'text' a docstring) For convenience, I've written the patch in two parts. The first one just changes the CONST_STACK_* macros to store the co_const indices instead of the constants themselves, the second one is the actual implementation of the new logic. Aside from simply having to store less objects around, this also makes co_consts contents closer together. This may help the cache a little bit. --------- I did run benchmarks multiple times, but it looked like all the results were random noise. That makes sense, since I didn't directly affect the runtime. The only consistently faster benchmark is: - regex_dna: 288 ms +- 7 ms -> 275 ms +- 5 ms: 1.05x faster I tried to measure the difference in compile time, but it too was lost in the noise. --------- I also compared size of compiled .pyc files in the Lib/ directory. The gains are mostly very small. _compat_pickle.cpython-37.pyc | 6554 -> 5851 | -10.7% sre_compile.cpython-37.pyc | 10275 -> 10025 | -2.43% hashlib.cpython-37.pyc | 6624 -> 6514 | -1.66% pstats.cpython-37.pyc | 21755 -> 21435 | -1.47% _markupbase.cpython-37.pyc | 7979 -> 7864 | -1.44% pydoc.cpython-37.pyc | 83899 -> 82712 | -1.41% _strptime.cpython-37.pyc | 15951 -> 15751 | -1.25% __future__.cpython-37.pyc | 4155 -> 4105 | -1.2% opcode.cpython-37.pyc | 5401 -> 5341 | -1.11% colorsys.cpython-37.pyc | 3299 -> 3263 | -1.09% signal.cpython-37.pyc | 2503 -> 2478 | -0.999% _osx_support.cpython-37.pyc | 9663 -> 9568 | -0.983% gettext.cpython-37.pyc | 13990 -> 13854 | -0.972% getpass.cpython-37.pyc | 4223 -> 4183 | -0.947% compare.cpython-37.pyc | 541 -> 536 | -0.924% warnings.cpython-37.pyc | 13328 -> 13208 | -0.9% platform.cpython-37.pyc | 27931 -> 27681 | -0.895% imaplib.cpython-37.pyc | 42019 -> 41653 | -0.871% webbrowser.cpython-37.pyc | 15836 -> 15702 | -0.846% this.cpython-37.pyc | 1253 -> 1243 | -0.798% rlcompleter.cpython-37.pyc | 5768 -> 5723 | -0.78% zipfile.cpython-37.pyc | 48024 -> 47672 | -0.733% imghdr.cpython-37.pyc | 4138 -> 4108 | -0.725% turtle.cpython-37.pyc | 131600 -> 130653 | -0.72% timeit.cpython-37.pyc | 11676 -> 11596 | -0.685% lzma.cpython-37.pyc | 11980 -> 11900 | -0.668% bz2.cpython-37.pyc | 11270 -> 11195 | -0.665% aifc.cpython-37.pyc | 25821 -> 25651 | -0.658% gzip.cpython-37.pyc | 16215 -> 16110 | -0.648% uuid.cpython-37.pyc | 20382 -> 20260 | -0.599% plistlib.cpython-37.pyc | 27354 -> 27191 | -0.596% cProfile.cpython-37.pyc | 4199 -> 4174 | -0.595% tarfile.cpython-37.pyc | 62437 -> 62076 | -0.578% sysconfig.cpython-37.pyc | 15819 -> 15728 | -0.575% profile.cpython-37.pyc | 13889 -> 13814 | -0.54% random.cpython-37.pyc | 19177 -> 19074 | -0.537% _threading_local.cpython-37.pyc | 6609 -> 6574 | -0.53% _dummy_thread.cpython-37.pyc | 4839 -> 4814 | -0.517% datetime.cpython-37.pyc | 53722 -> 53445 | -0.516% tracemalloc.cpython-37.pyc | 17218 -> 17131 | -0.505% // remaining 129 files are < 0.5% smaller, 33 of them didn't change their size ---------- components: Interpreter Core messages: 281820 nosy: Adrian Wielgosik priority: normal severity: normal status: open title: Remove unneeded folded consts after peephole type: resource usage versions: Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun Nov 27 08:07:32 2016 From: report at bugs.python.org (Nick Coghlan) Date: Sun, 27 Nov 2016 13:07:32 +0000 Subject: [New-bugs-announce] [issue28814] Deprecation notice on inspect.getargvalues() is incorrect Message-ID: <1480252052.94.0.16018093739.issue28814@psf.upfronthosting.co.za> New submission from Nick Coghlan: inspect.getargvalues() and inspect.formatargvalues() were deprecated in Python 3.5 as part of implementing issue 20438 This is incorrect, as these are *frame* introspection related functions, not callable introspection ones. The documentation and implementation layout is confusing though, as they're interleaved with the callable introspection operations. ---------- assignee: docs at python components: Documentation messages: 281824 nosy: docs at python, ncoghlan, yselivanov priority: normal severity: normal stage: needs patch status: open title: Deprecation notice on inspect.getargvalues() is incorrect type: enhancement versions: Python 3.5, Python 3.6, Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun Nov 27 11:16:18 2016 From: report at bugs.python.org (patrila) Date: Sun, 27 Nov 2016 16:16:18 +0000 Subject: [New-bugs-announce] [issue28815] test_socket fails if /proc/modules is existent but not readable Message-ID: <1480263378.99.0.648069549511.issue28815@psf.upfronthosting.co.za> New submission from patrila: Dear Python developers The test_socket test fails if /proc/modules is existent but not readable by the user (this is for example the case with the grsecurity patchset of the kernel). The method reading /proc/modules is isTipcAvailable(), which is not a test but a guard for other tests. It seems reasonable to return False in the case that /proc/modules is not readable (but existent). The method isTipcAvailable() already returns False if /proc/modules is non existent (but fails to return False if it's not readable). Attached a proposed test. Feel free to remove the EISDIR in case you feel uncomfortable and want a it be a "real" error. The patch should be applied to both Python-2.7 and Python-3.x. Kind regards Here is the inline version of the patch; it's also attached. diff -r 876bee0bd0ba Lib/test/test_socket.py --- a/Lib/test/test_socket.py Sat Nov 26 14:04:40 2016 -0800 +++ b/Lib/test/test_socket.py Sun Nov 27 17:00:55 2016 +0100 @@ -4779,12 +4779,21 @@ """ if not hasattr(socket, "AF_TIPC"): return False - if not os.path.isfile("/proc/modules"): - return False - with open("/proc/modules") as f: - for line in f: - if line.startswith("tipc "): - return True + try: + f = open("/proc/modules") + except IOError as e: + # It's ok if the file does not exist, is a directory or if we + # have not the permission to read it. In any other case it's a + # real error, so raise it again. + if e.errno in (ENOENT, EISDIR, EACCES): + return False + else: + raise + else: + with f: + for line in f: + if line.startswith("tipc "): + return True return False @unittest.skipUnless(isTipcAvailable(), ---------- files: test_socket_isTipcAvailable_proc_modules.patch keywords: patch messages: 281828 nosy: patrila priority: normal severity: normal status: open title: test_socket fails if /proc/modules is existent but not readable versions: Python 2.7, Python 3.3, Python 3.4, Python 3.5, Python 3.6, Python 3.7 Added file: http://bugs.python.org/file45666/test_socket_isTipcAvailable_proc_modules.patch _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Nov 28 04:45:40 2016 From: report at bugs.python.org (Decorater) Date: Mon, 28 Nov 2016 09:45:40 +0000 Subject: [New-bugs-announce] [issue28816] Document if zipimport can respect import hooks to load custom files from zip. Message-ID: <1480326340.52.0.39566018579.issue28816@psf.upfronthosting.co.za> New submission from Decorater: I am wondering so lets say for example if I was to make a json import hook (code below): from importlib.machinery import FileFinder import json import sys class ExtensionImporter(object): """Base Importer Class.""" def __init__(self, extension_list): self.extensions = extension_list def find_loader(self, name, path): """Filds some loaders for importing the module.""" self.path = path return self def load_module(self, fullname): if fullname in sys.modules: return sys.modules[fullname] return None class JsonImporter(ExtensionImporter): """JSON importer Class.""" def __init__(self): super(JsonImporter, self).__init__(['.json']) def load_module(self, fullname): """Loads modules found with the extension""" premodule = super(JsonImporter, self).load_module(fullname) if premodule is None: with open(self.path, 'r') as f: module = json.load(f) sys.modules[fullname] = module return module raise ImportError('Couldn't open path') extension_importers = [JsonImporter()] hook_list = [] for importer in extension_importers: hook_list.append((importer.find_loader, importer.extensions)) sys.path_hooks.insert(0, FileFinder.path_hook(*hook_list)) sys.path_importer_cache.clear() What if I had those json files in a zip file and used ZipImport on that zip file would it use that import hook that I shared above if all other import methods fail to import those json files (obvously they would fail though)? There should be documentation to explain if it supports user created import hooks that they create using importlib. This is because I would be very happy if zipimport supports my own import hook if any other method of importing files from that zip file fails but mine ends up working. It also allows people to load up many other formats (provided they know the format) as if it was a python script. (of if they compressed it in a custom file like for example a RAR file that WinRAR can create. And yes I would support a RARImport for rar files. ---------- components: Interpreter Core, Windows messages: 281850 nosy: Decorater, paul.moore, steve.dower, tim.golden, zach.ware priority: normal severity: normal status: open title: Document if zipimport can respect import hooks to load custom files from zip. versions: Python 3.4, Python 3.5, Python 3.6, Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Nov 28 05:41:44 2016 From: report at bugs.python.org (Ram Rachum) Date: Mon, 28 Nov 2016 10:41:44 +0000 Subject: [New-bugs-announce] [issue28817] Provide method PurePath.quote() that calls shlex.quote Message-ID: <1480329704.42.0.830196974731.issue28817@psf.upfronthosting.co.za> New submission from Ram Rachum: I have a a PurePath object like so: path = PurePath('/home/my awesome user/file.txt') I'm SSHing into a server and I want to remove the file. So I have to do this: ssh_client.run(f'/bin/rm {shlex.quote(str(path))}') Which is really long and ugly. I suggested in issue28623 that shlex.quote could just take a Path argument, and in issue28811 that __str__ would use shlex.quote, and it was rejected too. My next suggestion: Implement PurePath.quote() method that calls shlex.quote() on the path. What do you think? ---------- components: Library (Lib) messages: 281858 nosy: cool-RR, pitrou priority: normal severity: normal status: open title: Provide method PurePath.quote() that calls shlex.quote type: enhancement versions: Python 3.6 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Nov 28 06:03:40 2016 From: report at bugs.python.org (INADA Naoki) Date: Mon, 28 Nov 2016 11:03:40 +0000 Subject: [New-bugs-announce] [issue28818] simplify lookdict functions Message-ID: <1480331020.89.0.515440713784.issue28818@psf.upfronthosting.co.za> New submission from INADA Naoki: This patch reduces indirect access to improve readability. benchmark: Slower (14): - logging_format: 29.9 us +- 2.7 us -> 31.2 us +- 2.8 us: 1.04x slower - scimark_monte_carlo: 227 ms +- 5 ms -> 235 ms +- 16 ms: 1.03x slower - dulwich_log: 149 ms +- 10 ms -> 153 ms +- 14 ms: 1.03x slower - json_dumps: 23.9 ms +- 1.5 ms -> 24.5 ms +- 2.4 ms: 1.03x slower - scimark_lu: 490 ms +- 12 ms -> 501 ms +- 33 ms: 1.02x slower - deltablue: 16.4 ms +- 0.6 ms -> 16.7 ms +- 0.6 ms: 1.02x slower - call_simple: 13.3 ms +- 0.5 ms -> 13.6 ms +- 0.5 ms: 1.02x slower - sqlalchemy_declarative: 323 ms +- 11 ms -> 330 ms +- 26 ms: 1.02x slower - regex_compile: 389 ms +- 12 ms -> 394 ms +- 28 ms: 1.01x slower - python_startup: 16.4 ms +- 0.5 ms -> 16.6 ms +- 0.9 ms: 1.01x slower - regex_dna: 284 ms +- 7 ms -> 286 ms +- 20 ms: 1.01x slower - pidigits: 297 ms +- 7 ms -> 299 ms +- 17 ms: 1.01x slower - raytrace: 1.18 sec +- 0.02 sec -> 1.18 sec +- 0.03 sec: 1.00x slower - python_startup_no_site: 9.96 ms +- 0.25 ms -> 9.98 ms +- 0.45 ms: 1.00x slower Faster (11): - pickle_dict: 65.7 us +- 1.4 us -> 63.1 us +- 2.0 us: 1.04x faster - call_method_unknown: 19.4 ms +- 1.1 ms -> 18.8 ms +- 0.9 ms: 1.03x faster - xml_etree_parse: 248 ms +- 11 ms -> 244 ms +- 14 ms: 1.02x faster - xml_etree_generate: 225 ms +- 13 ms -> 221 ms +- 10 ms: 1.02x faster - call_method_slots: 16.9 ms +- 0.8 ms -> 16.6 ms +- 0.7 ms: 1.02x faster - logging_silent: 725 ns +- 41 ns -> 715 ns +- 35 ns: 1.01x faster - xml_etree_iterparse: 210 ms +- 16 ms -> 208 ms +- 12 ms: 1.01x faster - json_loads: 53.7 us +- 3.4 us -> 53.0 us +- 1.7 us: 1.01x faster - call_method: 17.2 ms +- 0.8 ms -> 17.0 ms +- 0.6 ms: 1.01x faster - nbody: 227 ms +- 14 ms -> 226 ms +- 8 ms: 1.00x faster - pickle_pure_python: 1.07 ms +- 0.04 ms -> 1.07 ms +- 0.07 ms: 1.00x faster Benchmark hidden because not significant (39): 2to3, chameleon, chaos, crypto_pyaes, django_template, fannkuch, float, genshi_text, genshi_xml, go, hexiom, html5lib, logging_simple, mako, meteor_contest, nqueens, pathlib, pickle, pickle_list, regex_effbot, regex_v8, richards, scimark_fft, scimark_sor, scimark_sparse_mat_mult, spectral_norm, sqlalchemy_imperative, sqlite_synth, sympy_expand, sympy_integrate, sympy_str, sympy_sum, telco, tornado_http, unpack_sequence, unpickle, unpickle_list, unpickle_pure_python, xml_etree_process ---------- components: Interpreter Core files: simplify-lookdict.patch keywords: patch messages: 281861 nosy: inada.naoki priority: normal severity: normal stage: patch review status: open title: simplify lookdict functions type: enhancement versions: Python 3.7 Added file: http://bugs.python.org/file45668/simplify-lookdict.patch _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Nov 28 07:34:30 2016 From: report at bugs.python.org (B. Liu) Date: Mon, 28 Nov 2016 12:34:30 +0000 Subject: [New-bugs-announce] [issue28819] tzinfo class spacing bug Message-ID: <1480336470.38.0.585299184386.issue28819@psf.upfronthosting.co.za> New submission from B. Liu: In Python 2.7, the following works: import datetime ZERO = timedelta(0) class UTC(datetime.tzinfo): """UTC""" # can be configured here def utcoffset(self, dt): return ZERO def tzname(self, dt): return "UTC" def dst(self, dt): return ZERO def extraMethod(self): return "here is an extra method" utc = UTC() epoch = datetime.datetime.utcfromtimestamp(0) print datetime.datetime.fromtimestamp(epoch, utc) But, the following does not work: import datetime ZERO = timedelta(0) class UTC(datetime.tzinfo): """UTC""" # can be configured here def utcoffset(self, dt): return ZERO def tzname(self, dt): return "UTC" def dst(self, dt): return ZERO def extraMethod(self): return "here is an extra method" utc = UTC() epoch = datetime.datetime.utcfromtimestamp(0) print datetime.datetime.fromtimestamp(epoch, utc) The difference is that there is a space after the class line. Is this an issue with grammar or some issue with the library's C code? If it is an issue with grammar, I appear to have not run into this issue with non-datetime-related code. If this not an issue, please correct me. Thanks! Brian ---------- components: Library (Lib) files: bug.py messages: 281867 nosy: bzliu94 priority: normal severity: normal status: open title: tzinfo class spacing bug Added file: http://bugs.python.org/file45670/bug.py _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Nov 28 07:47:25 2016 From: report at bugs.python.org (Rares Aioanei) Date: Mon, 28 Nov 2016 12:47:25 +0000 Subject: [New-bugs-announce] [issue28820] Typo in section 6 of the Python 3.4 documentation Message-ID: <1480337245.94.0.248137980963.issue28820@psf.upfronthosting.co.za> New submission from Rares Aioanei: In section 6.4.1. it's said "Although certain modules are designed to export only names that follow certain patterns when you use import *, it is still considered bad practise in production code." "practise" should be "practice", as it's a noun. ---------- assignee: docs at python components: Documentation messages: 281870 nosy: Rares Aioanei, docs at python priority: normal severity: normal status: open title: Typo in section 6 of the Python 3.4 documentation type: enhancement versions: Python 3.4 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Nov 28 11:22:16 2016 From: report at bugs.python.org (Florin Papa) Date: Mon, 28 Nov 2016 16:22:16 +0000 Subject: [New-bugs-announce] [issue28821] generate_opcode_h.py crash when run with python2 Message-ID: <1480350136.76.0.0346939338673.issue28821@psf.upfronthosting.co.za> New submission from Florin Papa: Hello, This is Florin Papa from the Dynamic Scripting Languages Optimizations team in Intel Corporation. In changeset 105360:46e2755b022c [1] the generate_opcode_h.py script was modified to use tokenize.open() in order to avoid a Resource Warning. The tokenize module does not have an 'open' attribute in python2, therefore the build will crash if python3 is not present on the system. The patch attached checks if the tokenize module has the 'open' attribute. If it does, the current implementation will be used. Otherwise, it will fall back to the old implementation. Thank you, Florin [1] https://hg.python.org/cpython/rev/46e2755b022c ---------- components: Build files: generate_opcode_h.patch keywords: patch messages: 281882 nosy: florin.papa priority: normal severity: normal status: open title: generate_opcode_h.py crash when run with python2 type: crash versions: Python 3.7 Added file: http://bugs.python.org/file45674/generate_opcode_h.patch _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Nov 28 11:29:18 2016 From: report at bugs.python.org (Xiang Zhang) Date: Mon, 28 Nov 2016 16:29:18 +0000 Subject: [New-bugs-announce] [issue28822] Fix indices handling in PyUnicode_FindChar Message-ID: <1480350558.9.0.582516213925.issue28822@psf.upfronthosting.co.za> New submission from Xiang Zhang: PyUnicode_FindChar declares in the doc it treats its *start* and *end* parameters as str[start:end], same as other APIs like PyUnicode_Find, PyUnicode_Count. But it doesn't allow negative indices like others so violates the doc. ---------- components: Interpreter Core files: PyUnicode_FindChar.patch keywords: patch messages: 281883 nosy: haypo, serhiy.storchaka, xiang.zhang priority: normal severity: normal stage: patch review status: open title: Fix indices handling in PyUnicode_FindChar type: behavior versions: Python 3.5, Python 3.6, Python 3.7 Added file: http://bugs.python.org/file45675/PyUnicode_FindChar.patch _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Nov 28 11:45:09 2016 From: report at bugs.python.org (Serhiy Storchaka) Date: Mon, 28 Nov 2016 16:45:09 +0000 Subject: [New-bugs-announce] [issue28823] Simplify compiling to BUILD_MAP_UNPACK Message-ID: <1480351509.39.0.820702291303.issue28823@psf.upfronthosting.co.za> New submission from Serhiy Storchaka: Currently the compiler produces BUILD_MAP_UNPACK with an argument at most 255. If needed to merge more than 255 dictionaries, BUILD_MAP_UNPACK is called several times. But this is unneeded complication since BUILD_MAP_UNPACK doesn't have a limitation on its argument. Seems this code is the remnants from the patches when there was not the opcode BUILD_MAP_UNPACK_WITH_CALL, and BUILD_MAP_UNPACK packed the position of function name in higher bits of its argument. Proposed patch simplifies the compiler, makes the bytecode faster if merge more than 255 dictionaries and more compact if merge more than 509 dictionaries. BUILD_MAP_UNPACK was introduced in issue2292. ---------- components: Interpreter Core files: BUILD_MAP_UNPACK-unlimited-arg.patch keywords: patch messages: 281885 nosy: Joshua.Landau, benjamin.peterson, brett.cannon, georg.brandl, ncoghlan, neil.g, serhiy.storchaka, yselivanov priority: normal severity: normal stage: patch review status: open title: Simplify compiling to BUILD_MAP_UNPACK type: enhancement versions: Python 3.7 Added file: http://bugs.python.org/file45676/BUILD_MAP_UNPACK-unlimited-arg.patch _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Nov 28 13:55:46 2016 From: report at bugs.python.org (tzickel) Date: Mon, 28 Nov 2016 18:55:46 +0000 Subject: [New-bugs-announce] [issue28824] os.environ should preserve the case of the OS keys ? Message-ID: <1480359346.4.0.287394822331.issue28824@psf.upfronthosting.co.za> New submission from tzickel: In Windows, python's os.environ currently handles the case sensitivity different that the OS. While it's true that the OS is case insensitive, it does preserve the case that you first set it as. For example: C:\Users\user>set aSD=Blah C:\Users\user>set asd aSD=Blah But in python: >>> import os >>> 'aSD' in os.environ.keys() False Today as more people pass environment variables to processes, it's better to behave as the OS does. Basically I think that os.environ (both in 2.7 and 3) should preserve the case as well (for when you need to access / iterate over the keys or set a key), but ignore it when you get a key. https://github.com/python/cpython/blob/b82a5a65caa5b0f0efccaf2bbea94f1eba19a54d/Lib/os.py#L733 ---------- components: Windows messages: 281906 nosy: larry, loewis, paul.moore, steve.dower, tim.golden, tzickel, zach.ware priority: normal severity: normal status: open title: os.environ should preserve the case of the OS keys ? versions: Python 2.7, Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Nov 28 14:17:08 2016 From: report at bugs.python.org (Victor Porton) Date: Mon, 28 Nov 2016 19:17:08 +0000 Subject: [New-bugs-announce] [issue28825] socket.SO_KEEPALIVE does not work on FreeBSD Message-ID: <1480360628.41.0.790647323663.issue28825@psf.upfronthosting.co.za> New submission from Victor Porton: When I connect telnet XXX 9000 to the server in attached file, the connection breaks after 5 min (and a few seconds), as if SO_KEEPALIVE were not specified. I run my server on FreeBSD 9.2-RELEASE-p15 (GENERIC) So SO_KEEPALIVE does not work in Python 2.7 and 3.5 on FreeBSD. Specifically, I need my TCP connection not to break in extended periods of time and cannot do this with Python. It seems that similar code does work with PHP (I am going to write a short PHP example), so it is likely a Python bug, not FreeBSD bug. ---------- components: FreeBSD, IO, Library (Lib) files: bug.py messages: 281911 nosy: koobs, porton priority: normal severity: normal status: open title: socket.SO_KEEPALIVE does not work on FreeBSD type: behavior versions: Python 2.7, Python 3.5 Added file: http://bugs.python.org/file45677/bug.py _______________________________________ Python tracker