, argc=4, argv=0x7fffffffdd88, init=, fini=, rtld_fini=, stack_end=0x7fffffffdd78) at ../csu/libc-start.c:291 #122 0x00005555556293aa in _start () [1] https://bitbucket.org/pypy/pypy/commits/c501282f4438fe2675e6bfd9cc38374fa55d3da0 ---------- components: Interpreter Core messages: 285228 nosy: gumblex priority: normal severity: normal status: open title: Crash on GC when compiling PyPy type: crash versions: Python 2.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed Jan 11 11:15:14 2017 From: report at bugs.python.org (Xiang Zhang) Date: Wed, 11 Jan 2017 16:15:14 +0000 Subject: [New-bugs-announce] [issue29243] --enable-optimizations makes common build commands always need to compile from scratch Message-ID: <1484151314.82.0.308427338149.issue29243@psf.upfronthosting.co.za> New submission from Xiang Zhang: In 3.6 we get --enable-optimizations. One thing I find annoyed with this flag is that it makes some build commands compile from scratch since it always clears the environment first. For example, with the commands listed at the top of Makefile: ./configure make make test make install It compiles 3 times and considering the optimized build needs more time every time, it lasts long. I am not sure this is a problem and feel free to close it. ---------- components: Build messages: 285233 nosy: xiang.zhang priority: normal severity: normal status: open title: --enable-optimizations makes common build commands always need to compile from scratch versions: Python 3.6, Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed Jan 11 13:11:47 2017 From: report at bugs.python.org (rdb) Date: Wed, 11 Jan 2017 18:11:47 +0000 Subject: [New-bugs-announce] [issue29244] Python 3.6 unnecessarily requires inttypes.h Message-ID: <1484158307.25.0.0951457804811.issue29244@psf.upfronthosting.co.za> New submission from rdb: Python 3.6 now requires inttypes.h on all platforms. However, this is not provided by MSVC 2010 and 2012, which is still used by some people who build extension modules for Python. MSVC 2010 does provide stdint.h, and replacing the inttypes.h include with an include to stdint.h seems to work fine. I would suggest a fix along the lines of this: #if defined(_MSC_VER) && _MSC_VER < 1800 #include #else #include #endif Alternatively, the HAVE_INTTYPES_H definition could be used to fall back to stdint.h, and it could be undefined for the MSVC build. ---------- components: Extension Modules messages: 285250 nosy: rdb priority: normal severity: normal status: open title: Python 3.6 unnecessarily requires inttypes.h versions: Python 3.6 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed Jan 11 14:44:31 2017 From: report at bugs.python.org (Kevin Bonham) Date: Wed, 11 Jan 2017 19:44:31 +0000 Subject: [New-bugs-announce] [issue29245] Can't write to NamedTemporaryFile Message-ID: <1484163871.38.0.591576246688.issue29245@psf.upfronthosting.co.za> New submission from Kevin Bonham: Python 3.6.0 (default, Dec 24 2016, 08:01:42) [GCC 4.2.1 Compatible Apple LLVM 8.0.0 (clang-800.0.42.1)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> from tempfile import NamedTemporaryFile >>> tmp = NamedTemporaryFile() >>> tmp.write("hello world!") Traceback (most recent call last): File "", line 1, in File "/usr/local/Cellar/python3/3.6.0/Frameworks/Python.framework/Versions/3.6/lib/python3.6/tempfile.py", line 483, in func_wrapper return func(*args, **kwargs) TypeError: a bytes-like object is required, not 'str' >>> type(tmp) The more verbose error points to issue #18879 (http://bugs.python.org/issue18879), which seems very similar to my problem, but is marked as resolved. --------------------------------------------------------------------------- TypeError Traceback (most recent call last) in () ----> 1 tmp_file.write("blah") /usr/local/Cellar/python3/3.6.0/Frameworks/Python.framework/Versions/3.6/lib/python3.6/tempfile.py in func_wrapper(*args, **kwargs) 481 @_functools.wraps(func) 482 def func_wrapper(*args, **kwargs): --> 483 return func(*args, **kwargs) 484 # Avoid closing the file as long as the wrapper is alive, 485 # see issue #18879. TypeError: a bytes-like object is required, not 'str' This also seems like it might be related to http://bugs.python.org/issue28867, though I'm getting the same behavior with TemporaryFile: >>> from tempfile import TemporaryFile >>> tmp2 = TemporaryFile() >>> tmp2.write("Hello World") Traceback (most recent call last): File "", line 1, in TypeError: a bytes-like object is required, not 'str' I can do: >>> with open(tmp.name, "w") as t: ... t.write("Hello world!") Is this intended behavior? The docs still say that these functions should return file-like objects. ---------- components: Library (Lib) messages: 285266 nosy: Kevin Bonham priority: normal severity: normal status: open title: Can't write to NamedTemporaryFile type: behavior versions: Python 3.6 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed Jan 11 18:03:00 2017 From: report at bugs.python.org (Spiro Sideris) Date: Wed, 11 Jan 2017 23:03:00 +0000 Subject: [New-bugs-announce] [issue29246] typing.Union raises RecursionError when comparing Union to other type Message-ID: <1484175780.67.0.919413735976.issue29246@psf.upfronthosting.co.za> New submission from Spiro Sideris: The typing.Union module raises a RecursionError when comparing a Union with no additional tree_args to a type that is not a Union. An example of the stack trace with the added test looks like the following: 0:00:00 [1/1] test_typing test test_typing failed -- Traceback (most recent call last): File "/Users/spiro/Development/open_source/cpython/Lib/test/test_typing.py", line 221, in test_union_compare_other self.assertNotEqual(Union, object) File "/Users/spiro/Development/open_source/cpython/Lib/unittest/case.py", line 827, in assertNotEqual if not first != second: File "/Users/spiro/Development/open_source/cpython/Lib/typing.py", line 760, in __eq__ return self._subs_tree() == other#return self._subs_tree() is not self and self._subs_tree() == other File "/Users/spiro/Development/open_source/cpython/Lib/typing.py", line 760, in __eq__ return self._subs_tree() == other#return self._subs_tree() is not self and self._subs_tree() == other File "/Users/spiro/Development/open_source/cpython/Lib/typing.py", line 760, in __eq__ return self._subs_tree() == other#return self._subs_tree() is not self and self._subs_tree() == other [Previous line repeated 233 more times] File "/Users/spiro/Development/open_source/cpython/Lib/typing.py", line 759, in __eq__ if not isinstance(other, _Union): RecursionError: maximum recursion depth exceeded in __instancecheck__ test_typing failed 1 test failed: test_typing Total duration: 316 ms Tests result: FAILURE The test compares a Union with no tree_args to another type. ---------- components: Library (Lib) files: spirowork.patch keywords: patch messages: 285279 nosy: spiside priority: normal severity: normal status: open title: typing.Union raises RecursionError when comparing Union to other type type: crash versions: Python 2.7, Python 3.3, Python 3.4, Python 3.5, Python 3.6, Python 3.7 Added file: http://bugs.python.org/file46264/spirowork.patch _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed Jan 11 23:23:54 2017 From: report at bugs.python.org (Nathaniel Smith) Date: Thu, 12 Jan 2017 04:23:54 +0000 Subject: [New-bugs-announce] [issue29247] Document return value of epoll.poll Message-ID: <1484195034.62.0.718564451527.issue29247@psf.upfronthosting.co.za> New submission from Nathaniel Smith: The documentation for select.epoll.poll doesn't document the return value at all, which is somewhat important information :-) I think it's a list of (fd, eventmask) tuples? https://docs.python.org/3.7/library/select.html#select.epoll.poll ---------- assignee: docs at python components: Documentation messages: 285285 nosy: docs at python, njs priority: normal severity: normal status: open title: Document return value of epoll.poll versions: Python 2.7, Python 3.5, Python 3.6, Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu Jan 12 03:09:20 2017 From: report at bugs.python.org (Eryk Sun) Date: Thu, 12 Jan 2017 08:09:20 +0000 Subject: [New-bugs-announce] [issue29248] os.readlink fails on Windows Message-ID: <1484208560.98.0.375529555783.issue29248@psf.upfronthosting.co.za> New submission from Eryk Sun: win_readlink in Modules/posixmodule.c mistakenly treats the PrintNameOffset field of the reparse data buffer as a number of characters instead of bytes. Thus, if the offset is non-zero, the value returned is incorrect stack garbage. For example, the following should return "C:\\ProgramData": >>> os.readlink(r'C:\Users\All Users') '\u6c20\u3012\u041f\x01\u2768\u60b2\u031b\x02\x05\x00\u031e\x06\u8c01\u4012' Craig Holmquist found this bug, as detailed in message 277385. He included a fix in the patch for issue 23407, but this should be addressed in the next maintenance release of 3.5 and 3.6. ---------- components: Library (Lib), Windows messages: 285294 nosy: eryksun, paul.moore, steve.dower, tim.golden, zach.ware priority: normal severity: normal stage: needs patch status: open title: os.readlink fails on Windows type: behavior versions: Python 3.5, Python 3.6, Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu Jan 12 05:19:54 2017 From: report at bugs.python.org (Jon Walsh) Date: Thu, 12 Jan 2017 10:19:54 +0000 Subject: [New-bugs-announce] [issue29249] Pathlib glob ** bug Message-ID: <1484216394.04.0.57434547579.issue29249@psf.upfronthosting.co.za> New submission from Jon Walsh: >>> from pathlib import Path >>> Path("a/b/c/d/e.txt").match('a/*/**/*') False ---------- components: Library (Lib) messages: 285297 nosy: Jon Walsh priority: normal severity: normal status: open title: Pathlib glob ** bug type: behavior versions: Python 3.5 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu Jan 12 05:42:45 2017 From: report at bugs.python.org (Eryk Sun) Date: Thu, 12 Jan 2017 10:42:45 +0000 Subject: [New-bugs-announce] [issue29250] islink and stat follow_symlinks are inconsistent on Windows Message-ID: <1484217765.9.0.444376233166.issue29250@psf.upfronthosting.co.za> New submission from Eryk Sun: win32_xstat_impl in Modules/posixmodule.c doesn't check the value of reparse_tag to limit traversal to only tags that are considered links. So we have cases in which os.path.islink returns False (e.g. junctions, pending the resolution of issue 23407), but os.stat and os.lstat disagree. For example: >>> os.path.islink(r'C:\Documents and Settings') False >>> s = os.stat(r'C:\Documents and Settings') >>> ls = os.lstat(r'C:\Documents and Settings') >>> s.st_ino == ls.st_ino False At its strictest, traversal would be limited to just symbolic links. It could be relaxed to also include junctions, and even further to include all tags with the name surrogate bit set [1]. This includes the following Microsoft tags IO_REPARSE_TAG_MOUNT_POINT (junctions) IO_REPARSE_TAG_SYMLINK IO_REPARSE_TAG_IIS_CACHE and non-Microsoft tags: IO_REPARSE_TAG_SOLUTIONSOFT IO_REPARSE_TAG_OSR_SAMPLE IO_REPARSE_TAG_QI_TECH_HSM IO_REPARSE_TAG_MAXISCALE_HSM I'm creating this issue as an enhancement for 3.7. The problem has existed for a long time, but it hasn't been a thorn in anyone's side as far as I know. [1]: https://msdn.microsoft.com/en-us/library/aa365511 ---------- components: Library (Lib), Windows messages: 285299 nosy: eryksun, paul.moore, steve.dower, tim.golden, zach.ware priority: normal severity: normal stage: needs patch status: open title: islink and stat follow_symlinks are inconsistent on Windows type: enhancement versions: Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu Jan 12 07:04:20 2017 From: report at bugs.python.org (Martin Panter) Date: Thu, 12 Jan 2017 12:04:20 +0000 Subject: [New-bugs-announce] [issue29251] Class __dict__ is only a mapping proxy Message-ID: <1484222660.7.0.65822313324.issue29251@psf.upfronthosting.co.za> New submission from Martin Panter: The __dict__ attribute of class objects is documented as being a (standard) dictionary, but implemented with a proxy object. I propose to clarify the documentation in ?Custom classes? under , and in . I believe my changes are also applicable to Python 2, as long as I point out the proxy is specific to ?new-style? classes. ---------- assignee: docs at python components: Documentation files: class-dict.patch keywords: patch messages: 285313 nosy: docs at python, martin.panter priority: normal severity: normal stage: patch review status: open title: Class __dict__ is only a mapping proxy type: behavior versions: Python 2.7, Python 3.5, Python 3.6, Python 3.7 Added file: http://bugs.python.org/file46268/class-dict.patch _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu Jan 12 07:27:27 2017 From: report at bugs.python.org (Decorater) Date: Thu, 12 Jan 2017 12:27:27 +0000 Subject: [New-bugs-announce] [issue29252] self in classes missinterpreted as a string. Message-ID: <1484224047.15.0.809127294576.issue29252@psf.upfronthosting.co.za> New submission from Decorater: So, I have a class (it subclasses the zipimporter for my import hook) that allows me to not worry about issues when the hook is appended to the beginning of the path hooks. The code to the class as follows: class OriginalZipImport(zipimport.zipimporter): """ Original zipimporter class. Modified for requirements of this import hook. """ # Sadly I have to manually specify the file extensions the original # zipimporter supports to bypass an AttributeError. # according to https://docs.python.org/3.6/library/zipimport.html # zipimport only allows importing ``.py`` and ``.pyc`` files from zips. extensions = ['.py', '.pyc'] def __init__(self, archivepath): super(OriginalZipImport, self.inst).__init__(self, archivepath) def find_loader(self, fullname, path=None): """ Original find_loader. """ self.inst.find_loader(fullname, path=path) def find_module(self, fullname, path=None): """ Original find_module. """ self.inst.find_module(self, fullname, path=path) def load_module(self, fullname): """ Original load_module. """ self.inst.load_module(self, fullname) However instead of working like the way I would expect it gives this traceback: Traceback (most recent call last): File "E:\Users\Elsword\Desktop\DecoraterBot\Async\\zipmaker\make_zip.py", line 11, in from pathlib import Path File "", line 961, in _find_and_load File "", line 946, in _find_and_load_unlocked File "", line 885, in _find_spec File "", line 1157, in find_spec File "", line 1129, in _get_spec File "", line 1273, in find_spec File "", line 1229, in _get_spec File "E:\Users\Elsword\Desktop\DecoraterBot\Async\zipmaker\py2pycx\_pycx_backend.py", line 180, in find_loader self.inst.find_loader(fullname, path=path) AttributeError: 'str' object has no attribute 'inst' Note: it does use a slightly modified make_zip.py that is different than the one that is on the python source trees. It is specifically for testing the hook to see if it works or not. The hook is supposed to work however it is confusing why it is doing this as I did not set self to be an string so it should allow me to do all of this just fine. ---------- components: Interpreter Core messages: 285316 nosy: Decorater priority: normal severity: normal status: open title: self in classes missinterpreted as a string. versions: Python 3.6, Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu Jan 12 08:51:51 2017 From: report at bugs.python.org (Erik Bray) Date: Thu, 12 Jan 2017 13:51:51 +0000 Subject: [New-bugs-announce] [issue29253] Fix test_asyncore tests on Cygwin Message-ID: <1484229111.27.0.864294562114.issue29253@psf.upfronthosting.co.za> New submission from Erik Bray: This patch works around a couple different problems with running the test_asyncore tests in Cygwin, both of which are due to bugs in Cygwin itself and not in Python. As such, I don't think Python should try to work around these issues in general, but I do think they're worth at least working around in the tests so that they can pass and/or skip. The first issue is more severe, as it actually causes the test run to hang indefinitely due to a bug in Cygwin that can cause connect() on a socket to block indefinitely. This occurs only in a situation where a "client" connects to a "server" that is not accept()-ing connections--a situation that occurs specifically in some of the asyncore tests where both the "client" and the "server" are in the same test process. This applies a known workaround to the tests only, allowing them to pass. The other bug simply causes a test failure--the bug is that getsockopt(SO_REUSEADDR) returns the wrong result after a setsockopt(SO_REUSEADDR) (the latter has the correct behavior; the value of the option just isn't reported back correctly after being set). In this case the relevant test is simply skipped on Cygwin. The patch is currently light on in-line documentation of the situation, but I can add that if it's otherwise approved. ---------- components: Tests files: 0001-Fix-test_asyncore-tests-on-Cygwin-by-working-around-.patch keywords: patch messages: 285326 nosy: erik.bray, zach.ware priority: normal severity: normal stage: patch review status: open title: Fix test_asyncore tests on Cygwin Added file: http://bugs.python.org/file46272/0001-Fix-test_asyncore-tests-on-Cygwin-by-working-around-.patch _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu Jan 12 11:42:13 2017 From: report at bugs.python.org (Suraj Kumar) Date: Thu, 12 Jan 2017 16:42:13 +0000 Subject: [New-bugs-announce] [issue29254] Documentation Error Message-ID: <1484239333.41.0.299474192201.issue29254@psf.upfronthosting.co.za> New submission from Suraj Kumar: Hi, I have found one documentation mistake in section 3.1.2. Strings saying """Note how the start is always included, and the end always excluded. This makes sure that s[:i] + s[i:] is always equal to s: >>>word = 'Python' >>> word[:2] + word[2:] 'Python' """ I think instead of 'the end always excluded' it should be 'the end always included'. If my observation is correct, please do the needful change. Thanks & Regards, Suraj ---------- assignee: docs at python components: Documentation messages: 285331 nosy: Suraj Kumar, docs at python priority: normal severity: normal status: open title: Documentation Error type: behavior versions: Python 3.6 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu Jan 12 18:35:07 2017 From: report at bugs.python.org (Nathaniel Smith) Date: Thu, 12 Jan 2017 23:35:07 +0000 Subject: [New-bugs-announce] [issue29255] selects.KqueueSelector behaves incorrectly when no fds are registered Message-ID: <1484264107.13.0.0489042381572.issue29255@psf.upfronthosting.co.za> New submission from Nathaniel Smith: When calling kevent(), selectors.KqueueSelector.select sets the "maxevents" argument to len(self._fd_to_key). So if there are no fds registered, it passes maxevents=0. It turns out that the kevent() API has a special case behavior for maxevents=0: it returns immediately, ignoring timeout. I have no idea why kevent() works this way, but it's specifically called out in the man page: The nevents argument determines the size of eventlist. When nevents is zero, kevent() will return immediately even if there is a timeout specified unlike select(2). https://www.freebsd.org/cgi/man.cgi?query=kqueue&sektion=2 It happens that asyncio doesn't run into this because asyncio always has at least one fd registered with its selector, but this causes problems for other users of the selectors module, e.g.: https://github.com/dabeaz/curio/issues/156 I believe fix would just be to add some code like: "if max_ev == 0: max_ev = 1" to selectors.KqueueSelector.select. ---------- components: Library (Lib) messages: 285352 nosy: njs priority: normal severity: normal status: open title: selects.KqueueSelector behaves incorrectly when no fds are registered versions: Python 3.5, Python 3.6, Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu Jan 12 18:42:18 2017 From: report at bugs.python.org (Nathaniel Smith) Date: Thu, 12 Jan 2017 23:42:18 +0000 Subject: [New-bugs-announce] [issue29256] Windows select() errors out when given no fds to select on, which breaks SelectSelector Message-ID: <1484264538.85.0.902649207203.issue29256@psf.upfronthosting.co.za> New submission from Nathaniel Smith: If SelectSelector.select() is called when there are no fds registered, then it ends up calling select.select([], [], [], timeout). On sensible operating systems, this is equivalent to time.sleep(timeout). On Windows, it raises an error. Asyncio manages to avoid hitting this due to the fact that it always has at least one fd registered, but it causes problems for other users of the selectors module, e.g.: https://github.com/dabeaz/curio/issues/75 I see two possible approaches to fixing this: 1) Modify SelectSelector to check for this case and call time.sleep() instead of select.select() when encountered. 2) Modify the select.select() wrapper so that it behaves consistently on all operating systems, by special-casing this situation on Windows. Option (2) seems nicer to me. ---------- components: Library (Lib) messages: 285353 nosy: njs priority: normal severity: normal status: open title: Windows select() errors out when given no fds to select on, which breaks SelectSelector versions: Python 3.5, Python 3.6, Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu Jan 12 19:48:36 2017 From: report at bugs.python.org (Gerald Britton) Date: Fri, 13 Jan 2017 00:48:36 +0000 Subject: [New-bugs-announce] [issue29257] Possible error in discussion of Abstract Base Classes and abstract properties Message-ID: <1484268516.3.0.750896526181.issue29257@psf.upfronthosting.co.za> New submission from Gerald Britton: I was rereading the 2.7 docs about abstract base classes the other day. I found this: "This defines a read-only property; you can also define a read-write abstract property using the ?long? form of property declaration:" along with an example. so I copied the example and put in a little surrounding code: from abc import ABCMeta, abstractproperty class C: __metaclass__ = ABCMeta def getx(self): pass def setx(self, value): pass x = abstractproperty(getx, setx) class D(C): @property def x(self):self._x d = D() print(d) When I ran this, I expected an exception, since I defined a read/write abstract property but only implemented the read operation. However, the example runs fine. That is the class D can be instantiated without error. Of course I cannot set the property since I didn't implement that part. Now, If I don't implement the property at all, I can' instantiate the class. I get: "TypeError: Can't instantiate abstract class D with abstract methods x" which is what I would expect. What I don't understand is why I don't get a similar error when I implement the read operation for the property but not the write operation. If this actually doesn't work (catching the non-implementation at instantiation time), then why is it documented this way? To me at least the doc implies that it *will* raise on the missing write property implementation. If ABCs are working as intended, can the documentation be changed to reflect that as per my experience above? If the documentation is correct, can the ABC implementation be modified to function that way? ---------- assignee: docs at python components: Documentation messages: 285356 nosy: docs at python, gbritton priority: normal severity: normal status: open title: Possible error in discussion of Abstract Base Classes and abstract properties versions: Python 2.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu Jan 12 23:47:54 2017 From: report at bugs.python.org (Anthony Sottile) Date: Fri, 13 Jan 2017 04:47:54 +0000 Subject: [New-bugs-announce] [issue29258] __init__.py required for pkgutil.walk_packages in python3 Message-ID: <1484282874.75.0.69511180509.issue29258@psf.upfronthosting.co.za> New submission from Anthony Sottile: PEP420 makes __init__.py files optional: https://docs.python.org/3/whatsnew/3.3.html#pep-420-implicit-namespace-packages Though it seems without them, pkgutil.walk_packages does not function as desired: https://docs.python.org/3/library/pkgutil.html#pkgutil.walk_packages Consider the following example: ``` $ tree foo foo ??? bar ? ??? baz.py ? ??? __init__.py ??? __init__.py ??? womp.py ``` And a test script # test.py ``` import pkgutil import foo for _, mod, _ in pkgutil.walk_packages(foo.__path__, foo.__name__ + '.'): print(mod) ``` In both python2 and python3 I get the following output: ``` $ python2.7 test.py foo.bar foo.bar.baz foo.womp $ python3.5 test.py foo.bar foo.bar.baz foo.womp ``` Removing the __init__.py files and only using python3, I get this: ``` $ find -name '__init__.*' -delete $ python3.5 test.py foo.bar ``` The modules are definitely importable: ``` $ python3.5 -c 'import foo.bar.baz' $ ``` Originally asked as a question on stackoverflow: http://stackoverflow.com/questions/41203765/init-py-required-for-pkgutil-walk-packages-in-python3 ---------- components: Library (Lib) messages: 285361 nosy: Anthony Sottile priority: normal severity: normal status: open title: __init__.py required for pkgutil.walk_packages in python3 type: behavior versions: Python 3.3, Python 3.4, Python 3.5, Python 3.6, Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Jan 13 07:24:34 2017 From: report at bugs.python.org (STINNER Victor) Date: Fri, 13 Jan 2017 12:24:34 +0000 Subject: [New-bugs-announce] [issue29259] Add tp_fastcall to PyTypeObject: support FASTCALL calling convention for all callable objects Message-ID: <1484310274.99.0.832491900546.issue29259@psf.upfronthosting.co.za> New submission from STINNER Victor: A new FASTCALL calling convention was added to Python 3.6. It allows to avoid the creation of a temporary tuple to pass positional arguments and a temporary dictionary to pass keyword arguments. A new METH_FASTCALL calling convention was added for C functions. Most functions now support fastcall, except objects with a __call__() method which have to go through slot_tp_call() which still requires a tuple and dictionary. I tried multiple implementations to support fast calls to call the __call__() method, but I had practical and technical issues. First, I tried to reuse the tp_call field to PyTypeObject: it can be a regular call (tuple/dict for arguments) or a fast call. I added a flag to the tp_flags field. It was tricky to support class inheritance, decide to set or clear the flag. But the real blocker issue is fAthat it is obviously breaks the backward compatibility: existing code calling directly tp_call with the regular calling convention will crash immediatly, and the error is not catched during compilation, even if the code is recompiled. I propose a different design: add a new tp_fastcall field to PyTypeObject and use a wrapper for tp_call when tp_fastcall is defined. If a type defines tp_fastcall but not, the tp_call wrapper "simply" calls tp_fastcall. Advantages: * The wrapper is trivial * Minor changes to PyType_Ready() to support inheritance (simple logic) * Fully backward compatible * If tp_call is called directly without keyword arguments, there is no overhead but a speedup! Inheritance: * If a type only defines tp_call, tp_fastcall is not inherited from the parent: tp_fastcall is set to NULL. * If a type only defines tp_fastcall: tp_fastcall is always use (tp_call uses the wrapper) * If a type defines tp_call and tp_fastcall, PyObject_Call() uses tp_call whereas _PyObject_FastCallDict() uses tp_fastcall. Functions of the C API will be modified to use tp_fastcall if available. The plan is then to patch most Python types to replace their tp_call with tp_fastcall. First, most important (common) types like Python and C functions, descriptors, and the various kinds of wrappers should be patched. Later, we should maybe discuss on a case by case basis to decide if it's worth it. I will try to run benchmark before any kind. ---------- messages: 285388 nosy: haypo, inada.naoki, serhiy.storchaka priority: normal pull_requests: 17 severity: normal status: open title: Add tp_fastcall to PyTypeObject: support FASTCALL calling convention for all callable objects type: performance versions: Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Jan 13 09:31:05 2017 From: report at bugs.python.org (STINNER Victor) Date: Fri, 13 Jan 2017 14:31:05 +0000 Subject: [New-bugs-announce] [issue29260] Use designated initializers to define PyTypeObject types Message-ID: <1484317865.58.0.725398845848.issue29260@psf.upfronthosting.co.za> New submission from STINNER Victor: Currently, PyTypeObject fields are set in order in the C code. Typical example: PyTypeObject PyMethod_Type = { PyVarObject_HEAD_INIT(&PyType_Type, 0) "method", sizeof(PyMethodObject), 0, (destructor)method_dealloc, /* tp_dealloc */ 0, /* tp_print */ 0, /* tp_getattr */ 0, /* tp_setattr */ 0, /* tp_reserved */ (reprfunc)method_repr, /* tp_repr */ 0, /* tp_as_number */ 0, /* tp_as_sequence */ 0, /* tp_as_mapping */ (hashfunc)method_hash, /* tp_hash */ method_call, /* tp_call */ 0, /* tp_str */ method_getattro, /* tp_getattro */ PyObject_GenericSetAttr, /* tp_setattro */ 0, /* tp_as_buffer */ Py_TPFLAGS_DEFAULT | Py_TPFLAGS_HAVE_GC, /* tp_flags */ method_doc, /* tp_doc */ (traverseproc)method_traverse, /* tp_traverse */ 0, /* tp_clear */ method_richcompare, /* tp_richcompare */ offsetof(PyMethodObject, im_weakreflist), /* tp_weaklistoffset */ 0, /* tp_iter */ 0, /* tp_iternext */ method_methods, /* tp_methods */ method_memberlist, /* tp_members */ method_getset, /* tp_getset */ 0, /* tp_base */ 0, /* tp_dict */ method_descr_get, /* tp_descr_get */ 0, /* tp_descr_set */ 0, /* tp_dictoffset */ 0, /* tp_init */ 0, /* tp_alloc */ method_new, /* tp_new */ }; The aligned comment is an old practice required to be able to see to which field correspond a line. This syntax usually produces a lot of zeros. The C standard (C99?) allows to use a more readable syntax: PyTypeObject PyMethod_Type = { PyVarObject_HEAD_INIT(&PyType_Type, 0) .tp_name = "method", .tp_basicsize = sizeof(PyMethodObject), .tp_dealloc = (destructor)method_dealloc, .tp_repr = (reprfunc)method_repr, .tp_hash = (hashfunc)method_hash, .tp_call = method_call, .tp_getattro = method_getattro, ... }; It seems like it's possible to start with positional fields and then switch to named fields, so PyVarObject_HEAD_INIT() still works. Maybe maybe PyVarObject_HEAD_INIT() should also evolve? Or maybe we need a new macro using the ".tp_xxx=...," syntax? INADA Naoki proposed to use this syntax in my pull request which adds a new tp_fastcall field which requires to add many zeros: https://github.com/python/cpython/pull/65#pullrequestreview-16566423 Example of change: @@ -370,6 +370,17 @@ PyTypeObject PyMethod_Type = { 0, /* tp_init */ 0, /* tp_alloc */ method_new, /* tp_new */ + 0, /* tp_free */ + 0, /* tp_is_gc */ + 0, /* tp_bases */ + 0, /* tp_mro */ + 0, /* tp_cache */ + 0, /* tp_subclasses */ + 0, /* tp_weaklist */ + 0, /* tp_del */ + 0, /* tp_version_tag */ + 0, /* tp_finalize */ + (fastternaryfunc)method_fastcall, /* tp_fastcall */ }; ---------- messages: 285399 nosy: haypo, inada.naoki priority: normal severity: normal status: open title: Use designated initializers to define PyTypeObject types versions: Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Jan 13 10:06:25 2017 From: report at bugs.python.org (Dave Jones) Date: Fri, 13 Jan 2017 15:06:25 +0000 Subject: [New-bugs-announce] [issue29261] Missing venv/scripts/common after "make install" Message-ID: <1484319985.48.0.948212892156.issue29261@psf.upfronthosting.co.za> New submission from Dave Jones: While attempting to diagnose something (unrelated to this issue) under python 3.6, I used the following steps to clone and build a non-root python installation: $ mkdir py36 $ hg clone https://hg.python.org/cpython $ cd cpython $ hg update 3.6 $ ./configure --prefix=/home/pi/py36 $ make $ make install Afterwards I attempted to set up a venv for further testing: $ /home/pi/py36/bin/python3 -m venv py36env This succeeded but, unfortunately, activating the venv proved impossible as the "activate" script was missing from its "bin" sub-dir. After a bit of investigation it appears that "make install" had copied "Lib/venv/scripts/posix" but not "Lib/venv/scripts/common" into the installation, hence the templates for "activate.csh" and "activate.fish" are present, but not the "activate" template. I had a search around for related issues but only found #22343 which explains why "activate" is no longer present (because it's moved from scripts/posix/ to scripts/common/) but I don't think is responsible for common not getting copied by "make install". It appears adding "venv/scripts/common" to LIBSUBDIRS in Makefile.pre.in fixes the issue (as in the attached patch) but I've no idea if this covers all necessary platforms (I'd guess Windows doesn't use the Makefile if #22343 fixed things for it?). ---------- components: Installation files: missing_venv_common.diff keywords: patch messages: 285403 nosy: Dave Jones priority: normal severity: normal status: open title: Missing venv/scripts/common after "make install" type: behavior versions: Python 3.6, Python 3.7 Added file: http://bugs.python.org/file46280/missing_venv_common.diff _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Jan 13 10:45:47 2017 From: report at bugs.python.org (flying sheep) Date: Fri, 13 Jan 2017 15:45:47 +0000 Subject: [New-bugs-announce] [issue29262] Provide a way to check for *real* typing.Union instances Message-ID: <1484322347.31.0.5780880295.issue29262@psf.upfronthosting.co.za> New submission from flying sheep: typing.Union prevents the use of `isinstance` and `issubclass` via a hook. This is presumably to prevent errors that would arise if someone tried to do issubclass(A, Union[A, B]) or isinstance(A(), Union[A, B]), because Union isn?t specified in the PEP to allow a check like this, and doesn?t implement it. (Instead it throws said error) However, as far as I can see there is no blessed way to check if an object was returned by Union.__getitem__(). A simple way that works is: sig = inspect.signature(f) ann = sig.parameters['arg1'].annotation is_an_union = isinstance(ann, typing._Union) but _Union is a private class, and an implementation detail. is there a blessed way to do this? If not, one should be added. ---------- components: Library (Lib) messages: 285410 nosy: flying sheep priority: normal severity: normal status: open title: Provide a way to check for *real* typing.Union instances versions: Python 3.6 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Jan 13 11:09:52 2017 From: report at bugs.python.org (STINNER Victor) Date: Fri, 13 Jan 2017 16:09:52 +0000 Subject: [New-bugs-announce] [issue29263] Implement LOAD_METHOD/CALL_METHOD for C functions Message-ID: <1484323792.44.0.217164349535.issue29263@psf.upfronthosting.co.za> New submission from STINNER Victor: The issue #29259 implements tp_fastcall on method_descriptor (PyMethodDescr_Type). According to http://bugs.python.org/issue26110#msg283093 it would allow to implement LOAD_METHOD and CALL_METHOD for C functions. ---------- messages: 285414 nosy: haypo, inada.naoki, yselivanov priority: normal severity: normal status: open title: Implement LOAD_METHOD/CALL_METHOD for C functions type: performance versions: Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Jan 13 12:27:39 2017 From: report at bugs.python.org (Brian Vandenberg) Date: Fri, 13 Jan 2017 17:27:39 +0000 Subject: [New-bugs-announce] [issue29264] sparc/ffi.c:440 error: 'asm' undeclared Message-ID: <1484328459.12.0.655564355741.issue29264@psf.upfronthosting.co.za> New submission from Brian Vandenberg: When building pythong 3.6.0 on solaris 10/sparc I'm seeing the following error: Modules/_ctypes/libffi/src/sparc/ffi.c:440:8: error: 'asm' undeclared (first use in this function) (...) If I force it to use -std=gnu99 then it seems to compile fine. ---------- components: ctypes messages: 285428 nosy: phantal priority: normal severity: normal status: open title: sparc/ffi.c:440 error: 'asm' undeclared type: compile error versions: Python 3.6 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Jan 13 12:48:34 2017 From: report at bugs.python.org (Brian Vandenberg) Date: Fri, 13 Jan 2017 17:48:34 +0000 Subject: [New-bugs-announce] [issue29265] test suite is attempting to spawn 258 child processes to run tests Message-ID: <1484329714.36.0.262857115906.issue29265@psf.upfronthosting.co.za> New submission from Brian Vandenberg: I'm attempting to build python 3.6.0 on sparc/solaris 10. After the initial configure/compile complete I ran "make test" and I see: $ make test running build running build_ext (...) running build_scripts copying and adjusting (...) changing mode of (...) renaming (...) (...) Run tests in parallel using 258 child processes I'm fairly sure the issue stems from the fact that each core on the machine has 8 "threads" and there's 32 cores (for a total of 256 virtual cores). Each core can execute 8 parallel tasks only in very specific circumstances. It's intended for use by things like lapack/atlas where you might be doing many computations on the same set of data. Outside of these more restricted circumstances each core can only handle 2 parallel tasks (or so I gathered from the documentation), so at best this machine could handle 64 backgrounded jobs though I normally restrict my builds to the actual core count or less. The most common way to get a "realistic" core count on these machines from shell scripts is: $ core_count=`kstat -m cpu_info | grep core_id | sort -u | wc -l` ... though I'm not sure how the test suite is determining the core count. I didn't see any mention of "kstat" anywhere. ---------- components: Tests messages: 285430 nosy: phantal priority: normal severity: normal status: open title: test suite is attempting to spawn 258 child processes to run tests type: behavior versions: Python 3.6 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Jan 13 13:43:15 2017 From: report at bugs.python.org (Brian Vandenberg) Date: Fri, 13 Jan 2017 18:43:15 +0000 Subject: [New-bugs-announce] [issue29266] test_create_connection_service_name fails if "http" isn't listed in /etc/services Message-ID: <1484332995.5.0.803330803747.issue29266@psf.upfronthosting.co.za> New submission from Brian Vandenberg: One of our solaris machines doesn't have an entry in /etc/services for "http". This is causing test_create_connection_service_name to fail. In my case I can just ignore that particular failure, but as a fix you might consider overtly getting valid service names from the host to test. Additionally, if you don't already have it you might consider adding a test that validates behavior when using a non-existent service. ---------- components: Tests messages: 285434 nosy: phantal priority: normal severity: normal status: open title: test_create_connection_service_name fails if "http" isn't listed in /etc/services type: behavior versions: Python 3.6 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Jan 13 13:53:57 2017 From: report at bugs.python.org (Brian Vandenberg) Date: Fri, 13 Jan 2017 18:53:57 +0000 Subject: [New-bugs-announce] [issue29267] Cannot override some flags in CFLAGS from the command-line Message-ID: <1484333637.07.0.801577004436.issue29267@psf.upfronthosting.co.za> New submission from Brian Vandenberg: Due to issue 29264 I was attempting to override the build default "-std=c99" with: /path/to/configure (...) CFLAGS=-std=gnu99 ... however, the configure script is written like this: CFLAGS_NODIST="$CFLAGS_NODIST -std=c99" This causes it to always override my attempt to override the build default. ---------- components: Build messages: 285435 nosy: phantal priority: normal severity: normal status: open title: Cannot override some flags in CFLAGS from the command-line type: behavior versions: Python 3.6 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Jan 13 14:43:48 2017 From: report at bugs.python.org (Brian Vandenberg) Date: Fri, 13 Jan 2017 19:43:48 +0000 Subject: [New-bugs-announce] [issue29268] test_spwd fails on solaris using NIS users Message-ID: <1484336628.67.0.439917676346.issue29268@psf.upfronthosting.co.za> New submission from Brian Vandenberg: On all our solaris 10 machines when I run a simple test program it never reports a failure when calling getspnam: #include #include int main( int, char** ) { spwd *asdf = getspnam( "some_user" ); if( NULL == sdf ) { perror( "getspnam" ); } return 0; } If I run the above program on our linux boxes it fails as expected, but on our solaris machines it produces the same information you'd see running "ypcat passwd | grep some_user". I suspect either there's a bug in the solaris implementation of getpwnam() or perhaps there's a configuration issue on our solaris machines, though it's also possible this is just how it behaves in Solaris (at least with NIS). As to whether anything should change for test_spwd -- I suspect this will get closed as "won't fix" but at least this report may help anyone else running into this failure. ---------- components: Tests messages: 285439 nosy: phantal priority: normal severity: normal status: open title: test_spwd fails on solaris using NIS users type: behavior versions: Python 3.6 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Jan 13 15:55:24 2017 From: report at bugs.python.org (Brian Vandenberg) Date: Fri, 13 Jan 2017 20:55:24 +0000 Subject: [New-bugs-announce] [issue29269] test_socket failing in solaris Message-ID: <1484340924.46.0.120687210952.issue29269@psf.upfronthosting.co.za> New submission from Brian Vandenberg: I started looking into this failure to see if I could figure out why but it looks like I'd have to spend more time than I have available to figure out the cause. Environment/setup: * air-gapped network (no internet access) * sparc / Solaris 10 * Built with gcc 6.3.0 * Altered configure script to change -std=c99 to -std=gnu99 (see issue 29264) * The only configure flags used were --prefix and --with-universal-archs=all When I run test_socket I see the following 4 failures; please note, I'm hand typing the results so I may typo something: ERROR: testCount (test.test_socket.SendfileUsingSendfileTest) Traceback: File "(...)/test_socket.py", line 5248, in testCount File "(...)/test_socket.py", line 5151, in recv_data MemoryError Error: testCount (test.test_socket.SendfileUsingSendfileTest) Traceback: File "(...)/test_socket.py", line 277, in _tearDown File "(...)/test_socket.py", line 289, in clientRun File "(...)/test_socket.py", line 5241, in _testCount File "(...)/Lib/socket.py", line 296, in _sendfile_use_sendfile socket.timeout: timed out Error: testWithTimeout (test.test_socket.SendfileUsingSendfileTest) Traceback: File "(...)/test_socket.py", line 5318, in testWithTimeout data = self.recv_data(conn) File "(...)/test_socket.py", line 5151, in recv_data chunk = conn.recv(self.BUFSIZE) MemoryError Error: testWithTimeout (test.test_socket.SendfileUsingSendfileTest) Traceback: File "(...)/test_socket.py", line 277, in _tearDown raise exc File "(...)/test_socket.py", line 289, in clientRun test_func() File "(...)/test_socket.py", line 5313, in _testWithTimeout sent = meth(file) File "(...)/Lib/socket.py", line 296, in _sendfile_use_sendfile socket.timeout: timed out Error: testCountWithOffset (test.test_socket.SendfileUsingSendfileTest) Traceback: File "(...)/test_socket.py", line 5287, in testCountWithOffset self.assertEqual(len(data), count) AssertionError: 4376231 != 100007 Ran 539 tests in 69.166s FAILED (failures=1, errors=4, skipped=324) test test_socket failed ---------- components: Tests messages: 285440 nosy: phantal priority: normal severity: normal status: open title: test_socket failing in solaris type: behavior versions: Python 3.6 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Jan 13 17:10:22 2017 From: report at bugs.python.org (Dave Jones) Date: Fri, 13 Jan 2017 22:10:22 +0000 Subject: [New-bugs-announce] [issue29270] super call in ctypes sub-class fails in 3.6 Message-ID: <1484345422.14.0.967631121608.issue29270@psf.upfronthosting.co.za> New submission from Dave Jones: While investigating a bug report in one of my libraries (https://github.com/waveform80/picamera/issues/355) I've come across a behaviour that appears in Python 3.6 but not prior versions. Specifically, calling super() in a sub-class of a ctypes scalar type appears to fail at the class definition stage. A minimal test case is as follows: import ctypes as ct class SuperTest(ct.c_uint32): def __repr__(self): return super().__repr__() This works happily under python 3.2, 3.4, and 3.5 (that I've tested), and also under 2.7 (with the appropriate modification to super's arguments). However, under 3.6 it elicits the following exception: Traceback (most recent call last): File "py36_ctypes.py", line 3, in class SuperTest(ct.c_uint32): TypeError: __class__ set to defining 'SuperTest' as Reading through the "What's New" list in 3.6, I thought this might be something to do with the PEP-487 implementation (given it modified class construction), but having read through the PEP and associated patches I'm not so sure as I can't see anything that affects the setting of the "__class__" attribute (but don't rule it out on that basis; I'm no expert!). I'll admit that sub-classing one of ctypes' scalar types is a little odd, but given this works in prior versions and there doesn't appear to be anything in the documentation banning the practice (that I've found?) this might constitute a bug? ---------- components: ctypes messages: 285444 nosy: Dave Jones priority: normal severity: normal status: open title: super call in ctypes sub-class fails in 3.6 type: behavior versions: Python 3.6 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Jan 13 17:11:24 2017 From: report at bugs.python.org (Yury Selivanov) Date: Fri, 13 Jan 2017 22:11:24 +0000 Subject: [New-bugs-announce] [issue29271] Task.current_task(None) returns unexpected result Message-ID: <1484345484.86.0.127501941011.issue29271@psf.upfronthosting.co.za> New submission from Yury Selivanov: Quoting Alberto Berdi from python-tulip mailing list: working with the Python 3.6 asyncio implemented in C, i've got what is for me a strange behavior of the asyncio.Task.current task function. Executing the following test: import asyncio async def coro(): print(asyncio.Task.current_task()) print(asyncio.Task.current_task(None)) print(asyncio.Task.current_task(loop=asyncio.get_event_loop())) loop = asyncio.get_event_loop() loop.run_until_complete(coro()) in Py3.5, I obtain the result: cb=[_run_until_complete_cb() at /usr/lib/python3.5/asyncio/base_events.py:176]> cb=[_run_until_complete_cb() at /usr/lib/python3.5/asyncio/base_events.py:176]> cb=[_run_until_complete_cb() at /usr/lib/python3.5/asyncio/base_events.py:176]> In Py3.6, i get: cb=[_run_until_complete_cb() at /usr/lib/python3.6/asyncio/base_events.py:176]> None cb=[_run_until_complete_cb() at /usr/lib/python3.6/asyncio/base_events.py:176]> What's going on here? It's still right to expect the same result from the three calls or something has changed? The documentation for the Task.current_task() function seems unchanged between the two version. My python3.6 has version: $ python3.6 -VV Python 3.6.0 (default, Dec 29 2016, 04:29:02) [GCC 6.2.1 20161215] ---------- assignee: yselivanov components: asyncio messages: 285445 nosy: gvanrossum, yselivanov priority: normal severity: normal stage: patch review status: open title: Task.current_task(None) returns unexpected result type: behavior versions: Python 3.6, Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Jan 13 20:37:00 2017 From: report at bugs.python.org (Brian Vandenberg) Date: Sat, 14 Jan 2017 01:37:00 +0000 Subject: [New-bugs-announce] [issue29272] test_logging hangs if /etc/hosts only aliases "localhost" to ::1 Message-ID: <1484357820.6.0.682177445577.issue29272@psf.upfronthosting.co.za> New submission from Brian Vandenberg: On some of the linux boxes on our (air-gapped, if that matters) network it looks like some of them were mis-configured and their /etc/hosts file looks something like this: $ cat /etc/hosts 127.0.0.1 snoopy.the.internal.domain snoopy localhost4 localhost4.localdomain4 ::1 localhost localhost.localdomain localhost6 localhost6.localdomain6 ... whereas for other linux machines not exhibiting this problem: $ cat /etc/hosts 127.0.0.1 woodstock.the.internal.domain woodstock localhost localhost.localdomain localhost4 localhost4.localdomain4 ::1 localhost localhost.localdomain localhost6 localhost6.localdomain6 It appears this is causing test_logging to hang while running SocketHandlerTest. Unfortunately I don't have administrative privileges so I'm not able to prove I'm right. I only have circumstantial evidence, but no machine I tested (about 20 machines) contradicts what I stated above. ---------- components: Tests messages: 285452 nosy: phantal priority: normal severity: normal status: open title: test_logging hangs if /etc/hosts only aliases "localhost" to ::1 type: behavior versions: Python 3.6 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sat Jan 14 02:25:54 2017 From: report at bugs.python.org (Martin Panter) Date: Sat, 14 Jan 2017 07:25:54 +0000 Subject: [New-bugs-announce] [issue29273] test___all__ alters utf8 locale setting Message-ID: <1484378754.56.0.202012710496.issue29273@psf.upfronthosting.co.za> New submission from Martin Panter: On a Linux computer I have LANG=en_US.utf8 set. Python 3 initializes the locale with this setting (see revision 43e32b2b4004): >>> locale.setlocale(locale.LC_CTYPE) # Get setting initialized by Python 'en_US.utf8' In Lib/test/test___all__.py, there is a special case to restore the locale after loading the ?readline? module (last changed in revision 37ed6eed0595). It does this by calling getlocale() and later setlocale(). However the getlocale() call translates the setting to UTF-8 in uppercase with a dash, and setlocale() keeps it that way: >>> locale.getlocale(locale.LC_CTYPE) ('en_US', 'UTF-8') >>> locale.setlocale(locale.LC_CTYPE, _) 'en_US.UTF-8' This means that the test framework may complain: Warning -- locale was modified by test___all__ One subtlety is that I only see the warning when the ?tkinter? module is unavailable. Otherwise, the test imports Lib/idlelib/iomenu.py, which calls setlocale(locale.LC_CTYPE, "") and restores the original ?utf8? spelling. The easiest fix would be to use setlocale(locale.LC_CTYPE) instead of getlocale() in test___all__. However I think it may be safe to remove this code entirely. The original version of the it was added by Guido in revision ba522c74a931 (year 2002). Three hours later is revision c7d5801bbae8, where Guido changed the ?readline? module to restore the locale after Readline library initialization. Since the module is fixed, I don?t think we need the test workaround, and I propose to remove it in Python 3, where it causes the above conflict. ---------- components: Tests files: readline-getlocale.patch keywords: patch messages: 285467 nosy: martin.panter priority: normal severity: normal stage: patch review status: open title: test___all__ alters utf8 locale setting type: behavior versions: Python 3.5, Python 3.6, Python 3.7 Added file: http://bugs.python.org/file46287/readline-getlocale.patch _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sat Jan 14 02:30:59 2017 From: report at bugs.python.org (Martin Panter) Date: Sat, 14 Jan 2017 07:30:59 +0000 Subject: [New-bugs-announce] =?utf-8?q?=5Bissue29274=5D_Change_=E2=80=9Cte?= =?utf-8?b?c3RzIGNhc2Vz4oCdIOKGkiDigJx0ZXN0IGNhc2Vz4oCd?= Message-ID: <1484379059.0.0.00296348115264.issue29274@psf.upfronthosting.co.za> New submission from Martin Panter: The ?unittest? documentation has ?tests cases? written a few times. This doesn?t seem right to me, but I thought I should get a second opinion in case I missed something. ---------- assignee: docs at python components: Documentation files: tests-cases.patch keywords: patch messages: 285468 nosy: docs at python, martin.panter priority: normal severity: normal stage: patch review status: open title: Change ?tests cases? ? ?test cases? versions: Python 2.7, Python 3.5, Python 3.6, Python 3.7 Added file: http://bugs.python.org/file46288/tests-cases.patch _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sat Jan 14 10:46:18 2017 From: report at bugs.python.org (Elizabeth Myers) Date: Sat, 14 Jan 2017 15:46:18 +0000 Subject: [New-bugs-announce] [issue29275] time module still has Y2K issues note Message-ID: <1484408778.39.0.526783015881.issue29275@psf.upfronthosting.co.za> New submission from Elizabeth Myers: It's 2017. I think it's time to remove the Y2K warning from this: https://docs.python.org/3/library/time.html :P. It's 17 years past the sell-by date for that notice. ---------- assignee: docs at python components: Documentation messages: 285489 nosy: Elizacat, docs at python priority: normal severity: normal status: open title: time module still has Y2K issues note versions: Python 2.7, Python 3.3, Python 3.4, Python 3.5, Python 3.6, Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sat Jan 14 10:52:38 2017 From: report at bugs.python.org (Ari) Date: Sat, 14 Jan 2017 15:52:38 +0000 Subject: [New-bugs-announce] [issue29276] HTMLParser in Python 2.7 doesn't recognize image tags wrapped up in link tags Message-ID: <1484409158.63.0.451865597774.issue29276@psf.upfronthosting.co.za> New submission from Ari: The following code produces incorrect results under Python 2.7.13. One would expect it to print 2 lines, "Encountered a start tag: a" and "Encountered a start tag: img". Yet it prints only "Encountered a start tag: a". from HTMLParser import HTMLParser class MyHTMLParser(HTMLParser): def handle_starttag(self, tag, attrs): print 'Encountered a start tag: %s' % tag parser = MyHTMLParser() parser.feed('

') Python 3.5.2 produces correct results on the same input and prints the expected "Encountered a start tag: a" and "Encountered a start tag: img". from html.parser import HTMLParser class MyHTMLParser(HTMLParser): def handle_starttag(self, tag, attrs): print("Encountered a start tag:", tag) parser = MyHTMLParser() parser.feed('

') ---------- components: Library (Lib) messages: 285490 nosy: Ari priority: normal severity: normal status: open title: HTMLParser in Python 2.7 doesn't recognize image tags wrapped up in link tags type: behavior versions: Python 2.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sat Jan 14 12:07:50 2017 From: report at bugs.python.org (miniflow) Date: Sat, 14 Jan 2017 17:07:50 +0000 Subject: [New-bugs-announce] [issue29277] os.getcwd failing on LOFS share Message-ID: <1484413670.89.0.750982203265.issue29277@psf.upfronthosting.co.za> New submission from miniflow: I am calling os.getcwd and am occasionally seeing failures inside my program. This is not due to files missing as the files on the share are static. Eror is: [Errno 2] No such file or directory Which is definitely not the case. Can any sort of trace be done at the time of failure to detect if the issue is with python or the OS? System is running Debian inside of an Illumos LX zone. ---------- components: IO, Library (Lib) messages: 285492 nosy: David Terk priority: normal severity: normal status: open title: os.getcwd failing on LOFS share type: crash versions: Python 2.7, Python 3.4 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun Jan 15 02:44:06 2017 From: report at bugs.python.org (Utku Gultopu) Date: Sun, 15 Jan 2017 07:44:06 +0000 Subject: [New-bugs-announce] [issue29278] Python 3.6 build fails with parallel make Message-ID: <1484466246.11.0.873853911133.issue29278@psf.upfronthosting.co.za> New submission from Utku Gultopu: Version Info ============ Linux 4.4.0-59-generic #80-Ubuntu SMP x86_64 x86_64 x86_64 GNU/Linux Issue ===== When the multiple jobs option (`make -j`) is specified, build fails after compiling the `structmember.c` file. Subsequent compilation attempts show the following message: gcc: internal compiler error: Killed (program cc1) Please submit a full bug report, with preprocessed source if appropriate. See for instructions. ---------- components: Build messages: 285503 nosy: ugultopu priority: normal severity: normal status: open title: Python 3.6 build fails with parallel make type: compile error versions: Python 3.6 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun Jan 15 03:17:47 2017 From: report at bugs.python.org (Arcadiy Ivanov) Date: Sun, 15 Jan 2017 08:17:47 +0000 Subject: [New-bugs-announce] [issue29279] --with-pydebug optimizes too much Message-ID: <1484468267.58.0.136858573865.issue29279@psf.upfronthosting.co.za> New submission from Arcadiy Ivanov: Around line 1480 of configure.ac: if test "$Py_DEBUG" = 'true' ; then # Optimization messes up debuggers, so turn it off for # debug builds. if "$CC" -v --help 2>/dev/null |grep -- -Og > /dev/null; then OPT="-g -Og -Wall $STRICT_PROTO" else OPT="-g -O0 -Wall $STRICT_PROTO" fi else OPT="-g $WRAP -O3 -Wall $STRICT_PROTO" fi The "-Og", unfortunately, optimizes too much resulting in local variables and arguments being optimized out: (gdb) info locals kwlist = {0x7ffff02c3318 "database", 0x7ffff02c3321 "timeout", 0x7ffff02c3329 "detect_types", 0x7ffff02c3336 "isolation_level", 0x7ffff02c3346 "check_same_thread", 0x7ffff02c3392 "factory", 0x7ffff02c3358 "cached_statements", 0x7ffff02c336a "uri", 0x0} database = 0x7ffff04f7da0 ":memory:" detect_types = 0 isolation_level = 0x1 factory = 0x7ffff04c7dc0 check_same_thread = 1 cached_statements = 4888694 uri = 0 timeout = 5 (gdb) info args self = args = 0x7fffeff6be28 kwargs = 0x0 When "self" is optimized out it's too much optimization for debug mode. ---------- components: Build messages: 285505 nosy: arcivanov priority: normal severity: normal status: open title: --with-pydebug optimizes too much versions: Python 3.6 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun Jan 15 14:03:25 2017 From: report at bugs.python.org (Arthur Vanwalleghen) Date: Sun, 15 Jan 2017 19:03:25 +0000 Subject: [New-bugs-announce] [issue29280] gdbm & ndbm support missing in Windows Message-ID: <1484507005.86.0.826464747916.issue29280@psf.upfronthosting.co.za> New submission from Arthur Vanwalleghen: When I tried to work with GDBM and Ndbm files I saw that their supporting modules were not found. This can be verified with Pydoc: Pydoc\Python36-32\lib \dbm(package)\Package Contents\gnu|ndbm ErrorDuringImport: problem in dbm.gnu - ModuleNotFoundError: No module named '_gdbm' ErrorDuringImport: problem in dbm.ndbm - ModuleNotFoundError: No module named '_dbm' I first noticed this in 3.5 and waited to see if it was fixed in 3.6. I am running Win8.1. I want to work with files created on Linux/Perl using GDBM or NDBM. I cannot find any perl support for Python's dbm.dumb. I am looking for an out-of-the-box support for simple key/data files. ---------- components: Windows messages: 285517 nosy: ArtVan, paul.moore, steve.dower, tim.golden, zach.ware priority: normal severity: normal status: open title: gdbm & ndbm support missing in Windows type: behavior versions: Python 3.5, Python 3.6 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Jan 16 04:19:59 2017 From: report at bugs.python.org (Alexey Popravka) Date: Mon, 16 Jan 2017 09:19:59 +0000 Subject: [New-bugs-announce] [issue29281] json.loads documentation missing "versionchanged" statement Message-ID: <1484558399.93.0.792457597374.issue29281@psf.upfronthosting.co.za> New submission from Alexey Popravka: json.loads function was changed in Python 3.6 to accept bytes and bytearrays, however documentation is missing `versionchanged` block describing this changes. ---------- assignee: docs at python components: Documentation messages: 285545 nosy: Alexey Popravka, docs at python priority: normal severity: normal status: open title: json.loads documentation missing "versionchanged" statement versions: Python 3.6, Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Jan 16 05:01:37 2017 From: report at bugs.python.org (Juraj Sukop) Date: Mon, 16 Jan 2017 10:01:37 +0000 Subject: [New-bugs-announce] [issue29282] Fused multiply-add: proposal to add math.fma() Message-ID: <1484560897.19.0.0955404368118.issue29282@psf.upfronthosting.co.za> New submission from Juraj Sukop: Fused multiply-add (henceforth FMA) is an operation which calculates the product of two numbers and then the sum of the product and a third number with just one floating-point rounding. More concretely: r = x*y + z The value of `r` is the same as if the RHS was calculated with infinite precision and the rounded to a 32-bit single-precision or 64-bit double-precision floating-point number [1]. Even though one FMA CPU instruction might be calculated faster than the two separate instructions for multiply and add, its main advantage comes from the increased precision of numerical computations that involve the accumulation of products. Examples which benefit from using FMA are: dot product [2], compensated arithmetic [3], polynomial evaluation [4], matrix multiplication, Newton's method and many more [5]. C99 includes [6] `fma` function to `math.h` and emulates the calculation if the FMA instruction is not present on the host CPU [7]. PEP 7 states that "Python versions greater than or equal to 3.6 use C89 with several select C99 features" and that "Future C99 features may be added to this list in the future depending on compiler support" [8]. This proposal is then about adding new `fma` function with the following signature to `math` module: math.fma(x, y, z) '''Return a float representing the result of the operation `x*y + z` with single rounding error, as defined by the platform C library. The result is the same as if the operation was carried with infinite precision and rounded to a floating-point number.''' Attached is a simple module for Python 3 demonstrating the fused multiply-add operation. On my machine, `example.py` prints: 40037.524591982365 horner_double 40037.48821639768 horner_fma 40037.49486325783 horner_compensated 40037.49486325783 horner_decimal [1] https://en.wikipedia.org/wiki/Multiply%E2%80%93accumulate_operation [2] S. Graillat, P. Langlois, N. Louvet. Accurate dot products with FMA. 2006 [3] S. Graillat, Accurate Floating Point Product and Exponentiation. 2007. [4] S. Graillat, P. Langlois, N. Louvet. Improving the compensated Horner scheme with a Fused Multiply and Add. 2006 [5] J.-M. Muller, N. Brisebarre, F. de Dinechin, C.-P. Jeannerod, V. Lef?vre, G. Melquiond, N. Revol, D. Stehl?, S. Torres. Handbook of Floating-Point Arithmetic. 2010. Chapter 5 [6] ISO/IEC 9899:TC3, "7.12.13.1 The fma functions", Committee Draft - Septermber 7, 2007 [7] https://git.musl-libc.org/cgit/musl/tree/src/math/fma.c [8] https://www.python.org/dev/peps/pep-0007/ ---------- components: Library (Lib) messages: 285547 nosy: juraj.sukop priority: normal severity: normal status: open title: Fused multiply-add: proposal to add math.fma() type: enhancement versions: Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Jan 16 06:14:45 2017 From: report at bugs.python.org (jan matejek) Date: Mon, 16 Jan 2017 11:14:45 +0000 Subject: [New-bugs-announce] [issue29283] duplicate README in site-packages Message-ID: <1484565285.81.0.453565653975.issue29283@psf.upfronthosting.co.za> New submission from jan matejek: Lib/site-packages directory now contains README.txt in addition to README. Both files are identical. One of them should probably go away? ---------- components: Installation messages: 285555 nosy: matejcik priority: normal severity: normal status: open title: duplicate README in site-packages type: enhancement versions: Python 3.6, Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Jan 16 11:42:44 2017 From: report at bugs.python.org (John Taylor) Date: Mon, 16 Jan 2017 16:42:44 +0000 Subject: [New-bugs-announce] [issue29284] Include thread_name_prefix in the concurrent.futures.ThreadPoolExecutor example 17.4.2.1 Message-ID: <1484584964.59.0.239435362381.issue29284@psf.upfronthosting.co.za> New submission from John Taylor: Please include how to use the thread_name_prefix method argument (new to Python 3.6) in the example: 17.4.2.1. ThreadPoolExecutor Example ---------- assignee: docs at python components: Documentation messages: 285572 nosy: docs at python, jftuga priority: normal severity: normal status: open title: Include thread_name_prefix in the concurrent.futures.ThreadPoolExecutor example 17.4.2.1 type: enhancement versions: Python 3.6 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Jan 16 12:06:40 2017 From: report at bugs.python.org (John Taylor) Date: Mon, 16 Jan 2017 17:06:40 +0000 Subject: [New-bugs-announce] [issue29285] Unicode errors occur inside of multi-line comments Message-ID: <1484586400.61.0.654989234385.issue29285@psf.upfronthosting.co.za> New submission from John Taylor: I am using Python 3.5.2 on OS X 10.11.6. The same error occurs with 3.5.2 on Windows 10. When I run the attached code, I get this error: SyntaxError: (unicode error) 'unicodeescape' codec can't decode bytes in position 23-24: truncated \uXXXX escape I think unicode processing should be ignored inside of multiline comments. Inside of my comments, I really want to use this text verbatim. ---------- components: Interpreter Core files: example.py messages: 285573 nosy: jftuga priority: normal severity: normal status: open title: Unicode errors occur inside of multi-line comments type: behavior versions: Python 3.5 Added file: http://bugs.python.org/file46303/example.py _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Jan 16 12:30:51 2017 From: report at bugs.python.org (STINNER Victor) Date: Mon, 16 Jan 2017 17:30:51 +0000 Subject: [New-bugs-announce] [issue29286] Use METH_FASTCALL in str methods Message-ID: <1484587851.78.0.529922329593.issue29286@psf.upfronthosting.co.za> New submission from STINNER Victor: Changes 27dc9a1c061e and 01b06ca45f64 converted the str (Unicode) methods to Argument Clinic, cool! But str methods taking more than one argument use positional-only arguments. Currently, Argument Clinic doesn't use METH_FASTCALL for these methods, but METH_VARARGS. There are two options: * Allow passing arguments as keywoards: str.replace(old='a', new='b') * Enhance Argument Clinic to use also METH_FASTCALL for functions using positional-only functions The goal is to speedup method calls. Example with str.replace(): $ ./python-patch -m perf timeit '"a".replace("x", "y")' --duplicate=100 --compare-to ./python-ref python-ref: ..................... 132 ns +- 1 ns python-patch: ..................... 102 ns +- 2 ns Median +- std dev: [python-ref] 132 ns +- 1 ns -> [python-patch] 102 ns +- 2 ns: 1.30x faster (-23%) ---------- messages: 285574 nosy: haypo, inada.naoki, larry, serhiy.storchaka priority: normal severity: normal status: open title: Use METH_FASTCALL in str methods type: performance versions: Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Jan 16 17:20:08 2017 From: report at bugs.python.org (Raymond Hettinger) Date: Mon, 16 Jan 2017 22:20:08 +0000 Subject: [New-bugs-announce] [issue29287] IDLE needs syntax highlighting for f-strings Message-ID: <1484605208.08.0.296182628472.issue29287@psf.upfronthosting.co.za> New submission from Raymond Hettinger: Follow the lead from Vim, MagicPython, and PyCharm. Needs separate colorization to make the expression distinct from the rest of the string. Needs close-brace matching. Would be desirable to have autocompletion as well. https://twitter.com/raymondh/status/819085176764366848 ---------- assignee: terry.reedy components: IDLE messages: 285585 nosy: rhettinger, terry.reedy priority: normal severity: normal stage: needs patch status: open title: IDLE needs syntax highlighting for f-strings type: enhancement versions: Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Jan 16 17:30:45 2017 From: report at bugs.python.org (Ilya Kulakov) Date: Mon, 16 Jan 2017 22:30:45 +0000 Subject: [New-bugs-announce] [issue29288] Lookup Error while importing idna from a worker thread Message-ID: <1484605845.11.0.283710954459.issue29288@psf.upfronthosting.co.za> New submission from Ilya Kulakov: See this post: https://github.com/kennethreitz/requests/issues/3578 The current workaround for requests is to have a no-op import somewhere in the code. However, that doesn't really work for us: our python and stdlib are bundled by pyqtdeploy as in Qt resources system and we frequently see this error. On certain machines. Workaround that seems to work more reliably is a no-op look up via ''.encode('ascii').decode('idna'). ---------- components: Library (Lib) messages: 285586 nosy: Ilya.Kulakov priority: normal severity: normal status: open title: Lookup Error while importing idna from a worker thread versions: Python 3.5 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Jan 16 18:24:44 2017 From: report at bugs.python.org (STINNER Victor) Date: Mon, 16 Jan 2017 23:24:44 +0000 Subject: [New-bugs-announce] [issue29289] Convert OrderedDict methods to Argument Clinic Message-ID: <1484609084.15.0.406928886689.issue29289@psf.upfronthosting.co.za> New submission from STINNER Victor: Attached patch converts methods using METH_VARARGS|METH_KEYWORDS calling convention to METH_FASTCALL using Argument Clinic. This calling convention is faster, and Argument Clinic provides better docstring. See also issue #29263 "Implement LOAD_METHOD/CALL_METHOD for C functions" and issue #29286 "Use METH_FASTCALL in str methods". ---------- components: Interpreter Core files: odict_clinic.patch keywords: patch messages: 285588 nosy: eric.snow, haypo, inada.naoki priority: normal severity: normal status: open title: Convert OrderedDict methods to Argument Clinic type: performance versions: Python 3.7 Added file: http://bugs.python.org/file46306/odict_clinic.patch _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Jan 16 22:08:41 2017 From: report at bugs.python.org (Steven D'Aprano) Date: Tue, 17 Jan 2017 03:08:41 +0000 Subject: [New-bugs-announce] [issue29290] argparse breaks long lines on NO-BREAK SPACE Message-ID: <1484622521.42.0.0580919403747.issue29290@psf.upfronthosting.co.za> New submission from Steven D'Aprano: argparse help incorrectly breaks long lines on U+u00A0 NO-BREAK SPACE. The attached script has been run on Python 3.5.3rc1 in a terminal window 80 columns wide, and it produces output:: usage: argparse_nobreak.py [-h] [--no-condensedxxxx] optional arguments: -h, --help show this help message and exit --no-condensedxxxx Disable default font-style: condensed. Also disables "M+ 1M" condensed monospace. I expected the last line should have broken just before the "M+ 1M", rather than in the middle. See also #20491. ---------- components: Library (Lib) files: argparse_nobreak.py messages: 285607 nosy: steven.daprano priority: normal severity: normal status: open title: argparse breaks long lines on NO-BREAK SPACE type: behavior versions: Python 3.5 Added file: http://bugs.python.org/file46310/argparse_nobreak.py _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Jan 17 03:04:31 2017 From: report at bugs.python.org (ipolcak) Date: Tue, 17 Jan 2017 08:04:31 +0000 Subject: [New-bugs-announce] [issue29291] Misleading text in the documentation of re library for non-greedy match Message-ID: <1484640271.49.0.239743289753.issue29291@psf.upfronthosting.co.za> New submission from ipolcak: The text about non-greedy match in the documentation for re library is misleading. The docs for py2.7 (https://docs.python.org/2.7/library/re.html) and 3.6 (https://docs.python.org/3.6/library/re.html) says: "The '*', '+', and '?' qualifiers are all greedy; they match as much text as possible. Sometimes this behaviour isn?t desired; if the RE <.*> is matched against b , it will match the entire string, and not just . Adding ? after the qualifier makes it perform the match in non-greedy or minimal fashion; as few characters as possible will be matched. Using the RE <.*?> will match only ." The docs for py3.4 (https://docs.python.org/3.4/library/re.html) offers a little bit different example: "The '*', '+', and '?' qualifiers are all greedy; they match as much text as possible. Sometimes this behaviour isn?t desired; if the RE <.*> is matched against '

title

', it will match the entire string, and not just '

'. Adding '?' after the qualifier makes it perform the match in non-greedy or minimal fashion; as few characters as possible will be matched. Using .*? in the previous expression will match only '

'." However, in reality if the non-greedy match is not successful, it might fallback to the greedy match, see: >>> import re >>> a = re.compile(r"<.?>") >>> a.match(" b ") <_sre.SRE_Match object; span=(0, 15), match=' b '> >>> a.search(" b ") <_sre.SRE_Match object; span=(0, 15), match=' b '> So the '<.?>' part of the regex matches ' b ' in this example. I propose to add to the documentation the following text: "However, note that even the non-greedy version can match additional text, for example consider the RE '(<.>)' to be matched against ' b '. The match is successful and the unnamed group contains ' b '." ---------- assignee: docs at python components: Documentation messages: 285619 nosy: docs at python, ipolcak priority: normal severity: normal status: open title: Misleading text in the documentation of re library for non-greedy match type: behavior versions: Python 2.7, Python 3.4, Python 3.6 _ Python tracker _ From report at bugs.python.org Tue Jan 17 03:06:20 2017 From: report at bugs.python.org (Xiang Zhang) Date: Tue, 17 Jan 2017 08:06:20 +0000 Subject: [New-bugs-announce] [issue29292] Missing a parameter in PyEval_EvalCodeEx doc Message-ID: <1484640380.78.0.065326592654.issue29292@psf.upfronthosting.co.za> New submission from Xiang Zhang: The signature of PyEval_EvalCodeEx now gets a "PyObject kwdefs" parameter but the doc doesn't mention it. ---------- assignee: docs at python components: Documentation messages: 285620 nosy: docs at python, xiang.zhang priority: normal severity: normal status: open title: Missing a parameter in PyEval_EvalCodeEx doc versions: Python 3.5, Python 3.6, Python 3.7 _ Python tracker _ From report at bugs.python.org Tue Jan 17 03:43:56 2017 From: report at bugs.python.org (Victor de la Fuente) Date: Tue, 17 Jan 2017 08:43:56 +0000 Subject: [New-bugs-announce] [issue29293] Missing parameter "n" on multiprocessing.Condition.notify() Message-ID: <1484642636.98.0.126375980974.issue29293@psf.upfronthosting.co.za> New submission from Victor de la Fuente: Versions: Darwin MacBook-Pro-de-Victor.local 16.3.0 Darwin Kernel Version 16.3.0: Thu Nov 17 20:23:58 PST 2016; root:xnu-3789.31.2~1/RELEASE_X86_64 x86_64 Python 3.6.0 (v3.6.0:41df79263a11, Dec 22 2016, 17:23:13) [GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] on darwin import multiprocessing as mp condition = mp.Condition() with condition: condition.notify(2) #Docs show notify(n=1) #TypeError: notify() takes 1 positional argument but 2 were given #Replacing the call with condition.notify(2): #TypeError: notify() got an unexpected keyword argument 'n' # Docs show signature: def notify(self, n=1) # But found this debugging: # multiprocessing/synchronize.py # line 211: class Condition(object): # ..... # line 271: def notify(self): #<-- ?There is no n paramater? # ..... ---------- components: Library (Lib) messages: 285623 nosy: Victor de la Fuente priority: normal severity: normal status: open title: Missing parameter "n" on multiprocessing.Condition.notify() type: compile error versions: Python 3.6 _____ Python tracker _ From report at bugs.python.org Tue Jan 17 04:12:38 2017 From: report at bugs.python.org (Thomas Heller) Date: Tue, 17 Jan 2017 09:12:38 +0000 Subject: [New-bugs-announce] [issue29294] ctypes.windll.LoadLibrary refuses unicode argument Message-ID: <1484644358.67.0.715247179434.issue29294@psf.upfronthosting.co.za> New submission from Thomas Heller: ctypes.windll.LoadLibrary refuses unicode argument; this is a regression in Python 2.7.13: Python 2.7.13 (v2.7.13:a06454b1afa1, Dec 17 2016, 20:42:59) [MSC v.1500 32 bit (Intel)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> from ctypes import windll >>> windll.LoadLibrary(u"kernel32.dll") Traceback (most recent call last): File "", line 1, in File "C:\Python27\lib\ctypes\init.py", line 440, in LoadLibrary return self._dlltype(name) File "C:\Python27\lib\ctypes\init.py", line 362, in init self._handle = _dlopen(self._name, mode) TypeError: LoadLibrary() argument 1 must be string, not unicode >>> It did work in Python 2.7.12 and before: Python 2.7.12 (v2.7.12:d33e0cf91556, Jun 27 2016, 15:19:22) [MSC v.1500 32 bit (Intel)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> from ctypes import windll >>> windll.LoadLibrary(u"kernel32") >>> ---------- components: ctypes messages: 285628 nosy: theller priority: normal severity: normal status: open title: ctypes.windll.LoadLibrary refuses unicode argument versions: Python 2.7 _ Python tracker _ From report at bugs.python.org Tue Jan 17 05:47:42 2017 From: report at bugs.python.org (INADA Naoki) Date: Tue, 17 Jan 2017 10:47:42 +0000 Subject: [New-bugs-announce] [issue29295] dict: Optimize PyDict_GetItemString() Message-ID: <1484650062.84.0.813029301463.issue29295@psf.upfronthosting.co.za> New submission from INADA Naoki: PyDict_GetItemString() is heavily used, especially from keyword argument parsing. Current implementation creates temporary string for key object. This patch avoid the temporary key string when passed C string is ASCII. This benchmark is based on a8563ef0eb8a, so PyDict_GetItemString() calls for parsing positional arguments is reduced already. $ ../python -m perf compare_to -G --min-speed 2 default.json patched.json Slower (1): - scimark_lu: 430 ms +- 21 ms -> 446 ms +- 23 ms: 1.04x slower (+4%) Faster (11): - telco: 24.2 ms +- 0.4 ms -> 21.8 ms +- 0.7 ms: 1.11x faster (-10%) - xml_etree_parse: 315 ms +- 17 ms -> 302 ms +- 14 ms: 1.04x faster (-4%) - logging_simple: 31.6 us +- 0.3 us -> 30.4 us +- 0.3 us: 1.04x faster (-4%) - mako: 41.6 ms +- 0.7 ms -> 40.3 ms +- 0.4 ms: 1.03x faster (-3%) - logging_format: 36.5 us +- 0.3 us -> 35.5 us +- 0.4 us: 1.03x faster (-3%) - float: 297 ms +- 4 ms -> 289 ms +- 4 ms: 1.03x faster (-3%) - scimark_monte_carlo: 276 ms +- 10 ms -> 269 ms +- 7 ms: 1.02x faster (-2%) - regex_effbot: 5.31 ms +- 0.37 ms -> 5.19 ms +- 0.06 ms: 1.02x faster (-2%) - pickle_pure_python: 1.32 ms +- 0.02 ms -> 1.29 ms +- 0.02 ms: 1.02x faster (-2%) - scimark_sor: 525 ms +- 9 ms -> 514 ms +- 8 ms: 1.02x faster (-2%) - richards: 180 ms +- 3 ms -> 176 ms +- 2 ms: 1.02x faster (-2%) Benchmark hidden because not significant (52): ... Performance difference of telco is bit surprising. Profiler shows the difference is from `print(t, file=outfil)` (here: https://github.com/python/performance/blob/master/performance/benchmarks/bm_telco.py#L79 ) Until most common builtin functions are converted to FASTCALL, this patch has significant performance gain. ---------- components: Interpreter Core files: dict_getitemascii.patch keywords: patch messages: 285631 nosy: inada.naoki priority: normal severity: normal status: open title: dict: Optimize PyDict_GetItemString() type: performance versions: Python 3.7 Added file: http://bugs.python.org/file46313/dict_getitemascii.patch _ Python tracker _ From report at bugs.python.org Tue Jan 17 06:34:31 2017 From: report at bugs.python.org (INADA Naoki) Date: Tue, 17 Jan 2017 11:34:31 +0000 Subject: [New-bugs-announce] [issue29296] convert print() to METH_FASTCALL Message-ID: <1484652871.69.0.785445814249.issue29296@psf.upfronthosting.co.za> New submission from INADA Naoki: Median +- std dev: [default] 24.2 ms +- 0.4 ms -> [patched] 19.2 ms +- 0.4 ms: 1.26x faster (-21%) ---------- files: print-fastcall.patch keywords: patch messages: 285632 nosy: haypo, inada.naoki priority: normal severity: normal status: open title: convert print() to METH_FASTCALL type: performance versions: Python 3.7 Added file: http://bugs.python.org/file46314/print-fastcall.patch ___________________ Python tracker _ From report at bugs.python.org Tue Jan 17 09:43:36 2017 From: report at bugs.python.org (Mo Ali) Date: Tue, 17 Jan 2017 14:43:36 +0000 Subject: [New-bugs-announce] [issue29297] python3 open() does not check argument type before attempting to read() or write() Message-ID: <1484664216.19.0.300485907535.issue29297@psf.upfronthosting.co.za> New submission from Mo Ali: Python3 open(), read(), or write()doesn't check argument type before action causing a hang. Would like to catch exceptions but not without an exception to return. See below. Python3.6: Python 3.6.0 (default, Dec 24 2016, 08:01:42) [GCC 4.2.1 Compatible Apple LLVM 8.0.0 (clang-800.0.42.1)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> test = False >>> with open(str(test)) as f: ... fail = f.read() ... python2.7: ??? python2 0 < 09:35:31 Python 2.7.13 (default, Dec 18 2016, 07:03:39) [GCC 4.2.1 Compatible Apple LLVM 8.0.0 (clang-800.0.42.1)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> test = False >>> with open(test) as f: ... fail = f.read() ... Traceback (most recent call last): File "", line 1, in TypeError: coercing to Unicode: need string or buffer, bool found ---------- messages: 285644 nosy: Mo Ali priority: normal severity: normal status: open title: python3 open() does not check argument type before attempting to read() or write() type: crash versions: Python 3.5 _ Python tracker ___________________ From report at bugs.python.org Tue Jan 17 09:52:49 2017 From: report at bugs.python.org (zachrahan) Date: Tue, 17 Jan 2017 14:52:49 +0000 Subject: [New-bugs-announce] [issue29298] argparse fails with required subparsers, un-named dest, and empty argv Message-ID: <1484664769.91.0.625725054847.issue29298@psf.upfronthosting.co.za> New submission from zachrahan: In python 3.6 (and several versions back), using argparse with required subparsers will cause an unhelpful TypeError if the 'dest' parameter is not explicitly specified, and no arguments are provided. Test case: import argparse parser = argparse.ArgumentParser() subparsers = parser.add_subparsers() subparsers.required = True args = parser.parse_args([]) Observed result: TypeError: sequence item 0: expected str instance, NoneType found If the line above is changed to: subparsers = parser.add_subparsers(dest='function') Then the following is printed to stderr: usage: python [-h] {} ... python: error: the following arguments are required: function This issue goes back at least several years: http://stackoverflow.com/questions/23349349/argparse-with-required-subparser/23354355 Though it seems odd to not specify a dest in the add_subparsers line, the pattern is not completely useless. The below works fine without setting a 'dest' in add_subparsers, except when argv is empty: sub1 = subparsers.add_parser('print') sub1.set_defaults(function=print) However, an empty argv produces the unexpected TypeError above. I'm not sure if argparse should provide a more useful exception in this case, or if there is a clean way to do the right thing without a dest specified. ---------- components: Library (Lib) messages: 285646 nosy: zachrahan priority: normal severity: normal status: open title: argparse fails with required subparsers, un-named dest, and empty argv type: behavior versions: Python 3.6 _ Python tracker _ From report at bugs.python.org Tue Jan 17 10:17:14 2017 From: report at bugs.python.org (STINNER Victor) Date: Tue, 17 Jan 2017 15:17:14 +0000 Subject: [New-bugs-announce] [issue29299] Argument Clinic: Fix signature of optional positional-only arguments Message-ID: <1484666234.33.0.914406658402.issue29299@psf.upfronthosting.co.za> New submission from STINNER Victor: When a function has only positional arguments and at least one argument is optional, the expected signature is: func(mandatory_arg1, mandatory_arg2[, optional_arg3[, optinal_arg4]]) For example, the signature of format() is inconsistent with its documentation. Signature: --- $ python3 -c 'help(format)'|cat Help on built-in function format in module builtins: format(value, format_spec='', /) Return value.format(format_spec) format_spec defaults to the empty string --- Documentation: --- .. function:: format(value[, format_spec]) --- Attached patch is a first attempt to fix the issue. The problem is that my heuristic to check if an argument is "optional" doesn't seem to work as expected in all cases. I chose to check if the C default is NULL. The problem is that some functions defines a C default to NULL whereas the Python default is set to a different value and is correct. Example with _io.FileIO.truncate: /[clinic input] _io.FileIO.truncate size as posobj: object = NULL / whereas the documentation says that the default is None: .. method:: truncate(size=None) It's easy to fix the default, but in this case my change doesn't fix the signature anymore since the C default is still NULL: /[clinic input] _io.FileIO.truncate size as posobj: object(c_default="NULL") = None / We need a different heuristic than C default is NULL, or we should fix functions where the heuristic fails. ---------- components: Argument Clinic files: ac_optional_positional.patch keywords: patch messages: 285650 nosy: haypo, larry, rhettinger, serhiy.storchaka priority: normal severity: normal status: open title: Argument Clinic: Fix signature of optional positional-only arguments versions: Python 3.7 Added file: http://bugs.python.org/file46316/ac_optional_positional.patch _ Python tracker _ From report at bugs.python.org Tue Jan 17 11:31:45 2017 From: report at bugs.python.org (STINNER Victor) Date: Tue, 17 Jan 2017 16:31:45 +0000 Subject: [New-bugs-announce] [issue29300] Modify the _struct module to use FASTCALL and Argument Clinic Message-ID: <1484670705.59.0.0193443230555.issue29300@psf.upfronthosting.co.za> New submission from STINNER Victor: Attached patch modify the _struct module to use FASTCALL and Argument Clinic. AC requires a summary line, so I duplicated the first sentence of iter_unpack() docstring: iter_unpack(fmt, buffer, /) Return an iterator yielding tuples unpacked from the given bytes. Return an iterator yielding tuples unpacked from the given bytes source according to the format string, like a repeated invocation of unpack_from(). Requires that the bytes length be a multiple of the format struct size. ---------- components: Argument Clinic files: struct_fastcall.patch keywords: patch messages: 285663 nosy: haypo, inada.naoki, larry, serhiy.storchaka priority: normal severity: normal status: open title: Modify the _struct module to use FASTCALL and Argument Clinic type: performance versions: Python 3.7 Added file: http://bugs.python.org/file46319/struct_fastcall.patch _ Python tracker _ From report at bugs.python.org Tue Jan 17 11:38:44 2017 From: report at bugs.python.org (STINNER Victor) Date: Tue, 17 Jan 2017 16:38:44 +0000 Subject: [New-bugs-announce] [issue29301] decimal: Use FASTCALL and/or Argument Clinic Message-ID: <1484671124.21.0.493200110844.issue29301@psf.upfronthosting.co.za> New submission from STINNER Victor: I'm currently working on the isuse #29259: "Add tp_fastcall to PyTypeObject: support FASTCALL calling convention for all callable objects". I used bm_telco of the performance benchmark suite to check which functions still require to create a temporary tuple for positional arguments and a temporary dict for keyword arguments. I found 3 remaining functions which have an impact on the result of the benchmark: * print(): optimized by the issue #29296 * _struct.unpack(): I just created the issue #29300 "Modify the _struct module to use FASTCALL and Argument Clinic" * _decimal.Decimal.quantize() I would like to know if Stephan would be ok to modify the _decimal module to use FASTCALL. I know that recently he reverted changes to keep the same code base on Python 3.5, 3.6 and 3.7. With 4 changes (tp_fastcall #29259, print #29296, unpack #29300 and this issue), bm_telco becomes 22% faster which is not negligible! 20.9 ms +- 0.5 ms => 16.4 ms +- 0.5 ms Attached decimal.patch patch is the strict minimum change to optimize bm_telco, but I would prefer to change all _decimal functions and methods using METH_VARARGS and METH_VARARGS|METH_KEYWORDS to convert them to METH_FASTCALL. The best would be to use Argument Clinic. AC exists since Python 3.5, so it should be possible to keep the same code base on Python 3.5-3.7, only generated code would be different. ---------- files: decimal.patch keywords: patch messages: 285665 nosy: haypo, skrah priority: normal severity: normal status: open title: decimal: Use FASTCALL and/or Argument Clinic type: performance versions: Python 3.7 Added file: http://bugs.python.org/file46320/decimal.patch _ Python tracker _ From report at bugs.python.org Tue Jan 17 17:49:42 2017 From: report at bugs.python.org (Alexander Mohr) Date: Tue, 17 Jan 2017 22:49:42 +0000 Subject: [New-bugs-announce] [issue29302] add contextlib.AsyncExitStack Message-ID: <1484693382.32.0.211448130443.issue29302@psf.upfronthosting.co.za> New submission from Alexander Mohr: ExitStack is a really useful class and would be a great to have an async version. I've gone ahead and created an implementation based on the existing Python 3.5.2 implementation. Let me know what you guys think. I think it would be possible to combine most of the two classes together if you guys think it would be useful. Let me know if I can/should create a github PR and where to do that. ---------- components: Library (Lib) files: exit_stack.py messages: 285687 nosy: thehesiod priority: normal severity: normal status: open title: add contextlib.AsyncExitStack type: enhancement versions: Python 3.6, Python 3.7 Added file: http://bugs.python.org/file46322/exit_stack.py ___________________ Python tracker _ From report at bugs.python.org Tue Jan 17 18:26:15 2017 From: report at bugs.python.org (anonymous2017) Date: Tue, 17 Jan 2017 23:26:15 +0000 Subject: [New-bugs-announce] [issue29303] asyncio.Lock.acquire() does not always yield Message-ID: <1484695575.39.0.807179616569.issue29303@psf.upfronthosting.co.za> New submission from anonymous2017: Details here: http://stackoverflow.com/questions/41708609/unfair-scheduling-bad-lock-optimization-in-asyncio-event-loop Essentially there should be a `yield` before this line otherwise a coroutine that only acquires and releases a lock will never yield to other coroutines. ---------- components: asyncio messages: 285688 nosy: anonymous2017, gvanrossum, yselivanov priority: normal severity: normal status: open title: asyncio.Lock.acquire() does not always yield type: behavior versions: Python 3.6 _ Python tracker ___________________ From report at bugs.python.org Tue Jan 17 21:36:13 2017 From: report at bugs.python.org (INADA Naoki) Date: Wed, 18 Jan 2017 02:36:13 +0000 Subject: [New-bugs-announce] [issue29304] dict: simplify lookup function Message-ID: <1484706973.14.0.775713839146.issue29304@psf.upfronthosting.co.za> New submission from INADA Naoki: All lookdict* functions are implemented like pseudo language: ``` lookup() if not collision: return result while True: perturb_shift() lookup() if not collision: return result ``` This patch changes it as: ``` while True: lookup() if not collision: return result perturb_shift() ``` It removes 100 lines of code. Good. But how about performance? When this patch is applied to 4a534c45bbf6: ``` $ ../python.patched -m perf compare_to default.json patched2.json -G --min-speed=2 Slower (4): - xml_etree_generate: 271 ms +- 6 ms -> 283 ms +- 9 ms: 1.04x slower (+4%) - nqueens: 263 ms +- 4 ms -> 272 ms +- 3 ms: 1.04x slower (+4%) - scimark_monte_carlo: 272 ms +- 10 ms -> 280 ms +- 14 ms: 1.03x slower (+3%) - scimark_lu: 435 ms +- 23 ms -> 446 ms +- 32 ms: 1.03x slower (+3%) Faster (7): - call_method: 15.2 ms +- 0.2 ms -> 14.7 ms +- 0.4 ms: 1.04x faster (-4%) - call_simple: 14.4 ms +- 0.2 ms -> 13.9 ms +- 0.3 ms: 1.04x faster (-4%) - xml_etree_iterparse: 227 ms +- 9 ms -> 219 ms +- 7 ms: 1.04x faster (-3%) - scimark_sor: 527 ms +- 10 ms -> 510 ms +- 11 ms: 1.03x faster (-3%) - call_method_slots: 14.7 ms +- 0.5 ms -> 14.3 ms +- 0.2 ms: 1.03x faster (-3%) - genshi_text: 90.2 ms +- 1.1 ms -> 87.8 ms +- 1.1 ms: 1.03x faster (-3%) - django_template: 403 ms +- 5 ms -> 394 ms +- 4 ms: 1.02x faster (-2%) Benchmark hidden because not significant (53): 2to3, ... ``` And when this patch applied to 1a97b10cb420 : ``` $ ../python.patched -m perf compare_to default.json patched.json -G --min-speed=2 Slower (6): - call_simple: 13.5 ms +- 0.5 ms -> 14.4 ms +- 0.4 ms: 1.07x slower (+7%) - xml_etree_generate: 270 ms +- 6 ms -> 287 ms +- 5 ms: 1.06x slower (+6%) - xml_etree_process: 240 ms +- 6 ms -> 247 ms +- 4 ms: 1.03x slower (+3%) - regex_compile: 429 ms +- 3 ms -> 440 ms +- 5 ms: 1.03x slower (+3%) - call_method_unknown: 16.1 ms +- 0.2 ms -> 16.5 ms +- 0.3 ms: 1.02x slower (+2%) - logging_simple: 31.2 us +- 0.4 us -> 32.0 us +- 0.3 us: 1.02x slower (+2%) Faster (8): - genshi_text: 90.6 ms +- 1.4 ms -> 87.6 ms +- 1.2 ms: 1.03x faster (-3%) - scimark_sor: 513 ms +- 11 ms -> 497 ms +- 12 ms: 1.03x faster (-3%) - genshi_xml: 200 ms +- 2 ms -> 194 ms +- 2 ms: 1.03x faster (-3%) - unpickle_pure_python: 857 us +- 21 us -> 835 us +- 13 us: 1.03x faster (-3%) - python_startup_no_site: 9.95 ms +- 0.02 ms -> 9.74 ms +- 0.02 ms: 1.02x faster (-2%) - json_dumps: 29.7 ms +- 0.4 ms -> 29.1 ms +- 0.4 ms: 1.02x faster (-2%) - xml_etree_iterparse: 225 ms +- 9 ms -> 220 ms +- 5 ms: 1.02x faster (-2%) - chameleon: 31.1 ms +- 0.3 ms -> 30.5 ms +- 0.5 ms: 1.02x faster (-2%) Benchmark hidden because not significant (50): 2to3, ... ``` I can't see any stable and significant performance regression. I'll try to create some micro benchmarks. ---------- files: dictlook-refactoring.patch keywords: patch messages: 285695 nosy: inada.naoki priority: normal severity: normal status: open title: dict: simplify lookup function Added file: http://bugs.python.org/file46324/dictlook-refactoring.patch _ Python tracker _ From report at bugs.python.org Wed Jan 18 03:41:01 2017 From: report at bugs.python.org (Babe Hardy) Date: Wed, 18 Jan 2017 08:41:01 +0000 Subject: [New-bugs-announce] [issue29305] encoding to ascii in client.py Message-ID: <1484728861.39.0.489354661051.issue29305@psf.upfronthosting.co.za> New submission from Babe Hardy: used urlopen('...{}'.format(v)).read() with v being a string when v is of a non-ascii format: >>UnicodeEncodeError: 'ascii' codec can't encode characters<< in line 984 of putrequest after changing >>self._output(request.encode('ascii'))<< into >>self._output(request.encode('utf-8'))<< my script worked again ---------- components: Library (Lib) messages: 285707 nosy: Babe Hardy priority: normal severity: normal status: open title: encoding to ascii in client.py type: compile error versions: Python 3.5 _ Python tracker _ From report at bugs.python.org Wed Jan 18 04:04:45 2017 From: report at bugs.python.org (STINNER Victor) Date: Wed, 18 Jan 2017 09:04:45 +0000 Subject: [New-bugs-announce] [issue29306] Check usage of Py_EnterRecursiveCall() and Py_LeaveRecursiveCall() in new FASTCALL functions Message-ID: <1484730285.0.0.960009012423.issue29306@psf.upfronthosting.co.za> New submission from STINNER Victor: I added the following new functions to Python 3.6: * _PyObject_FastCallDict() * _PyObject_FastCallKeywords() * _PyFunction_FastCallDict() * _PyFunction_FastCallKeywords() * _PyCFunction_FastCallDict() * _PyCFunction_FastCallKeywords() I'm not sure that these functions update correctly the "recursion_depth" counter using Py_EnterRecursiveCall() and Py_LeaveRecursiveCall(). ---------- components: Interpreter Core messages: 285709 nosy: haypo, inada.naoki priority: normal severity: normal status: open title: Check usage of Py_EnterRecursiveCall() and Py_LeaveRecursiveCall() in new FASTCALL functions type: behavior versions: Python 3.6, Python 3.7 _ Python tracker _ From report at bugs.python.org Wed Jan 18 05:56:23 2017 From: report at bugs.python.org (Jeroen Van Goey) Date: Wed, 18 Jan 2017 10:56:23 +0000 Subject: [New-bugs-announce] [issue29307] ModuleNotFoundError when using literal string interpolation with invalid format specifier Message-ID: <1484736983.43.0.107395329278.issue29307@psf.upfronthosting.co.za> New submission from Jeroen Van Goey: I installed Python 3.6 from J Fernyhough's PPA by doing sudo add-apt-repository ppa:jonathonf/python-3.6 sudo apt-get update sudo apt-get install python3.6 I made a string, using the new literal string interpolation, but I supplied an invalid format specifier. I not only got the expected "ValueError: Invalid format specifier", but also the unexpected "ModuleNotFoundError: No module named 'apt_pkg'". $ python3.6 Python 3.6.0 (default, Dec 29 2016, 21:40:36) [GCC 5.4.1 20161202] on linux Type "help", "copyright", "credits" or "license" for more information. >>> value = 4 * 20 >>> f'the value is {value:%A}' Traceback (most recent call last): File "", line 1, in ValueError: Invalid format specifier Error in sys.excepthook: Traceback (most recent call last): File "/usr/lib/python3/dist-packages/apport_python_hook.py", line 63, in apport_excepthook from apport.fileutils import likely_packaged, get_recent_crashes File "/usr/lib/python3/dist-packages/apport/init.py", line 5, in from apport.report import Report File "/usr/lib/python3/dist-packages/apport/report.py", line 30, in import apport.fileutils File "/usr/lib/python3/dist-packages/apport/fileutils.py", line 23, in from apport.packaging_impl import impl as packaging File "/usr/lib/python3/dist-packages/apport/packaging_impl.py", line 23, in import apt File "/usr/lib/python3/dist-packages/apt/init.py", line 23, in import apt_pkg ModuleNotFoundError: No module named 'apt_pkg' Original exception was: Traceback (most recent call last): File "", line 1, in ValueError: Invalid format specifier ---------- messages: 285716 nosy: jeroen-vangoey priority: normal severity: normal status: open title: ModuleNotFoundError when using literal string interpolation with invalid format specifier type: behavior versions: Python 3.6 _ Python tracker _ From report at bugs.python.org Wed Jan 18 09:39:37 2017 From: report at bugs.python.org (Jack Bennett) Date: Wed, 18 Jan 2017 14:39:37 +0000 Subject: [New-bugs-announce] [issue29308] venv should match virtualenv VIRTUAL_ENV_DISABLE_PROMPT config Message-ID: <1484750377.25.0.809506276992.issue29308@psf.upfronthosting.co.za> New submission from Jack Bennett: virtualenv has https://virtualenv.pypa.io/en/latest/reference/#envvar-VIRTUAL_ENV_DISABLE_PROMPT to block activate from trying to overwrite the prompt function. Users of venv can't do that, 3.6 added prompt but even None defaults to the current folder name. If I may suggest either respecting the virtualenv variable or perhaps prompt=False Specifically we've ran into this in powershell while making cmder stricter on the prompt over here: https://github.com/cmderdev/cmder/issues/1207 Thanks, Jack ---------- components: Library (Lib) messages: 285729 nosy: Jack Bennett priority: normal severity: normal status: open title: venv should match virtualenv VIRTUAL_ENV_DISABLE_PROMPT config type: behavior versions: Python 3.6 _ Python tracker _ From report at bugs.python.org Wed Jan 18 09:58:33 2017 From: report at bugs.python.org (=?utf-8?q?Romain_S=C3=A9bille?=) Date: Wed, 18 Jan 2017 14:58:33 +0000 Subject: [New-bugs-announce] [issue29309] Interpreter hang when interrupting a loop.run_in_executor() future Message-ID: <1484751513.93.0.173979168861.issue29309@psf.upfronthosting.co.za> New submission from Romain S?bille: Hi, I stumble on this today and can't find any reasons or previous issue as to why this happen so I thought of submitting the issue here. I attached a script (run_in_executor.py) to reproduce the problem, the original problem arise when waiting to read on a socket but I have the same result with time.sleep(). The output: $ python3.6 run_in_executor.py Start Going to sleep ^CClosing loop Traceback (most recent call last): File "run_in_executor.py", line 15, in loop.run_until_complete(sleep_in_executor(loop)) File "/usr/local/lib/python3.6/asyncio/base_events.py", line 454, in run_until_complete self.run_forever() File "/usr/local/lib/python3.6/asyncio/base_events.py", line 421, in run_forever self._run_once() File "/usr/local/lib/python3.6/asyncio/base_events.py", line 1389, in _run_once event_list = self._selector.select(timeout) File "/usr/local/lib/python3.6/selectors.py", line 445, in select fd_event_list = self._epoll.poll(timeout, max_ev) KeyboardInterrupt ^CError in atexit._run_exitfuncs: Traceback (most recent call last): File "/usr/local/lib/python3.6/concurrent/futures/thread.py", line 39, in _python_exit t.join() File "/usr/local/lib/python3.6/threading.py", line 1056, in join self._wait_for_tstate_lock() File "/usr/local/lib/python3.6/threading.py", line 1072, in _wait_for_tstate_lock elif lock.acquire(block, timeout): KeyboardInterrupt Current behavior: We have to send two ^C before we go back to the shell. Expected behavior: Only one ^C to go back to the shell. Python versions tested: - Python 3.4.2 - Python 3.4.6 - Python 3.5.3 - Python 3.6.0 >From what I have gathered the error is within the ThreadPoolExecutor cleaning function but I don't see why there are still living workers, the Executor.shutdown() take care of that. Unfortunately I don't really know what to look, is a AsyncIO bug ? A ThreadPoolExecutor bug ? Or something else. ---------- components: asyncio files: run_in_executor.py messages: 285732 nosy: gvanrossum, rsebille, yselivanov priority: normal severity: normal status: open title: Interpreter hang when interrupting a loop.run_in_executor() future type: behavior versions: Python 3.4, Python 3.5, Python 3.6 Added file: http://bugs.python.org/file46328/run_in_executor.py ___________________ Python tracker _ From report at bugs.python.org Wed Jan 18 11:09:17 2017 From: report at bugs.python.org (Jelle Zijlstra) Date: Wed, 18 Jan 2017 16:09:17 +0000 Subject: [New-bugs-announce] [issue29310] Document typing.NamedTuple default argument syntax Message-ID: <1484755757.54.0.970081096642.issue29310@psf.upfronthosting.co.za> New submission from Jelle Zijlstra: Document https://github.com/python/typing/pull/338 I should have a patch in the next few days; creating this issue to remind myself. ---------- assignee: Jelle Zijlstra components: Documentation messages: 285740 nosy: Jelle Zijlstra priority: normal severity: normal status: open title: Document typing.NamedTuple default argument syntax versions: Python 3.6, Python 3.7 _ Python tracker ___________________ From report at bugs.python.org Wed Jan 18 11:54:10 2017 From: report at bugs.python.org (STINNER Victor) Date: Wed, 18 Jan 2017 16:54:10 +0000 Subject: [New-bugs-announce] [issue29311] Argument Clinic: convert dict methods Message-ID: <1484758450.31.0.717694851278.issue29311@psf.upfronthosting.co.za> New submission from STINNER Victor: The dict type has multiple methods which use METH_VARARGS or METH_VARARGS|METH_KEYWORDS calling convention, whereras there is no a new faster METH_FASTCALL calling convention which avoids temporary tuple/dict to pass arguments. ---------- components: Argument Clinic messages: 285742 nosy: haypo, inada.naoki, larry priority: normal severity: normal status: open title: Argument Clinic: convert dict methods type: performance versions: Python 3.7 _ Python tracker _ From report at bugs.python.org Wed Jan 18 12:07:42 2017 From: report at bugs.python.org (STINNER Victor) Date: Wed, 18 Jan 2017 17:07:42 +0000 Subject: [New-bugs-announce] [issue29312] Use FASTCALL in dict.update() Message-ID: <1484759262.53.0.625567482548.issue29312@psf.upfronthosting.co.za> New submission from STINNER Victor: Follow-up of the issue #29311 "Argument Clinic: convert dict methods". The dict.update() method hs a special prototype: def update(arg=None **kw): ... I don't think that Argument Clinic supports it right now, so I propose to first optimize dict_update() to use METH_FASTCALL. Attached patch is a first step: convert kwnames tuple to a dict. A second step would be to avoid the temporary dict and update the dict using args + kwnames directly. ---------- files: dict_update_fastcall.patch keywords: patch messages: 285744 nosy: haypo, inada.naoki priority: normal severity: normal status: open title: Use FASTCALL in dict.update() type: enhancement versions: Python 3.7 Added file: http://bugs.python.org/file46330/dict_update_fastcall.patch _ Python tracker _ From report at bugs.python.org Wed Jan 18 14:38:39 2017 From: report at bugs.python.org (eszense) Date: Wed, 18 Jan 2017 19:38:39 +0000 Subject: [New-bugs-announce] [issue29313] msi by bdist_msi will fail execute install-scripts if space in present in python path Message-ID: <1484768319.35.0.107822358832.issue29313@psf.upfronthosting.co.za> New submission from eszense: If python.exe path contains space, msi generated with bdist_msi command with --install-script=abc.py will fail with msg: C:\\Program Files\\Python35\\python.exe: can't open file 'C:\\Program': [Error 2] No such file or directory Patch attached for your consideration ---------- components: Distutils files: patch.diff keywords: patch messages: 285749 nosy: dstufft, eric.araujo, eszense priority: normal severity: normal status: open title: msi by bdist_msi will fail execute install-scripts if space in present in python path type: crash Added file: http://bugs.python.org/file46333/patch.diff ___________________ Python tracker _ From report at bugs.python.org Wed Jan 18 15:07:47 2017 From: report at bugs.python.org (R. David Murray) Date: Wed, 18 Jan 2017 20:07:47 +0000 Subject: [New-bugs-announce] [issue29314] asyncio.async deprecation warning is missing stacklevel=2 Message-ID: <1484770067.77.0.974182324744.issue29314@psf.upfronthosting.co.za> New submission from R. David Murray: Title pretty much says it all. The warning isn't very helpful currently, since it just reports the line in tasks.py where the deprecation warning is generated. ---------- messages: 285750 nosy: r.david.murray priority: normal severity: normal status: open title: asyncio.async deprecation warning is missing stacklevel=2 versions: Python 3.5, Python 3.6, Python 3.7 _ Python tracker _ From report at bugs.python.org Wed Jan 18 15:21:15 2017 From: report at bugs.python.org (Mike Lissner) Date: Wed, 18 Jan 2017 20:21:15 +0000 Subject: [New-bugs-announce] [issue29315] \b requires raw strings or to be escaped. Update docs with that hint? Message-ID: <1484770875.58.0.864108991771.issue29315@psf.upfronthosting.co.za> New submission from Mike Lissner: I just ran into a funny corner case I imagine others are aware of. When you write "\b" in Python, it is a single character: "\x08". So if you try to write a regex like: words = '\b(.)\b' That won't work. But using a raw string will: words = r'\b(.)\b' As will escaping it in this horrible fashion: words = '\\b(.)\\b' I believe this doesn't affect any of the other regex flags, so I wonder if it's worth adding something to the docs to warn about this. I just spent a bunch of time trying to figure out why it seemed like \b wasn't working. A little tip in the docs would have gone a LONG way. ---------- assignee: docs at python components: Documentation messages: 285751 nosy: Mike.Lissner, docs at python priority: normal severity: normal status: open title: \b requires raw strings or to be escaped. Update docs with that hint? type: enhancement versions: Python 2.7, Python 3.3, Python 3.4, Python 3.5, Python 3.6, Python 3.7 _ Python tracker _ From report at bugs.python.org Wed Jan 18 16:20:07 2017 From: report at bugs.python.org (Guido van Rossum) Date: Wed, 18 Jan 2017 21:20:07 +0000 Subject: [New-bugs-announce] [issue29316] Can we keep typing.py provisional for the duration of the Python 3.6 release cycle? Message-ID: <1484774407.47.0.689245055951.issue29316@psf.upfronthosting.co.za> New submission from Guido van Rossum: The 3.6 what's new (https://docs.python.org/3/whatsnew/3.6.html#whatsnew36-typing) claims typing.py is no longer provisional, but I don't think it's quite ready yet. There are a number of things that I'd like to merge into typing.py over the duration of 3.6's lifetime. Nothing backwards incompatible, but definitely new features. See also https://github.com/python/typing/issues/247 ---------- messages: 285758 nosy: gvanrossum priority: normal severity: normal status: open title: Can we keep typing.py provisional for the duration of the Python 3.6 release cycle? _ Python tracker ___________________ From report at bugs.python.org Wed Jan 18 18:33:31 2017 From: report at bugs.python.org (mark) Date: Wed, 18 Jan 2017 23:33:31 +0000 Subject: [New-bugs-announce] [issue29317] test_copyxattr_symlinks fails Message-ID: <1484782411.92.0.520036190929.issue29317@psf.upfronthosting.co.za> New submission from mark: ====================================================================== ERROR: test_copyxattr_symlinks (test.test_shutil.TestShutil) ---------------------------------------------------------------------- Traceback (most recent call last): File "/opt/Python-3.6.0/Lib/test/test_shutil.py", line 505, in test_copyxattr_symlinks os.setxattr(src, 'trusted.foo', b'42') PermissionError: [Errno 1] Operation not permitted: '/tmp/tmpvlu10qdm/foo' ---------------------------------------------------------------------- The problem is that "trusted" attributes (in this case trusted.foo) are visible and accessible only to processes that have the CAP_SYS_ADMIN capability. The current "skipUnless" guard is wrong: being root (UID 0) doesn't necessarely imply you have CAP_SYS_ADMIN. For instance this test (and thus "make test") will always fail in a Docker container unless it's started with "--cap-add SYS_ADMIN" (which, in general, is not the best thing to do). ---------- components: Tests messages: 285764 nosy: marktay priority: normal severity: normal status: open title: test_copyxattr_symlinks fails versions: Python 3.6 _ Python tracker _ From report at bugs.python.org Thu Jan 19 06:33:01 2017 From: report at bugs.python.org (STINNER Victor) Date: Thu, 19 Jan 2017 11:33:01 +0000 Subject: [New-bugs-announce] [issue29318] Optimize _PyFunction_FastCallDict() for kwargs Message-ID: <1484825581.48.0.790581995103.issue29318@psf.upfronthosting.co.za> New submission from STINNER Victor: Example: --- def func(x, y): print(x, y) def proxy2(func, kw): func(kw) def proxy1(func, kw): proxy2(func, kw) --- The "proxy2(func, kw)" call in proxy1() is currently inefficient: _PyFunction_FastCallDict() converts the dictionary into a C array [key1, value1, key2, value2, ...] and then _PyEval_EvalCodeWithName() rebuilds the dictionary from the C array. Since "func(args, kw)" proxies are common in Python, especially to call the parent constructor when overriding init, I think that it would be interesting to optimize this code path. I first expected that it was a regression of FASTCALL, but Python < 3.6 doesn't optimize this code neither. ---------- components: Interpreter Core messages: 285773 nosy: haypo, inada.naoki priority: normal severity: normal status: open title: Optimize _PyFunction_FastCallDict() for kwargs type: performance versions: Python 3.7 ___________________ Python tracker _ From report at bugs.python.org Thu Jan 19 07:11:31 2017 From: report at bugs.python.org (Paul Moore) Date: Thu, 19 Jan 2017 12:11:31 +0000 Subject: [New-bugs-announce] [issue29319] Embedded 3.6.0 distribution cannot run pyz files Message-ID: <1484827891.73.0.559570067048.issue29319@psf.upfronthosting.co.za> New submission from Paul Moore: Trying to run a pyz file using the embedded distribution for 3.6.0, I get an error "Could not import runpy module". To reproduce, see below: >type .\main.py print('Hello, world') >zip test.pyz main.py adding: main.py (172 bytes security) (stored 0%) >unzip .\python-3.6.0-embed-amd64.zip ... >.\python.exe .\test.pyz Could not import runpy module ModuleNotFoundError: No module named 'runpy' Running with the standard interpreter works fine: >py -V Python 3.6.0 >py .\test.pyz Hello, world ---------- assignee: steve.dower components: Windows messages: 285783 nosy: paul.moore, steve.dower, tim.golden, zach.ware priority: normal severity: normal status: open title: Embedded 3.6.0 distribution cannot run pyz files type: behavior versions: Python 3.6 _ Python tracker ___________________ From report at bugs.python.org Thu Jan 19 07:53:40 2017 From: report at bugs.python.org (eszense) Date: Thu, 19 Jan 2017 12:53:40 +0000 Subject: [New-bugs-announce] [issue29320] bdist_msi install_script fail to execute if alt python location specified Message-ID: <1484830420.95.0.711337585743.issue29320@psf.upfronthosting.co.za> New submission from eszense: if install-script specified and alternative python location selected during installation, msiexec will fail with: "There is a problem with this Windows Installer package. A program required for this install to complete could not be run. Contact your support personnel or package vendor. This is due to the lack of PythonExeX Custom Action setting the path to python.exe Patch attacked for your consideration ---------- files: patch.txt messages: 285786 nosy: eszense priority: normal pull_requests: 18 severity: normal status: open title: bdist_msi install_script fail to execute if alt python location specified type: crash Added file: http://bugs.python.org/file46336/patch.txt ___________________ Python tracker _ From report at bugs.python.org Thu Jan 19 09:26:43 2017 From: report at bugs.python.org (RK-5wWm9h) Date: Thu, 19 Jan 2017 14:26:43 +0000 Subject: [New-bugs-announce] [issue29321] Wrong documentation for unicode and str comparison Message-ID: <1484836003.73.0.224345826816.issue29321@psf.upfronthosting.co.za> New submission from RK-5wWm9h: PROBLEM (IN BRIEF): In the currently published 2.7.13 The Python Language Reference manual, section 5.9 "Comparisons" (https://docs.python.org/2/reference/expressions.html#comparisons): "If both are numbers, they are converted to a common type. Otherwise, objects of different types always compare unequal..." This an incorrect (and misleading) statement. PROPOSED FIX: Insert a new sentence, to give this resulting text: "If both are numbers, they are converted to a common type. If one is str and the other unicode, they are compared as below. Otherwise, objects of different types always compare unequal..." DETAILS, JUSTIFICATION, CORRECTNESS, ETC: The behaviour that a str and a unicode object -- despite being objects of different types -- may compare equal, is explicitly stated several paragraphs later: "* Strings are compared lexicographically using the numeric equivalents (the result of the built-in function ord()) of their characters. Unicode and 8-bit strings are fully interoperable in this behavior. [4]" Text in the 2.7.13 The Python Standard Library (Library Reference manual) is careful to cover this unicode - str case (https://docs.python.org/2/library/stdtypes.html#comparisons): "Objects of different types, except different numeric types and different string types, never compare equal; such objects are ordered consistently but arbitrarily (so that sorting a heterogeneous array yields a consistent result)." IMPACT AND RELATED BUG: The current incorrect text is really misleading for anyone reading the Language Ref. It's easy to see the categorical statement and stop reading because your question has been answered. Further, the Library ref about unicode and str (The Python Standard Library (Library Reference manual) section 5.6 "Sequence Types": https://docs.python.org/2/library/stdtypes.html#sequence-types-str-unicode-list-tuple-bytearray-buffer-xrange), links here. Link text: "(For full details see Comparisons in the language reference.)". (Aside: That paragraph has a mistake similar to this present bug: it says "to compare equal, every element must compare equal and the two sequences must be of the same type"; I'll file a separate bug for it.) PS: First time reporting a Python bug; following https://docs.python.org/2/bugs.html. Hope I did ok! :-) ---------- assignee: docs at python components: Documentation messages: 285790 nosy: RK-5wWm9h, docs at python priority: normal severity: normal status: open title: Wrong documentation for unicode and str comparison type: behavior versions: Python 2.7 _ Python tracker _______________ From report at bugs.python.org Thu Jan 19 09:39:52 2017 From: report at bugs.python.org (Michael J) Date: Thu, 19 Jan 2017 14:39:52 +0000 Subject: [New-bugs-announce] [issue29322] SimpleCV error on Raspberry Pi Message-ID: <1484836792.51.0.0433994997007.issue29322@psf.upfronthosting.co.za> New submission from Michael J: Hello, I'm using a Microsoft LifeCam with SimpleCV on the ipython interpreter(or shell) on a Raspberry Pi 3 model B(with Raspbian, tell me if you need the exact OS version as it's quite complicated). I'm following the directions of a book: Raspberry Pi Cookbook by Simon Monk. I get this error: SimpleCV:1> c = Camera() SimpleCV:2> i = c.getImage() SimpleCV:3> i SimpleCV:3: SimpleCV:4> SimpleCV:4> i.show() --------------------------------------------------------------------------- IOError Traceback (most recent call last) /usr/local/lib/python2.7/dist-packages/SimpleCV/Shell/Shell.pyc in () ----> 1 i.show() /usr/local/lib/python2.7/dist-packages/SimpleCV/ImageClass.pyc in show(self, type) 5445 d = Display(displaytype='notebook') 5446 else: -> 5447 d = Display(self.size()) 5448 self.save(d) 5449 return d /usr/local/lib/python2.7/dist-packages/SimpleCV/Display.pyc in init(self, resolution, flags, title, displaytype, headless) 156 if not displaytype == 'notebook': 157 self.screen = pg.display.set_mode(resolution, flags) --> 158 scvLogo = SimpleCV.Image("simplecv").scale(32,32) 159 pg.display.set_icon(scvLogo.getPGSurface()) 160 if flags != pg.FULLSCREEN and flags != pg.NOFRAME: /usr/local/lib/python2.7/dist-packages/SimpleCV/ImageClass.pyc in init(self, source, camera, colorSpace, verbose, sample, cv2image) 785 self._bitmap = cv.LoadImage(self.filename, iscolor=cv.CV_LOAD_IMAGE_COLOR) 786 except: --> 787 self._pil = pil.open(self.filename).convert("RGB") 788 self._bitmap = cv.CreateImageHeader(self._pil.size, cv.IPL_DEPTH_8U, 3) 789 cv.SetData(self._bitmap, self._pil.tostring()) /usr/local/lib/python2.7/dist-packages/PIL/Image.pyc in open(fp, mode) 2280 2281 if filename: -> 2282 fp = builtins.open(filename, "rb") 2283 2284 try: IOError: [Errno 2] No such file or directory: '/usr/local/lib/python2.7/dist-packages/SimpleCV/sampleimages/simplecv.png' I(Michael J or "MJ" as I'm commonly called) have searched for the directory myself, and I can tell you, it's not there. Tell me if you need more information, as I will be glad to assist. I'm sorry for any inconvenience caused by me using ipython if it's unfamiliar. ---------- components: IO messages: 285791 nosy: MJ priority: normal severity: normal status: open title: SimpleCV error on Raspberry Pi type: behavior versions: Python 2.7 ___________________ Python tracker _ From report at bugs.python.org Thu Jan 19 09:40:31 2017 From: report at bugs.python.org (RK-5wWm9h) Date: Thu, 19 Jan 2017 14:40:31 +0000 Subject: [New-bugs-announce] [issue29323] Wrong documentation (Library) for unicode and str comparison Message-ID: <1484836831.64.0.870722472975.issue29323@psf.upfronthosting.co.za> New submission from RK-5wWm9h: PROBLEM (IN BRIEF): In the currently published 2.7.13 The Python Standard Library (Library Reference manual) section 5.6 "Sequence Types" (https://docs.python.org/2/library/stdtypes.html#sequence-types-str-unicode-list-tuple-bytearray-buffer-xrange): "to compare equal, ... the two sequences must be of the same type" This an incorrect (and misleading) statement, for the unicode and str case. PROPOSED FIX: Current full paragraph: "Sequence types also support comparisons. In particular, tuples and lists are compared lexicographically by comparing corresponding elements. This means that to compare equal, every element must compare equal and the two sequences must be of the same type and have the same length. (For full details see Comparisons in the language reference.)" Proposed replacement text: "Sequence types also support comparisons. In particular, tuples and lists are compared lexicographically by comparing corresponding elements. This means that to compare equal, every element must compare equal and the two sequences must be of the same type and have the same length. (Unicode and str are treated as the same type here; for full details see Comparisons in the language reference.)" DETAILS, JUSTIFICATION, CORRECTNESS, ETC: The current incorrect text is really misleading. The behaviour that a str and a unicode object -- despite being objects of different types -- may compare equal, is explicitly stated in the 2.7.13 The Python Language Reference manual, section 5.9 "Comparisons" (https://docs.python.org/2/reference/expressions.html#comparisons): "* Strings are compared lexicographically using the numeric equivalents (the result of the built-in function ord()) of their characters. Unicode and 8-bit strings are fully interoperable in this behavior. [4]" (Aside: Incidentally an earlier paragraph in the Language Ref fails to cover the unicode and str case; see separately filed bug Issue 29321.) ---------- assignee: docs at python components: Documentation messages: 285792 nosy: RK-5wWm9h, docs at python priority: normal severity: normal status: open title: Wrong documentation (Library) for unicode and str comparison type: behavior versions: Python 2.7 _ Python tracker ___________________ From report at bugs.python.org Thu Jan 19 10:01:06 2017 From: report at bugs.python.org (Christian Heimes) Date: Thu, 19 Jan 2017 15:01:06 +0000 Subject: [New-bugs-announce] [issue29324] test_aead_aes_gcm fails on Kernel 4.9 Message-ID: <1484838066.61.0.93462051202.issue29324@psf.upfronthosting.co.za> New submission from Christian Heimes: $ ./python -m test -m test_aead_aes_gcm test_socket Run tests sequentially 0:00:00 [1/1] test_socket test test_socket failed -- Traceback (most recent call last): File "/home/heimes/dev/python/cpython/Lib/test/support/init.py", line 556, in wrapper return func(*args, **kw) File "/home/heimes/dev/python/cpython/Lib/test/test_socket.py", line 5515, in test_aead_aes_gcm res = op.recv(assoclen + len(plain) + taglen) OSError: [Errno 22] Invalid argument The tests were written and passed under Linux Kernel 4.7. I was under the assumption that the API is stable. But the most recent version 4.9 has changed the API for AEAD mode slightly. libkcapi project maintain a well written documentation of the Kernel crypto API, https://github.com/smuellerDD/libkcapi/commit/be242c387b7030cbccae2c183107efa86d9a3cd6 Fedora 24 and 25 are affected. I'm going to update the tests. Downstream distributors: Feel free to disable the test in the mean time. The feature is not critical. ---------- assignee: christian.heimes components: Extension Modules messages: 285794 nosy: christian.heimes priority: normal severity: normal status: open title: test_aead_aes_gcm fails on Kernel 4.9 type: behavior versions: Python 3.6, Python 3.7 _ Python tracker _ From report at bugs.python.org Thu Jan 19 11:30:50 2017 From: report at bugs.python.org (Brian Vandenberg) Date: Thu, 19 Jan 2017 16:30:50 +0000 Subject: [New-bugs-announce] [issue29325] pysqlite: Evaluate removal of sqlite3_stmt_readonly Message-ID: <1484843450.51.0.0079411502064.issue29325@psf.upfronthosting.co.za> New submission from Brian Vandenberg: I'm not sure where to request changes to pysqlite, so my apologies if this isn't the right place. To begin with: I'll either end up building a newer version of sqlite myself or just accepting that pysqlite won't be part of this python installation. However, I thought it might be useful to know that use of the function "sqlite3_stmt_readonly" is the only thing tying pysqlite to the current minimum requirement to use sqlite 3.7.4. The currently available 'supported' version of sqlite for RHEL6 is 3.6.x and there's likely others out there who (for whatever reason) are stuck on an older release of sqlite and moving to the latest & greatest OS isn't [currently] feasible. ---------- components: Extension Modules messages: 285804 nosy: phantal priority: normal severity: normal status: open title: pysqlite: Evaluate removal of sqlite3_stmt_readonly type: enhancement _ Python tracker _ From report at bugs.python.org Thu Jan 19 11:54:32 2017 From: report at bugs.python.org (Steve Dower) Date: Thu, 19 Jan 2017 16:54:32 +0000 Subject: [New-bugs-announce] [issue29326] Blank lines in ._pth file are not ignored Message-ID: <1484844872.11.0.819349056222.issue29326@psf.upfronthosting.co.za> New submission from Steve Dower: If a python._pth file includes a blank line, it gets treated as '\n' which is then appended to the directory and used as an entry in sys.path. Empty lines should be ignored completely. ---------- components: Windows messages: 285809 nosy: paul.moore, steve.dower, tim.golden, zach.ware priority: low severity: normal stage: needs patch status: open title: Blank lines in ._pth file are not ignored type: behavior versions: Python 3.6, Python 3.7 ___________________ Python tracker _ From report at bugs.python.org Thu Jan 19 12:31:08 2017 From: report at bugs.python.org (Serhiy Storchaka) Date: Thu, 19 Jan 2017 17:31:08 +0000 Subject: [New-bugs-announce] [issue29327] SystemError in sorted(iterable=[]) Message-ID: <1484847068.03.0.460461228187.issue29327@psf.upfronthosting.co.za> New submission from Serhiy Storchaka: In Python 3.6: >>> sorted(iterable=[]) Traceback (most recent call last): File "", line 1, in SystemError: Objects/tupleobject.c:81: bad argument to internal function In Python 3.5 and 2.7: >>> sorted(iterable=[]) Traceback (most recent call last): File "", line 1, in TypeError: 'iterable' is an invalid keyword argument for this function ---------- components: Interpreter Core keywords: 3.6regression messages: 285817 nosy: serhiy.storchaka priority: normal severity: normal status: open title: SystemError in sorted(iterable=[]) type: behavior versions: Python 3.6, Python 3.7 _ Python tracker _ From report at bugs.python.org Thu Jan 19 13:24:32 2017 From: report at bugs.python.org (Elizabeth Myers) Date: Thu, 19 Jan 2017 18:24:32 +0000 Subject: [New-bugs-announce] [issue29328] struct module should support variable-length strings Message-ID: <1484850272.25.0.472309940141.issue29328@psf.upfronthosting.co.za> New submission from Elizabeth Myers: There was some discussion on python-ideas about this, and I figured it would be more productive to bring it here since to me this appears to be a glaring omission. The struct module has no capability to support variable-length strings; this includes null-terminated and Pascal-ish strings with a different integer datatype (usually in binary) specifying length. This unfortunate omission makes the struct module extremely unwieldy to use in situations where you need to unpack a lot of variable-length strings, especially iteratively; see https://mail.python.org/pipermail/python-ideas/2017-January/044328.html for why. For zero-terminated strings, it is essentially impossible. It's worth noting many modern protocols use variable-length strings, including DHCP. I therefore propose the following extensions to the struct module (details can be bikeshedded over :P): - Z (uppercase) format specifier (I did not invent this idea, see https://github.com/stendec/netstruct - although that uses $), which states the preceding whole-number datatype is the length of a string that follows. - z (lowercase) format specifier, which specifies a null-terminated (also known as C style) string. An optional length parameter can be added to specify the maximum search length. These two additions will make the struct module much more usable in a wider variety of contexts. ---------- components: Library (Lib) messages: 285828 nosy: Elizacat priority: normal severity: normal status: open title: struct module should support variable-length strings type: enhancement versions: Python 3.7 _ Python tracker ___________________ From report at bugs.python.org Thu Jan 19 13:55:38 2017 From: report at bugs.python.org (=?utf-8?q?Pekka_Kl=C3=A4rck?=) Date: Thu, 19 Jan 2017 18:55:38 +0000 Subject: [New-bugs-announce] [issue29329] Incorrect documentation for custom `hex()` support on Python 2 Message-ID: <1484852138.82.0.311515909069.issue29329@psf.upfronthosting.co.za> New submission from Pekka Kl?rck: Documentation of `hex()` on Python 2 says that custom objects need to implement `index` to support it. Based on my tests that doesn't work but `hex` is needed instead. Docs are at https://docs.python.org/2/library/functions.html?highlight=hex#hex and here's an example session: Python 2.7.6 (default, Oct 26 2016, 20:30:19) [GCC 4.8.4] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> class Hex(object): ... def index(self): ... return 255 ... >>> hex(Hex()) Traceback (most recent call last): File "", line 1, in TypeError: hex() argument can't be converted to hex >>> >>> class Hex(object): ... def hex(self): ... return hex(255) ... >>> hex(Hex()) '0xff' Assuming this is fixed, should probably note that with Python 3 you actually need to implement `index` and `hex` has no effect. ---------- messages: 285832 nosy: pekka.klarck priority: normal severity: normal status: open title: Incorrect documentation for custom `hex()` support on Python 2 ___________________ Python tracker _ From report at bugs.python.org Thu Jan 19 17:30:40 2017 From: report at bugs.python.org (saumitra paul) Date: Thu, 19 Jan 2017 22:30:40 +0000 Subject: [New-bugs-announce] [issue29330] slots needs documentation Message-ID: <1484865040.1.0.923136956691.issue29330@psf.upfronthosting.co.za> New submission from saumitra paul: As you said..assiging it to you :) ---------- messages: 285850 nosy: rhettinger, saumitra1978 priority: normal severity: normal status: open title: slots needs documentation versions: Python 2.7 _ Python tracker ___________________ From report at bugs.python.org Fri Jan 20 01:59:39 2017 From: report at bugs.python.org (Serhiy Storchaka) Date: Fri, 20 Jan 2017 06:59:39 +0000 Subject: [New-bugs-announce] [issue29331] Simplify argument parsing in sorted() and list.sort() Message-ID: <1484895579.74.0.0168401872287.issue29331@psf.upfronthosting.co.za> New submission from Serhiy Storchaka: Proposed patch simplifies argument parsing in sorted() and list.sort(). list.sort() now uses the support of keyword-only parameters in PyArg_ParseTupleAndKeywords(). Keyword arguments now are passed as is to list.sorted() in sorted(). ---------- components: Interpreter Core files: sort-arg-parse.patch keywords: patch messages: 285887 nosy: haypo, rhettinger, serhiy.storchaka priority: normal severity: normal stage: patch review status: open title: Simplify argument parsing in sorted() and list.sort() type: enhancement versions: Python 3.7 Added file: http://bugs.python.org/file46351/sort-arg-parse.patch _ Python tracker _ From report at bugs.python.org Fri Jan 20 03:29:05 2017 From: report at bugs.python.org (Wen Adam) Date: Fri, 20 Jan 2017 08:29:05 +0000 Subject: [New-bugs-announce] [issue29332] Uniform SelectSelector._select behavior Message-ID: <1484900945.96.0.01738502245.issue29332@psf.upfronthosting.co.za> New submission from Wen Adam: SelectSelector._select is differently on different platforms. On win32, SelectSelector._select is a unbound/bound method and pass instance as the first argument, but on nix, SelectSelector._select is a builtin_function_or_method(Although self.select still works) If someone want to replace(or patch) select, he must implement two select function(one is a function, the other is a instance method), Otherwise you will get?TypeError: select() takes at most 4 arguments (5 given)?. And I think make built-in funtion(like len, select.select) as a class attr/method is a black magic(it's still a function) and hard to maintain ---------- components: Library (Lib) files: uniform_select_behaviour.patch keywords: patch messages: 285891 nosy: Wen Adam, neologix priority: normal severity: normal status: open title: Uniform SelectSelector._select behavior type: behavior versions: Python 3.4, Python 3.5, Python 3.6, Python 3.7 Added file: http://bugs.python.org/file46352/uniform_select_behaviour.patch _ Python tracker _ From report at bugs.python.org Fri Jan 20 06:50:36 2017 From: report at bugs.python.org (Gereon Kaiping) Date: Fri, 20 Jan 2017 11:50:36 +0000 Subject: [New-bugs-announce] [issue29333] ConfigParser calls Interpolation.before_read after reading Message-ID: <1484913036.08.0.781257093151.issue29333@psf.upfronthosting.co.za> New submission from Gereon Kaiping: In its current implementation, `ConfigParser` calls its Interpolation's `before_read` method in the very last step of `_read`, when all properties have already been overwritten by their new uninterpolated values. I am developing a program with modular config files: It is possible to supply several configuration files on the command line, and they are all fed through the `.read` method. Now it would be amazing to use `read` time interpolation instead of `get` time interpolation to construct things like outputfilename = %(outputfilename)s_extension_from_this_ini_module By looking at the `Interpolation` class, it seems that behaviour like this should be supported by supplying a `before_read` as follows. def before_read(self, parser, section, option, value): L = [] interpolations = parser[section] self._interpolate_some( parser, option, L, value, section, interpolations, 1) return ''.join(L) However, this is not possible, because `before_read` is only called after* all values in the config file have been read and all old values in the ConfigParser object have been overridden. The attached file contains a subclass of `BasicInterpolation` and a subclass of `ConfigParser` which in concert allow me to write recursive property definitions as given above. The downside of this change is that (a) interpolation values can't be defined after they are used any more, and (b) it is not possible to hack this parser to accept multi-line option names as interpolations. (To me personally, both of these don't sound like useful features to begin with.) ---------- files: earlyinterpolation.py messages: 285902 nosy: Anaphory priority: normal severity: normal status: open title: ConfigParser calls Interpolation.before_read after reading Added file: http://bugs.python.org/file46355/earlyinterpolation.py _ Python tracker _ From report at bugs.python.org Fri Jan 20 07:13:47 2017 From: report at bugs.python.org (Greg Stark) Date: Fri, 20 Jan 2017 12:13:47 +0000 Subject: [New-bugs-announce] [issue29334] ssl.SSLObject method getpeercert() is buggy, do_handshake() is strange Message-ID: <1484914427.68.0.380715465195.issue29334@psf.upfronthosting.co.za> New submission from Greg Stark: In my experiments with the relatively new class SSLObject from the ssl module I've noticed the following behavior(s) which I think can be described as bugs. The getpeercert() method raises a ValueError exception "handshake not done" even after the handshake has successfully completed. If, however, I call the do_handshake() method after the handshake completes, then getpeercert() correctly runs and returns the peer's certificate. So now let's focus on do_handshake(). This method is basically undocumented, which I thought was ok because what it does should be obvious. It does seem to initiate a handshake if it's the first method call after the SSLObject is created. If called afterward, it doesn't outwardly appear to do anything, but as mentioned previously it does magically make the getpeercert() method start working. ---------- assignee: christian.heimes components: SSL messages: 285903 nosy: Greg Stark, christian.heimes priority: normal severity: normal status: open title: ssl.SSLObject method getpeercert() is buggy, do_handshake() is strange type: behavior versions: Python 3.5 _ Python tracker _ From report at bugs.python.org Fri Jan 20 15:48:08 2017 From: report at bugs.python.org (Zach Riggle) Date: Fri, 20 Jan 2017 20:48:08 +0000 Subject: [New-bugs-announce] [issue29335] Python 2.7 subprocess module does not check WIFSTOPPED on SIGCHLD Message-ID: <1484945288.47.0.935636853008.issue29335@psf.upfronthosting.co.za> New submission from Zach Riggle: The attached script hits some "This should never happen" code in the subprocess module. These lines here: https://github.com/python/cpython/blob/2.7/Lib/subprocess.py#L1036-L1038 The root cause is a lack of checking WIFSTOPPED and WSTOPSIG in the handler. When a process elects into being ptraced via PTRACE_TRACEME, it is stopped on the SIGSEGV instead of terminating, allowing the user to attach a debugger before the kernel destroys the process. This bug makes it impossible to wait on any process which crashes, which is set up to wait for a debugger. ---------- components: Library (Lib) files: bug.py messages: 285921 nosy: Zach Riggle priority: normal severity: normal status: open title: Python 2.7 subprocess module does not check WIFSTOPPED on SIGCHLD versions: Python 2.7 Added file: http://bugs.python.org/file46363/bug.py ___________________ Python tracker _ From report at bugs.python.org Fri Jan 20 22:30:45 2017 From: report at bugs.python.org (INADA Naoki) Date: Sat, 21 Jan 2017 03:30:45 +0000 Subject: [New-bugs-announce] [issue29336] merge tuples in module Message-ID: <1484969445.33.0.406694538272.issue29336@psf.upfronthosting.co.za> New submission from INADA Naoki: Tuples consists of None, True, False, str and bytes can be merged safely. To avoid memory leak, this patch shares such tuples in compile unit (module, in most case.) ---------- files: merge-tuples.patch keywords: patch messages: 285933 nosy: haypo, inada.naoki priority: normal severity: normal status: open title: merge tuples in module Added file: http://bugs.python.org/file46366/merge-tuples.patch _ Python tracker ___________________ From report at bugs.python.org Sat Jan 21 02:09:46 2017 From: report at bugs.python.org (Serhiy Storchaka) Date: Sat, 21 Jan 2017 07:09:46 +0000 Subject: [New-bugs-announce] [issue29337] BytesWarning at compile time Message-ID: <1484982586.78.0.422468246503.issue29337@psf.upfronthosting.co.za> New submission from Serhiy Storchaka: BytesWarning can be raised during compilation. $ ./python -Wa -b -c "lambda: 'a'; lambda: b'a'" sys:1: BytesWarning: Comparison between bytes and string sys:1: BytesWarning: Comparison between bytes and string sys:1: BytesWarning: Comparison between bytes and string sys:1: BytesWarning: Comparison between bytes and string _PyCode_ConstantKey() should produce keys that don't allow comparing bytes with strings nested in tuples or frozensets. Currently it returns (tuple, ('a',), ((str, 'a'),)) for ('a',) and (tuple, (b'a',), ((bytes, b'a'),)) for (b'a',). Key tuples have the same size and the same first element. Comparing second elements emits a BytesWarning. ---------- components: Interpreter Core messages: 285940 nosy: haypo, inada.naoki, serhiy.storchaka priority: normal severity: normal status: open title: BytesWarning at compile time type: behavior versions: Python 3.5, Python 3.6, Python 3.7 _ Python tracker _ From report at bugs.python.org Sat Jan 21 04:42:36 2017 From: report at bugs.python.org (Serhiy Storchaka) Date: Sat, 21 Jan 2017 09:42:36 +0000 Subject: [New-bugs-announce] [issue29338] Output the text signature in the help of a class Message-ID: <1484991756.99.0.970269270756.issue29338@psf.upfronthosting.co.za> New submission from Serhiy Storchaka: Pydoc outputs the text signature for C functions. It defines parameters and default values and is a part of function description. Help on built-in function format in module builtins: format(value, format_spec='', /) Return value.format(format_spec) format_spec defaults to the empty string When builtin or extension class is converted to Argument Clinic, the generated docstring and text signature of new or init methods usually is used as class docstring and text signature. But pydoc doesn't output the text signature for classes. The important part of information is lost. For example, for just converted builtin enumerate class: Help on class enumerate in module builtins: class enumerate(object) | Return an enumerate object. | | iterable | an object supporting iteration | | The enumerate object yields pairs containing a count (from start, which | defaults to zero) and a value yielded by the iterable argument. | | enumerate is useful for obtaining an indexed list: | (0, seq[0]), (1, seq[1]), (2, seq[2]), ... | | Methods defined here: | | getattribute(self, name, /) | Return getattr(self, name). | | iter(self, /) | Implement iter(self). | | new(*args, **kwargs) from builtins.type | Create and return a new object. See help(type) for accurate signature. | | next(self, /) | Implement next(self). | | reduce(...) | Return state information for pickling. The iterable and start parameters of the constructor are referred but not defined. ---------- components: Argument Clinic, Library (Lib) messages: 285946 nosy: larry, ncoghlan, rhettinger, serhiy.storchaka, yselivanov priority: normal severity: normal status: open title: Output the text signature in the help of a class type: enhancement versions: Python 3.7 ___________________ Python tracker _ From report at bugs.python.org Sat Jan 21 17:56:15 2017 From: report at bugs.python.org (12345 67890) Date: Sat, 21 Jan 2017 22:56:15 +0000 Subject: [New-bugs-announce] [issue29339] Interactive: Move to same indentation level as previous line Message-ID: <1485039375.56.0.473197770315.issue29339@psf.upfronthosting.co.za> New submission from 12345 67890: Not sure if anyone has ever thought about this, but it would be nice if the interactive interpreter set the same indentation level as the previous line. Of course this isn't urgent, but what do others think? ---------- messages: 285970 nosy: 12345 67890 priority: normal severity: normal status: open title: Interactive: Move to same indentation level as previous line type: enhancement _ Python tracker ___________________ From report at bugs.python.org Sat Jan 21 18:37:20 2017 From: report at bugs.python.org (Nick Sweeting) Date: Sat, 21 Jan 2017 23:37:20 +0000 Subject: [New-bugs-announce] [issue29340] SSL destructor segfaults in python3.6 threads when an unverified-cert connection is closed Message-ID: <1485041840.84.0.00558025824464.issue29340@psf.upfronthosting.co.za> New submission from Nick Sweeting: I mistakenly thought this bug was a pyOpenSSL bug, so I reported it there (https://github.com/pyca/pyopenssl/issues/588), but it looks like this is actually a bug in CPython. See the link for a nicer formatted bug report with more details. Code to reproduce: https://gist.github.com/pirate/6649314e02384274b29e04364c9d0c16 Crash dump: https://gist.github.com/pirate/e1485110093d2d9fc49596e6ff481777 Description: The bug is a segmentation fault when closing several secure websockets from inside concurrent python3.6 threads: ```pythohn 'python3.6 libssl_bug.py' terminated by signal SIGSEGV (Address boundary error) ``` It does not occur when doing it in a single thread, only when closing multiple threads at once. ```python threads = [] for _ in range(NUM_THREADS): t = SocketThread('wss://echo.websocket.org/', ssl_opt={'cert_reqs': 0}) t.start() threads.append(t) sleep(4) for t in threads: t.keep_running = False t.ws.close() # libssl segfaults on python3.6 when closing a wss:// connection with cert_reqs=0 t.join() ``` This is my first python bug report, so apologies if I didn't get the formatting right or if I'm missing some info. ---------- assignee: christian.heimes components: SSL, macOS files: libssl_bug.py messages: 285971 nosy: Nick Sweeting, christian.heimes, ned.deily, ronaldoussoren priority: normal severity: normal status: open title: SSL destructor segfaults in python3.6 threads when an unverified-cert connection is closed type: crash versions: Python 3.6 Added file: http://bugs.python.org/file46375/libssl_bug.py _ Python tracker _ From report at bugs.python.org Sun Jan 22 00:47:41 2017 From: report at bugs.python.org (Xiang Zhang) Date: Sun, 22 Jan 2017 05:47:41 +0000 Subject: [New-bugs-announce] [issue29341] Missing accepting path-like object in docstrings of os module functions Message-ID: <1485064061.59.0.597221589705.issue29341@psf.upfronthosting.co.za> New submission from Xiang Zhang: PathLike objects are added in 3.6 and they are mentioned in the documentation. But in some os module functions' docstrings, acceptable types of path parameter are mentioned and they are not altered to mention path-like object. For example: chown(path, uid, gid, , dir_fd=None, follow_symlinks=True) Change the owner and group id of path to the numeric uid and gid.\ path Path to be examined; can be string, bytes, or open-file-descriptor int. ---------- messages: 285988 nosy: xiang.zhang priority: normal severity: normal status: open title: Missing accepting path-like object in docstrings of os module functions versions: Python 3.6, Python 3.7 _ Python tracker _ From report at bugs.python.org Sun Jan 22 06:20:17 2017 From: report at bugs.python.org (Marian Beermann) Date: Sun, 22 Jan 2017 11:20:17 +0000 Subject: [New-bugs-announce] [issue29342] os.posix_fadvise misreports errors Message-ID: <1485084017.58.0.131551905677.issue29342@psf.upfronthosting.co.za> New submission from Marian Beermann: It has been observed that posix_fadvise will not report the original error if the syscall fails. Eg. https://bugs.alpinelinux.org/issues/6592 >>> os.posix_fadvise(-1, 0, 0, os.POSIX_FADV_DONTNEED) Traceback (most recent call last): File "", line 1, in OSError: [Errno 0] Error Should report EBADF >>> os.posix_fadvise(16, 0, 0, os.POSIX_FADV_DONTNEED) Traceback (most recent call last): File "", line 1, in OSError: [Errno 0] Error Ditto >>> os.posix_fadvise(0, 0, 0, 12345) Traceback (most recent call last): File "", line 1, in OSError: [Errno 0] Error Should be EINVAL ... This might be because unlike most syscall wrappers posix_fadvise does not set errno but only returns it. ---------- components: IO, Library (Lib) messages: 286002 nosy: enkore priority: normal severity: normal status: open title: os.posix_fadvise misreports errors versions: Python 3.6 _ Python tracker _ From report at bugs.python.org Sun Jan 22 07:11:57 2017 From: report at bugs.python.org (Christian Heimes) Date: Sun, 22 Jan 2017 12:11:57 +0000 Subject: [New-bugs-announce] [issue29343] sock.close() raises OSError EBADF when socket's fd is closed Message-ID: <1485087117.22.0.587639771074.issue29343@psf.upfronthosting.co.za> New submission from Christian Heimes: In Python 3.6 the behavior of socket's close method has changed. In Python 2.7, 3.5 and earlier, socket.close() ignored EBADF. Starting with Python 3.6 socket.close() raises an OSError exception when its internal fd has been closed. Python 2.7.12 (default, Sep 29 2016, 12:52:02) [GCC 6.2.1 20160916 (Red Hat 6.2.1-2)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import os, socket >>> sock = socket.socket() >>> os.close(sock.fileno()) >>> sock.close() Python 3.5.2 (default, Sep 14 2016, 11:28:32) [GCC 6.2.1 20160901 (Red Hat 6.2.1-1)] on linux Type "help", "copyright", "credits" or "license" for more information. >>> import socket >>> sock = socket.socket() >>> import os, socket >>> os.close(sock.fileno()) >>> sock.close() Python 3.6.0+ (3.6:ea0c488b9bac, Jan 14 2017, 14:08:17) [GCC 6.3.1 20161221 (Red Hat 6.3.1-1)] on linux Type "help", "copyright", "credits" or "license" for more information. >>> import os, socket >>> sock = socket.socket() >>> os.close(sock.fileno()) >>> sock.close() Traceback (most recent call last): File "", line 1, in File "/home/heimes/dev/python/python36/lib/python3.6/socket.py", line 417, in close self._real_close() File "/home/heimes/dev/python/python36/lib/python3.6/socket.py", line 411, in _real_close _ss.close(self) OSError: [Errno 9] Bad file descriptor Abstraction layers such as the socket class should ignore EBADF in close(). A debug message or a warning is ok, but an exception is wrong. Even the man page for close discourages it, http://man7.org/linux/man-pages/man2/close.2.html Note, however, that a failure return should be used only for diagnostic purposes (i.e., a warning to the application that there may still be I/O pending or there may have been failed I/O) or remedial purposes (e.g., writing the file once more or creating a backup). ---------- components: Library (Lib) keywords: 3.6regression messages: 286004 nosy: christian.heimes priority: normal severity: normal status: open title: sock.close() raises OSError EBADF when socket's fd is closed type: behavior versions: Python 3.6, Python 3.7 _ Python tracker _ From report at bugs.python.org Sun Jan 22 09:09:45 2017 From: report at bugs.python.org (Jeremy Bustamante) Date: Sun, 22 Jan 2017 14:09:45 +0000 Subject: [New-bugs-announce] [issue29344] sock_recv not detected a coroutine Message-ID: <1485094185.41.0.725056014669.issue29344@psf.upfronthosting.co.za> New submission from Jeremy Bustamante: Documemtation says sock_recv is a coroutine https://docs.python.org/3.6/library/asyncio-eventloop.html#low-level-socket-operations But following code says it isnt: import asyncio loop = asyncio.get_event_loop() asyncio.iscoroutinefunction(loop.sock_recv) False asyncio.iscoroutine(loop.sock_recv) False ---------- components: asyncio messages: 286011 nosy: Jeremy Bustamante, gvanrossum, yselivanov priority: normal severity: normal status: open title: sock_recv not detected a coroutine type: behavior versions: Python 3.6 ___________________ Python tracker _ From report at bugs.python.org Sun Jan 22 12:02:49 2017 From: report at bugs.python.org (Just a Person) Date: Sun, 22 Jan 2017 17:02:49 +0000 Subject: [New-bugs-announce] [issue29345] More lost updates with multiprocessing.Value and .Array Message-ID: <1485104569.63.0.130882514769.issue29345@psf.upfronthosting.co.za> New submission from Just a Person: Lately, I have been having trouble using the multiprocessing library's shared memory on Windows. Often, updating the .value property seems to fail for no reason. As shown in the attached video, changing ```if name == 'main':``` in the sample code from the documentation to ```if True:``` causes the program to not work. This issue does not arise under Linux as far as I can tell (testing the same code). Any explanation/fix would be helpful. ---------- components: Windows files: 2017-01-22-1140-25.mp4 messages: 286015 nosy: Just a Person, paul.moore, steve.dower, tim.golden, zach.ware priority: normal severity: normal status: open title: More lost updates with multiprocessing.Value and .Array type: behavior versions: Python 3.6 Added file: http://bugs.python.org/file46380/2017-01-22-1140-25.mp4 _ Python tracker ___________________ From report at bugs.python.org Sun Jan 22 12:47:34 2017 From: report at bugs.python.org (Eli Collins) Date: Sun, 22 Jan 2017 17:47:34 +0000 Subject: [New-bugs-announce] [issue29346] datetime.utcfromtimestamp() returns strange result for very large values Message-ID: <1485107254.51.0.0185006280427.issue29346@psf.upfronthosting.co.za> New submission from Eli Collins: I've found an odd behavior when passing very large values to ``datetime.datetime.utcfromtimestamp()`` and ``.fromtimestamp()`` under python 3.6. Under python 3.5, ``utcfromtimestamp(1<<40)`` would throw a ValueError that the year was out of range. Under python 3.6, this now returns a datetime in year 36812 (which seems reasonable given the input). The unexpected behavior occurs when increasing the bits passed: ``utcfromtimestamp(1<<41)`` returns a datetime with a smaller* year (6118). This pattern proceeds as the bits are increased, with the years increasing & then wrapping around again, up to the point where it exceeds time_t (at that point, python 3.6 throws the same OSError as 3.5). It looks to me like 3.6 dropped a bounds check somewhere, and is now truncating high bits off the resulting year? --- Attached is the "dump_timestamp_output.py" script that I was using to examine boundary behavior of utctimestamp() when I found this bug. System was running Linux Mint 18.1 x86_64, using the python 3.6.0 build from https://launchpad.net/~fkrull/+archive/ubuntu/deadsnakes (ubuntu's python 3.6.0 build also shows this behavior). ---------- components: Library (Lib) files: dump_timestamp_output.py messages: 286021 nosy: Eli Collins priority: normal severity: normal status: open title: datetime.utcfromtimestamp() returns strange result for very large values type: behavior versions: Python 3.6 Added file: http://bugs.python.org/file46384/dump_timestamp_output.py _ Python tracker _ From report at bugs.python.org Sun Jan 22 22:24:01 2017 From: report at bugs.python.org (Saida Dhanavath) Date: Mon, 23 Jan 2017 03:24:01 +0000 Subject: [New-bugs-announce] [issue29347] Python 2.7.8 is crashing while creating weakref for a given object. Message-ID: <1485141841.25.0.0993319189306.issue29347@psf.upfronthosting.co.za> New submission from Saida Dhanavath: We are using python 2.7.8 on Ubuntu 14.04 to host our services. In one of the crashes python interpreter got segmentation fault while initializing weakref for a given object. Please find snip of backtraces as given below. #0 0x00007f62aa86951a in clear_weakref (self=0x7f5a1ed17520) at Objects/weakrefobject.c:65 #1 proxy_dealloc (self=0x7f5a1ed17520) at Objects/weakrefobject.c:540 #2 0x00007f62aa869b8b in PyWeakref_NewProxy (ob=, callback=) at Objects/weakrefobject.c:855 #3 0x00007f62aa901e56 in weakref_proxy (self=, args=) at ./Modules/_weakref.c:73 #4 0x00007f62aa8a929b in call_function (oparg=, pp_stack=0x7f5d36661c90) at Python/ceval.c:4033 . . . Have tried to root cause the issue and found that PyWeakref_NewProxy at Objects/weakrefobject.c creates new isntance of PyWeakReference struct and does not intialize wr_prev and wr_next of new isntance. These pointers can have garbage and point to random memory locations. As per comment in the code there could be a race while creating new instance and some other thread could have created weakref by the time current thread returns from new_weakref function. If it finds weakref created, current thread destroys instance created by itself and uses the one created by some other thread. Python should not crash while destroying the isntance created in the same interpreter function. As per my understanding, both wr_prev and wr_next of PyWeakReference instance should be initialized to NULL to avoid segfault. ---------- components: Interpreter Core messages: 286044 nosy: dhanavaths priority: normal severity: normal status: open title: Python 2.7.8 is crashing while creating weakref for a given object. versions: Python 2.7 _ Python tracker _ From report at bugs.python.org Mon Jan 23 01:39:24 2017 From: report at bugs.python.org (Tang Weizhi) Date: Mon, 23 Jan 2017 06:39:24 +0000 Subject: [New-bugs-announce] [issue29348] this comment 'iff' should be 'if'? Message-ID: <1485153564.45.0.440343066023.issue29348@psf.upfronthosting.co.za> New submission from Tang Weizhi: I think this comment 'if' should be more clearly, or 'iff' is the abbreviation of 'if and only if'? ---------- assignee: docs at python components: Documentation messages: 286051 nosy: Tangwz, docs at python priority: normal pull_requests: 20 severity: normal status: open title: this comment 'iff' should be 'if'? type: enhancement versions: Python 2.7 ___________________ Python tracker _ From report at bugs.python.org Mon Jan 23 04:41:21 2017 From: report at bugs.python.org (Jim Fasarakis-Hilliard) Date: Mon, 23 Jan 2017 09:41:21 +0000 Subject: [New-bugs-announce] [issue29349] Update old Python 2 code in Docs/tools/extensions/patchlevel.py Message-ID: <1485164481.25.0.457345760711.issue29349@psf.upfronthosting.co.za> New submission from Jim Fasarakis-Hilliard: There's a hidden Python 2 print call in the script that only gets reached after you move the Doc/ folder outside the main CPython directory and run `make html`. Additionally, an obsolete way of assuring a file gets closed is used (changed to use `with` statement). P.S: Tagged this with the Documentation component, maybe build is more applicable but I wasn't sure. ---------- assignee: docs at python components: Documentation files: patchlevel.patch keywords: patch messages: 286065 nosy: Jim Fasarakis-Hilliard, docs at python priority: normal severity: normal status: open title: Update old Python 2 code in Docs/tools/extensions/patchlevel.py type: behavior versions: Python 3.4, Python 3.5, Python 3.6, Python 3.7 Added file: http://bugs.python.org/file46387/patchlevel.patch _ Python tracker _ From report at bugs.python.org Mon Jan 23 05:26:05 2017 From: report at bugs.python.org (Serhiy Storchaka) Date: Mon, 23 Jan 2017 10:26:05 +0000 Subject: [New-bugs-announce] [issue29350] Add support of multiple signatures Message-ID: <1485167165.19.0.665694725089.issue29350@psf.upfronthosting.co.za> New submission from Serhiy Storchaka: Some functions can be described by the single signature. See examples in msg285647. Selected examples: dict.pop(key) dict.pop(key, default) type(obj) type(name, bases, mapping) range(stop) range(start, stop, step=1) min(iterable, , key=identity) min(iterable, , default, key=identity) min(args, key=identity) I think the only way to resolve this problem is to add the support of multiple signatures in inspect, pydoc, Argument Clinic, etc. ---------- components: Library (Lib) messages: 286068 nosy: larry, rhettinger, serhiy.storchaka, yselivanov priority: normal severity: normal status: open title: Add support of multiple signatures type: enhancement versions: Python 3.7 _ Python tracker _______________ From report at bugs.python.org Mon Jan 23 08:25:03 2017 From: report at bugs.python.org (Markus Gerstel) Date: Mon, 23 Jan 2017 13:25:03 +0000 Subject: [New-bugs-announce] [issue29351] absolute imports for logging Message-ID: <1485177903.18.0.43823292649.issue29351@psf.upfronthosting.co.za> New submission from Markus Gerstel: Running 'import logging' causes at minimum 46 failing 'open' and 12 failing 'stat' calls because python looks for packages inside python/Lib/logging which will never be there, in particular: sys, os, time, cStringIO, traceback, warnings, weakref, collections, codecs, thread, threading, atexit. The impact of this is limited when python is installed locally, but noticeable when run on a networked file system. How to reproduce: run $ strace python -c "import logging;" 2>&1 | grep ENOENT | grep "\/logging\/" How to fix: Add 'from future import absolute_import' to all files in the logging directory. A relevant patch is attached. ---------- components: Library (Lib) files: 0001-absolute-import.patch keywords: patch messages: 286083 nosy: mgerstel priority: normal severity: normal status: open title: absolute imports for logging type: resource usage versions: Python 2.7 Added file: http://bugs.python.org/file46390/0001-absolute-import.patch _ Python tracker _ From report at bugs.python.org Mon Jan 23 11:49:16 2017 From: report at bugs.python.org (Akira Li) Date: Mon, 23 Jan 2017 16:49:16 +0000 Subject: [New-bugs-announce] [issue29352] provide the authorative source for s[i:j] negative slice indices (<-len(s)) behavior for standard sequences Message-ID: <1485190156.24.0.631194950251.issue29352@psf.upfronthosting.co.za> New submission from Akira Li: I've failed to find where the behavior for negative indices in s[i:j] expression (i, j < -len(s)) for standard sequences (str, list, etc) is formally defined. The observed behavior implemented in PySlice_GetIndicesEx(): If "len(s) + i" or "len(s) + j" is negative, use 0. [1] I don't see it in the docs. if (start < 0) start += length; if (start < 0) start = (step < 0) ? -1 : 0; ... if (stop < 0) stop += length; if (stop < 0) stop = (step < 0) ? -1 : 0; The tutorial mentions [2]: > out of range slice indexes are handled gracefully when used for > slicing" slice.indices() documentation says [3]: > Missing or out-of-bounds indices are handled in a manner consistent > with regular slices. Neither define it explicitly. The behavior for the upper boundary is defined explicitly [4]: > If i* or j is greater than ``len(s)``, use ``len(s)`` I've added the documentation patch that defines the behavior for the lower boundary too. [1] Objects/sliceobject.c [2] Doc/tutorial/introduction.rst [3] Doc/reference/datamodel.rst [4] Doc/library/stdtypes.rst ---------- assignee: docs at python components: Documentation files: docs-negative-slice-indices.patch keywords: patch messages: 286098 nosy: akira, docs at python priority: normal severity: normal status: open title: provide the authorative source for s[i:j] negative slice indices (<-len(s)) behavior for standard sequences versions: Python 3.5, Python 3.6, Python 3.7 Added file: http://bugs.python.org/file46393/docs-negative-slice-indices.patch _ Python tracker _ From report at bugs.python.org Mon Jan 23 13:58:25 2017 From: report at bugs.python.org (brokenenglish) Date: Mon, 23 Jan 2017 18:58:25 +0000 Subject: [New-bugs-announce] [issue29353] Incorrect handling of HTTP response with "Content-Type: message/rfc822" header Message-ID: <1485197905.85.0.319696153354.issue29353@psf.upfronthosting.co.za> New submission from brokenenglish: Hello. I found a bug that causes incorrect handling of some values of "Content-Type" header. When you retrieve any URL with "Content-Type: message/rfc822" header, additional payload is added to HTTPMessage. In some cases it causes annoing warnings. Here is a code from packages requests and urllib3 that can cause warning output: 1. Checking and raising excpection in urllib3: https://github.com/shazow/urllib3/blob/0fb5e083b2adf7618db8c26e8e50206de09dd845/urllib3/util/response.py#L61-L66 2. The same code packaged in requests: https://github.com/kennethreitz/requests/blob/362da46e9a46da6e86e1907f03014384ab210151/requests/packages/urllib3/util/response.py#L61-L66 3. Logging the error to output: https://github.com/kennethreitz/requests/blob/362da46e9a46da6e86e1907f03014384ab210151/requests/packages/urllib3/connectionpool.py#L402-L407 The issue arises from the class email.feedparser.FeedParser that handles HTTP and email headers in the same way. So when it handles headers with the message with content-type "message/" and some other, method _parsegen() tries to parse it like a message with another message (see comments to this condition: https://hg.python.org/cpython/file/tip/Lib/email/feedparser.py#l295). If headers-only argument is set to True, we can avoid some redundant checks on HTTP headers (see this condition: https://hg.python.org/cpython/file/tip/Lib/email/feedparser.py#l244). I read the comment above (https://hg.python.org/cpython/file/tip/Lib/email/feedparser.py#l241) but I am a newbie and I don't see any other solution to my problem for now. I can rewrite the patch if you show me a better way to fix this issue. A patch with the unit test is atached. ---------- files: rfc822_httpmessage_payload.patch keywords: patch messages: 286106 nosy: brokenenglish priority: normal severity: normal status: open title: Incorrect handling of HTTP response with "Content-Type: message/rfc822" header versions: Python 3.3, Python 3.4, Python 3.5, Python 3.6, Python 3.7 Added file: http://bugs.python.org/file46395/rfc822_httpmessage_payload.patch _____ Python tracker _ From report at bugs.python.org Mon Jan 23 16:50:38 2017 From: report at bugs.python.org (Elias Vanderstuyft) Date: Mon, 23 Jan 2017 21:50:38 +0000 Subject: [New-bugs-announce] [issue29354] Python 2.7.12: pydoc.help(lambda (a, ): lambda x: a) raises IndexError Message-ID: <1485208238.9.0.161996393795.issue29354@psf.upfronthosting.co.za> New submission from Elias Vanderstuyft: The following might be either a bug in the pydoc module or in the inspect module: $ python Python 2.7.12 (default, Nov 19 2016, 06:48:10) [GCC 5.4.0 20160609] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import pydoc >>> pydoc.help(lambda (a,): lambda x: a) # this fails Traceback (most recent call last): File "", line 1, in File "/usr/lib/python2.7/pydoc.py", line 1797, in call self.help(request) File "/usr/lib/python2.7/pydoc.py", line 1844, in help else: doc(request, 'Help on %s:') File "/usr/lib/python2.7/pydoc.py", line 1581, in doc pager(render_doc(thing, title, forceload)) File "/usr/lib/python2.7/pydoc.py", line 1576, in render_doc return title % desc + '\n\n' + text.document(object, name) File "/usr/lib/python2.7/pydoc.py", line 363, in document if inspect.isroutine(object): return self.docroutine(args) File "/usr/lib/python2.7/pydoc.py", line 1321, in docroutine args, varargs, varkw, defaults = inspect.getargspec(object) File "/usr/lib/python2.7/inspect.py", line 817, in getargspec args, varargs, varkw = getargs(func.func_code) File "/usr/lib/python2.7/inspect.py", line 791, in getargs args[i] = stack[0] IndexError: list index out of range >>> pydoc.help(lambda (a,): lambda x: x) # this succeeds >>> ---------- components: Library (Lib) messages: 286119 nosy: Elias Vanderstuyft priority: normal severity: normal status: open title: Python 2.7.12: pydoc.help(lambda (a,): lambda x: a) raises IndexError type: behavior versions: Python 2.7 _ Python tracker _ From report at bugs.python.org Mon Jan 23 23:38:13 2017 From: report at bugs.python.org (Ma Lin) Date: Tue, 24 Jan 2017 04:38:13 +0000 Subject: [New-bugs-announce] [issue29355] sqlite3: remove sqlite3_stmt_readonly() Message-ID: <1485232693.83.0.291296040478.issue29355@psf.upfronthosting.co.za> New submission from Ma Lin: In CPython 3.6.0, sqlite3 module uses sqlite3_stmt_readonly(), however this function is only available on SQLite 3.7.4+ (release on 2010-12-07). Then CPython 3.6.0 can not be compiled on RHEL6 (with SQLite 3.6.x), complaints: issue29098, issue29325. sqlite3_stmt_readonly() was introduced in 284676cf2ac8, it was used twice [1][2], but it seems that we can avoid using this function in both of them. [1] https://hg.python.org/cpython/file/3.6/Modules/_sqlite/cursor.c#l517 With palaviv's patches of issue28518, we can eliminate sqlite3_stmt_readonly() in here. [2] https://hg.python.org/cpython/file/3.6/Modules/_sqlite/cursor.c#l612 We can use the old way in here, the old way is even better IMO. This issue depends on issue28518. ---------- components: Library (Lib) messages: 286130 nosy: Ma Lin, berker.peksag, palaviv priority: normal severity: normal status: open title: sqlite3: remove sqlite3_stmt_readonly() type: enhancement versions: Python 3.6, Python 3.7 _ Python tracker _ From report at bugs.python.org Mon Jan 23 23:52:52 2017 From: report at bugs.python.org (=?utf-8?q?=C3=81d=C3=A1m_Szieberth?=) Date: Tue, 24 Jan 2017 04:52:52 +0000 Subject: [New-bugs-announce] [issue29356] TypeError by typing for Ellipsis Message-ID: <1485233572.17.0.570161775222.issue29356@psf.upfronthosting.co.za> New submission from ?d?m Szieberth: I would expect Ellipsis the same behavior as None in type hints instead of: TypeError: Union[arg, ...]: each arg must be a type. Got Ellipsis. ... would be much nicer than type(...) there. ---------- messages: 286134 nosy: ?d?m Szieberth priority: normal severity: normal status: open title: TypeError by typing for Ellipsis type: behavior versions: Python 3.6 ___________________ Python tracker _ From report at bugs.python.org Tue Jan 24 02:14:54 2017 From: report at bugs.python.org (Elazar Gershuni) Date: Tue, 24 Jan 2017 07:14:54 +0000 Subject: [New-bugs-announce] [issue29357] New NamedTuple syntax silently ignores method definitions Message-ID: <1485242094.09.0.593682639878.issue29357@psf.upfronthosting.co.za> New submission from Elazar Gershuni: The following does not work as expected: ``` from typing import NamedTuple class A(NamedTuple): a: int def repr(self): return 'some A' def spam(self): print('spam!') >>> a = A(5) >>> repr(a) # should be 'some A' 'A(a=5)' >>> a.spam() Traceback (most recent call last): File "", line 1, in AttributeError: 'A' object has no attribute 'spam' ``` ---------- components: Library (Lib) messages: 286147 nosy: elazar, gvanrossum, rhettinger priority: normal severity: normal status: open title: New NamedTuple syntax silently ignores method definitions type: behavior versions: Python 3.6 _ Python tracker ___________________ From report at bugs.python.org Tue Jan 24 03:25:29 2017 From: report at bugs.python.org (STINNER Victor) Date: Tue, 24 Jan 2017 08:25:29 +0000 Subject: [New-bugs-announce] [issue29358] Add tp_fastnew and tp_fastinit to PyTypeObject, 15-20% faster object instanciation Message-ID: <1485246329.89.0.254567368952.issue29358@psf.upfronthosting.co.za> New submission from STINNER Victor: After #29259 "Add tp_fastcall to PyTypeObject: support FASTCALL calling convention for all callable objects", the two last slots which still use the (args: tuple, kwargs: dict) calling convention are tp_new and tp_init, two major slots to instanciate objects. I implemented tp_fastnew/tp_fastinit on top of the issue #29259 pull request (tp_fastcall). The implementation is a WIP, it's just complete enough to start benchmarking the code. Example of benchmarks on the two types currently optimized in my WIP fast_init branch, list and _asyncio.Future: --- haypo at smithers$ ./python -m perf timeit -s 'List=list' 'List()' --duplicate=100 --compare-to=../default-ref/python Median +- std dev: [ref] 81.9 ns +- 0.2 ns -> [fast_init] 69.3 ns +- 0.4 ns: 1.18x faster (-15%) haypo at smithers$ ./python -m perf timeit -s 'List=list' 'List((1,2,3))' --duplicate=100 --compare-to=../default-ref/python Median +- std dev: [ref] 137 ns +- 6 ns -> [fast_init] 107 ns +- 0 ns: 1.28x faster (-22%) haypo at smithers$ ./python -m perf timeit -s 'import _asyncio, asyncio; Future=_asyncio.Future; loop=asyncio.get_event_loop()' 'Future(loop=loop)' --compare-to=../default-ref/python Median +- std dev: [ref] 411 ns +- 20 ns -> [fast_init] 355 ns +- 18 ns: 1.16x faster (-14%) --- The speedup of tp_fastnew + tp_fastinit is between 1.16x faster and 1.28x faster. The question is now if it is worth it. Warning: The code is not fully optimized and is likely to have subtle bugs. The pull request is not ready for a review, but you may take a look if you would like to have an idea of the required changes. The most tricky changes are made in typeobject.c to support backward compatibility (call tp_new if tp_fastnew is not set) and stable API (support Python 3.6 PyTypeObject without the two slots). Note: tp_fastnew and tp_fastinit slots are the expected end of my large FASTCALL optimization project. ---------- components: Interpreter Core messages: 286151 nosy: haypo, inada.naoki, rhettinger, serhiy.storchaka, yselivanov priority: normal pull_requests: 23 severity: normal status: open title: Add tp_fastnew and tp_fastinit to PyTypeObject, 15-20% faster object instanciation type: performance versions: Python 3.7 _ Python tracker _ From report at bugs.python.org Tue Jan 24 03:32:59 2017 From: report at bugs.python.org (Aivar Annamaa) Date: Tue, 24 Jan 2017 08:32:59 +0000 Subject: [New-bugs-announce] [issue29359] Deprecate string concatenation without plus Message-ID: <1485246779.15.0.108891177308.issue29359@psf.upfronthosting.co.za> New submission from Aivar Annamaa: How quickly will you notice the bug here: for sequence in ["Command", "MagicCommand", "Open", "Save", "SaveAs", "NewFile", "EditorTextCreated" "ShellCommand", "ShellInput", "ShowView", "HideView", "TextInsert", "TextDelete", ]: do_something(sequence) ? Given the title of the issue, its probably not so hard, but usually a list literal is not the first thing to suspect. Similar problem of missing comma may occur easily with multiline calls. If concatenation without plus was deprecated, these mistakes would be so easy to notice. ---------- messages: 286152 nosy: Aivar.Annamaa priority: normal severity: normal status: open title: Deprecate string concatenation without plus type: enhancement versions: Python 3.7 _ Python tracker _ From report at bugs.python.org Tue Jan 24 03:49:44 2017 From: report at bugs.python.org (STINNER Victor) Date: Tue, 24 Jan 2017 08:49:44 +0000 Subject: [New-bugs-announce] [issue29360] Don't check if all keys are strings in _PyStack_AsDict() Message-ID: <1485247784.31.0.540630884546.issue29360@psf.upfronthosting.co.za> New submission from STINNER Victor: While running the test suite on issues #29259 (tp_fastcall) and #29358 (tp_fastnew and tp_fastinit), a few test failed on the following assertion of _PyStack_AsDict(): assert(PyUnicode_CheckExact(key)); For example, test_dict calls dict({1: 2}) which must raise a TypeError. I'm not sure who is responsible to check if all keys are strings: _PyStack_AsDict(), PyArg_ParseXXX() or the final function? The check is already implemented in dict_init(), dict_update_common() calls PyArg_ValidateKeywordArguments(): >>> dict({1:2}) Traceback (most recent call last): File "", line 1, in TypeError: keyword arguments must be strings In Python 3.7, PyArg_ParseTupleAndKeywords(), _PyArg_ParseTupleAndKeywordsFast() and _PyArg_ParseStackAndKeywords() and PyArg_ValidateKeywordArguments() raise an error if a key of keyword argumens is not a string: if (!PyUnicode_Check(key)) { PyErr_SetString(PyExc_TypeError, "keywords must be strings"); return cleanreturn(0, &freelist); } IMHO the CALL_FUNCTION_EX instruction and the FASTCALL calling convention must not check if all keys are string: the check must only be done in the function (which can be a type constructor like dict_init()), for performance. Almost all functions use one the PyArg_ParseXXX() function. Only very special cases like dict_init() use more specific code to handle keyword arguments. By the way, _PyObject_FastCallKeywords() already contains the following comment: /* kwnames must only contains str strings, no subclass, and all keys must be unique: these checks are implemented in Python/ceval.c and _PyArg_ParseStackAndKeywords(). */ Note: Technically, I know that it's possible to put non-string keys in a dictionary and in a type dictionary. PyPy (and other Python implementations?) don't support non-string in type dictionary. Do you know use cases where func(**kwargs) must accept non-string keys? At the end, this long issue is a simple patch replacing an assertion with a comment in _PyStack_AsDict(): see attached patch ;-) ---------- components: Interpreter Core files: pystack_asdict.patch keywords: patch messages: 286157 nosy: haypo priority: normal severity: normal status: open title: Don't check if all keys are strings in _PyStack_AsDict() versions: Python 3.7 Added file: http://bugs.python.org/file46403/pystack_asdict.patch ___________________ Python tracker _ From report at bugs.python.org Tue Jan 24 04:56:41 2017 From: report at bugs.python.org (Reto Cavelti) Date: Tue, 24 Jan 2017 09:56:41 +0000 Subject: [New-bugs-announce] [issue29361] bug in pyserial: serialposix.py Message-ID: <1485251801.63.0.344501122232.issue29361@psf.upfronthosting.co.za> New submission from Reto Cavelti: bug in pyserial: serialposix.py line 50: # set custom divisor buf[6] = buf[7] / baudrate TypeError: integer argument expected, got float fix: do cast: buf[6] = int(buf[7] / baudrate) ---------- components: Library (Lib) messages: 286168 nosy: rc priority: normal severity: normal status: open title: bug in pyserial: serialposix.py type: crash versions: Python 3.5 _ Python tracker ___________________ From report at bugs.python.org Tue Jan 24 10:35:21 2017 From: report at bugs.python.org (STINNER Victor) Date: Tue, 24 Jan 2017 15:35:21 +0000 Subject: [New-bugs-announce] [issue29362] regrtest: don't fail immediately if a child does crash Message-ID: <1485272121.72.0.342410390691.issue29362@psf.upfronthosting.co.za> New submission from STINNER Victor: When regrtest is used with -jN, regrtest spawn child processes to run tests. If a child crashs, regrtest master process immedialtely fails. It would prefer to be able to continue to run following tests. It's not because a single test crashed that all tests would crash. Example of output when a crash occurs: ----------------- (...) 0:02:45 [118/404/3] test_bytes crashed -- running: test_multiprocessing_spawn (67 sec), test_io (54 sec), test_subprocess (76 sec) python: Objects/abstract.c:2186: _Py_CheckFunctionResult: Assertion `(callable != NULL) ^ (where != NULL)' failed. Fatal Python error: Aborted Current thread 0x00007f9210f11440 (most recent call first): File "/home/haypo/prog/python/git_cpython/Lib/unittest/case.py", line 177 in handle File "/home/haypo/prog/python/git_cpython/Lib/unittest/case.py", line 728 in assertRaises File "/home/haypo/prog/python/git_cpython/Lib/test/test_bytes.py", line 636 in test_maketrans (...) File "/home/haypo/prog/python/git_cpython/Lib/runpy.py", line 85 in _run_code File "/home/haypo/prog/python/git_cpython/Lib/runpy.py", line 193 in _run_module_as_main Traceback (most recent call last): File "/home/haypo/prog/python/git_cpython/Lib/runpy.py", line 193, in _run_module_as_main "main", mod_spec) (...) File "/home/haypo/prog/python/git_cpython/Lib/test/libregrtest/main.py", line 411, in run_tests run_tests_multiprocess(self) File "/home/haypo/prog/python/git_cpython/Lib/test/libregrtest/runtest_mp.py", line 221, in run_tests_multiprocess raise Exception(msg) Exception: Child error on test_bytes: Exit code -6 ----------------- ---------- components: Tests messages: 286194 nosy: haypo priority: normal severity: normal status: open title: regrtest: don't fail immediately if a child does crash type: enhancement versions: Python 3.7 ___________________ Python tracker _ From report at bugs.python.org Tue Jan 24 13:06:33 2017 From: report at bugs.python.org (Ethan Furman) Date: Tue, 24 Jan 2017 18:06:33 +0000 Subject: [New-bugs-announce] [issue29363] allow Python code to determine class help text Message-ID: <1485281193.78.0.719790622563.issue29363@psf.upfronthosting.co.za> New submission from Ethan Furman: >From issue29338, msg286139: -------------------------- It is easy to fix the [pydoc] test by adding missed lines. But I'm not sure that output the (correct) signature of enum classes makes the help better. Color(value, names=None, , module=None, qualname=None, type=None, start=1) Ethan, what are your thoughts? ---------- components: Argument Clinic messages: 286199 nosy: ethan.furman, larry, ncoghlan, rhettinger, serhiy.storchaka, xiang.zhang, yselivanov priority: normal severity: normal status: open title: allow Python code to determine class help text type: behavior versions: Python 3.7 _ Python tracker ___________________ From report at bugs.python.org Tue Jan 24 13:54:25 2017 From: report at bugs.python.org (Jason Matthew) Date: Tue, 24 Jan 2017 18:54:25 +0000 Subject: [New-bugs-announce] [issue29364] msilib Fetch raises MSIError rather than return None Message-ID: <1485284065.77.0.143549147327.issue29364@psf.upfronthosting.co.za> New submission from Jason Matthew: Fairly new to MSIs here. Working towards extracting the same information which is available via Orca graphical interfaces. I believe I've hit a bug within _msi.View class, namely the inability to safely determine the number of records in my result. Examples I've found depicting how to obtain query results lead me to believe _msi.View.Fetch() will return None if no more rows exist. My development using python 2.7.11, Windows 7 enterprise and 2.7.13, Windows Server 2016, has shown Fetch() will raise a msilib.MSIError when no more rows are available. MSIError is commonly used within msilib which means explicit error handling (or leniency in this case) may incorrectly suppress other error scenarios. Entering a bug with the hope either a patch addresses this issue or documentation can be updated to reflect proper usage. Example provided below. >>> import msilib >>> db = msilib.OpenDatabase("msi-1.0.0.8.msi", msilib.MSIDBOPEN_READONLY) >>> v = db.OpenView("select from Property") >>> v.Execute(None) >>> while 1: ... r = v.Fetch() ... if not r:break ... print r.GetString(1) ... UpgradeCode WixSharp_UI_INSTALLDIR WixSharp_InstallDialogs WIXUI_INSTALLDIR Manufacturer ProductCode ProductLanguage ProductName ProductVersion SecureCustomProperties Traceback (most recent call last): File "", line 2, in _msi.MSIError: unknown error 103 ---------- components: Library (Lib) messages: 286204 nosy: Jason Matthew priority: normal severity: normal status: open title: msilib Fetch raises MSIError rather than return None type: behavior versions: Python 2.7 _ Python tracker _ From report at bugs.python.org Tue Jan 24 21:30:14 2017 From: report at bugs.python.org (Gregory P. Smith) Date: Wed, 25 Jan 2017 02:30:14 +0000 Subject: [New-bugs-announce] [issue29365] parallel make test or make install can fail building pgen when configure --enable-optimizations is used Message-ID: <1485311414.76.0.958809336193.issue29365@psf.upfronthosting.co.za> New submission from Gregory P. Smith: If you do a make -j12 test or make install before having run a separate 'make all' the build _can_ fail trying to build pgen. (it seems to be parallel make related, it won't always fail) Even if you have run a separate 'make all', the make test and make install steps are likely to rebuild a ton of stuff and rerun the profile generation stage. Making the suggested "make, make test, make install" process effectively useless or at least take 3x longer than it should. something along the lines of: oss/cpython$ make -j12 install gcc -pthread -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes Parser/acceler.o Parser/grammar1.o Parser/listnode.o Parser/node.o Parser/parser.o Parser/parsetok.o Parser/bitset.o Parser/metagrammar.o Parser/firstsets.o Parser/grammar.o Parser/pgen.o Objects/obmalloc.o Python/mysnprintf.o Python/pyctype.o Parser/tokenizer_pgen.o Parser/printgrammar.o Parser/pgenmain.o -lpthread -ldl -lutil -o Parser/pgen Parser/grammar1.o: In function `PyGrammar_LabelRepr': /home/gps/oss/cpython/build27/../2.7/Parser/grammar1.c:35: undefined reference to `__gcov_indirect_call_profiler' Parser/grammar1.o: In function `PyGrammar_FindDFA': /home/gps/oss/cpython/build27/../2.7/Parser/grammar1.c:13: undefined reference to `__gcov_indirect_call_profiler' Parser/grammar1.o:(.data+0x80): undefined reference to `__gcov_merge_add' Parser/grammar1.o: In function `_GLOBAL__sub_I_65535_0_PyGrammar_FindDFA': /home/gps/oss/cpython/build27/../2.7/Parser/grammar1.c:57: undefined reference to `__gcov_init' ... a bunch more similar errors omitted ... Parser/pgenmain.o: In function `_GLOBAL__sub_I_65535_0_Py_Exit': /home/gps/oss/cpython/build27/../2.7/Parser/pgenmain.c:173: undefined reference to `__gcov_init' collect2: error: ld returned 1 exit status make: * [Parser/pgen] Error 1 make: * Waiting for unfinished jobs.... Parser/pgen ../2.7/Grammar/Grammar Include/graminit.h Python/graminit.c make[2]: Parser/pgen: Command not found make[2]: * [Include/graminit.h] Error 127 make[2]: * Waiting for unfinished jobs.... make[2]: Leaving directory `/home/gps/oss/cpython/build27' make[1]: * [build_all_generate_profile] Error 2 make[1]: Leaving directory `/home/gps/oss/cpython/build27' make: * [profile-opt] Error 2 ---------- components: Build messages: 286223 nosy: gregory.p.smith priority: normal severity: normal status: open title: parallel make test or make install can fail building pgen when configure --enable-optimizations is used type: compile error versions: Python 2.7, Python 3.6, Python 3.7 _ Python tracker _ From report at bugs.python.org Tue Jan 24 21:43:12 2017 From: report at bugs.python.org (raylu) Date: Wed, 25 Jan 2017 02:43:12 +0000 Subject: [New-bugs-announce] [issue29366] os.listdir has inconsistent behavior when run on a non-directory Message-ID: <1485312192.81.0.934042138535.issue29366@psf.upfronthosting.co.za> New submission from raylu: According to https://github.com/python/cpython/blob/4ed71dae9d23a412f9f73d3a0b1f4be37543e49e/Lib/test/test_unicode_file_functions.py#L106 listdir can sometimes return FileNotFoundError on Windows. Tangentially related: https://hg.python.org/cpython/rev/385c2ec78f16 ---------- components: Library (Lib), Windows messages: 286225 nosy: paul.moore, raylu, steve.dower, tim.golden, zach.ware priority: normal severity: normal status: open title: os.listdir has inconsistent behavior when run on a non-directory _ Python tracker _ From report at bugs.python.org Wed Jan 25 04:29:36 2017 From: report at bugs.python.org (STINNER Victor) Date: Wed, 25 Jan 2017 09:29:36 +0000 Subject: [New-bugs-announce] [issue29367] python-gdb: display wrapper_call() Message-ID: <1485336576.45.0.391174330106.issue29367@psf.upfronthosting.co.za> New submission from STINNER Victor: python-gdb.py handles various Python calls, but not descriptors. Attached patch adds code to display calls to wrapperobject objects. The patch adds the first "" line of the traceback: --- (gdb) py-bt Traceback (most recent call first): File "/home/haypo/prog/python/git_cpython/Lib/encodings/utf_8.py", line 41, in getregentry streamwriter=StreamWriter, File "/home/haypo/prog/python/git_cpython/Lib/encodings/init.py", line 122, in search_function entry = getregentry() --- I wrote this patch while working on the issue #29358. Object instanciation is full of wrappers. A better debugger helps to me understand my bugs ;-) Example to get such tracecback: --- haypo at selma$ gdb -args ./python -m test -v test_sys (gdb) b wrapper_call (gdb) run (gdb) py-bt Traceback (most recent call first): File "/home/haypo/prog/python/default/Lib/enum.py", line 66, in init super().init() File "/home/haypo/prog/python/default/Lib/enum.py", line 117, in prepare enum_dict = _EnumDict() File "/home/haypo/prog/python/default/Lib/enum.py", line 509, in class Enum(metaclass=EnumMeta): File "", line 205, in _call_with_frames_removed (...) --- I guess that the next step is to write a test ;-) ---------- files: gdb_wrapper_call.patch keywords: patch messages: 286233 nosy: haypo, inada.naoki priority: normal severity: normal status: open title: python-gdb: display wrapper_call() type: enhancement versions: Python 3.7 Added file: http://bugs.python.org/file46411/gdb_wrapper_call.patch _ Python tracker _ From report at bugs.python.org Wed Jan 25 04:55:50 2017 From: report at bugs.python.org (Serhiy Storchaka) Date: Wed, 25 Jan 2017 09:55:50 +0000 Subject: [New-bugs-announce] [issue29368] Optimize unpickling list-like objects Message-ID: <1485338150.33.0.560817507263.issue29368@psf.upfronthosting.co.za> New submission from Serhiy Storchaka: According to PEP 307 the extend method can be used for appending list items to the object. listitems Optional, and new in this PEP. If this is not None, it should be an iterator (not a sequence!) yielding successive list items. These list items will be pickled, and appended to the object using either obj.append(item) or obj.extend(list_of_items). This is primarily used for list subclasses, but may be used by other classes as long as they have append() and extend() methods with the appropriate signature. (Whether append() or extend() is used depends on which pickle protocol version is used as well as the number of items to append, so both must be supported.) Proposed patch makes the extend method be used in the APPENDS opcode. To avoid breaking existing code the use of the extend method is optional. Microbenchmark: $ ./python -m timeit -s "import pickle, collections; p = pickle.dumps(collections.deque([None]10000), 4)" -- "pickle.loads(p)" Unpatched: 100 loops, best of 5: 2.02 msec per loop Patched: 500 loops, best of 5: 833 usec per loop ---------- components: Library (Lib) files: pickle-appends-extend.patch keywords: patch messages: 286237 nosy: alexandre.vassalotti, serhiy.storchaka priority: normal severity: normal stage: patch review status: open title: Optimize unpickling list-like objects type: performance versions: Python 3.7 Added file: http://bugs.python.org/file46412/pickle-appends-extend.patch _ Python tracker _ From report at bugs.python.org Wed Jan 25 07:33:14 2017 From: report at bugs.python.org (INADA Naoki) Date: Wed, 25 Jan 2017 12:33:14 +0000 Subject: [New-bugs-announce] [issue29369] Use Py_IDENTIFIER in Python-ast.c Message-ID: <1485347594.14.0.436037009882.issue29369@psf.upfronthosting.co.za> New submission from INADA Naoki: While I'm looking Python's memory usage, I found some dicts which key is not interned "_fields", "_ast" and "modules". ---------- components: Interpreter Core files: ast-identifier.patch keywords: patch messages: 286244 nosy: inada.naoki priority: normal severity: normal status: open title: Use Py_IDENTIFIER in Python-ast.c versions: Python 3.7 Added file: http://bugs.python.org/file46413/ast-identifier.patch ___________________ Python tracker _ From report at bugs.python.org Wed Jan 25 08:01:21 2017 From: report at bugs.python.org (=?utf-8?b?0JTQuNC70Y/QvSDQn9Cw0LvQsNGD0LfQvtCy?=) Date: Wed, 25 Jan 2017 13:01:21 +0000 Subject: [New-bugs-announce] [issue29370] "./configure --enable-optimizations && make install" does not install files in ${BINDIR} and ${LIBDIR} Message-ID: <1485349281.44.0.908942370293.issue29370@psf.upfronthosting.co.za> New submission from ????? ????????: With Python 3.5.3 "./configure --enable-shared && make install DESTDIR=/tmp/ZZ1" does install files in /tmp/ZZ1/usr/local/bin/ and /tmp/ZZ1/usr/local/bin/ . "./configure --enable-optimizations --enable-shared && make install DESTDIR=/tmp/ZZ2" does not install files in /tmp/ZZ2/usr/local/bin/ and /tmp/ZZ2/usr/local/bin/ . ---------- components: Installation messages: 286247 nosy: dilyan.palauzov priority: normal severity: normal status: open title: "./configure --enable-optimizations && make install" does not install files in ${BINDIR} and ${LIBDIR} type: behavior versions: Python 3.5 _ Python tracker _ From report at bugs.python.org Wed Jan 25 08:36:09 2017 From: report at bugs.python.org (Marco Buttu) Date: Wed, 25 Jan 2017 13:36:09 +0000 Subject: [New-bugs-announce] [issue29371] Typo in doctest documentation Message-ID: <1485351369.68.0.185486175403.issue29371@psf.upfronthosting.co.za> New submission from Marco Buttu: >From the doctest documentation [1]: "Symbolic names for the flags are supplied as module constants, which can be or'ed together and passed to various functions." Is there a typo in "...which can be or'ed together..."? Maybe "collected together"? [1] https://docs.python.org/3/library/doctest.html#option-flags ---------- assignee: docs at python components: Documentation messages: 286254 nosy: docs at python, marco.buttu priority: normal severity: normal status: open title: Typo in doctest documentation versions: Python 2.7, Python 3.3, Python 3.4, Python 3.5, Python 3.6, Python 3.7 _ Python tracker _______________ From report at bugs.python.org Wed Jan 25 11:03:44 2017 From: report at bugs.python.org (Lee Griffiths) Date: Wed, 25 Jan 2017 16:03:44 +0000 Subject: [New-bugs-announce] [issue29372] RotatingFileHandler rotates empty logfile if it emits a large string Message-ID: <1485360224.38.0.947637323441.issue29372@psf.upfronthosting.co.za> New submission from Lee Griffiths: If you write a really big string to an empty file, and that string is > maxBytes, then `RotatingFileHandler` (or even `BaseRotatingHandler`) will roll that empty file into the backup queue. I think it should instead use that empty file for the mega-string. By not doing so it "wastes" a slot in the backup queue. It's a very minor issue: I doubt it really happens IRL, but I noticed it when extending RotatingFileHandler in py2.7 to add gzip stuff and my test cases used a really small size (16 bytes!) Here's a test file: #!/usr/bin/env python3 # coding=utf-8 import logging import os import tempfile from logging.handlers import RotatingFileHandler class MockRecord(object): def init(self, msg): self.msg = msg self.stack_info = None self.exc_info = None self.exc_text = None def getMessage(self): return self.msg def test_file_rollover_from_mega_string(temp_dir_path): # This is a pretty weird test. # It tests that writing a huge string to a blank file causes the blank # file to be archived and the huge string written to the next log file. # # Normally the log files would have a large max bytes so we'd have to # be writing a giant string to an empty file for this to happen. # But, even if it does, it's what BaseRotatingHandler does, so... log_path = os.path.join(temp_dir_path, "mylog.log") handler = RotatingFileHandler(log_path, maxBytes=16, backupCount=5) handler.setFormatter(logging.Formatter()) with open(log_path) as log: assert log.read() == "" # ---------------------------------------------- handler.emit(MockRecord("There once was a test from bitbucket")) with open(log_path) as log: log_read = log.read() assert log_read == "There once was a test from bitbucket\n" with open(log_path + ".1") as log: log_read = log.read() assert log_read == "" # ---------------------------------------------- handler.emit(MockRecord("11 chars")) with open(log_path) as log: log_read = log.read() assert log_read == "11 chars\n" with open(log_path + ".1") as log: log_read = log.read() assert log_read == "There once was a test from bitbucket\n" with open(log_path + ".2") as log: log_read = log.read() assert log_read == "" handler.close() test_file_rollover_from_mega_string(tempfile.mkdtemp()) and here's a patch that I think will fix it: ~/src/others/cpython (master %=)$ cat empty_rollover.patch diff --git a/Lib/logging/handlers.py b/Lib/logging/handlers.py index 7d77973..0dabfd7 100644 --- a/Lib/logging/handlers.py +++ b/Lib/logging/handlers.py @@ -186,7 +186,11 @@ class RotatingFileHandler(BaseRotatingHandler): if self.maxBytes > 0: # are we rolling over? msg = "%s\n" % self.format(record) self.stream.seek(0, 2) #due to non-posix-compliant Windows feature - if self.stream.tell() + len(msg) >= self.maxBytes: + size = self.stream.tell() + if size == 0: + # No point rolling-over an empty file + return 0 + elif size + len(msg) >= self.maxBytes: return 1 return 0 ---------- components: Library (Lib) messages: 286265 nosy: Poddster, vinay.sajip priority: normal severity: normal status: open title: RotatingFileHandler rotates empty logfile if it emits a large string type: resource usage versions: Python 2.7, Python 3.7 _ Python tracker _ From report at bugs.python.org Wed Jan 25 12:21:16 2017 From: report at bugs.python.org (Zac Medico) Date: Wed, 25 Jan 2017 17:21:16 +0000 Subject: [New-bugs-announce] [issue29373] subprocess.Popen os.close(p2cread) in _execute_child can cause garbage collection of self.stdout to close a reused file descriptor Message-ID: <1485364876.35.0.214175144503.issue29373@psf.upfronthosting.co.za> New submission from Zac Medico: In Popen _execute_child method, os.close(p2cread) closes a file descriptor which is referenced by self.stdout with closefd=True, created by this code: self.stdout = io.open(c2pread, 'rb', bufsize) When self.stdout is finally garbage collected, it can close a file descriptor which has been reused since the os.close(p2cread) call. In the rare cases when this happens, it results in a traceback like the following: Traceback (most recent call last): File "/usr/lib64/python3.4/subprocess.py", line 1418, in _execute_child part = _eintr_retry_call(os.read, errpipe_read, 50000) File "/usr/lib64/python3.4/subprocess.py", line 491, in _eintr_retry_call return func(args) OSError: [Errno 9] Bad file descriptor I have observed this issue with python 3.4.5, and the code flaw also appears to be present in 3.5, 3.6, and 3.7. This sort of garbage collection issue has been highlighted in the first comment of http://bugs.python.org/issue16140, but there is still a remaining problem with this os.close(p2cread) call. ---------- messages: 286268 nosy: zmedico priority: normal severity: normal status: open title: subprocess.Popen os.close(p2cread) in _execute_child can cause garbage collection of self.stdout to close a reused file descriptor versions: Python 3.4, Python 3.5, Python 3.6, Python 3.7 ___________________ Python tracker _ From report at bugs.python.org Wed Jan 25 12:25:31 2017 From: report at bugs.python.org (Petr MOTEJLEK) Date: Wed, 25 Jan 2017 17:25:31 +0000 Subject: [New-bugs-announce] [issue29374] Doc bug: signal.sigwait and signal.sigtimedwait do not work outside the Main thread Message-ID: <1485365131.36.0.15146613889.issue29374@psf.upfronthosting.co.za> New submission from Petr MOTEJLEK: Hi, The documentation for signal.signal() clearly states that it is only supposed to be called on MainThread However, it does not say so for the signal.sigwait() and neither signal.sigtimedwait() I think this is an error on the documentation side of things (unless I misread it). When either signal.sigwait or signal.sigtimedwait are called outside MainThread, they simply never catch any signals (signal.sigwait blocks indefinitely) I did not test this on Windows, but on both Linux and OS X the behavior is the same Consider the below simple code import signal import os def sigwait(): print("Send me a signal, my PID is {p}".format(p=os.getpid())) print("Got the signal: {i}".format(i=signal.sigwait((signal.SIGUSR1,)))) If sigwait() is called on MainThread and the process receives SIGUSR1, "Got the signal: ..." gets printed. However, calling sigwait in a different thread blocks the thread indefinitely. The behavior is the same with signal.sigtimedwait() as well ---------- assignee: docs at python components: Documentation messages: 286269 nosy: docs at python, petr at motejlek.net priority: normal severity: normal status: open title: Doc bug: signal.sigwait and signal.sigtimedwait do not work outside the Main thread type: behavior versions: Python 3.4, Python 3.5, Python 3.6, Python 3.7 _ Python tracker _ From report at bugs.python.org Wed Jan 25 15:14:44 2017 From: report at bugs.python.org (Jonathan Guthrie) Date: Wed, 25 Jan 2017 20:14:44 +0000 Subject: [New-bugs-announce] [issue29375] httplib: wrong Host header when connecting to IPv6 link-local address Message-ID: <1485375284.1.0.310369100275.issue29375@psf.upfronthosting.co.za> New submission from Jonathan Guthrie: This is related to issue 5111. An IPv6 link-local address must include a scope specifier as part of the address passed to the HTTPConnection or HTTPSConnection constructor, that scope specifier is not being stripped from the address passed in the HTTP 1.1 Host: header line. So, suppose I was attempting to connect to an HTTP server on fe80::8aae:1dff:fea4:29c8. That's a link-local address, so I must give the scope specifier, which is the interface I can use to connect to that address. So, the address I would pass to the HTTPConnection constructor might look like "fe80::8aae:1dff:fea4:29c8%eth0". The appropriate Host line in the HTTP request would be "Host: [fe80::8aae:1dff:fea4:29c8]" but it is actually "Host: [fe80::8aae:1dff:fea4:29c8%eth0]" Compliant HTTP servers reject requests including this line as malformed. ---------- messages: 286276 nosy: JonathanGuthrie priority: normal severity: normal status: open title: httplib: wrong Host header when connecting to IPv6 link-local address type: behavior versions: Python 2.7, Python 3.5 _ Python tracker ___________________ From report at bugs.python.org Wed Jan 25 18:10:08 2017 From: report at bugs.python.org (Tom Myers) Date: Wed, 25 Jan 2017 23:10:08 +0000 Subject: [New-bugs-announce] [issue29376] threading._DummyThread.repr raises AssertionError Message-ID: <1485385808.95.0.00162143215773.issue29376@psf.upfronthosting.co.za> New submission from Tom Myers: _DummyThread objects have _is_stopped = False, but _tstate_lock = None. This causes an AssertionError in is_alive and repr . File ".../cp34/lib/python3.4/threading.py", line 814, in repr self.is_alive() # easy way to get ._is_stopped set when appropriate File ".../cp34/lib/python3.4/threading.py", line 1110, in is_alive self._wait_for_tstate_lock(False) File ".../cp34/lib/python3.4/threading.py", line 1066, in _wait_for_tstate_lock assert self._is_stopped AssertionError ---------- components: Library (Lib) messages: 286285 nosy: Tom Myers priority: normal severity: normal status: open title: threading._DummyThread.repr raises AssertionError versions: Python 3.4 _ Python tracker _ From report at bugs.python.org Thu Jan 26 05:16:18 2017 From: report at bugs.python.org (Manuel Krebber) Date: Thu, 26 Jan 2017 10:16:18 +0000 Subject: [New-bugs-announce] [issue29377] Add the 'wrapper_descriptor' type to the types module Message-ID: <1485425778.09.0.153066086326.issue29377@psf.upfronthosting.co.za> New submission from Manuel Krebber: There currently is no type in the types module for the slot wrappers/wrapper_descriptor types. I would like to have something like WrapperDescriptor = type(object.init) added to it (or maybe even add it to MethodType). This would be helpful to check more easily if some object is a method. I can create a pull request for this if desired. ---------- components: Library (Lib) messages: 286304 nosy: Wheerd priority: normal severity: normal status: open title: Add the 'wrapper_descriptor' type to the types module type: enhancement _ Python tracker _ From report at bugs.python.org Thu Jan 26 11:25:04 2017 From: report at bugs.python.org (Kaeptm Blaubaer) Date: Thu, 26 Jan 2017 16:25:04 +0000 Subject: [New-bugs-announce] [issue29378] Invalid example in documentation for PyErr_Fetch Message-ID: <1485447904.37.0.693014852804.issue29378@psf.upfronthosting.co.za> New submission from Kaeptm Blaubaer: In the example are the pointers to pointers to PyObject too many, because then PyErr_Fetch and PyErr_Restore would get too many pointers to pointers. ---------- assignee: docs at python components: Documentation messages: 286321 nosy: Kaeptm Blaubaer, docs at python priority: normal severity: normal status: open title: Invalid example in documentation for PyErr_Fetch _ Python tracker _ From report at bugs.python.org Thu Jan 26 12:09:47 2017 From: report at bugs.python.org (Piotr Dobrogost) Date: Thu, 26 Jan 2017 17:09:47 +0000 Subject: [New-bugs-announce] [issue29379] Custom handlers not used when passing "context" argument to urllib2.urlopen() Message-ID: <1485450587.01.0.396469985739.issue29379@psf.upfronthosting.co.za> New submission from Piotr Dobrogost: When urllib2.urlopen() is passed "context" argument the new opener is being built (https://hg.python.org/cpython/file/a06454b1afa1/Lib/urllib2.py#l147) and used instead custom opener which might had been already installed with urllib2.install_opener(). This might lead to problems with proxies if custom ProxyHandler was used in custom opener ? see http://stackoverflow.com/q/36089694/95735 as an example. ---------- components: Library (Lib) messages: 286325 nosy: piotr.dobrogost priority: normal severity: normal status: open title: Custom handlers not used when passing "context" argument to urllib2.urlopen() type: behavior versions: Python 2.7 _ Python tracker _ From report at bugs.python.org Fri Jan 27 08:47:35 2017 From: report at bugs.python.org (Mirko Tebaldi (Realtebo)) Date: Fri, 27 Jan 2017 13:47:35 +0000 Subject: [New-bugs-announce] [issue29380] A lot of warning while executing make install Message-ID: <1485524855.82.0.489793661635.issue29380@psf.upfronthosting.co.za> New submission from Mirko Tebaldi (Realtebo): /usr/src/Python-3.6.0/Modules/_ctypes/libffi/src/x86/ffi64.c: In function 'classify_argument': /usr/src/Python-3.6.0/Modules/_ctypes/libffi/src/x86/ffi64.c:195:18: warning: suggest braces around empty body in an 'else' statement [-Wempty-body] FFI_ASSERT (0); ^ /usr/src/Python-3.6.0/Modules/_ctypes/libffi/src/x86/ffi64.c:224:16: warning: comparison between signed and unsigned integer expressions [-Wsign-compare] for (i = 0; i < words; i++) ^ /usr/src/Python-3.6.0/Modules/_ctypes/libffi/src/x86/ffi64.c:245:20: warning: comparison between signed and unsigned integer expressions [-Wsign-compare] for (i = 0; i < num; i++) ^ /usr/src/Python-3.6.0/Modules/_ctypes/libffi/src/x86/ffi64.c:264:20: warning: comparison between signed and unsigned integer expressions [-Wsign-compare] for (i = 1; i < words; i++) ^ /usr/src/Python-3.6.0/Modules/_ctypes/libffi/src/x86/ffi64.c:270:16: warning: comparison between signed and unsigned integer expressions [-Wsign-compare] for (i = 0; i < words; i++) ^ /usr/src/Python-3.6.0/Modules/_ctypes/libffi/src/x86/ffi64.c: In function 'examine_argument': /usr/src/Python-3.6.0/Modules/_ctypes/libffi/src/x86/ffi64.c:323:17: warning: comparison between signed and unsigned integer expressions [-Wsign-compare] for (i = 0; i < n; ++i) ^ /usr/src/Python-3.6.0/Modules/_ctypes/libffi/src/x86/ffi64.c: In function 'ffi_call': /usr/src/Python-3.6.0/Modules/_ctypes/libffi/src/x86/ffi64.c:484:18: warning: comparison between signed and unsigned integer expressions [-Wsign-compare] for (j = 0; j < n; j++, a += 8, size -= 8) ^ /usr/src/Python-3.6.0/Modules/_ctypes/libffi/src/x86/ffi64.c: In function 'ffi_closure_unix64_inner': /usr/src/Python-3.6.0/Modules/_ctypes/libffi/src/x86/ffi64.c:659:18: warning: comparison between signed and unsigned integer expressions [-Wsign-compare] for (j = 0; j < n; j++, a += 8) ^ ---------- messages: 286355 nosy: Mirko Tebaldi (Realtebo) priority: normal severity: normal status: open title: A lot of warning while executing make install type: compile error versions: Python 3.6 _ Python tracker _ From report at bugs.python.org Fri Jan 27 12:20:02 2017 From: report at bugs.python.org (Jim Fasarakis-Hilliard) Date: Fri, 27 Jan 2017 17:20:02 +0000 Subject: [New-bugs-announce] [issue29381] Tutorial documentation contains undefined reference Message-ID: <1485537602.74.0.811037724367.issue29381@psf.upfronthosting.co.za> New submission from Jim Fasarakis-Hilliard: After moving a certain chunk of the 'interpreter.rst' contents to 'appendix.rst' in issue16827 the reference to #! in the section '2.2.3. Source Code Encoding' is currently confusing for new readers. Attached patches reword the sentence to remove that reference for 3.4+ and 2.7. ---------- assignee: docs at python components: Documentation files: interpreter_tut.patch keywords: patch messages: 286378 nosy: Jim Fasarakis-Hilliard, docs at python priority: normal severity: normal status: open title: Tutorial documentation contains undefined reference versions: Python 2.7, Python 3.4, Python 3.5, Python 3.6, Python 3.7 Added file: http://bugs.python.org/file46438/interpreter_tut.patch ___________________ Python tracker _ From report at bugs.python.org Fri Jan 27 20:03:15 2017 From: report at bugs.python.org (d s) Date: Sat, 28 Jan 2017 01:03:15 +0000 Subject: [New-bugs-announce] [issue29382] IDLE search/replace/replace+find is (still?) using the 'hilite' tag instead of 'hit' Message-ID: <1485565395.44.0.960082340731.issue29382@psf.upfronthosting.co.za> New submission from d s: Searching for this issue turned up a bunch of results claiming it's fixed, but it isn't, at least not on my windows 10 machine. I tried replacing 'hit' with 'itdunwork' in colorizer.py, searchbase.py, replace.py, searchengine.py, config.py, search.py, configdialog.py, and of course config-highlight.cfg, but it didn't work. ---------- assignee: terry.reedy components: IDLE, Windows messages: 286393 nosy: d s2, paul.moore, steve.dower, terry.reedy, tim.golden, zach.ware priority: normal severity: normal status: open title: IDLE search/replace/replace+find is (still?) using the 'hilite' tag instead of 'hit' type: behavior versions: Python 3.5, Python 3.6 _ Python tracker ___________________ From report at bugs.python.org Fri Jan 27 22:04:15 2017 From: report at bugs.python.org (INADA Naoki) Date: Sat, 28 Jan 2017 03:04:15 +0000 Subject: [New-bugs-announce] [issue29383] Reduce temporary unicode object while adding descriptors Message-ID: <1485572655.18.0.389188121452.issue29383@psf.upfronthosting.co.za> New submission from INADA Naoki: add_methods(), add_members(), and add_getset() creates PyUnicode from C string 3 times, and calls PyUnicode_InternInplace 2 times. 1. PyDict_GetItemString() at first. (PyUnicode_FromString() is called). 2. In middle, descr_new() calls PyUnicode_InternFromString(). 3. PyDict_SetItemString() at last. (creates unicode and intern it). Skipping (2) is require adding new private APIs to pass PyUnicodeObject. But I don't think it worth enough. (I'll try it later.) So this patch only remove last temporary unicode. (3 PyUnicode_FromString + 2 PyUnicode_InternInplace) becomes (2 PyUnicode_FromString + 2 PyUnicode_InternInplace). It seems ~1% startup speedup (without site). $ ./python -m performance.benchmarks.bm_python_startup --no-site default: python_startup_no_site: Median +- std dev: 12.7 ms +- 0.1 ms patched: python_startup_no_site: Median +- std dev: 12.6 ms +- 0.1 ms While speedup is small, this patch removes time to think "How large this overhead of GetItemString + SetItemString pair?" while reading code :) Additionally, this patch removes this comment in PyDict_SetItemString: - PyUnicode_InternInPlace(&kv); / XXX Should we really? / SetItemString is used to add something to namespace. Changing this behavior affects too widely. So we should do it. ---------- assignee: inada.naoki components: Interpreter Core files: descr-remove-getitemstring.patch keywords: patch messages: 286394 nosy: inada.naoki priority: normal severity: normal status: open title: Reduce temporary unicode object while adding descriptors versions: Python 3.7 Added file: http://bugs.python.org/file46441/descr-remove-getitemstring.patch _ Python tracker _ From report at bugs.python.org Sat Jan 28 23:06:55 2017 From: report at bugs.python.org (Martin Panter) Date: Sun, 29 Jan 2017 04:06:55 +0000 Subject: [New-bugs-announce] [issue29384] Unused beos build scripts Message-ID: <1485662815.62.0.566203367253.issue29384@psf.upfronthosting.co.za> New submission from Martin Panter: Modules/ld_so_beos and Modules/ar_beos are no longer used since Be OS support was dropped from the configure script for Python 3: https://hg.python.org/cpython/diff/a5e984eda45a/configure.in I propose to also remove the scripts. ---------- components: Build files: beos-scripts.patch keywords: patch messages: 286426 nosy: martin.panter priority: normal severity: normal stage: patch review status: open title: Unused beos build scripts type: enhancement versions: Python 3.7 Added file: http://bugs.python.org/file46447/beos-scripts.patch _ Python tracker _ From report at bugs.python.org Sun Jan 29 00:36:00 2017 From: report at bugs.python.org (Dhiraj) Date: Sun, 29 Jan 2017 05:36:00 +0000 Subject: [New-bugs-announce] [issue29385] Sockets Crashes or Memory Corruption Message-ID: <1485668160.59.0.222592776098.issue29385@psf.upfronthosting.co.za> New submission from Dhiraj: When Python Server receives a string of '/xff' (5 or more) from a page after completing a handshake, the tab immediately crashes. There are variations of this string with other characters mixed in that also cause an immediate close of python server. To reproduce: 1. Run the websockets.py server 2. Then open websockets.html in browser The PY server gets Close/Crash ---------- components: Windows files: REPRO.ZIP messages: 286429 nosy: Dhiraj_Mishra, paul.moore, steve.dower, tim.golden, zach.ware priority: normal severity: normal status: open title: Sockets Crashes or Memory Corruption versions: Python 2.7, Python 3.3, Python 3.4, Python 3.5, Python 3.6, Python 3.7 Added file: http://bugs.python.org/file46448/REPRO.ZIP _ Python tracker _ From report at bugs.python.org Sun Jan 29 01:28:41 2017 From: report at bugs.python.org (Gabriel McManus) Date: Sun, 29 Jan 2017 06:28:41 +0000 Subject: [New-bugs-announce] [issue29386] select.epoll.poll may behave differently if timeout = -1 vs timeout = None Message-ID: <1485671321.44.0.0475790676768.issue29386@psf.upfronthosting.co.za> New submission from Gabriel McManus: The select module epoll.poll method takes a "timeout" parameter which is documented as having a default value of -1 [1]. If no timeout (or None) is passed to epoll.poll, then a value of -1 is passed to the epoll_wait system call. But if a timeout of -1 is passed to epoll.poll, then a value of -1000 is passed to epoll_wait. This is because the timeout is converted from seconds to milliseconds. Before Python 3.5, if a negative timeout was passed to epoll.poll then -1 was passed to epoll_wait [2]. The Linux epoll_wait documentation doesn't specify the behaviour if timeout < -1. Linux itself behaves the same for all negative timeouts: never time out. But on Illumos, timeout < -1 currently times out immediately, and only timeout == -1 never times out [3]. Some code does pass -1 to select.epoll.poll expecting it to never time out. For example, selectors.EpollSelector [4]. I suggest restoring the pre-3.5 behaviour: epoll.poll should use -1 if the given timeout is negative. I discovered this because ipython3 uses selectors.EpollSelector on Illumos, and it was using 100% cpu while waiting for input because epoll_wait was returning immediately. To demonstrate the issue you can run: strace python3.5 -c 'import select; select.epoll().poll(timeout=-1)' & On Illumos this completes immediately, and the output contains the -1000 timeout: epoll_wait(3, [], 1023, -1000) = 0 On Linux, it will block. If you then kill the python process with SIGTERM, strace should print the interrupted epoll_wait call, revealing the -1000 timeout: epoll_wait(3, ... 299a070, 1023, -100) = -1 EINTR (Interrupted system call) [1] https://github.com/python/cpython/blob/b9e40ed1bcce127893e40dd355087cda7187ac27/Modules/selectmodule.c#L1489 [2] https://github.com/python/cpython/commit/02e27d1301ea680dce9c3013010e3befedf9628a [3] https://github.com/joyent/illumos-joyent/issues/136 [4] https://github.com/python/cpython/blob/8228a2b306844a213eddb4fb908c1925840ff67e/Lib/selectors.py#L428 ---------- components: Extension Modules messages: 286431 nosy: Gabriel McManus priority: normal severity: normal status: open title: select.epoll.poll may behave differently if timeout = -1 vs timeout = None type: behavior versions: Python 3.5, Python 3.6, Python 3.7 ___________________ Python tracker _ From report at bugs.python.org Sun Jan 29 18:53:58 2017 From: report at bugs.python.org (Martin Panter) Date: Sun, 29 Jan 2017 23:53:58 +0000 Subject: [New-bugs-announce] [issue29387] Tabs vs spaces FAQ out of date Message-ID: <1485734038.81.0.0566253928841.issue29387@psf.upfronthosting.co.za> New submission from Martin Panter: The Windows FAQ mentions the ?python -t? command-line option, but in Python 3 this option is undocumented (and I understand has no effect): /media/disk/home/proj/python/cpython/Doc/faq/windows.rst:303: WARNING: unknown option: -t Also, the reference to the tabnanny script is wrong. It exists as a module (Lib/tabnanny.py), not in Tools/scripts/. This aspect also applies to 2.7. ---------- assignee: docs at python components: Documentation, Windows files: tabnanny.patch keywords: patch messages: 286460 nosy: docs at python, martin.panter, paul.moore, steve.dower, tim.golden, zach.ware priority: normal severity: normal stage: patch review status: open title: Tabs vs spaces FAQ out of date versions: Python 2.7, Python 3.5, Python 3.6, Python 3.7 Added file: http://bugs.python.org/file46450/tabnanny.patch _ Python tracker ___________________ From report at bugs.python.org Sun Jan 29 21:05:32 2017 From: report at bugs.python.org (MaxR) Date: Mon, 30 Jan 2017 02:05:32 +0000 Subject: [New-bugs-announce] [issue29388] regex mismatch in the simple cases Message-ID: <1485741932.85.0.668510587664.issue29388@psf.upfronthosting.co.za> New submission from MaxR: When I compile and match, for example: pattern = re.compile(r""" ([a-z]|\d)+$ # ends with letter or number only """, re.VERBOSE | re.IGNORECASE) print(re.match(pattern, 'abc')) result is correct: <_sre.SRE_Match object; span=(0, 3), match='abc'> but then I use the same pattern with another string: print(re.match(pattern, 'abc.fds')) result is: None I tried to reformulate the pattern to the same, for example: pattern = re.compile(r""" (a|b|c|d|e|f|g|h|i|j|k|l|m|n|o|p|q|r|s|t|u|v|w|x|y|z|\d)+$ # ends with letter or number only """, re.VERBOSE | re.IGNORECASE) or to: pattern = re.compile(r""" [a-z\d]+$ # ends with letter or number only """, re.VERBOSE | re.IGNORECASE) or to: pattern = = re.compile(r""" ([a-z]|\d)+$ # ends with letter or number only """, re.VERBOSE | re.IGNORECASE) but the result is all the time the same - None And ?hat is the double strange for the last three pattern - None is result also for print(re.match(pattern, 'abc')) I checked this patterns on the site regex101.com and all of them should work good and right It's achtung! ---------- components: Regular Expressions messages: 286464 nosy: MaxR, ezio.melotti, mrabarnett priority: normal severity: normal status: open title: regex mismatch in the simple cases type: behavior versions: Python 3.5 _ Python tracker _ From report at bugs.python.org Mon Jan 30 04:02:05 2017 From: report at bugs.python.org (Shmuel Amar) Date: Mon, 30 Jan 2017 09:02:05 +0000 Subject: [New-bugs-announce] [issue29389] math.isclose signature contains incorrect start parameter Message-ID: <1485766925.53.0.41080948643.issue29389@psf.upfronthosting.co.za> New submission from Shmuel Amar: documentation of math.isclose() signature on https://docs.python.org/3/library/math.html#math.isclose is as follows: math.isclose(a, b, , rel_tol=1e-09, abs_tol=0.0) the third star '' argument is not allowed inside the function: https://hg.python.org/cpython/file/tip/Modules/clinic/mathmodule.c.h#l511 or mentioned in PEP485 here: https://www.python.org/dev/peps/pep-0485/#implementation and does not work when trying provide more than 2 positional values: >>> import math >>> math.isclose(1,2,3, rel_tol=5.) Traceback (most recent call last): File "", line 1, in TypeError: Function takes at most 2 positional arguments (3 given) So IMHO to solve this remove the positional argument on the signature of isclose() as it misleading. ---------- assignee: docs at python components: Documentation messages: 286474 nosy: Shmuel Amar, docs at python priority: normal severity: normal status: open title: math.isclose signature contains incorrect start parameter versions: Python 3.5, Python 3.6, Python 3.7 ___________________ Python tracker _ From report at bugs.python.org Mon Jan 30 05:05:09 2017 From: report at bugs.python.org (Michael Kesper) Date: Mon, 30 Jan 2017 10:05:09 +0000 Subject: [New-bugs-announce] [issue29390] Python Tutorial should introduce iterators and generators Message-ID: <1485770709.65.0.205484403801.issue29390@psf.upfronthosting.co.za> New submission from Michael Kesper: Please look at http://stackoverflow.com/questions/41932287/how-can-i-create-a-loop-for-these-if-statements/41932494?noredirect=1#41932494 For beginners, it would be good to introduce the concepts of 'pythonic' dealing with sequences (iterators) and streams (generators) as soon as possible and everywhere where sequenceable data structures are discussed. It is a common idiom in other languages to access members of sequences with counters, risking off-by-one errors or IndexErrors. If beginners aren't introduced to the 'right' concepts soon enough, they might have a hard time figuring out the 'iterator' way. Iterators should at least be mentioned in https://docs.python.org/3/tutorial/datastructures.html#looping-techniques. ---------- assignee: docs at python components: Documentation messages: 286480 nosy: docs at python, mkesper priority: normal severity: normal status: open title: Python Tutorial should introduce iterators and generators type: enhancement _ Python tracker _ From report at bugs.python.org Mon Jan 30 10:53:13 2017 From: report at bugs.python.org (Glenn Clark) Date: Mon, 30 Jan 2017 15:53:13 +0000 Subject: [New-bugs-announce] [issue29391] Windows Defender finds trojan Spursint in Numpy for Py36 Win64 Message-ID: <1485791593.43.0.325442187579.issue29391@psf.upfronthosting.co.za> New submission from Glenn Clark: Windows Defender trojan Spursint Numpy Python3.6 Python 3.4 Win64 ---------- components: Windows messages: 286492 nosy: Glenn Clark, paul.moore, steve.dower, tim.golden, zach.ware priority: normal severity: normal status: open title: Windows Defender finds trojan Spursint in Numpy for Py36 Win64 type: security versions: Python 3.6 _ Python tracker ___________________ From report at bugs.python.org Mon Jan 30 12:30:26 2017 From: report at bugs.python.org (Frazer McLean) Date: Mon, 30 Jan 2017 17:30:26 +0000 Subject: [New-bugs-announce] [issue29392] msvcrt.locking crashes python Message-ID: <1485797426.48.0.513516354213.issue29392@psf.upfronthosting.co.za> New submission from Frazer McLean: On 3.5.2 and 3.5.3, but not 3.4.3 or 2.7.10 (Windows 64-bit), the third line causes Python to crash with exit code 0xC0000417. import msvcrt fp = open('file.txt', 'w') msvcrt.locking(fp.fileno(), msvcrt.LK_NBLCK, -1) print('End') I came across this in the portalocker module. I don't think -1 is a valid third argument, but it doesn't raise an exception on Python 3.4 or 2.7. ---------- components: Windows messages: 286495 nosy: RazerM, paul.moore, steve.dower, tim.golden, zach.ware priority: normal severity: normal status: open title: msvcrt.locking crashes python type: crash versions: Python 3.5 ___________________ Python tracker _ From report at bugs.python.org Mon Jan 30 21:44:14 2017 From: report at bugs.python.org (Cooper Lees) Date: Tue, 31 Jan 2017 02:44:14 +0000 Subject: [New-bugs-announce] [issue29393] Enabiling IPv6 by default Message-ID: <1485830654.03.0.638312196963.issue29393@psf.upfronthosting.co.za> New submission from Cooper Lees: Can we discuss the possibility of enabling IPv6 by default now during build. I think enough of the world now NEED IPv6 so people should all be building their Python with it. ---------- components: Build messages: 286506 nosy: cooperlees priority: normal severity: normal status: open title: Enabiling IPv6 by default type: behavior versions: Python 3.7 _ Python tracker ___________________ From report at bugs.python.org Tue Jan 31 02:19:43 2017 From: report at bugs.python.org (Maximilian Blochberger) Date: Tue, 31 Jan 2017 07:19:43 +0000 Subject: [New-bugs-announce] [issue29394] Cannot tunnel TLS connection through TLS connection Message-ID: <1485847183.69.0.499542154053.issue29394@psf.upfronthosting.co.za> New submission from Maximilian Blochberger: I have the following scenario: Client ? Proxy ? Target. The following two scenarios are working perfectly fine: 1) Establishing a TLS-secured connection to the proxy and then tunnel traffic through that connection to the target. This results in the proxy being able to observe and manipulate the traffic in both directions. It protects against an adversary who has no control over the proxy, e.g. it prevents observers from learning that you are using a proxy (if the IP/port is not known) and from reading the actual traffic. 2) Establish a non-secured connection to the proxy and then tunnel TLS-secured traffic through that connection to the target. That prevents the proxy from being able to observe or manipulate the traffic. Although an observer could learn that you are using a proxy and what target you are connecting to. Now what I tried was to establish a TLS-secured connection to the proxy and then to establish a TLS-secured tunnel to the target, effectively resulting in two layers of TLS in between the client and the proxy. This would protect from an observer learning that you are using a proxy and where you connect to (the proxy still knows) but preventing the proxy from observing and manipulating the actual traffic to the target. This does not work in Python 3.6. The TLS-secured connection to the proxy is straight forward and can be easily done with ssl.SSLContext.wrap_socket(). The TCP connection between the proxy and the target can then be established by issuing an HTTP CONNECT request. The response can than be read without closing the connection as done in http.client.HTTPConnection._tunnel(). Now my idea was to call ssl.SSLContext.wrap_socket() again (with a different context for the target) and send traffic through that. Unfortunately the TLS handshake fails with the error message "unknown protocol". I looked into the actual traffic transmitted and realised that the handshake was performed in plain text ??so effectively stripping the TLS layer that was established already ? which results in the proxy server not knowing how to handle the traffic (as it is not TLS-secured) aborting the connection (and reporting a fatal TLS alert). This leads to the conclusion that another call to ssl.SSLContext.wrap_socket() will override a previous call of the same function (different context object though). I think this is unexpected behaviour. It might be easier to handle such scenarios if a tunnel would be a separate http.client.HTTP(S)Connection object, see issue #24964. This would also allow to handle ssl-specific calls such as ssl.SSLSocket.getpeercert() as each layer probably uses different certificates. ---------- assignee: christian.heimes components: SSL messages: 286517 nosy: Maximilian Blochberger, christian.heimes priority: normal severity: normal status: open title: Cannot tunnel TLS connection through TLS connection type: behavior versions: Python 3.6 _ Python tracker _ From report at bugs.python.org Tue Jan 31 04:03:23 2017 From: report at bugs.python.org (=?utf-8?q?Christian_W=C3=BCste?=) Date: Tue, 31 Jan 2017 09:03:23 +0000 Subject: [New-bugs-announce] =?utf-8?q?=5Bissue29395=5D_Edit_with_IDLE_3?= =?utf-8?q?=2E6_don=C2=B4t_work_if_Name_of_File_=22code=2Epy=22?= Message-ID: <1485853403.69.0.0659896818047.issue29395@psf.upfronthosting.co.za> New submission from Christian W?ste: As soon as I specify a file "code.py", it can not be opened via the right click > Edit with IDLE 3.6 (64-Bit). The content of the file is completely irrelevant. All other Python files located in the same folder as "code.py" can not be opened. If I rename the file in for example coder.py everything works. Previously, Python version 3.5.3 was installed. It is also possible that it is attached to my system. ---------- assignee: terry.reedy components: IDLE messages: 286518 nosy: terry.reedy, wueste87 priority: normal severity: normal status: open title: Edit with IDLE 3.6 don?t work if Name of File "code.py" type: behavior versions: Python 3.6 _ Python tracker _ From report at bugs.python.org Tue Jan 31 05:53:07 2017 From: report at bugs.python.org (Silvio Ricardo Cordeiro) Date: Tue, 31 Jan 2017 10:53:07 +0000 Subject: [New-bugs-announce] [issue29396] Re-opening /dev/tty breaks readline Message-ID: <1485859987.68.0.026952717617.issue29396@psf.upfronthosting.co.za> New submission from Silvio Ricardo Cordeiro: The following code works on python2 (tested with 2.7.6), but breaks readline for python3 (tested with 3.4.3): >>> import sys, readline >>> sys.stdin = open('/dev/tty', 'r') >>> input("> ") # press "7" >>> input("> ") # press "up" key Re-opening /dev/tty as stdin is a very useful technique. For example, one could debug a script that takes a file as stdin by re-opening the stdin as /dev/tty and using pdb.set_trace(). See bug report for python2 in 2002 here: https://bugs.python.org/issue512981 ---------- components: Library (Lib) messages: 286521 nosy: silvioricardoc priority: normal severity: normal status: open title: Re-opening /dev/tty breaks readline versions: Python 3.4 _ Python tracker _ From report at bugs.python.org Tue Jan 31 09:21:40 2017 From: report at bugs.python.org (zhaw_munt) Date: Tue, 31 Jan 2017 14:21:40 +0000 Subject: [New-bugs-announce] [issue29397] linux/random.h present but cannot be compiled Message-ID: <1485872500.79.0.0659257887821.issue29397@psf.upfronthosting.co.za> New submission from zhaw_munt: configure: WARNING: linux/random.h: present but cannot be compiled configure: WARNING: linux/random.h: check for missing prerequisite headers? configure: WARNING: linux/random.h: see the Autoconf documentation configure: WARNING: linux/random.h: section "Present But Cannot Be Compiled" configure: WARNING: linux/random.h: proceeding with the compiler's result configure: WARNING: ## --------------------------------------- ## configure: WARNING: ## Report this to https://bugs.python.org/ ## configure: WARNING: ## --------------------------------------- ## 2.6.18-194.el5 #1 SMP Fri Apr 2 14:58:14 EDT 2010 x86_64 x86_64 x86_64 GNU/Linu (CentOS 5.5) Buildings seems to work, at the end I have the binary but I've noticed that at least* _socket is missing. Other libraries such as "_bz2" and "_sqlite" are there (as I've installed their corresponding devel package) ---------- messages: 286533 nosy: zhaw_munt priority: normal severity: normal status: open title: linux/random.h present but cannot be compiled type: compile error versions: Python 3.5 _ Python tracker _ From report at bugs.python.org Tue Jan 31 09:56:33 2017 From: report at bugs.python.org (zeroinside) Date: Tue, 31 Jan 2017 14:56:33 +0000 Subject: [New-bugs-announce] [issue29398] memory corruption in xxlimited Message-ID: <1485874593.96.0.823310722961.issue29398@psf.upfronthosting.co.za> New submission from zeroinside: Hello I found incorrect gc behavior in xxlimited module. After an hour of investigation, I'm still not sure its security related problem. I have a partial control on RBP register, depends of memory layout. GDB: Starting program: /usr/bin/python3.6 [Thread debugging using libthread_db enabled] Using host libthread_db library "/usr/lib/libthread_db.so.1". Python 3.6.0 (default, Jan 16 2017, 12:12:55) [GCC 6.3.1 20170109] on linux Type "help", "copyright", "credits" or "license" for more information. >>> import xxlimited >>> a=xxlimited.new() >>> del a Program received signal SIGSEGV, Segmentation fault. 0x00007ffff73d612d in PyArena_Free () from /usr/lib/libpython3.6m.so.1.0 (gdb) info reg rax 0x7ffff7457270 140737341911664 rbx 0x7ffff7f812b8 140737353618104 rcx 0x62aa00 6466048 rdx 0x7ffff7457270 140737341911664 rsi 0x1 1 rdi 0x7ffff7f81300 140737353618176 rbp 0x500000a29 0x500000a29 rsp 0x7fffffffe210 0x7fffffffe210 r8 0x7ffff7f81000 140737353617408 r9 0x1c 28 r10 0x1b 27 r11 0x12300 74496 r12 0x7ffff7f812b8 140737353618104 r13 0x6fafd0 7319504 r14 0x7ffff3e7b570 140737285436784 r15 0x7ffff3e7b5a0 140737285436832 rip 0x7ffff73d612d 0x7ffff73d612d eflags 0x10202 [ IF RF ] cs 0x33 51 ss 0x2b 43 ds 0x0 0 es 0x0 0 fs 0x0 0 gs 0x0 0 root@/home/zeroinside/python/pyburst3.6/crash $ /usr/local/bin/python3.6 xlimited_poc.py ASAN:DEADLYSIGNAL ================================================================= ==5082==ERROR: AddressSanitizer: SEGV on unknown address 0x000000000008 (pc 0x000000557469 bp 0x0fe0603ea23d sp 0x7fff76e98c20 T0) ==5082==The signal is caused by a WRITE memory access. ==5082==Hint: address points to the zero page. #0 0x557468 in PyObject_GC_UnTrack /home/zeroinside/Downloads/Python-3.6.0/Modules/gcmodule.c:1699:9 #1 0x66d0af in subtype_dealloc /home/zeroinside/Downloads/Python-3.6.0/Objects/typeobject.c:1133:5 #2 0x61e557 in _PyDict_DelItem_KnownHash /home/zeroinside/Downloads/Python-3.6.0/Objects/dictobject.c:1641:5 #3 0x7970c0 in _PyEval_EvalFrameDefault /home/zeroinside/Downloads/Python-3.6.0/Python/ceval.c:2187:19 #4 0x7aef44 in PyEval_EvalFrameEx /home/zeroinside/Downloads/Python-3.6.0/Python/ceval.c:718:12 #5 0x7aef44 in _PyEval_EvalCodeWithName /home/zeroinside/Downloads/Python-3.6.0/Python/ceval.c:4119 #6 0x79571c in PyEval_EvalCodeEx /home/zeroinside/Downloads/Python-3.6.0/Python/ceval.c:4140:12 #7 0x79571c in PyEval_EvalCode /home/zeroinside/Downloads/Python-3.6.0/Python/ceval.c:695 #8 0x5295e7 in run_mod /home/zeroinside/Downloads/Python-3.6.0/Python/pythonrun.c:980:9 #9 0x5295e7 in PyRun_FileExFlags /home/zeroinside/Downloads/Python-3.6.0/Python/pythonrun.c:933 #10 0x527e75 in PyRun_SimpleFileExFlags /home/zeroinside/Downloads/Python-3.6.0/Python/pythonrun.c:396:13 #11 0x55340c in run_file /home/zeroinside/Downloads/Python-3.6.0/Modules/main.c:320:11 #12 0x55340c in Py_Main /home/zeroinside/Downloads/Python-3.6.0/Modules/main.c:780 #13 0x519776 in main /home/zeroinside/Downloads/Python-3.6.0/./Programs/python.c:69:11 #14 0x7f0300e01290 in __libc_start_main (/usr/lib/libc.so.6+0x20290) #15 0x432179 in _start (/usr/local/bin/python3.6+0x432179) AddressSanitizer can not provide additional info. SUMMARY: AddressSanitizer: SEGV /home/zeroinside/Downloads/Python-3.6.0/Modules/gcmodule.c:1699:9 in PyObject_GC_UnTrack ==5082==ABORTING ---------- components: Build files: xlimited.py messages: 286536 nosy: zeroinside priority: normal severity: normal status: open title: memory corruption in xxlimited type: security versions: Python 3.6 Added file: http://bugs.python.org/file46465/xlimited.py _ Python tracker _ From report at bugs.python.org Tue Jan 31 12:14:06 2017 From: report at bugs.python.org (Elli Pirelli) Date: Tue, 31 Jan 2017 17:14:06 +0000 Subject: [New-bugs-announce] [issue29399] python3.dll export forwarders not resolved in all situations due to missing python3?.dll dependency and DLL search path behavior Message-ID: <1485882846.24.0.14620001579.issue29399@psf.upfronthosting.co.za> New submission from Elli Pirelli: Let me first apologize if this issue has already been reported and discussed before. I would be surprised if there are not a few developers that had to deal with this issue before, but i was unable to find an existing bug report or discussion about this. So, here it goes... Importing/using Py_* symbols from python3.dll instead directly from python3?.dll can be a problem when python3.dll and python3?.dll reside in a directory that is not part of the standard DLL search path. An example would be an application that tries loading Python from a sub directory of the application. The functions exported in python3.dll are export forwarders referencing the respective functions in python3?.dll. python3.dll itself has no import dependency on python3?.dll. Without an import dependency on python3?.dll, this DLL will be loaded when one of the export forwarders in python3.dll will be resolved, for example when calling GetProcAddress("Py_Main"). However, in such a case an altered DLL search path will be completely ignored, and python3?.dll will only be looked for in the standard DLL search paths. Since the sub directory with the python DLLs is not part of the standard DLL search paths, the resolution of the export fowarder(s) fails. Trying to alter the DLL search path has no effect. Loading python3.dll with LoadLibraryEx and the flag LOAD_WITH_ALTERED_SEARCH_PATH does not help. Neither does calling SetDllDirectory or AddDllDirectory before loading python3.dll and/or before invoking GetProcAddress. Note that loading python3.dll itself works fine. However, resolving one of the exports with GetProcAddress will fail if python3?.dll cannot be found in the standard DLL search path. I don't know, if there is a way to make it work properly... As of now the only two options out of this dilemma are either to put the python DLLs (and assorted files) in the same directory as the application exe (ugly mess) or to load python3?.dll instead of python3.dll (which sort of defeats the purpose of having python3.dll, i guess). My suggestion for future Python 3 versions is to let python3.dll have a DLL dependency on python3?.dll. Linking python3.dll to python3?.dll (i.e., loading python3.dll will cause python3?.dll being loaded) allows using altered DLL search paths. ---------- messages: 286544 nosy: Elli Pirelli priority: normal severity: normal status: open title: python3.dll export forwarders not resolved in all situations due to missing python3?.dll dependency and DLL search path behavior type: behavior versions: Python 3.5, Python 3.6, Python 3.7 _ Python tracker _ From report at bugs.python.org Tue Jan 31 13:20:52 2017 From: report at bugs.python.org (George King) Date: Tue, 31 Jan 2017 18:20:52 +0000 Subject: [New-bugs-announce] [issue29400] Instruction level tracing via sys.settrace Message-ID: <1485886852.16.0.854514177291.issue29400@psf.upfronthosting.co.za> New submission from George King: I have recently put some effort into developing a code coverage tool that shows correct results for intraline branches. In order to get intraline trace data, I patched CPython, adding an optional "trace instructions" flag to sys.settrace. The patch is fairly simple. http://github.com/gwk/cpython/tree/inst-tracing The crucial detail in ceval is here: @@ -4521,6 +4523,9 @@ maybe_call_line_trace(Py_tracefunc func, PyObject obj, frame->f_lineno = line; result = call_trace(func, obj, tstate, frame, PyTrace_LINE, Py_None); } + else if (tstate->inst_tracing) { + result = call_trace(func, obj, tstate, frame, PyTrace_INSTRUCTION, Py_None); + } The call to maybe_call_line_trace is guarded by several checks, so I do not see how this would significantly impact performance, except for when tracing is enabled. I built python3.6 using clang -O3 -flto (I saw lots of test failures during the --with-optimizations PGO tests so I opted for a simpler build), with and without my patch. I then ran `python3 -m performance run --fast` and manually compared the results. The patched version was in most cases slightly slower but within the standard deviation of the normal version. I would appreciate feedback before going any further with this. I looked at haypo's perf project but was unsure how to use it to run a more detailed comparison. First, will this feature be rejected on performance grounds or some other obvious reason? * Second, does setting `tstate->inst_tracing` in PyEval_SetTraceInstructions require more elaborate guards similar to those in PyEval_SetTrace? I do not understand the intricacies of that function. I am also curious about backwards compatibility. This adds an optional argument to settrace (haypo's recommendation) and some other things that appear to be under Py_LIMITED_API or internal (but I am not sure). * Is a change like this eligible for 3.6.x or would it have to wait until 3.7? * Would it be better to add a completely new function, e.g. sys.settrace_instructions instead of changing the signature of settrace? Lastly, I have a few ideas for enhancing this patch that will require some more work on my end to verify their utility. In particular: * Remove the `else` to always trace with PyTrace_INSTRUCTION, rather than as an alternate to PyTrace_LINE as it does now. * Pass the previous instruction offset as the trace function argument. This would give the tracer complete "edge" information. (In my coverage tool's trace function I track the previous edge as a nonlocal, but it would be better/faster to do it automatically in maybe_call_line_trace). * Finally, if all of this works, it might also be useful (as a performance enhancement) to add a PyTrace_BRANCH option, which would omit runs of opcodes inbetween branches, as these edges tend not be be useful, and even counterproductive (at least for my use case). However it might be too tricky to get all the cases right. ---------- components: Interpreter Core messages: 286546 nosy: George.King, haypo priority: normal severity: normal status: open title: Instruction level tracing via sys.settrace type: enhancement versions: Python 3.6 ___________________ Python tracker _ From report at bugs.python.org Tue Jan 31 15:10:17 2017 From: report at bugs.python.org (Gerrit Holl) Date: Tue, 31 Jan 2017 20:10:17 +0000 Subject: [New-bugs-announce] [issue29401] Python3.6 docs should not have a "what's new in Python 3.7" page Message-ID: <1485893417.06.0.864310985594.issue29401@psf.upfronthosting.co.za> New submission from Gerrit Holl: I believe this page is not supposed to exist: https://docs.python.org/3.6/whatsnew/3.7.html Nor this one: https://docs.python.org/3.5/whatsnew/3.6.html Neither are maintained, nor would I expect them to be. ---------- assignee: docs at python components: Documentation messages: 286549 nosy: Gerrit.Holl, docs at python priority: normal severity: normal status: open title: Python3.6 docs should not have a "what's new in Python 3.7" page versions: Python 3.5, Python 3.6 _ Python tracker _______________ From report at bugs.python.org Tue Jan 31 16:34:54 2017 From: report at bugs.python.org (Terry J. Reedy) Date: Tue, 31 Jan 2017 21:34:54 +0000 Subject: [New-bugs-announce] [issue29402] Problem with Checkbutton and duplicate last name components Message-ID: <1485898494.74.0.656128703551.issue29402@psf.upfronthosting.co.za> New submission from Terry J. Reedy: Report and code from George Trojan on python-list. import tkinter class GUI(tkinter.Tk): def init(self): tkinter.Tk.init(self) frame = tkinter.Frame(self) for tag in ('A', 'B'): w = tkinter.Checkbutton(frame, text=tag) w.pack(side='top', padx=10, pady=10) print(w) frame.pack(side='top') frame = tkinter.Frame(self) for tag in ('C', 'D'): w = tkinter.Checkbutton(frame, text=tag) w.pack(side='top', padx=10, pady=10) print(w) frame.pack(side='top') gui = GUI() gui.mainloop() In 3.5, each Checkbutton has unique last name component. .2028654224384.2028654606600 .2028654224384.2028654608224 .2028654606656.2028654606824 .2028654606656.2028654608336 Clicking any box checks or unchecks exactly that box. In 3.6, last name components are duplicated .!frame.!checkbutton .!frame.!checkbutton2 .!frame2.!checkbutton .!frame2.!checkbutton2 Clicking any box checks or unchecks both button with 'same name'. I verified that the _w strings passed in tk.call are the full unique names. MRAB reported that adding a tk variable attribute fixed the problem. I notice that Brian Oakley said the same thing in the similar issue #25684, though that is marked for 2.7 and 3.5 also. ---------- keywords: 3.6regression messages: 286552 nosy: serhiy.storchaka, terry.reedy priority: normal severity: normal stage: test needed status: open title: Problem with Checkbutton and duplicate last name components type: behavior versions: Python 3.6, Python 3.7 _ Python tracker _ From report at bugs.python.org Tue Jan 31 18:01:55 2017 From: report at bugs.python.org (Aaron Gallagher) Date: Tue, 31 Jan 2017 23:01:55 +0000 Subject: [New-bugs-announce] [issue29403] mock's autospec's behavior on method-bound builtin functions is broken Message-ID: <1485903715.23.0.569013669104.issue29403@psf.upfronthosting.co.za> New submission from Aaron Gallagher: Cython will, in the right circumstances, offer a MethodType instance where im_func is a builtin function. Any instance of MethodType is automatically assumed to be a python-defined function (more specifically, a function that has an inspectable signature), but _set_signature was still conservative in its assumptions. As a result _set_signature would return early with None instead of a mock since the im_func had no inspectable signature. This causes problems deeper inside mock, as _set_signature is assumed to always return a mock, and nothing checked its return value. In similar corner cases, autospec will simply not check the spec of the function, so _set_signature is amended to now return early with the original, not-wrapped mock object. There is another None early return in _set_signature, but it seems.. harder to trigger? It seems more appropriate to raise an exception there if this is being fixed. ---------- components: Library (Lib) messages: 286565 nosy: Aaron Gallagher priority: normal severity: normal status: open title: mock's autospec's behavior on method-bound builtin functions is broken type: behavior versions: Python 2.7, Python 3.3, Python 3.4, Python 3.5, Python 3.6, Python 3.7 ___________________ Python tracker _ From report at bugs.python.org Tue Jan 31 19:00:11 2017 From: report at bugs.python.org (Vincent Pelletier) Date: Wed, 01 Feb 2017 00:00:11 +0000 Subject: [New-bugs-announce] [issue29404] "TypeError: 'int' does not have the buffer interface" on memoryview over bytearray Message-ID: <1485907211.51.0.510192590668.issue29404@psf.upfronthosting.co.za> New submission from Vincent Pelletier: Quoting a github gist[1] (not mine, I found it while googling for this error) clearly illustrating the issue: # Python 2 >>> b = bytearray(8) >>> v = memoryview(b) >>> v[0] = 42 Traceback (most recent call last): File "", line 1, in TypeError: 'int' does not have the buffer interface >>> b bytearray(b'\x00\x00\x00\x00\x00\x00\x00\x00') # Python 3 >>> b = bytearray(8) >>> v = memoryview(b) >>> v[0] = 42 >>> b bytearray(b'\x00\x00\x00\x00\x00\x00\x00') This is especially annoying as on python3 bytearray.setitem only accepts integers, making working py2 code totally incompatible with py3 in a 2to3-undetectable way. [1] https://gist.github.com/superbobry/b90c0cc7c44beaa51ef9 ---------- messages: 286567 nosy: vpelletier priority: normal severity: normal status: open title: "TypeError: 'int' does not have the buffer interface" on memoryview over bytearray type: behavior versions: Python 2.7 _ Python tracker ___________________ From report at bugs.python.org Tue Jan 31 19:26:25 2017 From: report at bugs.python.org (Milt Epstein) Date: Wed, 01 Feb 2017 00:26:25 +0000 Subject: [New-bugs-announce] [issue29405] improve csv.Sniffer().sniff() behavior Message-ID: <1485908785.08.0.552892497325.issue29405@psf.upfronthosting.co.za> New submission from Milt Epstein: I'm trying to use csv.Sniffer().sniff(sample_data) to determine the delimiter on a number of input files. Through some trial and error, many "Could not determine delimiter" errors, and analyzing how this routine works/behaves, I settled on sample_data being some number of lines of the input file, particularly 30. This value seems to allow the routine to work more frequently, although not always, particularly on short input files. I realize the way this routine works is somewhat idiosyncratic, and it won't be so easy to improve it generally, but there's one simple change that occurred to me that would help in some cases. Currently the function _guess_delimiter() in csv.py contains the following lines: # build a list of possible delimiters modeList = modes.items() total = float(chunkLength iteration) So total is increased by chunkLength on each iteration. The problem occurs when total becomes greater than the length of sample_data, that is, the iteration would go beyond the end of sample_data. That reading is handled fine, it's truncated at the end of sample_data, but total is needlessly set too high. My suggested change is to add the following two lines after the above: if total > len(data): total = float(len(data)) ---------- components: Library (Lib) messages: 286570 nosy: mepstein priority: normal severity: normal status: open title: improve csv.Sniffer().sniff() behavior type: behavior versions: Python 3.5 _ Python tracker _ From report at bugs.python.org Tue Jan 31 19:39:05 2017 From: report at bugs.python.org (Alexander Mohr) Date: Wed, 01 Feb 2017 00:39:05 +0000 Subject: [New-bugs-announce] [issue29406] asyncio SSL contexts leak sockets after calling close with certain Apache servers Message-ID: <1485909545.04.0.650924943503.issue29406@psf.upfronthosting.co.za> New submission from Alexander Mohr: with the attached code note how HttpClient.connection_lost callback is never called for the madis server. The madis server is an apache server, I tried with the OSX apache server and could not reproduce the issue so it seems something particular about their apache version or configuration. This is a pretty critical issue as close() does not release the socket. ---------- components: asyncio files: scratch_1.py messages: 286573 nosy: gvanrossum, thehesiod, yselivanov priority: normal severity: normal status: open title: asyncio SSL contexts leak sockets after calling close with certain Apache servers type: resource usage versions: Python 3.5 Added file: http://bugs.python.org/file46469/scratch_1.py _ Python tracker _ From report at bugs.python.org Tue Jan 31 20:09:07 2017 From: report at bugs.python.org (Berker Peksag) Date: Wed, 01 Feb 2017 01:09:07 +0000 Subject: [New-bugs-announce] [issue29407] Remove redundant ensure_future() calls in factorial example Message-ID: <1485911347.12.0.0497985806483.issue29407@psf.upfronthosting.co.za> New submission from Berker Peksag: Unless I'm missing something, ensure_future() calls at https://docs.python.org/3.6/library/asyncio-task.html#example-parallel-execution-of-tasks are redundant and can be removed. Off-topic: Can we use the async/await keywords in asyncio docs now? ---------- assignee: docs at python components: Documentation, asyncio files: ensure_future.diff keywords: patch messages: 286576 nosy: berker.peksag, docs at python, gvanrossum, yselivanov priority: normal severity: normal stage: patch review status: open title: Remove redundant ensure_future() calls in factorial example type: enhancement versions: Python 3.5, Python 3.6, Python 3.7 Added file: http://bugs.python.org/file46471/ensure_future.diff _ Python tracker _ From report at bugs.python.org Tue Jan 31 20:20:08 2017 From: report at bugs.python.org (Daniel Nunes) Date: Wed, 01 Feb 2017 01:20:08 +0000 Subject: [New-bugs-announce] [issue29408] Add timestamp-based dependency system to build_clib Message-ID: <1485912008.84.0.925111372259.issue29408@psf.upfronthosting.co.za> New submission from Daniel Nunes: Summary: - Added newer_pairwise_group function to distutils.dep_util - Added documentation and tests for previous function - Added timestamp-based dependency system to build_libraries in build_clib - Added tests for the above build_clib is sorely missing one thing in distutils - checking outdated dependencies when compiling. This makes it so that, when building a static c library, all objects are always all compiled. The function to create the static library itself does implement something of this sort, but since all objects have been recompiled it has no practical effect (everything still runs every time). This is a small patch designed to fix that, with a couple of qol improvements along the way. Two new keys were added to the build_info dictionary: obj_deps - Allows specifying extra dependencies for each source so they can be recognized as being newer than the object being compiled. cflags - This just allows the user to provide an extra set of flags to the compiler. That is pretty much it, a simple (but important) quality of life improvement for those of us who need to compile static libs with python. Example: setup( ... libraries=[( 'examplelib', {'sources': 'example.c', 'obj_deps': {'': 'global.h', # use an empty string to define global (for all sources) dependencies 'example.c': 'example.h'} } )] ... ) Documentation for the new function is done but there is no documentation for build_clib itself. After submitting this I'll start working on it and will wait to submit it until this patch is accepted/rejected (so I can know what to include). Thanks! ---------- components: Distutils files: build_clib.patch keywords: patch messages: 286577 nosy: daniel.nunes, dstufft, eric.araujo priority: normal severity: normal status: open title: Add timestamp-based dependency system to build_clib type: enhancement versions: Python 3.7 Added file: http://bugs.python.org/file46472/build_clib.patch _ Python tracker _ From report at bugs.python.org Tue Jan 31 22:06:20 2017 From: report at bugs.python.org (Eryk Sun) Date: Wed, 01 Feb 2017 03:06:20 +0000 Subject: [New-bugs-announce] [issue29409] Implement PEP 529 for io.FileIO Message-ID: <1485918380.71.0.564330613462.issue29409@psf.upfronthosting.co.za> New submission from Eryk Sun: PEP 529 isn't implemented for io.FileIO, and I think it should be. If a UTF-8 path is passed to open(), it ends up calling C _open instead of decoding the path and calling C _wopen. Also, if a pathlike object is passed to io.FileIO, it calls PyUnicode_FSConverter on it, converting it to UTF-8 and then passing it to _open. For example: >>> p = r'C:\Temp\????' >>> os.path.exists(p) True >>> open(p.encode()) Traceback (most recent call last): File "", line 1, in FileNotFoundError: [Errno 2] No such file or directory: b'C:\\Temp\\\xce\xb1\xce\xb2\xcf\x88\xce\xb4' >>> io.FileIO(pathlib.Path(p)) Traceback (most recent call last): File "", line 1, in FileNotFoundError: [Errno 2] No such file or directory: WindowsPath('C:/Temp/????') The Windows implementation should mirror the POSIX implementation via PyUnicode_FSDecoder. ---------- components: IO, Windows messages: 286585 nosy: eryksun, paul.moore, steve.dower, tim.golden, zach.ware priority: normal severity: normal stage: needs patch status: open title: Implement PEP 529 for io.FileIO type: behavior versions: Python 3.6, Python 3.7 _ Python tracker _