From report at bugs.python.org Fri Sep 1 00:01:34 2017 From: report at bugs.python.org (elmar) Date: Fri, 01 Sep 2017 04:01:34 +0000 Subject: [New-bugs-announce] [issue31318] Windows importlib.util.find_spec("re") results in AttributeError: module 'importlib' has no attribute 'util' Message-ID: <1504238494.92.0.410752076324.issue31318@psf.upfronthosting.co.za> New submission from elmar: Hi There, I found the following bug, but have now idea where to search to fix this: On Windows calling a python script from the command line with the line importlib.util.find_spec("re") in it or using the standard python shell and executing importlib.util.find_spec("re") causes an AttributeError: module 'importlib' has no attribute 'util'. Surprisingly using ipython and calling the same script with %run or executing importlib.util.find_spec("re") in the ipython shell works fine. C:\Users\buchere>python Python 3.6.2 (v3.6.2:5fd33b5, Jul 8 2017, 04:14:34) [MSC v.1900 32 bit (Intel)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> import importlib >>> importlib.util.find_spec("re") Traceback (most recent call last): File "", line 1, in AttributeError: module 'importlib' has no attribute 'util' >>> exit() ipython: C:\Users\buchere>ipython Python 3.6.2 (v3.6.2:5fd33b5, Jul 8 2017, 04:14:34) [MSC v.1900 32 bit (Intel)] Type 'copyright', 'credits' or 'license' for more information IPython 6.1.0 -- An enhanced Interactive Python. Type '?' for help. In [1]: import importlib In [2]: importlib.util.find_spec("re") Out[2]: ModuleSpec(name='re', loader=<_frozen_importlib_external.SourceFileLoade r object at 0x01EB6750>, origin='c:\\users\\buchere\\appdata\\local\\programs\\p ython\\python36-32\\lib\\re.py') In [3]: The same code works fine in the Darwin and Linux standard python shell. This error is really specific to the Windows standard python shell. ---------- messages: 301083 nosy: bue priority: normal severity: normal status: open title: Windows importlib.util.find_spec("re") results in AttributeError: module 'importlib' has no attribute 'util' _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Sep 1 02:56:13 2017 From: report at bugs.python.org (Raymond Hettinger) Date: Fri, 01 Sep 2017 06:56:13 +0000 Subject: [New-bugs-announce] [issue31319] Add __main__.py to IDLE Message-ID: <1504248973.71.0.350017401117.issue31319@psf.upfronthosting.co.za> New submission from Raymond Hettinger: Currently, I start IDLE with: $ python -m idlelib.idle It would be nice add __main__.py to simplify this to: $ python -m idlelib And perhaps, more boldly rename the directory so that we get: $ python -m idle ---------- assignee: terry.reedy components: IDLE messages: 301086 nosy: rhettinger, terry.reedy priority: normal severity: normal status: open title: Add __main__.py to IDLE type: enhancement versions: Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Sep 1 10:39:37 2017 From: report at bugs.python.org (STINNER Victor) Date: Fri, 01 Sep 2017 14:39:37 +0000 Subject: [New-bugs-announce] [issue31320] test_ssl logs a traceback Message-ID: <1504276777.85.0.978024096958.issue31320@psf.upfronthosting.co.za> New submission from STINNER Victor: It would be nice to not log such traceback: haypo at selma$ ./python -m test test_ssl -m test.test_ssl.ThreadedTests.test_echo Run tests sequentially 0:00:00 load avg: 0.69 [1/1] test_ssl Exception in thread Thread-16: Traceback (most recent call last): File "/home/haypo/prog/python/master/Lib/threading.py", line 917, in _bootstrap_inner self.run() File "/home/haypo/prog/python/master/Lib/test/test_ssl.py", line 1936, in run if not self.wrap_conn(): File "/home/haypo/prog/python/master/Lib/test/test_ssl.py", line 1881, in wrap_conn self.sock, server_side=True) File "/home/haypo/prog/python/master/Lib/ssl.py", line 401, in wrap_socket _context=self, _session=session) File "/home/haypo/prog/python/master/Lib/ssl.py", line 808, in __init__ self.do_handshake() File "/home/haypo/prog/python/master/Lib/ssl.py", line 1061, in do_handshake self._sslobj.do_handshake() File "/home/haypo/prog/python/master/Lib/ssl.py", line 683, in do_handshake self._sslobj.do_handshake() OSError: [Errno 0] Error 1 test OK. Total duration: 205 ms Tests result: SUCCESS ---------- components: Tests messages: 301107 nosy: alex, christian.heimes, dstufft, haypo, janssen priority: normal severity: normal status: open title: test_ssl logs a traceback versions: Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Sep 1 10:50:17 2017 From: report at bugs.python.org (STINNER Victor) Date: Fri, 01 Sep 2017 14:50:17 +0000 Subject: [New-bugs-announce] [issue31321] traceback.clear_frames() doesn't clear *all* frames Message-ID: <1504277417.4.0.438645016632.issue31321@psf.upfronthosting.co.za> New submission from STINNER Victor: traceback.clear_frames(): "Clears the local variables of all the stack frames in a traceback tb by calling the clear() method of each frame object." https://docs.python.org/dev/library/traceback.html#traceback.clear_frames That's wrong: it only calls frame.clear() on the top frame of each traceback object. Output of attached clear_frames_bug.py script: --- locals: frame locals {'z': 3} frame locals {'y': 2} frame locals {'x': 1} clear_frames() locals: frame locals {} frame locals {'y': 2} frame locals {'x': 1} --- As you can see, only the locals of the func3() frame, where the exception was raised, is clear. Locals of other frames are not cleared. Attached PR fixes the issue. Output with the fix: --- locals: frame locals {'z': 3} frame locals {'y': 2} frame locals {'x': 1} clear_frames() locals: frame locals {} frame locals {} frame locals {} --- ---------- messages: 301111 nosy: haypo priority: normal severity: normal status: open title: traceback.clear_frames() doesn't clear *all* frames versions: Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Sep 1 10:51:16 2017 From: report at bugs.python.org (Pritish Patil) Date: Fri, 01 Sep 2017 14:51:16 +0000 Subject: [New-bugs-announce] [issue31322] SimpleNamespace deep copy Message-ID: <1504277476.43.0.203831960923.issue31322@psf.upfronthosting.co.za> New submission from Pritish Patil: I am new here and not sure how I can suggest this properly. When using nested SimpleNamespaces, a making a copy by using new_NS=SimpleNamespace(**namespace.__dict__.copy()) only copies the highest level namespace. This is expected in python as shallow copies are preferred. But, a nested deep copy function would be nice, and will allow easier use. I suggest a simple def my_namespace_copy(namespace): '''Recursively deep copies nested namespaces''' new_NS=SimpleNamespace(**namespace.__dict__.copy()) for i in new_NS.__dict__.keys(): if(type(new_NS.__dict__[i]) == types.SimpleNamespace): new_NS.__setattr__(i, my_namespace_copy(new_NS.__getattribute__(i))) return new_NS I am not sure of the exact implementation of the class and guess this would need some appropriate modifications. I suggest this be added at SimpleNameSpace.__copy__ or at SimpleNameSpace.__deepcopy__ ---------- assignee: docs at python components: Documentation, Library (Lib) messages: 301112 nosy: Pritish Patil, docs at python priority: normal severity: normal status: open title: SimpleNamespace deep copy type: enhancement versions: Python 3.5, Python 3.6, Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Sep 1 10:58:24 2017 From: report at bugs.python.org (STINNER Victor) Date: Fri, 01 Sep 2017 14:58:24 +0000 Subject: [New-bugs-announce] [issue31323] test_ssl: reference cycle between ThreadedEchoServer and its ConnectionHandler Message-ID: <1504277904.95.0.0934980312796.issue31323@psf.upfronthosting.co.za> New submission from STINNER Victor: While testing bpo-31234 fix, I noticed that some test_ssl tests using ThreadedEchoServer create reference cycles with ConnectionHandler: ConnectionHandler fills ThreadedEchoServer.conn_errors with exceptions, but exceptions keep an alive traceback object which contains frame locals. I tried to use traceback.clear_frames(), but this function is broken: see bpo-31321. Attached PR works around the issue by storing errors are text rather than objects. The PR prevents warnings about dangling threads. ---------- components: Tests messages: 301116 nosy: haypo priority: normal severity: normal status: open title: test_ssl: reference cycle between ThreadedEchoServer and its ConnectionHandler type: resource usage versions: Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Sep 1 11:11:02 2017 From: report at bugs.python.org (STINNER Victor) Date: Fri, 01 Sep 2017 15:11:02 +0000 Subject: [New-bugs-announce] [issue31324] support._match_test() used by test.bisect is very inefficient Message-ID: <1504278662.09.0.172151395724.issue31324@psf.upfronthosting.co.za> New submission from STINNER Victor: support._match_test() uses a nested loop calling fnmatch.fnmatchcase(). This function creates a temporary regular expression object. The cache of the re module works around the performance... if the length of support.match_tests fits into the cache. But when the list is longer, the function becomes dead slow... def _match_test(test): global match_tests if match_tests is None: return True test_id = test.id() for match_test in match_tests: if fnmatch.fnmatchcase(test_id, match_test): return True for name in test_id.split("."): if fnmatch.fnmatchcase(name, match_test): return True return False Maybe we should build a giant regex matching test_id at each, but cache the regex since support.match_tests can be modified anytime. I implemented this once, but I lost the code :-) Currently, it's possible to match 3 things: * test method name: test_exit * test class name: SysModuleTest * full test id: test.test_sys.SysModuleTest.test_exit It's also possible to use "*" joker character in a test name. I would like to keep these convenient CLI. ---------- messages: 301122 nosy: haypo, serhiy.storchaka priority: normal severity: normal status: open title: support._match_test() used by test.bisect is very inefficient versions: Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Sep 1 11:15:22 2017 From: report at bugs.python.org (Robin) Date: Fri, 01 Sep 2017 15:15:22 +0000 Subject: [New-bugs-announce] [issue31325] req_rate is a namedtuple type rather than instance Message-ID: <1504278922.84.0.0240979311873.issue31325@psf.upfronthosting.co.za> New submission from Robin: > Finally, urllib/robotparser.py appears to contain a bug in the > following: > > req_rate = collections.namedtuple('req_rate', > 'requests seconds') > entry.req_rate = req_rate > entry.req_rate.requests = int(numbers[0]) > entry.req_rate.seconds = int(numbers[1]) > > As I read it, this should fail as the req_rate is immutable. Ref: https://news.ycombinator.com/item?id=15136961 They are mistaken in the nature of the bug, which is that req_rate is a namedtuple type, rather than an instance. As such, it is actually mutable, causing the assignments to not fail. Every entry creates a separate req_rate type, so it all works, but not in the way it should. ---------- components: Library (Lib) messages: 301127 nosy: gvx priority: normal pull_requests: 3307 severity: normal status: open title: req_rate is a namedtuple type rather than instance type: behavior versions: Python 2.7, Python 3.3, Python 3.4, Python 3.5, Python 3.6, Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Sep 1 12:45:47 2017 From: report at bugs.python.org (STINNER Victor) Date: Fri, 01 Sep 2017 16:45:47 +0000 Subject: [New-bugs-announce] [issue31326] concurrent.futures: ProcessPoolExecutor.shutdown(wait=True) should wait for the call queue thread Message-ID: <1504284347.13.0.276187725111.issue31326@psf.upfronthosting.co.za> New submission from STINNER Victor: concurrent.futures.ProcessPoolExecutor.shutdown(wait=True) doesn't wait for the call queue thread, and so test_concurrent_futures randomly emits warnings about dangling threads, especially with PR 3138 of bpo-31234. Attached PR fixes the issue. ---------- messages: 301135 nosy: haypo priority: normal severity: normal status: open title: concurrent.futures: ProcessPoolExecutor.shutdown(wait=True) should wait for the call queue thread type: resource usage versions: Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Sep 1 13:24:09 2017 From: report at bugs.python.org (Jerry Kramskoy) Date: Fri, 01 Sep 2017 17:24:09 +0000 Subject: [New-bugs-announce] [issue31327] bug in dateutil\tz\tz.py Message-ID: <1504286649.82.0.482783873664.issue31327@psf.upfronthosting.co.za> New submission from Jerry Kramskoy: Running python 3.6.2 on Windows 10. The following method causes presents a timetstamp value of -3600 (i.e. DST adjustment of one hour) which causes time.localtime() to raise an OS Errno 22. def _naive_is_dst(self, dt): timestamp = _datetime_to_timestamp(dt) return time.localtime(timestamp + time.timezone).tm_isdst Reproduce with python shell... import time z=time.localtime(-3600) Traceback (most recent call last): Python Shell, prompt 7, line 1 builtins.OSError: [Errno 22] Invalid argument The documentation doesn't specify a legal value range for input to localtime. Unfortunately,this causes AWS boto3 support for dynamodb to crash. Cheers, Jerry ---------- components: Library (Lib) messages: 301143 nosy: jerrykramskoy priority: normal severity: normal status: open title: bug in dateutil\tz\tz.py type: crash versions: Python 3.6 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Sep 1 14:09:59 2017 From: report at bugs.python.org (Segev Finer) Date: Fri, 01 Sep 2017 18:09:59 +0000 Subject: [New-bugs-announce] [issue31328] _sha3 is missing from Setup.dist Message-ID: <1504289399.64.0.916435234758.issue31328@psf.upfronthosting.co.za> New submission from Segev Finer: _sha3 is missing a commented out line from Setup.dist like the rest of the modules. It's really only used if someone wants to edit Setup.dist to build with the _sha3 module statically. Mostly a trivial change to add it in. ---------- components: Build, Extension Modules messages: 301144 nosy: Segev Finer priority: normal severity: normal status: open title: _sha3 is missing from Setup.dist type: behavior versions: Python 3.6, Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Sep 1 15:24:07 2017 From: report at bugs.python.org (Terry J. Reedy) Date: Fri, 01 Sep 2017 19:24:07 +0000 Subject: [New-bugs-announce] [issue31329] Add idlelib module entry to doc Message-ID: <1504293847.95.0.524257228924.issue31329@psf.upfronthosting.co.za> New submission from Terry J. Reedy: If this were a new section in idle.rst, it would appear with the IDLE doc displayed by Help IDLE. Being present there has its +s and -s. Would have to see how it works here as well as online. In any case, it would be good to have something indexed under I in Modules index. Content: Document python -m idlelib as command line start up. List other entry points. Otherwise describe as 'private implementation modules', as in idlelib.__init__. ---------- messages: 301145 nosy: terry.reedy priority: normal severity: normal stage: needs patch status: open title: Add idlelib module entry to doc type: enhancement versions: Python 3.6, Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Sep 1 23:19:37 2017 From: report at bugs.python.org (Antony Lee) Date: Sat, 02 Sep 2017 03:19:37 +0000 Subject: [New-bugs-announce] [issue31330] argparse.RawTextHelpFormatter does not maintain lines separated by more than one newline Message-ID: <1504322377.77.0.797544251654.issue31330@psf.upfronthosting.co.za> New submission from Antony Lee: The doc for argparse.RawTextHelpFormatter states that it "maintains whitespace for all sorts of help text, including argument descriptions." But the following example (which outputs "Foo", "Bar", and "Baz", each separated by exactly one empty line) shows that paragraphs separated by more than one empty line will only be separated in the output text by a single empty line. import argparse parser = argparse.ArgumentParser( prog='PROG', formatter_class=argparse.RawTextHelpFormatter, description='''\ Foo Bar Baz''') parser.print_help() ---------- components: Library (Lib) messages: 301158 nosy: Antony.Lee priority: normal severity: normal status: open title: argparse.RawTextHelpFormatter does not maintain lines separated by more than one newline versions: Python 3.6 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sat Sep 2 19:52:22 2017 From: report at bugs.python.org (Terry J. Reedy) Date: Sat, 02 Sep 2017 23:52:22 +0000 Subject: [New-bugs-announce] [issue31331] IDLE: Move prompts with input. Message-ID: <1504396342.39.0.90441409742.issue31331@psf.upfronthosting.co.za> New submission from Terry J. Reedy: When this program (from thread by Steven D'Aprano) on python-list is run import time from threading import Timer def do_work(): x = 2 + 2 print("It is", time.asctime(), "and 2+2 is", x) def go(): Timer(10, do_work, ()).start() # schedule it in one minute The response, if it occurs while one in entering a statement, 'pushes down' the entry in progress. ======================= RESTART: F:\Python\mypy\tem.py ======================= >>> go() >>> It is Sat Sep 2 19:42:10 2017 and 2+2 is 4 #<== output a = ( 12, # <== 2nd line of entry in progress The prompt should be pushed down too. ---------- assignee: terry.reedy components: IDLE messages: 301177 nosy: terry.reedy priority: normal severity: normal stage: test needed status: open title: IDLE: Move prompts with input. type: behavior versions: Python 3.6, Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sat Sep 2 23:40:17 2017 From: report at bugs.python.org (Tatsunori Uchino) Date: Sun, 03 Sep 2017 03:40:17 +0000 Subject: [New-bugs-announce] [issue31332] Building modules by Clang with Microsoft CodeGen Message-ID: <1504410017.01.0.560044317715.issue31332@psf.upfronthosting.co.za> New submission from Tatsunori Uchino: Microsoft released a new compiler for VC++ that complies with more of modern C++ than the standard compiler. https://blogs.msdn.microsoft.com/vcblog/2016/03/31/clang-with-microsoft-codegen-march-2016-released/ Is it possible to support it by creating a new script based on Lib/distutils/msvccompiler.py to Lib/distutils and adding a new item in the following dictionary in Lib/distutils/ccompiler.py? compiler_class = { 'unix': ('unixccompiler', 'UnixCCompiler', "standard UNIX-style compiler"), 'msvc': ('_msvccompiler', 'MSVCCompiler', "Microsoft Visual C++"), 'cygwin': ('cygwinccompiler', 'CygwinCCompiler', "Cygwin port of GNU C Compiler for Win32"), 'mingw32': ('cygwinccompiler', 'Mingw32CCompiler', "Mingw32 port of GNU C Compiler for Win32"), 'bcpp': ('bcppcompiler', 'BCPPCompiler', "Borland C++ Compiler"), } The executable of it is at C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\ClangC2\bin\amd64. The help message of it is: OVERVIEW: clang LLVM compiler USAGE: clang.exe [options] OPTIONS: -### Print (but do not run) the commands to run for this compilation --analyze Run the static analyzer -arcmt-migrate-emit-errors Emit ARC errors even if the migrator can fix them -arcmt-migrate-report-output Output path for the plist report -B2 Location of backend dll -bigobj Generate extended COFF format --cuda-device-only Do device-side CUDA compilation only --cuda-host-only Do host-side CUDA compilation only --cuda-path= CUDA installation path -cxx-isystem Add directory to the C++ SYSTEM include search path -c Only run preprocess, compile, and assemble steps -d2 Additional arguments to forward to the backend -dD Print macro definitions in -E mode in addition to normal output -dependency-dot Filename to write DOT-formatted header dependencies to -dependency-file Filename (or -) to write dependency output to -dM Print macro definitions in -E mode instead of normal output -emit-ast Emit Clang AST files for source inputs -emit-llvm Use the LLVM representation for assembler and object files -E Only run the preprocessor -faltivec Enable AltiVec vector initializer syntax -fansi-escape-codes Use ANSI escape codes for diagnostics -fapple-kext Use Apple's kernel extensions ABI -fapple-pragma-pack Enable Apple gcc-compatible #pragma pack handling -fapplication-extension Restrict code to those available for App Extensions -fasync-exceptions Catch SEH exceptions in catch-alls -fblocks Enable the 'blocks' language feature -fborland-extensions Accept non-standard constructs supported by the Borland compiler -fbuild-session-file= Use the last modification time of as the build session timestamp -fbuild-session-timestamp=