From report at bugs.python.org Sun Oct 1 02:36:42 2017 From: report at bugs.python.org (Serhiy Storchaka) Date: Sun, 01 Oct 2017 06:36:42 +0000 Subject: [New-bugs-announce] [issue31655] SimpleNamespace accepts non-string keyword names Message-ID: <1506839802.01.0.213398074469.issue31655@psf.upfronthosting.co.za> New submission from Serhiy Storchaka : Usually non-string keyword names are rejected. >>> def f(**kwargs): pass ... >>> f(**{0:0}) Traceback (most recent call last): File "", line 1, in TypeError: f() keywords must be strings >>> dict(**{0:0}) Traceback (most recent call last): File "", line 1, in TypeError: keywords must be strings There are checks in multiple places that satisfy this: in _PyEval_EvalCodeWithName(), PyArg_ValidateKeywordArguments(), PyArg_ParseTupleAndKeywords(), etc. But SimpleNamespace is an exception. >>> from types import SimpleNamespace >>> SimpleNamespace(**{0:0}) namespace() Non-string keys are omitted in the repr. Wouldn't be better to add also the check that keyword names for SimpleNamespace constructor are strings? I don't know how classify this issue, as a bug or an enhancement. Based on the StackOverflow question: https://stackoverflow.com/questions/46164770/accepting-integers-as-keys-of-kwargs. ---------- components: Interpreter Core messages: 303450 nosy: haypo, serhiy.storchaka priority: normal severity: normal status: open title: SimpleNamespace accepts non-string keyword names _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun Oct 1 11:01:16 2017 From: report at bugs.python.org (Matthias Gilch) Date: Sun, 01 Oct 2017 15:01:16 +0000 Subject: [New-bugs-announce] [issue31656] Bitwise operations for bytes-type Message-ID: <1506870076.79.0.213398074469.issue31656@psf.upfronthosting.co.za> New submission from Matthias Gilch : I've seen that the bytes type does not support bitwise operations, but I think users would expect that those operations will work. ---------- components: Interpreter Core messages: 303464 nosy: MGilch priority: normal severity: normal status: open title: Bitwise operations for bytes-type type: enhancement _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun Oct 1 11:08:33 2017 From: report at bugs.python.org (diana) Date: Sun, 01 Oct 2017 15:08:33 +0000 Subject: [New-bugs-announce] [issue31657] unit test for optimization levels does not cover __debug__ case Message-ID: <1506870513.15.0.213398074469.issue31657@psf.upfronthosting.co.za> New submission from diana : There are currently three supported optimization levels (0, 1, and 2). Briefly summarized, they do the following. 0: no optimizations 1: remove assert statements and __debug__ blocks 2: remove docstrings, assert statements, and __debug__ blocks The current compile() tests for optimization levels in Lib/test/test_builtin.py covers the assert and docstring cases, but it doesn't test that __debug__ code blocks are included or excluded based on the optimization level. For example, if you change Python/compile.c to always include __debug__ blocks regardless of the optimization level, the existing compile() tests will continue to pass. $ git diff Python/compile.c diff --git a/Python/compile.c b/Python/compile.c index 280ddc39e3..d65df098bb 100644 --- a/Python/compile.c +++ b/Python/compile.c @@ -4143,7 +4143,7 @@ expr_constant(struct compiler *c, expr_ty e) /* optimize away names that can't be reassigned */ id = PyUnicode_AsUTF8(e->v.Name.id); if (id && strcmp(id, "__debug__") == 0) - return !c->c_optimize; + return 1; return -1; case NameConstant_kind: { PyObject *o = e->v.NameConstant.value; ---------- messages: 303465 nosy: diana priority: normal severity: normal status: open title: unit test for optimization levels does not cover __debug__ case type: enhancement _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun Oct 1 16:51:11 2017 From: report at bugs.python.org (Craig Holmquist) Date: Sun, 01 Oct 2017 20:51:11 +0000 Subject: [New-bugs-announce] [issue31658] xml.sax.parse won't accept path objects Message-ID: <1506891071.57.0.213398074469.issue31658@psf.upfronthosting.co.za> New submission from Craig Holmquist : >>> import xml.sax >>> import pathlib [...] >>> xml.sax.parse(pathlib.Path('path/to/file'), handler) Traceback (most recent call last): File "", line 1, in File "/usr/lib/python3.6/xml/sax/__init__.py", line 33, in parse parser.parse(source) File "/usr/lib/python3.6/xml/sax/expatreader.py", line 105, in parse source = saxutils.prepare_input_source(source) File "/usr/lib/python3.6/xml/sax/saxutils.py", line 355, in prepare_input_source if source.getCharacterStream() is None and source.getByteStream() is None: AttributeError: 'PosixPath' object has no attribute 'getCharacterStream' ---------- components: Library (Lib), XML messages: 303490 nosy: craigh priority: normal severity: normal status: open title: xml.sax.parse won't accept path objects type: behavior versions: Python 3.6 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun Oct 1 20:47:34 2017 From: report at bugs.python.org (INADA Naoki) Date: Mon, 02 Oct 2017 00:47:34 +0000 Subject: [New-bugs-announce] [issue31659] ssl module should not use textwrap for wrapping PEM format. Message-ID: <1506905254.99.0.213398074469.issue31659@psf.upfronthosting.co.za> New submission from INADA Naoki : Current DER_cert_to_PEM_cert() uses textwrap like this: def DER_cert_to_PEM_cert(der_cert_bytes): """Takes a certificate in binary DER format and returns the PEM version of it as a string.""" f = str(base64.standard_b64encode(der_cert_bytes), 'ASCII', 'strict') return (PEM_HEADER + '\n' + textwrap.fill(f, 64) + '\n' + PEM_FOOTER + '\n') But textwrap is designed to break lines at word boundary. So textwrap.fill for base64 text may be slow. And `import textwrap` is little slow too. We can use simple slicing instead of textwrap. ---------- messages: 303505 nosy: inada.naoki priority: normal severity: normal status: open title: ssl module should not use textwrap for wrapping PEM format. _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun Oct 1 23:12:27 2017 From: report at bugs.python.org (Stephen Moore) Date: Mon, 02 Oct 2017 03:12:27 +0000 Subject: [New-bugs-announce] [issue31660] sys.executable different in os.execv'd python3.6 virtualenv session in python2 vs python3 Message-ID: <1506913947.94.0.213398074469.issue31660@psf.upfronthosting.co.za> New submission from Stephen Moore : Hi, I've come across a problem whereby if you do an os.execv to a python3.6 virtualenv python inside python2.7 vs python3.6 then the resulting python session has a different sys.executable. Where if you os.execv from python2.7 the sys.executable is equal to the virtualenv python Whereas from python3.6 the resulting python session has a sys.executable of the system python. An example of this in play can be found at https://gist.github.com/delfick/d750dc83e3b28e90cef8e2bfbd5b175a Note that I don't see the problem if the virtualenv is based on python2.7 (https://gist.github.com/delfick/f1ad6872e2614189a7d98f2583ffc564) ---------- components: macOS messages: 303507 nosy: Stephen Moore, ned.deily, ronaldoussoren priority: normal severity: normal status: open title: sys.executable different in os.execv'd python3.6 virtualenv session in python2 vs python3 versions: Python 2.7, Python 3.6 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Oct 2 02:14:07 2017 From: report at bugs.python.org (Serhiy Storchaka) Date: Mon, 02 Oct 2017 06:14:07 +0000 Subject: [New-bugs-announce] [issue31661] Issues with request rate in robotparser Message-ID: <1506924847.15.0.213398074469.issue31661@psf.upfronthosting.co.za> New submission from Serhiy Storchaka : There are issues in implementing support of request rate in robotparser. req_rate = collections.namedtuple('req_rate', 'requests seconds') entry.req_rate = req_rate entry.req_rate.requests = int(numbers[0]) entry.req_rate.seconds = int(numbers[1]) First, a new namedtuple type is created for every entry. This is slow even with recent namedtuple optimizations, and is much slower in 3.6. This wastes a memory, since new type is created for every entry. This is definitely wrong, since req_rate is set to a namedtuple type instead of namedtuple instance. And there is a question why a namedtuple is used here at all. Other classes in this module are not namedtuples. ---------- components: Library (Lib) messages: 303508 nosy: XapaJIaMnu, berker.peksag, serhiy.storchaka priority: normal severity: normal status: open title: Issues with request rate in robotparser type: behavior versions: Python 3.6, Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Oct 2 04:12:16 2017 From: report at bugs.python.org (Anselm Kruis) Date: Mon, 02 Oct 2017 08:12:16 +0000 Subject: [New-bugs-announce] [issue31662] trivial typos in Tools/msi/uploadrelease.bat Message-ID: <1506931936.93.0.213398074469.issue31662@psf.upfronthosting.co.za> New submission from Anselm Kruis : There are 3 trivial typos in Tools/msi/uploadrelease.bat: "godo" instead of "goto" in lines 25 to 28. ---------- components: Build messages: 303513 nosy: anselm.kruis, steve.dower priority: normal severity: normal status: open title: trivial typos in Tools/msi/uploadrelease.bat type: behavior versions: Python 3.5, Python 3.6, Python 3.7, Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Oct 2 05:24:09 2017 From: report at bugs.python.org (Basanna Badami) Date: Mon, 02 Oct 2017 09:24:09 +0000 Subject: [New-bugs-announce] [issue31663] pyautogui.typewrite() method doesn't work as expected. Message-ID: <1506936249.54.0.213398074469.issue31663@psf.upfronthosting.co.za> New submission from Basanna Badami : Tried to run pyautogui.typewrite('Hello World') On IDLE terminal to type 'Hello World' string to an open notepad(say). As, i've interchanged my mouse click options(on windows OS), instead of 'Hello World' string being print to to the open notepad, i could see a right click on it. The method pyautogui.typewrite() doesn't function as per mouse options settings on my machine. ---------- components: Windows messages: 303515 nosy: basanna007, paul.moore, steve.dower, tim.golden, zach.ware priority: normal severity: normal status: open title: pyautogui.typewrite() method doesn't work as expected. type: behavior versions: Python 3.6 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Oct 2 05:24:48 2017 From: report at bugs.python.org (Serhiy Storchaka) Date: Mon, 02 Oct 2017 09:24:48 +0000 Subject: [New-bugs-announce] [issue31664] Add support of new crypt methods Message-ID: <1506936288.65.0.213398074469.issue31664@psf.upfronthosting.co.za> New submission from Serhiy Storchaka : Proposed PR adds support of three new methods in the crypt module. 1. Blowfish. It is considered as strong as SSH512 for crypt() purpose. There are several variants of this method: '2', '2a', '2b' and '2y'. '2y' looks the same as '2b', other variants have different flaws. All four are supported on FreeBSD. '2b' is the only method available on OpenBSD, hence this change also fixes crypt on OpenBSD (see issue25287). Blowfish is not supported in glibc, but it is added in some Linux distributions (not in Ubuntu). The most strong of the available variants is chosen. 2. Extended DES. In contrary to traditional default algorithm it uses salt longer than 2 characters. It is supported on FreeBSD. 3. NT-Hash. It doesn't use salt and is compatible with Microsoft's NT scheme. It is supported on FreeBSD. mksalt() now takes the log_rounds argument for Blowfish. I'm not sure this is the best solution. And what should be a default value? ---------- messages: 303516 nosy: serhiy.storchaka priority: normal severity: normal status: open title: Add support of new crypt methods _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Oct 2 15:26:06 2017 From: report at bugs.python.org (Terry J. Reedy) Date: Mon, 02 Oct 2017 19:26:06 +0000 Subject: [New-bugs-announce] [issue31665] Edit "Setting [windows] environmental variables" Message-ID: <1506972366.59.0.213398074469.issue31665@psf.upfronthosting.co.za> New submission from Terry J. Reedy : Python Setup and Usage, 3.3.1. Excursus: Setting environment variables https://docs.python.org/3/using/windows.html#excursus-setting-environment-variables The word 'excursus' is so rare and archaic that this is my first encounter with it. Could we drop it? The last paragraph starts with "To permanently modify the default environment variables, click Start and search for ?edit environment variables?,". This part is fine on Windows 10 as it brings up "Edit the system environment variables" and "Edit environmental variables for your account". Both take one to an Environmental variables dialog. The former requires than one be or become an admin user (via UAC dialog). The paragraph continues "or open System properties, Advanced system settings and click the Environment Variables button. In this dialog, you can add or modify User and System variables. To change System variables, you need non-restricted access to your machine (i.e. Administrator rights)." On Windows 10, at least, this is confusing or wrong. Control Panel has 'System' linked to the System dialog. This has 'Advanced systems settings' linked to System Properties, but one only get there by being or becoming (via UAC) an admin user. This has the Environmental Properties button mentioned above. It opens Environmental Variables for the admin user you are or are acting as. This route cannot change EVs for non-admin users. I have the impression that this has changed since Win 7. ---------- assignee: docs at python components: Documentation, Windows messages: 303560 nosy: docs at python, paul.moore, steve.dower, terry.reedy, tim.golden, zach.ware priority: normal severity: normal stage: needs patch status: open title: Edit "Setting [windows] environmental variables" type: behavior versions: Python 3.6, Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Oct 2 15:42:56 2017 From: report at bugs.python.org (Scott Tucholka) Date: Mon, 02 Oct 2017 19:42:56 +0000 Subject: [New-bugs-announce] [issue31666] Pandas_datareader Error Message - ModuleNotFoundError: No module named 'pandas_datareader' with module in folder Message-ID: <1506973376.41.0.213398074469.issue31666@psf.upfronthosting.co.za> Change by Scott Tucholka : ---------- components: Library (Lib), Windows nosy: Scott Tucholka, paul.moore, steve.dower, tim.golden, zach.ware priority: normal severity: normal status: open title: Pandas_datareader Error Message - ModuleNotFoundError: No module named 'pandas_datareader' with module in folder versions: Python 3.6 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Oct 2 16:04:55 2017 From: report at bugs.python.org (linkid) Date: Mon, 02 Oct 2017 20:04:55 +0000 Subject: [New-bugs-announce] [issue31667] Wrong links in the gettext.NullTranslations class Message-ID: <1506974695.65.0.213398074469.issue31667@psf.upfronthosting.co.za> New submission from linkid : In gettext.NullTranslations class doc [0], links to gettext and ngettext methods are not consistent. [0] https://docs.python.org/3/library/gettext.html#the-nulltranslations-class ---------- assignee: docs at python components: Documentation messages: 303563 nosy: docs at python, linkid priority: normal severity: normal status: open title: Wrong links in the gettext.NullTranslations class type: enhancement versions: Python 3.5, Python 3.6, Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Oct 3 00:35:20 2017 From: report at bugs.python.org (fireattack) Date: Tue, 03 Oct 2017 04:35:20 +0000 Subject: [New-bugs-announce] [issue31668] "fixFirefoxAnchorBug" function in doctools.js causes navigating problem in Py3 doc in Chrome Message-ID: <1507005320.05.0.213398074469.issue31668@psf.upfronthosting.co.za> New submission from fireattack : Problem This is a regression bug/flaw in Sphinx's doctools.js, a JS file used in its base template, and therefore got inherited to Python 3's documentation website. Python 2's documentation website is not affected because it's based on an older version of Sphinx. There is a function in doctools.js: /** * workaround a firefox stupidity * see: https://bugzilla.mozilla.org/show_bug.cgi?id=645075 */ fixFirefoxAnchorBug : function() { if (document.location.hash) window.setTimeout(function() { document.location.href += ''; }, 10); } This function was supposed to fix an anchor bug in Firefox (see comment). It *used to* have a condition of $.browser outside, so it will only be applied to Firefox; but it was removed in JQuery so it was removed here as well. Therefore, this function now applies to all the browsers. Unfortunately, it causes navigating problem in Chrome now, when you use back and forward. The problem STR (Chrome only): 1. Open a link with hash (anchor), e.g. https://docs.python.org/3/library/stdtypes.html#text-sequence-type-str 2. Scroll away from the anchor position. 3. Click any external link (means the link that is not an anchor of current page). 4. Hit "back" button in the browser. What happened: When you navigating back, it doesn't go to your previous position. Instead, it goes to the anchor's location. What it should do: It should go to your previous position. Ironically, it won't cause this problem in Firefox, despite it's supposed to be a fix for (a different) anchor bug in Firefox. Comments I reported it to Sphinx as well: https://github.com/sphinx-doc/sphinx/issues/3549 but didn't get any response so far. Please keep in mind the Firefox anchor bug mentioned above will only happen if the anchor is assigned with
. From my observation we don't really use
in HTML, so I don't think this workaround function is really necessary here to begin with. ---------- assignee: docs at python components: Documentation messages: 303576 nosy: docs at python, fireattack priority: normal severity: normal status: open title: "fixFirefoxAnchorBug" function in doctools.js causes navigating problem in Py3 doc in Chrome type: behavior versions: Python 3.4, Python 3.5, Python 3.6, Python 3.7, Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Oct 3 02:02:28 2017 From: report at bugs.python.org (Thomas Jollans) Date: Tue, 03 Oct 2017 06:02:28 +0000 Subject: [New-bugs-announce] [issue31669] string.Template: cods, docs and PEP all disagree on definition of identifier Message-ID: <1507010548.18.0.213398074469.issue31669@psf.upfronthosting.co.za> New submission from Thomas Jollans : string.Template matches a $identifier with the regex [_a-z][_a-z0-9]* and re.IGNORECASE. This matches S, ? and s, but not ?. https://github.com/python/cpython/blob/master/Lib/string.py#L78 (this code came up on python-dev) The docs specify "any case-insensitive ASCII alphanumeric string (including underscores) that starts with an underscore or ASCII letter.". This includes S and s, but neither ? nor ?. https://docs.python.org/3/library/string.html#template-strings The docs refer to PEP 292, which specifies "By default, 'identifier' must spell a Python identifier [...]" This includes S, ?, s and ?. https://www.python.org/dev/peps/pep-0292/ It's not entirely clear what the correct behaviour is (probably accepting any Python identifier). In any case, the current behaviour of string.Template is a bit silly, but changing it might break stuff. ---------- components: Library (Lib) messages: 303577 nosy: tjollans priority: normal severity: normal status: open title: string.Template: cods, docs and PEP all disagree on definition of identifier type: behavior versions: Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Oct 3 02:22:26 2017 From: report at bugs.python.org (Brad Nelson) Date: Tue, 03 Oct 2017 06:22:26 +0000 Subject: [New-bugs-announce] [issue31670] Associate .wasm with application/wasm Message-ID: <1507011746.68.0.213398074469.issue31670@psf.upfronthosting.co.za> New submission from Brad Nelson : WebAssembly is tentatively spec'ed to require the application/wasm mime type for download sources: https://github.com/WebAssembly/design/blob/master/Web.md The mimetype module should associate .wasm with this mimetype to allow users of SimpleHTTPServer etc. to be able to easily use WebAssembly. ---------- components: IO messages: 303578 nosy: flagxor priority: normal severity: normal status: open title: Associate .wasm with application/wasm type: enhancement versions: Python 2.7, Python 3.4, Python 3.5, Python 3.6, Python 3.7, Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Oct 3 07:00:30 2017 From: report at bugs.python.org (INADA Naoki) Date: Tue, 03 Oct 2017 11:00:30 +0000 Subject: [New-bugs-announce] [issue31671] IntFlag makes re.compile slower Message-ID: <1507028430.84.0.213398074469.issue31671@psf.upfronthosting.co.za> New submission from INADA Naoki : flags exposed by re module are IntFlag from Python 3.4. Since it is passed to underlying sre_parse and sre_compile, tests in loop (e.g. `flags & SRE_FLAG_IGNORECASE`) calls IntFlag.__and__ and creates new instance everytime. It makes re.compile() slower. ---------- components: Library (Lib) messages: 303594 nosy: inada.naoki priority: normal severity: normal status: open title: IntFlag makes re.compile slower versions: Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Oct 3 07:15:40 2017 From: report at bugs.python.org (INADA Naoki) Date: Tue, 03 Oct 2017 11:15:40 +0000 Subject: [New-bugs-announce] [issue31672] string.Template should use re.ASCII flag Message-ID: <1507029340.05.0.213398074469.issue31672@psf.upfronthosting.co.za> New submission from INADA Naoki : Currently, strings.Template uses re.IGNORECASE without re.ASCII: idpattern = r'[_a-z][_a-z0-9]*' flags = _re.IGNORECASE [a-z] matches against '?' (0x131, DOTLESS I) and '?' (0x17f, LONG S). It is not intentional, and it makes re.compile slower. ---------- components: Regular Expressions messages: 303596 nosy: ezio.melotti, inada.naoki, mrabarnett priority: normal severity: normal status: open title: string.Template should use re.ASCII flag versions: Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Oct 3 07:35:06 2017 From: report at bugs.python.org (Serhiy Storchaka) Date: Tue, 03 Oct 2017 11:35:06 +0000 Subject: [New-bugs-announce] [issue31673] Fix the name of Tkinter's adderrorinfo method Message-ID: <1507030506.61.0.213398074469.issue31673@psf.upfronthosting.co.za> New submission from Serhiy Storchaka : When the _tkinter module was converted to Argument Clinic in issue20168, the name of the adderrorinfo method was written as "adderrinfo". This error was left unnoticed because there are no tests for this method besides bigmem tests. Today I have ran bigmem tests first time (they require 64-bit system and large amount of memory) and have found this typo. Seems these tests are not ran on any buildbots. ---------- assignee: serhiy.storchaka components: Tkinter messages: 303597 nosy: serhiy.storchaka priority: normal severity: normal stage: patch review status: open title: Fix the name of Tkinter's adderrorinfo method type: behavior versions: Python 3.6, Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Oct 3 07:44:09 2017 From: report at bugs.python.org (STINNER Victor) Date: Tue, 03 Oct 2017 11:44:09 +0000 Subject: [New-bugs-announce] [issue31674] Buildbots: random "Failed to connect to github.com port 443: Connection timed out" errors Message-ID: <1507031049.66.0.213398074469.issue31674@psf.upfronthosting.co.za> New submission from STINNER Victor : Sometimes, the "git" step fails on buildbots. ARMv7 Ubuntu 3.x just failed on build 1481, atTue Oct 3 11:14:22 2017: http://buildbot.python.org/all/builders/ARMv7%20Ubuntu%203.x/builds/1481/steps/git/logs/stdio git fetch -t https://github.com/python/cpython.git master (...) fatal: unable to access 'https://github.com/python/cpython.git/': Failed to connect to github.com port 443: Connection timed out ---------- components: Tests keywords: buildbot messages: 303598 nosy: haypo priority: normal severity: normal status: open title: Buildbots: random "Failed to connect to github.com port 443: Connection timed out" errors versions: Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Oct 3 08:04:23 2017 From: report at bugs.python.org (Serhiy Storchaka) Date: Tue, 03 Oct 2017 12:04:23 +0000 Subject: [New-bugs-announce] [issue31675] Tkinter: memory leak in splitlines() and split() Message-ID: <1507032263.31.0.213398074469.issue31675@psf.upfronthosting.co.za> New submission from Serhiy Storchaka : There is a memory leak in Tkinter's methods splitlines() and split() when pass a string larger than 2 GiB (when encoded to UTF-8). ---------- components: Tkinter messages: 303599 nosy: serhiy.storchaka priority: normal severity: normal stage: patch review status: open title: Tkinter: memory leak in splitlines() and split() type: resource usage versions: Python 2.7, Python 3.6, Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Oct 3 08:46:07 2017 From: report at bugs.python.org (Serhiy Storchaka) Date: Tue, 03 Oct 2017 12:46:07 +0000 Subject: [New-bugs-announce] [issue31676] Strange failure in test_cgitb Message-ID: <1507034767.15.0.213398074469.issue31676@psf.upfronthosting.co.za> New submission from Serhiy Storchaka : I have found a strange failure when run the bigmem tests. 0:20:19 load avg: 1.04 [116/407/1] test_cgitb test test_cgitb failed -- Traceback (most recent call last): File "/home/serhiy/py/cpython/Lib/test/test_cgitb.py", line 23, in test_html raise ValueError("Hello World") ValueError: Hello World During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/serhiy/py/cpython/Lib/test/test_cgitb.py", line 27, in test_html html = cgitb.html(sys.exc_info()) File "/home/serhiy/py/cpython/Lib/cgitb.py", line 117, in html records = inspect.getinnerframes(etb, context) File "/home/serhiy/py/cpython/Lib/inspect.py", line 1483, in getinnerframes frameinfo = (tb.tb_frame,) + getframeinfo(tb, context) File "/home/serhiy/py/cpython/Lib/inspect.py", line 1445, in getframeinfo lines, lnum = findsource(frame) File "/home/serhiy/py/cpython/Lib/inspect.py", line 780, in findsource module = getmodule(object, file) File "/home/serhiy/py/cpython/Lib/inspect.py", line 739, in getmodule f = getabsfile(module) File "/home/serhiy/py/cpython/Lib/inspect.py", line 708, in getabsfile _filename = getsourcefile(object) or getfile(object) File "/home/serhiy/py/cpython/Lib/inspect.py", line 693, in getsourcefile if os.path.exists(filename): File "/home/serhiy/py/cpython/Lib/genericpath.py", line 19, in exists os.stat(path) ValueError: embedded null byte 0:20:19 load avg: 1.04 [117/407/2] test_csv -- test_cgitb failed It is not reproduced when run test_cgitb separately. ---------- messages: 303608 nosy: ezio.melotti, haypo, michael.foord, serhiy.storchaka priority: normal severity: normal status: open title: Strange failure in test_cgitb type: behavior _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Oct 3 08:58:02 2017 From: report at bugs.python.org (INADA Naoki) Date: Tue, 03 Oct 2017 12:58:02 +0000 Subject: [New-bugs-announce] [issue31677] email.header uses re.IGNORECASE without re.ASCII Message-ID: <1507035482.04.0.213398074469.issue31677@psf.upfronthosting.co.za> New submission from INADA Naoki : email.header has this pattern: https://github.com/python/cpython/blob/85c0b8941f0c8ef3ed787c9d504712c6ad3eb5d3/Lib/email/header.py#L34-L43 # Match encoded-word strings in the form =?charset?q?Hello_World?= ecre = re.compile(r''' =\? # literal =? (?P[^?]*?) # non-greedy up to the next ? is the charset \? # literal ? (?P[qb]) # either a "q" or a "b", case insensitive \? # literal ? (?P.*?) # non-greedy up to the next ?= is the encoded string \?= # literal ?= ''', re.VERBOSE | re.IGNORECASE | re.MULTILINE) Since only 's' and 'i' has other lower case character, this is not a real bug. But using re.ASCII is more safe. Additionally, email.util has same pattern from 10 years ago, and it is not used by anywhere. It should be removed. ---------- components: Regular Expressions messages: 303612 nosy: ezio.melotti, inada.naoki, mrabarnett priority: normal severity: normal status: open title: email.header uses re.IGNORECASE without re.ASCII versions: Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Oct 3 09:47:42 2017 From: report at bugs.python.org (=?utf-8?b?0JzQuNGF0LDQuNC7INCc0LjRgNC+0L3QvtCy?=) Date: Tue, 03 Oct 2017 13:47:42 +0000 Subject: [New-bugs-announce] [issue31678] Incorrect C Function name for timedelta Message-ID: <1507038462.28.0.213398074469.issue31678@psf.upfronthosting.co.za> New submission from ?????? ??????? : Documentation: PyDateTime_DELTA_GET_MICROSECOND https://docs.python.org/3/c-api/datetime.html#c.PyDateTime_DELTA_GET_MICROSECOND Should be: PyDateTime_DELTA_GET_MICROSECONDS ---------- assignee: docs at python components: Documentation messages: 303616 nosy: docs at python, ?????? ??????? priority: normal severity: normal status: open title: Incorrect C Function name for timedelta versions: Python 3.6 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Oct 3 13:04:18 2017 From: report at bugs.python.org (John Brearley) Date: Tue, 03 Oct 2017 17:04:18 +0000 Subject: [New-bugs-announce] [issue31679] pydot missing write, write_png, etc Message-ID: <1507050258.25.0.213398074469.issue31679@psf.upfronthosting.co.za> New submission from John Brearley : I have successfully installed Graphviz tool from http://graphviz.org, updated my PATH variable appropriately and can generate .PNG files using Python module Graphviz with WinPython 3.6.1. However, I cannot get anywhere using the pydot V1.2.3 module in the same environment. The online help suggests that there there are write & write_png members but I can find no trace of them. My sample script is attached. The error I get is: Traceback (most recent call last): File "ML_ex2.py", line 109, in graph.write(path=out_fn, format="png") AttributeError: 'list' object has no attribute 'write' ---------- components: Extension Modules files: ML_ex2.py messages: 303632 nosy: jbrearley priority: normal severity: normal status: open title: pydot missing write, write_png, etc type: behavior versions: Python 3.6 Added file: https://bugs.python.org/file47186/ML_ex2.py _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Oct 3 15:39:02 2017 From: report at bugs.python.org (Serhiy Storchaka) Date: Tue, 03 Oct 2017 19:39:02 +0000 Subject: [New-bugs-announce] [issue31680] Expose curses library name and version on Python level Message-ID: <1507059542.78.0.213398074469.issue31680@psf.upfronthosting.co.za> New submission from Serhiy Storchaka : The name and the version of the curses library are needed for work around bugs in old versions or skip tests on platforms that provide broken curses library (like OpenBSD providing ncurses 5.7). The curses and _curses modules provide the attribute "version" (= b'2.2'), but this is just a version of the module. ---------- components: Extension Modules messages: 303640 nosy: serhiy.storchaka, twouters priority: normal severity: normal status: open title: Expose curses library name and version on Python level type: enhancement versions: Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Oct 3 15:39:44 2017 From: report at bugs.python.org (Elvis Pranskevichus) Date: Tue, 03 Oct 2017 19:39:44 +0000 Subject: [New-bugs-announce] [issue31681] pkgutil.get_data() leaks open files in Python 2.7 Message-ID: <1507059584.22.0.213398074469.issue31681@psf.upfronthosting.co.za> New submission from Elvis Pranskevichus : pkgutil.get_data() does this: def get_data(self, pathname): return open(pathname, "rb").read() which very obviously leaks open file descriptors. This has been fixed in 3.x by https://github.com/python/cpython/commit/1ab58dfb12dedb7, but never backported to 2.7. ---------- components: Library (Lib) messages: 303642 nosy: Elvis.Pranskevichus priority: normal severity: normal status: open title: pkgutil.get_data() leaks open files in Python 2.7 type: resource usage versions: Python 2.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Oct 3 17:36:59 2017 From: report at bugs.python.org (Monica) Date: Tue, 03 Oct 2017 21:36:59 +0000 Subject: [New-bugs-announce] [issue31682] Exception: Cannot import `win32api`! Message-ID: <1507066619.6.0.213398074469.issue31682@psf.upfronthosting.co.za> New submission from Monica : Hello, I'm a newbie at Python. I am trying to download Twister to be able to run hardware tests, and it gives me Exception: Cannot import `win32api`!. I downloaded 'win32_221' and added it to C:\Python27\Scripts It is still giving me the same error. This is what I got TWISTER_PATH is set to `C:\Users\user\Documents\Twister-git_hub_branch\client\executionprocess`. Traceback (most recent call last): File "C:\Users\user\Documents\Twister-git_hub_branch\client\executionprocess\ExecutionProcess.py", line 1563, in raise Exception('Cannot import `win32api`!\n') Exception: Cannot import `win32api`! ---------- components: Installation messages: 303650 nosy: maolvera priority: normal severity: normal status: open title: Exception: Cannot import `win32api`! versions: Python 2.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Oct 3 17:40:12 2017 From: report at bugs.python.org (Oren Milman) Date: Tue, 03 Oct 2017 21:40:12 +0000 Subject: [New-bugs-announce] [issue31683] a stack overflow on windows in faulthandler._fatal_error() Message-ID: <1507066812.21.0.213398074469.issue31683@psf.upfronthosting.co.za> New submission from Oren Milman : On my 64-bit Windows 10, the echo here would print -1073741571: python -c "import faulthandler; faulthandler._fatal_error(b'a' * 2 ** 22)" echo %errorlevel% This is code c00000fd, which windbg describes as 'Stack overflow'. This happens because Py_FatalError() (in Python/pylifecycle.c) does the following (on Windows only): len = strlen(msg); /* Convert the message to wchar_t. This uses a simple one-to-one conversion, assuming that the this error message actually uses ASCII only. If this ceases to be true, we will have to convert. */ buffer = alloca( (len+1) * (sizeof *buffer)); for( i=0; i<=len; ++i) buffer[i] = msg[i]; Note that (IIUC) running the aforementioned cmd wouldn't cause a post-mortem debugger to pop-up, because faulthandler_fatal_error_py() (in Modules/faulthandler.c) first calls faulthandler_suppress_crash_report(), and then calls Py_FatalError(). ---------- components: Extension Modules messages: 303651 nosy: Oren Milman, haypo priority: normal severity: normal status: open title: a stack overflow on windows in faulthandler._fatal_error() type: crash versions: Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Oct 3 17:46:21 2017 From: report at bugs.python.org (Aaron Meurer) Date: Tue, 03 Oct 2017 21:46:21 +0000 Subject: [New-bugs-announce] [issue31684] Scientific formatting of decimal 0 different from float 0 Message-ID: <1507067181.62.0.213398074469.issue31684@psf.upfronthosting.co.za> New submission from Aaron Meurer : >>> '{:+.19e}'.format(0.) '+0.0000000000000000000e+00' >>> import decimal >>> '{:+.19e}'.format(decimal.Decimal(0)) '+0.0000000000000000000e+19' Note the decimal uses e+19 instead of e+00. Obviously it's still mathematically correct, but it's annoying to have anything other than e+00 for a 0 value. ---------- messages: 303653 nosy: Aaron.Meurer priority: normal severity: normal status: open title: Scientific formatting of decimal 0 different from float 0 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Oct 3 18:11:19 2017 From: report at bugs.python.org (Monica) Date: Tue, 03 Oct 2017 22:11:19 +0000 Subject: [New-bugs-announce] [issue31685] Cannot connect to CE path `127.0.0.1:8010` Message-ID: <1507068679.1.0.213398074469.issue31685@psf.upfronthosting.co.za> New submission from Monica : Is there any reason it cannot connect to CE path? C:\Users\Documents\Twister-git_hub_branch\client\executionprocess>ExecutionProcess.py -u user -e EP-name -s 127.0.0.1:8010 TWISTER_PATH is set to `C:\Users\Documents\Twister-git_hub_branch\client\executionprocess`. Catching `Ctrl + C` signal OK. Portable mode: Appending `C:\Users\Documents\Twister-git_hub_branch` to python path. EP Info: Connecting to the Central Engine... *ERROR* Cannot connect to CE path `127.0.0.1:8010`! Exiting! Cannot connect to server! Farewell! ---------- components: Demos and Tools messages: 303656 nosy: maolvera priority: normal severity: normal status: open title: Cannot connect to CE path `127.0.0.1:8010` type: compile error versions: Python 2.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Oct 3 19:48:08 2017 From: report at bugs.python.org (Jake Lever) Date: Tue, 03 Oct 2017 23:48:08 +0000 Subject: [New-bugs-announce] [issue31686] GZip library doesn't properly close files Message-ID: <1507074488.46.0.213398074469.issue31686@psf.upfronthosting.co.za> New submission from Jake Lever : The attached code is designed to output compressed data to a gzip file. It creates two GzipFile objects, but only one is really used. It seems that it doesn't clean up and close the file properly. In Py2, the attached code will fail. The output file (debug.txt.gz) will be corrupt. However if the self.unused line is commented out, it works and the file is not corrupted. In Py3, a sys.exit call at the end is required to see the same behaviour. As example output, when the sys.exit is include in Py3, the output is below and the file is corrupt. Compressor.__init__ UnusedClass.write UnusedClass.write UnusedClass.write UnusedClass.write UnusedClass.write UnusedClass.write Compressor.compressToFile And when the sys.exit is commented out, the console output is below and the file is not corrupt. Compressor.__init__ UnusedClass.write UnusedClass.write UnusedClass.write UnusedClass.write UnusedClass.write UnusedClass.write Compressor.compressToFile UnusedClass.write UnusedClass.write UnusedClass.write ---------- files: gzipBug.py messages: 303662 nosy: Jake Lever priority: normal severity: normal status: open title: GZip library doesn't properly close files type: behavior versions: Python 2.7, Python 3.6 Added file: https://bugs.python.org/file47189/gzipBug.py _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed Oct 4 05:36:35 2017 From: report at bugs.python.org (STINNER Victor) Date: Wed, 04 Oct 2017 09:36:35 +0000 Subject: [New-bugs-announce] [issue31687] test_semaphore_tracker() of test_multiprocessing_spawn fails randomly (race condition?) Message-ID: <1507109795.91.0.213398074469.issue31687@psf.upfronthosting.co.za> New submission from STINNER Victor : Recent example of the slow and busy (load of 8.05 with 2 CPUs) x86 Gentoo Refleaks 3.6 buildbot: http://buildbot.python.org/all/builders/x86%20Gentoo%20Refleaks%203.6/builds/111/steps/test/logs/stdio == CPU count: 2 ... 1:41:09 load avg: 8.05 [170/405/1] test_multiprocessing_spawn failed -- running: test_asyncio (1118 sec) beginning 6 repetitions 123456 /buildbot/buildarea/3.6.ware-gentoo-x86.refleak/build/Lib/unittest/case.py:633: ResourceWarning: unclosed file <_io.BufferedReader name=22> outcome.errors.clear() test test_multiprocessing_spawn failed -- Traceback (most recent call last): File "/buildbot/buildarea/3.6.ware-gentoo-x86.refleak/build/Lib/test/_test_multiprocessing.py", line 4091, in test_semaphore_tracker _multiprocessing.sem_unlink(name2) AssertionError: OSError not raised 1:41:12 load avg: 7.97 [171/405/1] test_posixpath passed -- running: test_asyncio (1122 sec) beginning 6 repetitions 123456 ...... ---------- components: Tests messages: 303678 nosy: haypo priority: normal severity: normal status: open title: test_semaphore_tracker() of test_multiprocessing_spawn fails randomly (race condition?) versions: Python 3.6, Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed Oct 4 08:33:34 2017 From: report at bugs.python.org (abdullah patel) Date: Wed, 04 Oct 2017 12:33:34 +0000 Subject: [New-bugs-announce] [issue31688] scope error Message-ID: <1507120414.78.0.213398074469.issue31688@psf.upfronthosting.co.za> New submission from abdullah patel : there should be an error when this code is run. as I have not defined the arguments (I was told by my computer science teacher that it should not work in theory) and parameters but it clearly works. ---------- assignee: terry.reedy components: IDLE files: pythonerror.jpg messages: 303682 nosy: abdullah patel, terry.reedy priority: normal severity: normal status: open title: scope error type: behavior versions: Python 3.6 Added file: https://bugs.python.org/file47190/pythonerror.jpg _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed Oct 4 08:53:19 2017 From: report at bugs.python.org (Allen Riddell) Date: Wed, 04 Oct 2017 12:53:19 +0000 Subject: [New-bugs-announce] [issue31689] random.choices does not work with negative weights Message-ID: <1507121599.13.0.213398074469.issue31689@psf.upfronthosting.co.za> New submission from Allen Riddell : Code to reproduce problem: population = list(range(10)) weights = list(-1 * w for w in range(10)) [random.choices(population, weights) for _ in range(1000)] will raise IndexError: 358 bisect = _bisect.bisect 359 total = cum_weights[-1] --> 360 return [population[bisect(cum_weights, random() * total)] for i in range(k)] 361 362 ## -------------------- real-valued distributions ------------------- IndexError: list index out of range ---------- components: Library (Lib) messages: 303683 nosy: ariddell priority: normal severity: normal status: open title: random.choices does not work with negative weights versions: Python 3.6 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed Oct 4 10:02:56 2017 From: report at bugs.python.org (Serhiy Storchaka) Date: Wed, 04 Oct 2017 14:02:56 +0000 Subject: [New-bugs-announce] [issue31690] Make RE "a", "L" and "u" inline flags local Message-ID: <1507125776.04.0.213398074469.issue31690@psf.upfronthosting.co.za> New submission from Serhiy Storchaka : Currently re supports local inline flags. 'a(?i:b)' matches 'a' cases-sensitively, but 'b' case-insensitively. But flags 'a' and 'L' can't be scoped to a subpattern. The 'u' flag currently just redundant, it doesn't make effect in string patterns, and is not allowed in bytes patterns. They can be applied only to the whole pattern. I think it would be nice to make them local. The example of the problem that this can solve is issue31672. Currently '[a-z]' in Unicode case-insensitive mode matches not only Latin letters from ;a' to 'z' and from 'A' to 'Z', but also characters '?', '?', '?' and '?' which are equivalent to 'i', 's' and 'k' correspondingly. With local 'a' and 'u' flags you can use ASCII and Unicode ranges in the same pattern. I'm working on the patch. ---------- assignee: serhiy.storchaka components: Library (Lib), Regular Expressions messages: 303693 nosy: ezio.melotti, mrabarnett, serhiy.storchaka priority: normal severity: normal stage: needs patch status: open title: Make RE "a", "L" and "u" inline flags local type: enhancement versions: Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed Oct 4 11:09:59 2017 From: report at bugs.python.org (Ivan Pozdeev) Date: Wed, 04 Oct 2017 15:09:59 +0000 Subject: [New-bugs-announce] [issue31691] Include missing info on required build steps and how to build installer Message-ID: <1507129799.44.0.213398074469.issue31691@psf.upfronthosting.co.za> New submission from Ivan Pozdeev : Current build instructions no not mention required preliminary steps before building the solution and don't say how to build the installer package - which is the ultimate goal of a build. Being on XP at the moment, I cannot check the validity of build instructions for other branches. Maybe they can use this info as well. ---------- components: Build files: 0001-Add-missing-build-steps-in-build-procedure.patch keywords: patch messages: 303700 nosy: Ivan.Pozdeev priority: normal severity: normal status: open title: Include missing info on required build steps and how to build installer versions: Python 2.7, Python 3.4 Added file: https://bugs.python.org/file47191/0001-Add-missing-build-steps-in-build-procedure.patch _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed Oct 4 11:42:48 2017 From: report at bugs.python.org (Iryna Shcherbina) Date: Wed, 04 Oct 2017 15:42:48 +0000 Subject: [New-bugs-announce] [issue31692] Test `test_huntrleaks` fails in debug build with COUNT_ALLOCS Message-ID: <1507131768.17.0.213398074469.issue31692@psf.upfronthosting.co.za> New submission from Iryna Shcherbina : The newly added `test_huntrleaks` test is failing on Python 2.7.14 debug build with COUNT_ALLOCS. ====================================================================== FAIL: test_huntrleaks (test.test_regrtest.ArgsTestCase) ---------------------------------------------------------------------- Traceback (most recent call last): File "/builddir/build/BUILD/Python-2.7.14/Lib/test/test_regrtest.py", line 511, in test_huntrleaks self.check_leak(code, 'references') File "/builddir/build/BUILD/Python-2.7.14/Lib/test/test_regrtest.py", line 489, in check_leak self.assertIn(line2, output) AssertionError: 'test_regrtest_huntrleaks leaked [1, 1, 1] references, sum=3\n' not found in 'Run tests sequentially\n0:00:00 load avg: 0.63 [1/1] test_regrtest_huntrleaks\nbeginning 6 repetitions\n123456\n......\ntest_regrtest_huntrleaks leaked [93, 93, 93] references, sum=279\n1 test failed:\n test_regrtest_huntrleaks\n\nTotal duration: 32 ms\nTests result: FAILURE\n[53092 refs]\n' ---------------------------------------------------------------------- On Python 2.7.14 debug build *without* COUNT_ALLOCS the test passes fine, therefore I am not sure if there is a leak detected, or is it just COUNT_ALLOCS messing with the test and it should be skipped. ---------- components: Tests messages: 303705 nosy: ishcherb priority: normal severity: normal status: open title: Test `test_huntrleaks` fails in debug build with COUNT_ALLOCS versions: Python 2.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed Oct 4 12:02:55 2017 From: report at bugs.python.org (Barry A. Warsaw) Date: Wed, 04 Oct 2017 16:02:55 +0000 Subject: [New-bugs-announce] [issue31693] Document Py_GETENV Message-ID: <1507132975.16.0.213398074469.issue31693@psf.upfronthosting.co.za> New submission from Barry A. Warsaw : I already added a section to the C API docs for some useful macros. I didn't know about this one when I did that, but it seems like a useful macro to document, so I will do that. ---------- assignee: barry components: Documentation messages: 303707 nosy: barry priority: normal severity: normal status: open title: Document Py_GETENV versions: Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed Oct 4 14:52:39 2017 From: report at bugs.python.org (Tzu-ping Chung) Date: Wed, 04 Oct 2017 18:52:39 +0000 Subject: [New-bugs-announce] [issue31694] Running Windows installer with LauncherOnly=1 should not register the version as installed Message-ID: <1507143159.66.0.213398074469.issue31694@psf.upfronthosting.co.za> New submission from Tzu-ping Chung : After running python-3.6.2-amd64.exe /quiet LauncherOnly=1 Python 3.6.2 is registered as installed, even though nothing is actually installed. Running the installer again at this point shows an error indicating ?this product is already installed?, and does not show me the usual repair/modify/uninstall options. I have to run uninstallation with Control Panel before the installer works again. ---------- components: Installation messages: 303727 nosy: uranusjr priority: normal severity: normal status: open title: Running Windows installer with LauncherOnly=1 should not register the version as installed type: behavior versions: Python 3.6 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed Oct 4 15:22:19 2017 From: report at bugs.python.org (Serhiy Storchaka) Date: Wed, 04 Oct 2017 19:22:19 +0000 Subject: [New-bugs-announce] [issue31695] Improve bigmem tests Message-ID: <1507144939.6.0.213398074469.issue31695@psf.upfronthosting.co.za> New submission from Serhiy Storchaka : I think that the process of running bigmem tests should be improved. 1. Every bigmem test should run in separate process. It's effect on memory fragmentation shouldn't affect other tests. 2. Only one bigmem test should be run at the same time. Otherwise several parallel bigmem tests can exhaust RAM and lead to swapping. In multiprocessing mode the process running a bigmem test should send a signal to other processes, wait until they finished their current tests and paused, run a bigmem test, and send a signal to other processes that they can continue. 3. The hard limit of addressed memory should be set for every bigmem test, according to the declared requirements. It is better to crash one test and report the failure, than hang on swapping. 4. All other tests should be run with the limit set to 2 GiB (or 1 GiB?). This will help to find memory consuming tests which should be made bigmem tests. 5. The maxrss of the process running a bigmem test should be reported in verbose mode after finishing a test. ---------- components: Tests messages: 303729 nosy: haypo, serhiy.storchaka priority: normal severity: normal status: open title: Improve bigmem tests type: enhancement _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed Oct 4 16:56:25 2017 From: report at bugs.python.org (Benjamin Peterson) Date: Wed, 04 Oct 2017 20:56:25 +0000 Subject: [New-bugs-announce] [issue31696] don't mention GCC in sys.version when built with Clang Message-ID: <1507150585.16.0.213398074469.issue31696@psf.upfronthosting.co.za> New submission from Benjamin Peterson : When built with clang, Python reports that it was built with something like "GCC 4.2.1 Compatible Clang 4.0.0 (tags/RELEASE_400/rc1)". This is because clang pretends to be GCC 4.2.1 for the purposes of the __VERSION__ preprocessor macro. We should use __clang_version__ when clang is being used. (Possibly we should simply use the first line of "$CC --version" as the compiler version of record rather than writing preprocessor tests in getcompiler.c.) ---------- components: Build keywords: easy (C) messages: 303731 nosy: benjamin.peterson priority: normal severity: normal status: open title: don't mention GCC in sys.version when built with Clang versions: Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed Oct 4 19:47:30 2017 From: report at bugs.python.org (coady) Date: Wed, 04 Oct 2017 23:47:30 +0000 Subject: [New-bugs-announce] [issue31697] Regression in futures.as_completed with ProcessPoolExecutor. Message-ID: <1507160850.08.0.213398074469.issue31697@psf.upfronthosting.co.za> New submission from coady : `futures.as_completed` is not generating process-based futures in completed order. Reproduces with 3.7a0. ---------- files: test.py messages: 303733 nosy: coady priority: normal severity: normal status: open title: Regression in futures.as_completed with ProcessPoolExecutor. versions: Python 3.7 Added file: https://bugs.python.org/file47193/test.py _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu Oct 5 00:28:25 2017 From: report at bugs.python.org (Jelle Zijlstra) Date: Thu, 05 Oct 2017 04:28:25 +0000 Subject: [New-bugs-announce] [issue31698] Add REQ_NAME to the node.h API Message-ID: <1507177705.75.0.213398074469.issue31698@psf.upfronthosting.co.za> New submission from Jelle Zijlstra : See https://github.com/python/cpython/pull/1669#pullrequestreview-67229284 ---------- assignee: Jelle Zijlstra components: Interpreter Core messages: 303739 nosy: Jelle Zijlstra, yselivanov priority: low severity: normal status: open title: Add REQ_NAME to the node.h API versions: Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu Oct 5 03:41:45 2017 From: report at bugs.python.org (Thomas Moreau) Date: Thu, 05 Oct 2017 07:41:45 +0000 Subject: [New-bugs-announce] [issue31699] Deadlocks in `concurrent.futures.ProcessPoolExecutor` with pickling error Message-ID: <1507189305.87.0.213398074469.issue31699@psf.upfronthosting.co.za> New submission from Thomas Moreau : When using `concurrent.futures.ProcessPoolExecutor` with objects that are not picklable or unpicklable, several situations results in a deadlock, with the interpreter freezed. This is the case for different scenario, for instance these three : https://gist.github.com/tomMoral/cc27a938d669edcf0286c57516942369 The different pickling/unpickling error and their effect should be tested in `test_concurrent_futures.py` ---------- components: Library (Lib) messages: 303745 nosy: tomMoral priority: normal pull_requests: 3866 severity: normal status: open title: Deadlocks in `concurrent.futures.ProcessPoolExecutor` with pickling error versions: Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu Oct 5 04:35:17 2017 From: report at bugs.python.org (Sebastian Rittau) Date: Thu, 05 Oct 2017 08:35:17 +0000 Subject: [New-bugs-announce] [issue31700] one-argument version for Generator.typing Message-ID: <1507192517.86.0.213398074469.issue31700@psf.upfronthosting.co.za> New submission from Sebastian Rittau : Currently typing.Generator requires three arguments: Generator[YieldType, SendType, ReturnType]. At least for me, passing values to a generator is a very rare case. I suggest to allow only one argument to be passed to Generator: Generator[YieldType], where the other arguments default to None. This makes the common case more readable and is less error-prone. (I always forget the second and third argument, since that use case is so uncommon for me.) ---------- messages: 303748 nosy: srittau priority: normal severity: normal status: open title: one-argument version for Generator.typing versions: Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu Oct 5 05:59:11 2017 From: report at bugs.python.org (Fynn Be) Date: Thu, 05 Oct 2017 09:59:11 +0000 Subject: [New-bugs-announce] [issue31701] faulthandler dumps 'Windows fatal exception: code 0xe06d7363' Message-ID: <1507197551.3.0.213398074469.issue31701@psf.upfronthosting.co.za> New submission from Fynn Be : c++ extension compiled with MSVC 14 using python 3.6.2 on Windows 10 x64 Whenever a C++ exception is thrown in an extension the faulthandler dumps a traceback to it, even if it is caught. ---------- components: Windows messages: 303751 nosy: Fynn Be, haypo, paul.moore, steve.dower, tim.golden, zach.ware priority: normal severity: normal status: open title: faulthandler dumps 'Windows fatal exception: code 0xe06d7363' type: behavior versions: Python 3.6 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu Oct 5 08:32:52 2017 From: report at bugs.python.org (Serhiy Storchaka) Date: Thu, 05 Oct 2017 12:32:52 +0000 Subject: [New-bugs-announce] [issue31702] Allow to specify the number of rounds for SHA-* hashing in crypt Message-ID: <1507206772.04.0.213398074469.issue31702@psf.upfronthosting.co.za> New submission from Serhiy Storchaka : Blowfish salt should contain the binary logarithm of the number of rounds (from 4 to 31) (see issue31664). SHA-* salt can contain an explicit number of rounds in the form '$rounds={value}$'. It is bound to the range from 1000 to 999999999, the default is 5000. I propose to allow to specify the number of rounds in generated salt for SHA-* methods as well as for Blowfish. For unifying interface we can specify the number of rounds instead of its logarithm for Blowfish, and calculate the logarithm internally. The question is what to do with the value that is not a power of two for Blowfish. Should we raise an error or silently replace it with the upper power of two? ---------- components: Library (Lib) messages: 303760 nosy: christian.heimes, dstufft, gregory.p.smith, jafo, serhiy.storchaka priority: normal severity: normal status: open title: Allow to specify the number of rounds for SHA-* hashing in crypt type: enhancement versions: Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu Oct 5 09:41:53 2017 From: report at bugs.python.org (STINNER Victor) Date: Thu, 05 Oct 2017 13:41:53 +0000 Subject: [New-bugs-announce] [issue31703] [EASY] Running test_builtin twice fails on input tty tests Message-ID: <1507210913.82.0.213398074469.issue31703@psf.upfronthosting.co.za> New submission from STINNER Victor : haypo at selma$ ./python -m test test_builtin test_builtin -v ====================================================================== FAIL: test_input_tty_non_ascii (test.test_builtin.PtyTests) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/haypo/prog/python/master/Lib/test/test_builtin.py", line 1717, in test_input_tty_non_ascii self.check_input_tty("prompt?", b"quux\xe9", "utf-8") File "/home/haypo/prog/python/master/Lib/test/test_builtin.py", line 1708, in check_input_tty self.assertEqual(input_result, expected) AssertionError: 'quux' != 'quux\udce9' - quux + quux\udce9 ? + ====================================================================== FAIL: test_input_tty_non_ascii_unicode_errors (test.test_builtin.PtyTests) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/haypo/prog/python/master/Lib/test/test_builtin.py", line 1721, in test_input_tty_non_ascii_unicode_errors self.check_input_tty("prompt?", b"quux\xe9", "ascii") File "/home/haypo/prog/python/master/Lib/test/test_builtin.py", line 1708, in check_input_tty self.assertEqual(input_result, expected) AssertionError: 'quux' != 'quux\udce9' - quux + quux\udce9 ? + ---------- components: Tests keywords: easy messages: 303765 nosy: haypo priority: normal severity: normal status: open title: [EASY] Running test_builtin twice fails on input tty tests versions: Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu Oct 5 09:52:00 2017 From: report at bugs.python.org (=?utf-8?b?w4FsdmFybyBNdcOxb3o=?=) Date: Thu, 05 Oct 2017 13:52:00 +0000 Subject: [New-bugs-announce] [issue31704] HTTP check lowercase response from proxy Message-ID: <1507211520.07.0.213398074469.issue31704@psf.upfronthosting.co.za> New submission from ?lvaro Mu?oz : Recently faced an issue with a proxy responding in lowercase http, which caused this: ProtocolError('Connection aborted.', BadStatusLine('http/1.1 200 connection established\r\n',)) Changing the string to uppercase before checking if it starts with HTTP fixes this issue and allows to use this proxy. I know that the proxy is at fault here, but seeing as other applications (web browsers, office suite, text editors, etc.) still work behind this proxy, I think it might be a reasonable fix to have... ---------- components: Library (Lib) messages: 303768 nosy: alvaromunoz priority: normal pull_requests: 3869 severity: normal status: open title: HTTP check lowercase response from proxy type: behavior versions: Python 2.7, Python 3.4, Python 3.5, Python 3.6, Python 3.7, Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu Oct 5 15:58:24 2017 From: report at bugs.python.org (Charalampos Stratakis) Date: Thu, 05 Oct 2017 19:58:24 +0000 Subject: [New-bugs-announce] [issue31705] test_sha256 from test_socket fails on ppc64le arch Message-ID: <1507233504.45.0.213398074469.issue31705@psf.upfronthosting.co.za> New submission from Charalampos Stratakis : The issue is reproducible on a CentOS 7.4 on ppc64le architecture. It passes successfully on other arch's (however the other power pc arch's might also be affected). How to reproduce: Compile python 3.6 from source. Run ./python3 -m test --verbose test_socket And it fails with: ERROR: test_sha256 (test.test_socket.LinuxKernelCryptoAPI) ---------------------------------------------------------------------- Traceback (most recent call last): File "/root/test/cpython/Lib/test/test_socket.py", line 5424, in test_sha256 op.sendall(b"abc") OSError: [Errno 126] Required key not available ---------- components: Tests messages: 303783 nosy: cstratak priority: normal severity: normal status: open title: test_sha256 from test_socket fails on ppc64le arch versions: Python 3.6, Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu Oct 5 16:12:52 2017 From: report at bugs.python.org (=?utf-8?q?Fran=C3=A7ois_Freitag?=) Date: Thu, 05 Oct 2017 20:12:52 +0000 Subject: [New-bugs-announce] [issue31706] urlencode should accept generator as values for mappings when doseq=True Message-ID: <1507234372.12.0.213398074469.issue31706@psf.upfronthosting.co.za> New submission from Fran?ois Freitag : The urlencode documentation states that: The value element in itself can be a sequence and in that case, if the optional parameter doseq is evaluates to True, individual key=value pairs separated by '&' are generated for each element of the value sequence for the key. Passing a generator as the value gives unexpected results: >>> from urllib.parse import urlencode >>> def gen(): ... yield from range(2) >>> urlencode({'a': gen()}, doseq=True) 'a=%3Cgenerator+object+gen+at+0x7f35ff78db48%3E' A list gives: >>> urlencode({'a': [0, 1]}, doseq=True) 'a=0&a=1' ---------- components: Library (Lib) messages: 303784 nosy: freitafr priority: normal severity: normal status: open title: urlencode should accept generator as values for mappings when doseq=True type: enhancement _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu Oct 5 19:08:26 2017 From: report at bugs.python.org (Ben Burrill) Date: Thu, 05 Oct 2017 23:08:26 +0000 Subject: [New-bugs-announce] [issue31707] Irrational fractions Message-ID: <1507244906.43.0.213398074469.issue31707@psf.upfronthosting.co.za> New submission from Ben Burrill : fractions.Fraction enforces its numerator and denominator to be rational. This is a good idea for purely numeric fractions, but the abstractions that fractions.Fraction offers would also be useful for more abstract fractions. Some places where this might be useful would be fractions of polynomial types or fractions with abstract irrational numeric types, like F(SquareRoot(3), 2). This could be accomplished by having a more general Fraction superclass or by removing the requirement of rationality from fractions.Fraction. It is not trivial to create a subclass of Fraction that does this because the operators are hardcoded to use Fraction and initiation is done in __new__. ---------- components: Library (Lib) messages: 303788 nosy: Ben Burrill priority: normal severity: normal status: open title: Irrational fractions type: behavior versions: Python 3.6 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Oct 6 00:23:32 2017 From: report at bugs.python.org (Yury Selivanov) Date: Fri, 06 Oct 2017 04:23:32 +0000 Subject: [New-bugs-announce] [issue31708] Allow use of asynchronous generator expressions in synchronous functions Message-ID: <1507263812.11.0.213398074469.issue31708@psf.upfronthosting.co.za> Change by Yury Selivanov : ---------- assignee: yselivanov components: Interpreter Core nosy: yselivanov priority: normal severity: normal status: open title: Allow use of asynchronous generator expressions in synchronous functions type: behavior versions: Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Oct 6 00:23:47 2017 From: report at bugs.python.org (Yury Selivanov) Date: Fri, 06 Oct 2017 04:23:47 +0000 Subject: [New-bugs-announce] [issue31709] Drop support for asynchronous __aiter__ Message-ID: <1507263827.12.0.213398074469.issue31709@psf.upfronthosting.co.za> New submission from Yury Selivanov : As discussed in issue 27243, we want to drop support of asynchronous __aiter__ in Python 3.7. Together with issue 30406, this will enable us to add support for using asynchronous generator expressions in synchronous functions (issue 31708) ---------- assignee: yselivanov components: Interpreter Core messages: 303796 nosy: ncoghlan, yselivanov priority: normal severity: normal status: open title: Drop support for asynchronous __aiter__ versions: Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Oct 6 01:14:44 2017 From: report at bugs.python.org (pmpp) Date: Fri, 06 Oct 2017 05:14:44 +0000 Subject: [New-bugs-announce] [issue31710] setup.py: _ctypes won't getbuilt when system ffi is only in $PREFIX Message-ID: <1507266884.06.0.213398074469.issue31710@psf.upfronthosting.co.za> New submission from pmpp : --with-system-ffi is mandatory for linux build but no way is provided ( eg --with-ffi-includes= --with-ffi-libs= ) so setup.py can detect libffi already built in $PREFIX and $EPREFIX/lib. even if cflags/ldflags are ok , _ctypes build will be skipped with reason: INFO: Could not locate ffi libs and/or headers test condition: crystax-ndk toward android-19 on ubuntu xenial 32 bits ---------- components: Cross-Build messages: 303799 nosy: Alex.Willmer, pmpp priority: normal severity: normal status: open title: setup.py: _ctypes won't getbuilt when system ffi is only in $PREFIX type: compile error versions: Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Oct 6 05:43:00 2017 From: report at bugs.python.org (=?utf-8?q?J=C3=B6rn_Heissler?=) Date: Fri, 06 Oct 2017 09:43:00 +0000 Subject: [New-bugs-announce] [issue31711] ssl.SSLSocket.send(b"") fails Message-ID: <1507282980.19.0.213398074469.issue31711@psf.upfronthosting.co.za> New submission from J?rn Heissler : Traceback (most recent call last): File "client.py", line 10, in conn.send(b'') File "/usr/lib/python3.6/ssl.py", line 941, in send return self._sslobj.write(data) File "/usr/lib/python3.6/ssl.py", line 642, in write return self._sslobj.write(data) ssl.SSLEOFError: EOF occurred in violation of protocol (_ssl.c:2074) This error is not what I expected. I expected a noop instead. My guess is, that python calls SSL_write (3.6 branch, _ssl.c:2038) with that empty buffer. The manpage states: "When calling SSL_write() with num=0 bytes to be sent the behaviour is undefined." This undefined behaviour should either be documented in python, or defined to either raise an exception (ValueError?) or defined as a noop. I'd prefer the latter. ---------- assignee: christian.heimes components: SSL files: client.py messages: 303810 nosy: christian.heimes, joernheissler priority: normal severity: normal status: open title: ssl.SSLSocket.send(b"") fails type: behavior versions: Python 3.6 Added file: https://bugs.python.org/file47194/client.py _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Oct 6 07:39:10 2017 From: report at bugs.python.org (l4mer) Date: Fri, 06 Oct 2017 11:39:10 +0000 Subject: [New-bugs-announce] [issue31712] subprocess with stderr=subprocess.STDOUT hang Message-ID: <1507289950.65.0.213398074469.issue31712@psf.upfronthosting.co.za> New submission from l4mer : I am using ssh mux + authorized key. So can exectute commands without passwod. 1. disconnect mux by: ssh user at localhost -O exit 2. run simple script import subprocess if __name__ == '__main__': cmd = ["ssh", "user at localhost", "id"] try: buf = subprocess.check_output(cmd, stderr=subprocess.STDOUT) except subprocess.CalledProcessError as e: buf = e.output 3. This always hang in read(3, fcntl(3, F_GETFL) = 0 (flags O_RDONLY) fstat(3, {st_mode=S_IFIFO|0600, st_size=0, ...}) = 0 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f036b5f1000 lseek(3, 0, SEEK_CUR) = -1 ESPIPE (Illegal seek) fstat(3, {st_mode=S_IFIFO|0600, st_size=0, ...}) = 0 munmap(0x7f036b5f1000, 4096) = 0 fstat(3, {st_mode=S_IFIFO|0600, st_size=0, ...}) = 0 lseek(3, 0, SEEK_CUR) = -1 ESPIPE (Illegal seek) read(3, "uid=0(", 6) = 6 fstat(3, {st_mode=S_IFIFO|0600, st_size=0, ...}) = 0 lseek(3, 0, SEEK_CUR) = -1 ESPIPE (Illegal seek) read(3, "root) ", 6) = 6 fstat(3, {st_mode=S_IFIFO|0600, st_size=0, ...}) = 0 lseek(3, 0, SEEK_CUR) = -1 ESPIPE (Illegal seek) read(3, "gid=0(r", 7) = 7 fstat(3, {st_mode=S_IFIFO|0600, st_size=0, ...}) = 0 lseek(3, 0, SEEK_CUR) = -1 ESPIPE (Illegal seek) read(3, "oot) gro", 8) = 8 fstat(3, {st_mode=S_IFIFO|0600, st_size=0, ...}) = 0 lseek(3, 0, SEEK_CUR) = -1 ESPIPE (Illegal seek) read(3, "ups=0(roo", 9) = 9 fstat(3, {st_mode=S_IFIFO|0600, st_size=0, ...}) = 0 lseek(3, 0, SEEK_CUR) = -1 ESPIPE (Illegal seek) read(3, "t)\n", 10) = 3 read(3, 0x7f036b480653, 7) = ? ERESTARTSYS (To be restarted if SA_RESTART is set) --- SIGCHLD {si_signo=SIGCHLD, si_code=CLD_EXITED, si_pid=21620, si_status=0, si_utime=1, si_stime=0} --- read(3, 4. After ctr+c read(3, ^CProcess 21619 detached Traceback (most recent call last): File "test.py", line 6, in buf = subprocess.check_output(cmd, stderr=subprocess.STDOUT) File "/usr/lib/python2.7/subprocess.py", line 567, in check_output output, unused_err = process.communicate() File "/usr/lib/python2.7/subprocess.py", line 791, in communicate wigig at wigig-Latitude-E5270:~$ stdout = _eintr_retry_call(self.stdout.read) File "/usr/lib/python2.7/subprocess.py", line 476, in _eintr_retry_call return func(*args) BTW, when MUX is created it always works. ---------- messages: 303815 nosy: l4mer priority: normal severity: normal status: open title: subprocess with stderr=subprocess.STDOUT hang type: behavior versions: Python 2.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Oct 6 09:37:29 2017 From: report at bugs.python.org (matthewlweber) Date: Fri, 06 Oct 2017 13:37:29 +0000 Subject: [New-bugs-announce] [issue31713] python3 python-config script generates invalid includes Message-ID: <1507297049.36.0.213398074469.issue31713@psf.upfronthosting.co.za> New submission from matthewlweber : Related to https://bugs.python.org/issue22907 If building in a path that starts with /usr, the includedir=$(echo "@includedir@" | sed "s#^$prefix_build#$prefix_real#") assignment in the python-config.sh ends up having the path it processes ran through a path substitution once before this line is executed because the @includedir@ in the python-config.sh.in is set to the string '${prefix}/include'. ${prefix} is assigned just above includedir in python-config.sh to prefix=$(echo "$prefix_build" | sed "s#^$prefix_build#$prefix_real#") I believe we need to update the includedir to includedir=$(echo "@includedir@") Or rename the prefix variable in python-config.sh so that there isn't a naming conflict if a string is passed in via @includedir@ with that variable. Without fixing this you end up with multiple /usr substitutions in the includedir string, each replaced with the real path. ie resulting in an invalid path. ---------- components: Cross-Build messages: 303819 nosy: Alex.Willmer, matthewlweber priority: normal severity: normal status: open title: python3 python-config script generates invalid includes versions: Python 3.4 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Oct 6 10:22:12 2017 From: report at bugs.python.org (Serhiy Storchaka) Date: Fri, 06 Oct 2017 14:22:12 +0000 Subject: [New-bugs-announce] [issue31714] Improve re documentation Message-ID: <1507299732.49.0.213398074469.issue31714@psf.upfronthosting.co.za> New submission from Serhiy Storchaka : The proposed PR improves the documentation of re module and Regular Expression HOWTO. * Clarify the effect of the LOCALE flag. * Remove outdated statements. * Add an example of escaping a replacement string. * Add or correct some references. * Improve and fix a formatting. ---------- assignee: docs at python components: Documentation, Regular Expressions messages: 303822 nosy: akuchling, barry, docs at python, ezio.melotti, mrabarnett, serhiy.storchaka priority: normal severity: normal stage: patch review status: open title: Improve re documentation type: enhancement versions: Python 3.6, Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Oct 6 11:06:33 2017 From: report at bugs.python.org (Bradley Meck) Date: Fri, 06 Oct 2017 15:06:33 +0000 Subject: [New-bugs-announce] [issue31715] Add mimetype for extension .mjs Message-ID: <1507302393.84.0.213398074469.issue31715@psf.upfronthosting.co.za> New submission from Bradley Meck : I propose to add a mapping of file extension .mjs to mime type "text/javascript". The "text/javascript" MIME is registered in https://www.iana.org/assignments/media-types, was moved to *should* as the MIME in HTML ( https://github.com/whatwg/html/pull/3096), and is being updated by https://datatracker.ietf.org/doc/draft-bfarias-javascript-mjs/ This extension is being used by Node.js for support of ECMAScript Modules (ESM). ---------- components: Library (Lib) messages: 303824 nosy: Bradley Meck priority: normal severity: normal status: open title: Add mimetype for extension .mjs type: enhancement versions: Python 2.7, Python 3.6, Python 3.7, Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Oct 6 11:26:29 2017 From: report at bugs.python.org (Mor Haviv) Date: Fri, 06 Oct 2017 15:26:29 +0000 Subject: [New-bugs-announce] [issue31716] os.path.isdir returns true for dots Message-ID: <1507303589.7.0.213398074469.issue31716@psf.upfronthosting.co.za> New submission from Mor Haviv : I uploaded this as a question on Stack Overflow and I suspect it might be a bug. Here is the link for the Stack Overflow question: https://stackoverflow.com/questions/46608731/python-os-path-isdir-returns-true-for-dots/46608842#46608842 The problem itself (copied from what I uploaded on Stack Overflow): I'm programming my own shell in python. Right now I'm trying to implement the `cd` command to my shell. The function that performs this command has several variables: `self.current_dir = "C:\\"` - The default value, it changes depends on the user's input using the cd command `dir = "..."` - The requested directory that the user types. "..." is an example for an input that causes the problem. Here is my code: def command_cd(self, dir): if os.path.isdir(self.shell.current_dir + dir): self.shell.current_dir = self.shell.current_dir + dir + "\\" The problem is that for some strange reason, `os.path.isdir(self.shell.current_dir + dir)` returns `True` when the user types dots (Just like the example inputs for the variables which I gave above). The problem occurs even if you change the amount of dots (even above 5 dots) and I really have no idea what causes it. There's obviously no folder named `...` or anything like it. **If my problem isn't clear enough please comment and I'll edit it** ---------- components: Windows messages: 303827 nosy: morha13, paul.moore, steve.dower, tim.golden, zach.ware priority: normal severity: normal status: open title: os.path.isdir returns true for dots type: behavior versions: Python 2.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Oct 6 11:53:09 2017 From: report at bugs.python.org (Phillip) Date: Fri, 06 Oct 2017 15:53:09 +0000 Subject: [New-bugs-announce] [issue31717] Socket documentation threading misstep? Message-ID: <1507305189.16.0.213398074469.issue31717@psf.upfronthosting.co.za> New submission from Phillip : Very small, but, https://docs.python.org/2/howto/sockets.html https://docs.python.org/3/howto/sockets.html have : while True: # accept connections from outside (clientsocket, address) = serversocket.accept() # now do something with the clientsocket # in this case, we'll pretend this is a threaded server ct = client_thread(clientsocket) ct.run() and for some reason I really want it to be ct.start() ---------- assignee: docs at python components: Documentation messages: 303831 nosy: apoplexy, docs at python priority: normal severity: normal status: open title: Socket documentation threading misstep? versions: Python 2.7, Python 3.4, Python 3.5, Python 3.6, Python 3.7, Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Oct 6 14:09:48 2017 From: report at bugs.python.org (Oren Milman) Date: Fri, 06 Oct 2017 18:09:48 +0000 Subject: [New-bugs-announce] [issue31718] some methods of uninitialized io.IncrementalNewlineDecoder objects raise SystemError Message-ID: <1507313388.78.0.213398074469.issue31718@psf.upfronthosting.co.za> New submission from Oren Milman : Given an uninitialized IncrementalNewlineDecoder: uninitialized = io.IncrementalNewlineDecoder.__new__(io.IncrementalNewlineDecoder) each of the following calls would raise a SystemError ('null argument to internal routine'): uninitialized.getstate() uninitialized.setstate((b'foo', 0)) uninitialized.reset() In contrast, the following call would raise a ValueError ('IncrementalNewlineDecoder.__init__ not called'): uninitialized.decode(b'bar') ISTM that getstate(), setstate(), and reset() should have the same behavior as decode(). (Though i think that including the actual type name in the error message would be better, as it could be a subclass of IncrementalNewlineDecoder). ---------- components: IO messages: 303842 nosy: Oren Milman priority: normal severity: normal status: open title: some methods of uninitialized io.IncrementalNewlineDecoder objects raise SystemError type: behavior versions: Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Oct 6 16:35:01 2017 From: report at bugs.python.org (STINNER Victor) Date: Fri, 06 Oct 2017 20:35:01 +0000 Subject: [New-bugs-announce] [issue31719] [2.7] test_regrtest.test_crashed() fails on s390x Message-ID: <1507322101.2.0.213398074469.issue31719@psf.upfronthosting.co.za> New submission from STINNER Victor : On the s390x architecture, the test_regrtest.test_crashed() fails because Python doesn't crash. test.support._crash_python() calls ctypes.string_at(0) to crash Python. In debug mode, ctypes.string_at(0) fails with an assertion error and the process is killed with the SIGABRT signal. In release mode, ctypes.string_at(0) returns an empty string but doesn't crash. Attached PR adds a new _testcapi._read_null() function which does crash in a reliable way on s390x and x86 :-) The function uses the C "volatile" keyword to prevent compiler optimizations. I copied the code from my faulthandler._read_null() function in the master branch which was battle tested. It does crash on all platforms... except of AIX. ---------- components: Tests messages: 303854 nosy: haypo priority: normal severity: normal status: open title: [2.7] test_regrtest.test_crashed() fails on s390x versions: Python 2.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sat Oct 7 09:39:30 2017 From: report at bugs.python.org (Tzu-ping Chung) Date: Sat, 07 Oct 2017 13:39:30 +0000 Subject: [New-bugs-announce] [issue31720] msilib.MSIError is spelled MsiError in documentation Message-ID: <1507383570.37.0.213398074469.issue31720@psf.upfronthosting.co.za> New submission from Tzu-ping Chung : The title describes it all. ---------- assignee: docs at python components: Documentation messages: 303877 nosy: docs at python, uranusjr priority: normal severity: normal status: open title: msilib.MSIError is spelled MsiError in documentation versions: Python 2.7, Python 3.4, Python 3.5, Python 3.6, Python 3.7, Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sat Oct 7 10:19:24 2017 From: report at bugs.python.org (Oren Milman) Date: Sat, 07 Oct 2017 14:19:24 +0000 Subject: [New-bugs-announce] [issue31721] assertion failure in FutureObj_finalize() after setting _log_traceback to True Message-ID: <1507385964.19.0.213398074469.issue31721@psf.upfronthosting.co.za> New submission from Oren Milman : The following code causes an assertion failure in FutureObj_finalize() (in Modules/_asynciomodule.c): import asyncio asyncio.Future()._log_traceback = True Maybe we should allow Python code to only set it to False, and raise a ValueError in case Python code tries to set it to True? (PR 2050 made _log_traceback writable. Are there any usecases for setting it to True from Python code?) ---------- components: asyncio messages: 303878 nosy: Oren Milman, yselivanov priority: normal severity: normal status: open title: assertion failure in FutureObj_finalize() after setting _log_traceback to True type: crash versions: Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sat Oct 7 10:26:41 2017 From: report at bugs.python.org (Serhiy Storchaka) Date: Sat, 07 Oct 2017 14:26:41 +0000 Subject: [New-bugs-announce] [issue31722] _io.IncrementalNewlineDecoder doesn't inherit codecs.IncrementalDecoder Message-ID: <1507386401.5.0.213398074469.issue31722@psf.upfronthosting.co.za> New submission from Serhiy Storchaka : It is documented that io.IncrementalNewlineDecoder inherits codecs.IncrementalDecoder. And the Python implementation does. But the C implementation doesn't. >>> issubclass(_pyio.IncrementalNewlineDecoder, codecs.IncrementalDecoder) True >>> issubclass(_io.IncrementalNewlineDecoder, codecs.IncrementalDecoder) False >>> issubclass(io.IncrementalNewlineDecoder, codecs.IncrementalDecoder) False ---------- components: IO, Library (Lib) messages: 303879 nosy: benjamin.peterson, serhiy.storchaka, stutzbach priority: normal severity: normal status: open title: _io.IncrementalNewlineDecoder doesn't inherit codecs.IncrementalDecoder type: behavior versions: Python 2.7, Python 3.6, Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sat Oct 7 11:27:27 2017 From: report at bugs.python.org (Oren Milman) Date: Sat, 07 Oct 2017 15:27:27 +0000 Subject: [New-bugs-announce] [issue31723] refleaks in zipimport when calling zipimporter.__init__() more than once Message-ID: <1507390047.35.0.213398074469.issue31723@psf.upfronthosting.co.za> New submission from Oren Milman : The following code causes refleaks: import zipimport zi = zipimport.zipimporter.__new__(zipimport.zipimporter) zi.__init__('bar.zip') zi.__init__('bar.zip') zi.__init__('bar.zip\\foo') This is because zipimport_zipimporter___init___impl() (in Modules/zipimport.c) doesn't decref (if needed) before assigning to `self->files`, `self->archive` and `self->prefix`. I would open a PR to fix this soon. Should i add a test to test_zipimport? If yes, could you point out some similar refcount test to help me write this test? ---------- components: Extension Modules messages: 303883 nosy: Oren Milman priority: normal severity: normal status: open title: refleaks in zipimport when calling zipimporter.__init__() more than once type: resource usage versions: Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun Oct 8 01:44:01 2017 From: report at bugs.python.org (Zachary Ware) Date: Sun, 08 Oct 2017 05:44:01 +0000 Subject: [New-bugs-announce] [issue31724] test_xmlrpc should use something other than buildbot.python.org Message-ID: <1507441441.34.0.213398074469.issue31724@psf.upfronthosting.co.za> New submission from Zachary Ware : With the upgrade to buildbot 0.9, the xmlrpc interface to buildbot.python.org is removed, causing test_xmlrpc to fail. The test should hit a different host, possibly on pythontest.net? ---------- components: Tests messages: 303894 nosy: loewis, zach.ware priority: normal severity: normal stage: needs patch status: open title: test_xmlrpc should use something other than buildbot.python.org versions: Python 2.7, Python 3.6, Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun Oct 8 03:16:44 2017 From: report at bugs.python.org (Rick J. Pelleg) Date: Sun, 08 Oct 2017 07:16:44 +0000 Subject: [New-bugs-announce] [issue31725] Turtle/tkinter: NameError crashes ipython with "Tcl_AsyncDelete: async handler deleted by the wrong thread" Message-ID: <1507447004.91.0.213398074469.issue31725@psf.upfronthosting.co.za> New submission from Rick J. Pelleg : On Windows 10 Education, ran: ipython Python 3.6.3 (v3.6.3:2c5fed8, Oct 3 2017, 17:26:49) [MSC v.1900 32 bit (Intel)] Type 'copyright', 'credits' or 'license' for more information IPython 6.2.1 -- An enhanced Interactive Python. Type '?' for help. ---- After "from turtle import *" and several simple turtle commands: In [36]: fill() --------------------------------------------------------------------------- NameError Traceback (most recent call last) in () ----> 1 fill() NameError: name 'fill' is not defined Tcl_AsyncDelete: async handler deleted by the wrong thread Exception ignored in: > Traceback (most recent call last): File "d:\users\yuval\appdata\local\programs\python\python36-32\lib\tkinter\__init__.py", line 3501, in __del__ self.tk.call('image', 'delete', self.name) ---------- components: Tkinter messages: 303898 nosy: Rick J. Pelleg priority: normal severity: normal status: open title: Turtle/tkinter: NameError crashes ipython with "Tcl_AsyncDelete: async handler deleted by the wrong thread" versions: Python 3.6 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun Oct 8 04:08:56 2017 From: report at bugs.python.org (Franck Pommereau) Date: Sun, 08 Oct 2017 08:08:56 +0000 Subject: [New-bugs-announce] [issue31726] Missing token.COMMENT Message-ID: <1507450136.57.0.213398074469.issue31726@psf.upfronthosting.co.za> New submission from Franck Pommereau : Module token does not have constant COMMENT. But tokenize produces it and it appears in tok_name. ---------- components: Library (Lib) messages: 303901 nosy: fpom priority: normal severity: normal status: open title: Missing token.COMMENT type: enhancement versions: Python 2.7, Python 3.4 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun Oct 8 13:08:41 2017 From: report at bugs.python.org (Jonathan) Date: Sun, 08 Oct 2017 17:08:41 +0000 Subject: [New-bugs-announce] [issue31727] FTP_TLS errors when Message-ID: <1507482521.5.0.213398074469.issue31727@psf.upfronthosting.co.za> New submission from Jonathan : Using Python 3.6.3. The below issue only happens with FTP_TLS. It works fine via a plain FTP connection. I am connecting to an FTP to using `ftplib` via FTP_TLS. ftps = ftplib.FTP_TLS('example.com', timeout=5) # TLS is more secure than SSL ftps.ssl_version = ssl.PROTOCOL_TLS # login before securing control channel ftps.login('username at example.com', 'password') # switch to secure data connection ftps.prot_p() # Explicitly set Passive mode ftps.set_pasv(True) This all works. I can send `ftps.mkd('mydir')` (make directory) and `ftps.cwd('mydir')` (change working dir), and both work fine. But if I send **any** of these (they're all basically synonyms as far as I can tell): ftps.dir() ftps.nlst() ftps.retrlines('LIST') ftps.retrlines('MLSD') Then I get back an exception (below also includes all ftplib debug info as well; generally matches up with what FileZilla shows too): *cmd* 'AUTH TLS' *put* 'AUTH TLS\r\n' *get* '234 AUTH TLS OK.\n' *resp* '234 AUTH TLS OK.' *cmd* 'USER username at example.com' *put* 'USER username at example.com\r\n' *get* '331 User username at example.com OK. Password required\n' *resp* '331 User username at example.com OK. Password required' *cmd* 'PASS ******************************' *put* 'PASS ******************************\r\n' *get* '230 OK. Current restricted directory is /\n' *resp* '230 OK. Current restricted directory is /' *cmd* 'PBSZ 0' *put* 'PBSZ 0\r\n' *get* '200 PBSZ=0\n' *resp* '200 PBSZ=0' *cmd* 'PROT P' *put* 'PROT P\r\n' *get* '200 Data protection level set to "private"\n' *resp* '200 Data protection level set to "private"' *cmd* 'MKD mydir' *put* 'MKD mydir\r\n' *get* '257 "mydir" : The directory was successfully created\n' *resp* '257 "mydir" : The directory was successfully created' *cmd* 'CWD mydir' *put* 'CWD mydir\r\n' *get* '250 OK. Current directory is /mydir\n' *resp* '250 OK. Current directory is /mydir' *cmd* 'TYPE A' *put* 'TYPE A\r\n' *get* '200 TYPE is now ASCII\n' *resp* '200 TYPE is now ASCII' *cmd* 'PASV' *put* 'PASV\r\n' *get* '227 Entering Passive Mode (8,8,8,8,8,8)\n' *resp* '227 Entering Passive Mode (8,8,8,8,8,8)' *cmd* 'MLSD' *put* 'MLSD\r\n' *get* '150 Accepted data connection\n' *resp* '150 Accepted data connection' Traceback (most recent call last): File "c:\my_script.py", line 384, in run_ftps ftps.retrlines('MLSD') File "c:\libs\Python36\lib\ftplib.py", line 485, in retrlines conn.unwrap() File "C:\libs\Python36\lib\ssl.py", line 1051, in unwrap s = self._sslobj.unwrap() File "C:\libs\Python36\lib\ssl.py", line 698, in unwrap return self._sslobj.shutdown() OSError: [Errno 0] Error The same FTP command (LIST) works fine via filezilla. The closest thing I can find with googling is this: https://bugs.python.org/msg253161 - and I'm not sure if it's related or relevant. Short version: What does "OSError: [Errno 0] Error" actually mean, and how do I list my directory contents? ---------- messages: 303914 nosy: jonathan-lp priority: normal severity: normal status: open title: FTP_TLS errors when versions: Python 3.6 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun Oct 8 16:16:27 2017 From: report at bugs.python.org (Oren Milman) Date: Sun, 08 Oct 2017 20:16:27 +0000 Subject: [New-bugs-announce] [issue31728] crashes in _elementtree due to unsafe decrefs of Element.text and Element.tail Message-ID: <1507493787.35.0.213398074469.issue31728@psf.upfronthosting.co.za> New submission from Oren Milman : The following code causes the interpreter to crash: import xml.etree.ElementTree class X: def __del__(self): elem.clear() elem = xml.etree.ElementTree.Element('elem') elem.text = X() elem.clear() This is because _elementtree_Element_clear_impl() decrefs self->text in an unsafe manner. For the same reason, but for self->tail, a crash would happen if we replaced 'elem.text = X()' with 'elem.tail = X()'. Similarly, the following code also causes the interpreter to crash: import xml.etree.ElementTree class X: def __del__(self): elem.clear() elem = xml.etree.ElementTree.Element('elem') elem.text = X() elem.text = X() This is because element_text_setter() decrefs self->text in an unsafe manner. element_tail_setter() does the same for self->tail, so again, if we replaced 'elem.text = X()' with 'elem.tail = X()', we would also get a crash. ---------- components: XML messages: 303917 nosy: Oren Milman priority: normal severity: normal status: open title: crashes in _elementtree due to unsafe decrefs of Element.text and Element.tail type: crash versions: Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun Oct 8 16:51:11 2017 From: report at bugs.python.org (gene@nlc.co.nz) Date: Sun, 08 Oct 2017 20:51:11 +0000 Subject: [New-bugs-announce] [issue31729] multiprocesssing.Pool.map_async() undocumented Message-ID: <1507495871.16.0.213398074469.issue31729@psf.upfronthosting.co.za> New submission from gene at nlc.co.nz : To monitor how much of my multiprocess.Pool is completed I am forced to use undocumented features which I fear mat changed in the future. Especially result._number_left which looks like a private variable. Can you please document the result from Pool.map_async() and make the _number_left a proper method, e.g result.number_completed() result = pool.map_async(process_employee, gen_emps, chunksize=1) while not result.ready(): left = result._number_left It is not possible to pass Pipes, Queues or Shared memory to Pool workers in order for them to signal back to the master when them have completed a job. ---------- components: Library (Lib) messages: 303918 nosy: gene at nlc.co.nz priority: normal severity: normal status: open title: multiprocesssing.Pool.map_async() undocumented type: enhancement versions: Python 2.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun Oct 8 16:54:49 2017 From: report at bugs.python.org (gene@nlc.co.nz) Date: Sun, 08 Oct 2017 20:54:49 +0000 Subject: [New-bugs-announce] [issue31730] list unhashable, can not be use as key to dict Message-ID: <1507496089.59.0.213398074469.issue31730@psf.upfronthosting.co.za> New submission from gene at nlc.co.nz : A list can no be used as the key to a dict, apparently because it is "unhashable": TypeError: unhashable type: 'list'. The code must exist to hash object like this a tuple is hashable and can be constructed from a list. ---------- components: Interpreter Core messages: 303919 nosy: gene at nlc.co.nz priority: normal severity: normal status: open title: list unhashable, can not be use as key to dict type: enhancement versions: Python 2.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Oct 9 03:13:25 2017 From: report at bugs.python.org (STINNER Victor) Date: Mon, 09 Oct 2017 07:13:25 +0000 Subject: [New-bugs-announce] [issue31731] test_io hangs on Message-ID: <1507533205.95.0.213398074469.issue31731@psf.upfronthosting.co.za> New submission from STINNER Victor : test_io is running since longer than 5 hours on x86 Gentoo Refleaks 2.7: http://buildbot.python.org/all/#/builders/78/builds/1/steps/4/logs/stdio 2:12:11 load avg: 3.26 [402/403] test_contains passed -- running: test_io (1739 sec) beginning 6 repetitions 123456 ...... running: test_io (1769 sec) running: test_io (1799 sec) (...) running: test_io (19529 sec) http://buildbot.python.org/all/#/builders/78/builds/1/steps/4/logs/stdio ---------- messages: 303924 nosy: haypo priority: normal severity: normal status: open title: test_io hangs on _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Oct 9 08:29:21 2017 From: report at bugs.python.org (STINNER Victor) Date: Mon, 09 Oct 2017 12:29:21 +0000 Subject: [New-bugs-announce] [issue31732] Add TRACE level to the logging module Message-ID: <1507552161.59.0.213398074469.issue31732@psf.upfronthosting.co.za> New submission from STINNER Victor : The logging module has already 5 log levels: CRITICAL, ERROR, WARNING, INFO, DEBUG. https://docs.python.org/dev/library/logging.html#logging-levels (I don't count NOTSET, I don't consider it to be "usable", at least not in an application or a library.) For a large application, sometimes we need to a 6th level since DEBUG might be too verbose for some customers (random value: 1 MiB of log per minute). The OpenStack project has 2 additional logging levels: TRACE and AUDIT. See the Oslo Log project used by all OpenStack projects: https://github.com/openstack/oslo.log/blob/master/oslo_log/handlers.py#L39 The typical usage for TRACE is to log a Python traceback when an exception is logged: logger.debug("Oops: something bad happened! %s", exc) for line in my_exception_formatter(exc): logger.trace(line) In software development, "trace" term is commonly associated to "debug traces", as in "How to get debug traces?". Or more commonly in Python: "traceback" or "stack trace". An additional level helps to filter logs, but also to colorize logs differently. -- I don't propose to add a new AUDIT level since it's not well defined and it's less popular in OpenStack. For example, the Oslo Log module adds a new logger.trace() method, but no logger.audit(). ---------- components: Library (Lib) messages: 303945 nosy: haypo priority: normal severity: normal status: open title: Add TRACE level to the logging module versions: Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Oct 9 09:44:19 2017 From: report at bugs.python.org (STINNER Victor) Date: Mon, 09 Oct 2017 13:44:19 +0000 Subject: [New-bugs-announce] [issue31733] [2.7] Add PYTHONSHOWREFCOUNT environment variable to Python 2.7 Message-ID: <1507556659.49.0.213398074469.issue31733@psf.upfronthosting.co.za> New submission from STINNER Victor : When Python 2.7 is built in debug mode, it displays the total number of references when the program finishes or after each statement in the interactive interpreter. Example: haypo at selma$ ./python Python 2.7.14+ (heads/2.7:cc4b6f1c62, Oct 9 2017, 15:30:11) [GCC 7.2.1 20170915 (Red Hat 7.2.1-2)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> 1+1 2 [45025 refs] >>> 2+2 4 [45025 refs] >>> [45025 refs] [21655 refs] While I'm Python core developer working on reference leaks for years, I *never* used this value. I only use "python -m test -R 3:3 test_xxx" to track reference leaks. I really liked Python 3.4 which made this option an opt-in, python3 -X showrefcount: bpo-17323 (commit 1f8898a5916b942c86ee8716c37d2db388e7bf2f). I propose to change Python 2.7 behaviour to not print this value by default, but add a new PYTHONSHOWREFCOUNT environment variable to display it. The value is displayed if PYTHONSHOWREFCOUNT is set. Other similar existing variables: * PYTHONTHREADDEBUG * PYTHONDUMPREFS * PYTHONMALLOCSTATS https://docs.python.org/2.7/using/cmdline.html#debug-mode-variables I prefer to add a new environment style rather than adding a new -X command line option. IMHO an environment variable is less intrusive. For example, older Python 2.7 versions completely ignore unknown environment variables, whereas "python2.7 -X showrefcount ..." currently logs the "-X is reserved for implementation-specific arguments" message into stderr. Attached PR adds PYTHONSHOWREFCOUNT. ---------- messages: 303952 nosy: haypo priority: normal severity: normal status: open title: [2.7] Add PYTHONSHOWREFCOUNT environment variable to Python 2.7 versions: Python 2.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Oct 9 09:54:05 2017 From: report at bugs.python.org (Oren Milman) Date: Mon, 09 Oct 2017 13:54:05 +0000 Subject: [New-bugs-announce] [issue31734] crash or SystemError in sqlite3.Cache in case it is uninitialized or partially initialized Message-ID: <1507557245.08.0.213398074469.issue31734@psf.upfronthosting.co.za> New submission from Oren Milman : The following code causes a crash: import sqlite3 cache = sqlite3.Cache.__new__(sqlite3.Cache) cache.get(None) This is because pysqlite_cache_get() (in Modules/_sqlite/cache.c) assumes that the Cache object is initialized, and so it passes self->mapping to PyDict_GetItem(), which assumes it is not NULL, and crashes. Also, the following code causes a SystemError ('null argument to internal routine'), as well as refleaks in the deallocation of the Cache object: import sqlite3 cache = sqlite3.Cache(str) try: cache.__init__() except TypeError: pass cache.get(None) This is because pysqlite_cache_init() first sets self->factory to NULL, and only then parses its arguments, so in case it fails to parse the arguments (e.g. due to a wrong number of arguments) we are left with a partially initialized Cache object. While we are here, we should also fix refleaks that occur when sqlite3.Cache.__init__() is called more than once. ---------- components: Extension Modules messages: 303958 nosy: Oren Milman priority: normal severity: normal status: open title: crash or SystemError in sqlite3.Cache in case it is uninitialized or partially initialized type: crash versions: Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Oct 9 11:04:44 2017 From: report at bugs.python.org (Paul Pinterits) Date: Mon, 09 Oct 2017 15:04:44 +0000 Subject: [New-bugs-announce] [issue31735] Documentation incorrectly states how descriptors are invoked Message-ID: <1507561484.86.0.213398074469.issue31735@psf.upfronthosting.co.za> New submission from Paul Pinterits : The [descriptor howto](https://docs.python.org/3/howto/descriptor.html#invoking-descriptors) states: "For example, obj.d looks up d in the dictionary of obj. If d defines the method __get__(), then d.__get__(obj) is invoked [...]" This is not true - the descriptor obtained from obj's dictionary is never invoked. If it was, the following two snippets would produce output: class Class: pass obj = Class() obj.__dict__['d'] = property(lambda: print('called')) _ = obj.d # nothing is printed. class Obj: @property def d(self): print('called') _ = Obj.d # nothing is printed. ---------- assignee: docs at python components: Documentation messages: 303968 nosy: Paul Pinterits, docs at python priority: normal severity: normal status: open title: Documentation incorrectly states how descriptors are invoked versions: Python 2.7, Python 3.4, Python 3.5, Python 3.6, Python 3.7, Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Oct 9 13:11:10 2017 From: report at bugs.python.org (Ben Bolker) Date: Mon, 09 Oct 2017 17:11:10 +0000 Subject: [New-bugs-announce] [issue31736] PEP 485 tiny typo Message-ID: <1507569070.77.0.213398074469.issue31736@psf.upfronthosting.co.za> New submission from Ben Bolker : In two places in https://www.python.org/dev/peps/pep-0485/ floating point numbers are incorrectly formatted: 1-e8 and 1-e9 rather than 1e-8 and 1e-9 ("absolute tolerance default", para. 3 and "relative tolerance default", para. 1) ---------- messages: 303987 nosy: bbolker priority: normal severity: normal status: open title: PEP 485 tiny typo type: enhancement _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Oct 9 13:49:43 2017 From: report at bugs.python.org (Richard Gibson) Date: Mon, 09 Oct 2017 17:49:43 +0000 Subject: [New-bugs-announce] [issue31737] Documentation renders incorrectly Message-ID: <1507571383.25.0.213398074469.issue31737@psf.upfronthosting.co.za> New submission from Richard Gibson : The content at docs.python.org seems to be inserting language-dependent "smart quotes" in code blocks, which mangles backslashes and sequences like `'''`. Observed at https://docs.python.org/3/reference/lexical_analysis.html#string-and-bytes-literals , which renders longstring ::= ???? longstringitem* ???? | ????? longstringitem* ????? instead of longstring ::= "'''" longstringitem* "'''" | '"""' longstringitem* '"""' and stringescapeseq ::= ?" instead of stringescapeseq ::= "\" , and looks even worse in other languages: longstring ::= ? ?? ? longstringitem* ? ?? ? | ? ?? ?? longstringitem* ? ?? ?? longstring ::= ???? longstringitem* ???? | ????? longstringitem* ????? Running `make html` locally produces the desired output, so whatever's going on appears specific to the public site. ---------- assignee: docs at python components: Documentation messages: 303988 nosy: docs at python, gibson042 priority: normal severity: normal status: open title: Documentation renders incorrectly versions: Python 2.7, Python 3.4, Python 3.5, Python 3.6, Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Oct 9 14:06:22 2017 From: report at bugs.python.org (Liel Fridman) Date: Mon, 09 Oct 2017 18:06:22 +0000 Subject: [New-bugs-announce] [issue31738] Lib/site.py: method `abs_paths` is not documented Message-ID: <1507572382.24.0.213398074469.issue31738@psf.upfronthosting.co.za> Change by Liel Fridman : ---------- components: Tests nosy: lielfr priority: normal severity: normal status: open title: Lib/site.py: method `abs_paths` is not documented versions: Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Oct 9 15:27:58 2017 From: report at bugs.python.org (Nathaniel Manista) Date: Mon, 09 Oct 2017 19:27:58 +0000 Subject: [New-bugs-announce] [issue31739] socket.close recommended but not demonstrated in same-page example code Message-ID: <1507577278.77.0.213398074469.issue31739@psf.upfronthosting.co.za> New submission from Nathaniel Manista : https://docs.python.org/3.7/library/socket.html#socket.socket.close says "it is recommended to close() [sockets] explicitly, or to use a with statement around them", but in the example code on the same page at https://docs.python.org/3.7/library/socket.html#example correct use of the .close() method seems to be missing. ---------- assignee: docs at python components: Documentation messages: 303993 nosy: Nathaniel Manista, docs at python priority: normal severity: normal status: open title: socket.close recommended but not demonstrated in same-page example code type: enhancement versions: Python 3.6, Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Oct 9 16:32:17 2017 From: report at bugs.python.org (Oren Milman) Date: Mon, 09 Oct 2017 20:32:17 +0000 Subject: [New-bugs-announce] [issue31740] refleaks when calling sqlite3.Connection.__init__() more than once Message-ID: <1507581137.79.0.213398074469.issue31740@psf.upfronthosting.co.za> New submission from Oren Milman : The following code causes refleaks: import sqlite3 connection = sqlite3.Connection.__new__(sqlite3.Connection) connection.__init__('foo') connection.__init__('foo') This is because pysqlite_connection_init() (in Modules/_sqlite/connection.c) doesn't decref (if needed) before assigning to various fields of `self`. I would open a PR to fix this soon. ---------- components: Extension Modules messages: 303997 nosy: Oren Milman priority: normal severity: normal status: open title: refleaks when calling sqlite3.Connection.__init__() more than once type: resource usage versions: Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Oct 9 17:48:05 2017 From: report at bugs.python.org (Chris Caron) Date: Mon, 09 Oct 2017 21:48:05 +0000 Subject: [New-bugs-announce] [issue31741] backports import path can not be overridden in Windows (Linux works fine) Message-ID: <1507585685.64.0.213398074469.issue31741@psf.upfronthosting.co.za> New submission from Chris Caron : The Windows install of Python 2.7.13 defaults it's install into **C:\Python27**. It creates several subdirectories within this; one of which is **C:\Python27\lib\site-packages\backports**. So just out of the box (without this script installed): ```python import backports print(backports.__path__) # C:\Python27\lib\site-packages\backports ``` I have a custom package that sets the sys.path(0, "new path") entry which works great in Linux. I ship all my modules and settings in this path. This to work great in Windows too 'except' for the reference to `backports`. Consider the block of code: ```python import sys from os import getcwd from os.path import join sys.path.insert(0, join(getcwd(), 'MyLibraries')) # Now if we import backports, we should find the one in our MyLibraries directory import backports print(backports.__path__) # Nope... :( # C:\Python27\lib\site-packages\backports # What's weird is it's not entirely broken... you can do the following: import chardet print(chardet.__path__) # Prints my path to MyLibraries/chardet # So the path is correct and other libraries are coming in correctly ``` What's really weird, is (in Windows) even if i delete the **C:\Python27\lib\site-packages\backports**, it still imports the module; so it's like part of the Python27 build in some way. `pip install` with the backports module I want works great because it drops the correct package in this 'lib/site-packages\backports' directory which is constantly being referenced. >From what I can see the Microsoft version of Python27 can't seem to stop referencing (even when told not to) the _backports_ location. Linux works perfectly (as intended) but having customers port to this isn't an option for me. ---------- components: Windows messages: 304001 nosy: Chris Caron, paul.moore, steve.dower, tim.golden, zach.ware priority: normal severity: normal status: open title: backports import path can not be overridden in Windows (Linux works fine) type: behavior versions: Python 2.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Oct 9 21:47:27 2017 From: report at bugs.python.org (Nick Coghlan) Date: Tue, 10 Oct 2017 01:47:27 +0000 Subject: [New-bugs-announce] [issue31742] Default to emitting FutureWarning for provisional APIs Message-ID: <1507600047.89.0.213398074469.issue31742@psf.upfronthosting.co.za> New submission from Nick Coghlan : Chatting to Donald Stufft and Amber Brown about the way we currently handle provisional APIs, I think they have a legitimate concern that we may not be gathering adequately informed consent from users when they depend on APIs that we've reserved the right to change in the future. So what I'm now wondering is whether or not it may be worth amending PEP 411 to say that all provisional modules should include an import-time snippet like the following: import __main__ if not getattr(__main__, f"use_provisional_{__name__}"): import warnings warnings.warn(FutureWarning, f"The {__name__} module is currently a provisional API. New features might be added and API may change even between minor releases if deemed necessary.") The exact wording of the warning would match the caveat used in that module's API documentation (the variant above comes from the "typing" module) The idea here would be that *application* developers would get a FutureWarning whenever a provisional module was imported, unless they set "use_provisional_" in __main__. Storing the marker attribute in __main__ would be intended to serve two purposes: - it's always going to be imported anyway, so the runtime cost of the new snippet in the provisional modules will be low - it sends a clear indication to library authors that they *shouldn't* be setting the flag implicitly as a side-effect of import (framework authors, by contrast, may decide to set it, since they're more in control of the overall application behaviour than library authors) ---------- messages: 304006 nosy: dstufft, gvanrossum, ncoghlan priority: normal severity: normal stage: needs patch status: open title: Default to emitting FutureWarning for provisional APIs type: enhancement versions: Python 3.6, Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Oct 10 07:00:17 2017 From: report at bugs.python.org (=?utf-8?q?Ante_Peri=C4=87?=) Date: Tue, 10 Oct 2017 11:00:17 +0000 Subject: [New-bugs-announce] [issue31743] Proportional Width Font on Generated Python Docs PDFs Message-ID: <1507633217.5.0.213398074469.issue31743@psf.upfronthosting.co.za> New submission from Ante Peri? : Fonts in the generated documentation PDFs, for all Python versions, are proportional width in code blocks, unlike in HTML version, where they are fixed-width. Example https://docs.python.org/2.7/archives/python-2.7.14-docs-pdf-a4.zip ---------- assignee: docs at python components: Documentation files: Screenshot 2017-10-10 12.41.34.png messages: 304024 nosy: docs at python, synthmeat priority: normal severity: normal status: open title: Proportional Width Font on Generated Python Docs PDFs type: enhancement versions: Python 2.7 Added file: https://bugs.python.org/file47203/Screenshot 2017-10-10 12.41.34.png _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Oct 10 08:17:41 2017 From: report at bugs.python.org (Brian Sidebotham) Date: Tue, 10 Oct 2017 12:17:41 +0000 Subject: [New-bugs-announce] [issue31744] Python 2.7.14 Fails to compile on CentOS/RHEL7 Message-ID: <1507637861.45.0.213398074469.issue31744@psf.upfronthosting.co.za> New submission from Brian Sidebotham : I am getting the following error when trying to compile Python 2.7.14 (and previous 2.7 versions) on CentOS and RHEL7. The main problem is that CONFIG_ARGS does not exist. I've included the build. This RPATH is correct - it has to go through the RPM SPEC file so needs escaping like this to work. Python 3.6.2 builds find like this, but Python 2.7 is failing to build. Have I missed something? LDFLAGS='-Wl,-rpath=$\\$$ORIGIN/../lib' \ ./configure --prefix=/opt/wsl \ --enable-shared \ --with-system-ffi \ --with-ensurepip=yes \ --enable-optimizations ... ranlib libpython2.7.a Modules/posixmodule.o: In function `posix_tmpnam': /opt/jenkins/workspace/Build_WSL_Python2_on_RHEL7/BUILD/Python-2.7.14/./Modules/posixmodule.c:7642: warning: the use of `tmpnam_r' is dangerous, better use `mkstemp' Modules/posixmodule.o: In function `posix_tempnam': /opt/jenkins/workspace/Build_WSL_Python2_on_RHEL7/BUILD/Python-2.7.14/./Modules/posixmodule.c:7589: warning: the use of `tempnam' is dangerous, better use `mkstemp' gcc -pthread -Wl,-rpath=RIGIN/../lib -fprofile-generate -Xlinker -export-dynamic -o python \ Modules/python.o \ -L. -lpython2.7 -lpthread -ldl -lutil -lm LD_LIBRARY_PATH=/opt/jenkins/workspace/Build_WSL_Python2_on_RHEL7/BUILD/Python-2.7.14 ./python -E -S -m sysconfig --generate-posix-vars ;\ if test $? -ne 0 ; then \ echo "generate-posix-vars failed" ; \ rm -f ./pybuilddir.txt ; \ exit 1 ; \ fi Traceback (most recent call last): File "./setup.py", line 33, in COMPILED_WITH_PYDEBUG = ('--with-pydebug' in sysconfig.get_config_var("CONFIG_ARGS")) TypeError: argument of type 'NoneType' is not iterable make[2]: *** [sharedmods] Error 1 make[2]: Leaving directory `/opt/jenkins/workspace/Build_WSL_Python2_on_RHEL7/BUILD/Python-2.7.14' make[1]: *** [build_all_generate_profile] Error 2 make[1]: Leaving directory `/opt/jenkins/workspace/Build_WSL_Python2_on_RHEL7/BUILD/Python-2.7.14' make: *** [profile-opt] Error 2 error: Bad exit status from /var/tmp/rpm-tmp.sFvDid (%build) -bash-4.2# cd build/lib.linux-x86_64-2.7/ -bash-4.2# ls _sysconfigdata.py _sysconfigdata.pyc -bash-4.2# cat _sysconfigdata.py # system configuration generated and used by the sysconfig module build_time_vars = {'AC_APPLE_UNIVERSAL_BUILD': 0, 'AIX_GENUINE_CPLUSPLUS': 0, 'AR': 'ar', 'ARFLAGS': 'rc', 'ATHEOS_THREADS': 0, 'BASECFLAGS': '-fno-strict-aliasing', 'BASEMODLIBS': '', 'BEOS_THREADS': 0, 'BINDIR': '/opt/wsl/bin', 'BINLIBDEST': '/opt/wsl/lib/python2.7', 'BLDLIBRARY': '-L. -lpython2.7', 'BLDSHARED': 'gcc -pthread -shared -Wl,-rpath=RIGIN/../lib -fprofile-generate', 'BUILDEXE': '', 'BUILDPYTHON': 'python', 'CC': 'gcc -pthread', 'CCSHARED': '-fPIC', 'CFLAGS': '-fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes', 'CFLAGSFORSHARED': '-fPIC', 'CONFIGFILES': 'configure configure.ac acconfig.h pyconfig.h.in Makefile.pre.in', 'CONFINCLUDEDIR': '/opt/wsl/include', 'CONFINCLUDEPY': '/opt/wsl/include/python2.7', 'COREPYTHONPATH': ':plat-linux2:lib-tk:lib-old', 'COVERAGE_INFO': '/opt/jenkins/workspace/Build_WSL_Python2_on_RHEL7/BUILD/Python-2.7.14/coverage.info', 'COVERAGE_REPORT': '/opt/jenkins/workspace/Build_WSL_Python2_on_RHEL7/BUILD/Python-2.7.14/lcov-report', 'COVERAGE_REPORT_OPTIONS': '--no-branch-coverage --title "CPython lcov report"', 'CPPFLAGS': '-I. -IInclude -I./Include', 'CXX': 'g++ -pthread', 'C_THREADS': 0, 'DESTDIRS': '/opt/wsl /opt/wsl/lib /opt/wsl/lib/python2.7 /opt/wsl/lib/python2.7/lib-dynload', 'DESTLIB': '/opt/wsl/lib/python2.7', 'DESTPATH': '', 'DESTSHARED': '/opt/wsl/lib/python2.7/lib-dynload', 'DIRMODE': 755, 'DIST': 'README ChangeLog configure configure.ac acconfig.h pyconfig.h.in Makefile.pre.in Include Lib Misc Demo Ext-dummy', 'DISTDIRS': 'Include Lib Misc Demo Ext-dummy', 'DISTFILES': 'README ChangeLog configure configure.ac acconfig.h pyconfig.h.in Makefile.pre.in', 'DLINCLDIR': '.', 'DLLLIBRARY': '', 'DOUBLE_IS_ARM_MIXED_ENDIAN_IEEE754': 0, 'DOUBLE_IS_BIG_ENDIAN_IEEE754': 0, 'DOUBLE_IS_LITTLE_ENDIAN_IEEE754': 1, 'DYNLOADFILE': 'dynload_shlib.o', 'ENABLE_IPV6': 1, 'ENSUREPIP': 'upgrade', 'EXE': '', 'EXEMODE': 755, 'EXTRAMACHDEPPATH': '', 'EXTRAPLATDIR': '', 'EXTRATESTOPTS': '', 'EXTRA_CFLAGS': '', 'FILEMODE': 644, 'FLOCK_NEEDS_LIBBSD': 0, 'GETPGRP_HAVE_ARG': 0, 'GETTIMEOFDAY_NO_TZ': 0, 'GITBRANCH': '', 'GITTAG': '', 'GITVERSION': '', 'GLHACK': '-Dclear=__GLclear', 'GNULD': 'yes', 'HAVE_ACOSH': 1, 'HAVE_ADDRINFO': 1, 'HAVE_ALARM': 1, 'HAVE_ALLOCA_H': 1, 'HAVE_ALTZONE': 0, 'HAVE_ASINH': 1, 'HAVE_ASM_TYPES_H': 1, 'HAVE_ATANH': 1, 'HAVE_ATTRIBUTE_FORMAT_PARSETUPLE': 0, 'HAVE_BIND_TEXTDOMAIN_CODESET': 1, 'HAVE_BLUETOOTH_BLUETOOTH_H': 0, 'HAVE_BLUETOOTH_H': 0, 'HAVE_BROKEN_NICE': 0, 'HAVE_BROKEN_PIPE_BUF': 0, 'HAVE_BROKEN_POLL': 0, 'HAVE_BROKEN_POSIX_SEMAPHORES': 0, 'HAVE_BROKEN_PTHREAD_SIGMASK': 0, 'HAVE_BROKEN_SEM_GETVALUE': 0, 'HAVE_BROKEN_UNSETENV': 0, 'HAVE_C99_BOOL': 1, 'HAVE_CHFLAGS': 0, 'HAVE_CHOWN': 1, 'HAVE_CHROOT': 1, 'HAVE_CLOCK': 1, 'HAVE_COMPUTED_GOTOS': 1, 'HAVE_CONFSTR': 1, 'HAVE_CONIO_H': 0, 'HAVE_COPYSIGN': 1, 'HAVE_CTERMID': 1, 'HAVE_CTERMID_R': 0, 'HAVE_CURSES_H': 1, 'HAVE_CURSES_IS_TERM_RESIZED': 1, 'HAVE_CURSES_RESIZETERM': 1, 'HAVE_CURSES_RESIZE_TERM': 1, 'HAVE_DECL_ISFINITE': 1, 'HAVE_DECL_ISINF': 1, 'HAVE_DECL_ISNAN': 1, 'HAVE_DECL_TZNAME': 0, 'HAVE_DEVICE_MACROS': 1, 'HAVE_DEV_PTC': 0, 'HAVE_DEV_PTMX': 1, 'HAVE_DIRECT_H': 0, 'HAVE_DIRENT_H': 1, 'HAVE_DLFCN_H': 1, 'HAVE_DLOPEN': 1, 'HAVE_DUP2': 1, 'HAVE_DYNAMIC_LOADING': 1, 'HAVE_EPOLL': 1, 'HAVE_ERF': 1, 'HAVE_ERFC': 1, 'HAVE_ERRNO_H': 1, 'HAVE_EXECV': 1, 'HAVE_EXPM1': 1, 'HAVE_FCHDIR': 1, 'HAVE_FCHMOD': 1, 'HAVE_FCHOWN': 1, 'HAVE_FCNTL_H': 1, 'HAVE_FDATASYNC': 1, 'HAVE_FINITE': 1, 'HAVE_FLOCK': 1, 'HAVE_FORK': 1, 'HAVE_FORKPTY': 1, 'HAVE_FPATHCONF': 1, 'HAVE_FSEEK64': 0, 'HAVE_FSEEKO': 1, 'HAVE_FSTATVFS': 1, 'HAVE_FSYNC': 1, 'HAVE_FTELL64': 0, 'HAVE_FTELLO': 1, 'HAVE_FTIME': 1, 'HAVE_FTRUNCATE': 1, 'HAVE_GAI_STRERROR': 1, 'HAVE_GAMMA': 1, 'HAVE_GCC_ASM_FOR_X87': 1, 'HAVE_GETADDRINFO': 1, 'HAVE_GETCWD': 1, 'HAVE_GETC_UNLOCKED': 1, 'HAVE_GETENTROPY': 0, 'HAVE_GETGROUPS': 1, 'HAVE_GETHOSTBYNAME': 0, 'HAVE_GETHOSTBYNAME_R': 1, 'HAVE_GETHOSTBYNAME_R_3_ARG': 0, 'HAVE_GETHOSTBYNAME_R_5_ARG': 0, 'HAVE_GETHOSTBYNAME_R_6_ARG': 1, 'HAVE_GETITIMER': 1, 'HAVE_GETLOADAVG': 1, 'HAVE_GETLOGIN': 1, 'HAVE_GETNAMEINFO': 1, 'HAVE_GETPAGESIZE': 1, 'HAVE_GETPEERNAME': 1, 'HAVE_GETPGID': 1, 'HAVE_GETPGRP': 1, 'HAVE_GETPID': 1, 'HAVE_GETPRIORITY': 1, 'HAVE_GETPWENT': 1, 'HAVE_GETRESGID': 1, 'HAVE_GETRESUID': 1, 'HAVE_GETSID': 1, 'HAVE_GETSPENT': 1, 'HAVE_GETSPNAM': 1, 'HAVE_GETTIMEOFDAY': 1, 'HAVE_GETWD': 1, 'HAVE_GRP_H': 1, 'HAVE_HSTRERROR': 1, 'HAVE_HYPOT': 1, 'HAVE_IEEEFP_H': 0, 'HAVE_INET_ATON': 1, 'HAVE_INET_PTON': 1, 'HAVE_INITGROUPS': 1, 'HAVE_INT32_T': 1, 'HAVE_INT64_T': 1, 'HAVE_INTTYPES_H': 1, 'HAVE_IO_H': 0, 'HAVE_KILL': 1, 'HAVE_KILLPG': 1, 'HAVE_KQUEUE': 0, 'HAVE_LANGINFO_H': 1, 'HAVE_LARGEFILE_SUPPORT': 0, 'HAVE_LCHFLAGS': 0, 'HAVE_LCHMOD': 0, 'HAVE_LCHOWN': 1, 'HAVE_LGAMMA': 1, 'HAVE_LIBDL': 1, 'HAVE_LIBDLD': 0, 'HAVE_LIBIEEE': 0, 'HAVE_LIBINTL_H': 1, 'HAVE_LIBREADLINE': 1, 'HAVE_LIBRESOLV': 0, 'HAVE_LIBUTIL_H': 0, 'HAVE_LINK': 1, 'HAVE_LINUX_NETLINK_H': 1, 'HAVE_LINUX_TIPC_H': 1, 'HAVE_LOG1P': 1, 'HAVE_LONG_DOUBLE': 1, 'HAVE_LONG_LONG': 1, 'HAVE_LSTAT': 1, 'HAVE_MAKEDEV': 1, 'HAVE_MEMMOVE': 1, 'HAVE_MEMORY_H': 1, 'HAVE_MKFIFO': 1, 'HAVE_MKNOD': 1, 'HAVE_MKTIME': 1, 'HAVE_MMAP': 1, 'HAVE_MREMAP': 1, 'HAVE_NCURSES_H': 1, 'HAVE_NDIR_H': 0, 'HAVE_NETPACKET_PACKET_H': 1, 'HAVE_NICE': 1, 'HAVE_OPENPTY': 1, 'HAVE_OSX105_SDK': 0, 'HAVE_PATHCONF': 1, 'HAVE_PAUSE': 1, 'HAVE_PLOCK': 0, 'HAVE_POLL': 1, 'HAVE_POLL_H': 1, 'HAVE_PROCESS_H': 0, 'HAVE_PROTOTYPES': 1, 'HAVE_PTH': 0, 'HAVE_PTHREAD_ATFORK': 1, 'HAVE_PTHREAD_DESTRUCTOR': 0, 'HAVE_PTHREAD_H': 1, 'HAVE_PTHREAD_INIT': 0, 'HAVE_PTHREAD_SIGMASK': 1, 'HAVE_PTY_H': 1, 'HAVE_PUTENV': 1, 'HAVE_READLINK': 1, 'HAVE_REALPATH': 1, 'HAVE_RL_CALLBACK': 1, 'HAVE_RL_CATCH_SIGNAL': 1, 'HAVE_RL_COMPLETION_APPEND_CHARACTER': 1, 'HAVE_RL_COMPLETION_DISPLAY_MATCHES_HOOK': 1, 'HAVE_RL_COMPLETION_MATCHES': 1, 'HAVE_RL_COMPLETION_SUPPRESS_APPEND': 1, 'HAVE_RL_PRE_INPUT_HOOK': 1, 'HAVE_RL_RESIZE_TERMINAL': 1, 'HAVE_ROUND': 1, 'HAVE_SELECT': 1, 'HAVE_SEM_GETVALUE': 1, 'HAVE_SEM_OPEN': 1, 'HAVE_SEM_TIMEDWAIT': 1, 'HAVE_SEM_UNLINK': 1, 'HAVE_SETEGID': 1, 'HAVE_SETEUID': 1, 'HAVE_SETGID': 1, 'HAVE_SETGROUPS': 1, 'HAVE_SETITIMER': 1, 'HAVE_SETLOCALE': 1, 'HAVE_SETPGID': 1, 'HAVE_SETPGRP': 1, 'HAVE_SETREGID': 1, 'HAVE_SETRESGID': 1, 'HAVE_SETRESUID': 1, 'HAVE_SETREUID': 1, 'HAVE_SETSID': 1, 'HAVE_SETUID': 1, 'HAVE_SETVBUF': 1, 'HAVE_SHADOW_H': 1, 'HAVE_SIGACTION': 1, 'HAVE_SIGINTERRUPT': 1, 'HAVE_SIGNAL_H': 1, 'HAVE_SIGRELSE': 1, 'HAVE_SNPRINTF': 1, 'HAVE_SOCKADDR_SA_LEN': 0, 'HAVE_SOCKADDR_STORAGE': 1, 'HAVE_SOCKETPAIR': 1, 'HAVE_SPAWN_H': 1, 'HAVE_SSIZE_T': 1, 'HAVE_STATVFS': 1, 'HAVE_STAT_TV_NSEC': 1, 'HAVE_STAT_TV_NSEC2': 0, 'HAVE_STDARG_PROTOTYPES': 1, 'HAVE_STDINT_H': 1, 'HAVE_STDLIB_H': 1, 'HAVE_STRDUP': 1, 'HAVE_STRFTIME': 1, 'HAVE_STRINGS_H': 1, 'HAVE_STRING_H': 1, 'HAVE_STROPTS_H': 0, 'HAVE_STRUCT_STAT_ST_BIRTHTIME': 0, 'HAVE_STRUCT_STAT_ST_BLKSIZE': 1, 'HAVE_STRUCT_STAT_ST_BLOCKS': 1, 'HAVE_STRUCT_STAT_ST_FLAGS': 0, 'HAVE_STRUCT_STAT_ST_GEN': 0, 'HAVE_STRUCT_STAT_ST_RDEV': 1, 'HAVE_STRUCT_TM_TM_ZONE': 1, 'HAVE_SYMLINK': 1, 'HAVE_SYSCONF': 1, 'HAVE_SYSEXITS_H': 1, 'HAVE_SYS_AUDIOIO_H': 0, 'HAVE_SYS_BSDTTY_H': 0, 'HAVE_SYS_DIR_H': 0, 'HAVE_SYS_EPOLL_H': 1, 'HAVE_SYS_EVENT_H': 0, 'HAVE_SYS_FILE_H': 1, 'HAVE_SYS_LOADAVG_H': 0, 'HAVE_SYS_LOCK_H': 0, 'HAVE_SYS_MKDEV_H': 0, 'HAVE_SYS_MODEM_H': 0, 'HAVE_SYS_NDIR_H': 0, 'HAVE_SYS_PARAM_H': 1, 'HAVE_SYS_POLL_H': 1, 'HAVE_SYS_RANDOM_H': 0, 'HAVE_SYS_RESOURCE_H': 1, 'HAVE_SYS_SELECT_H': 1, 'HAVE_SYS_SOCKET_H': 1, 'HAVE_SYS_STATVFS_H': 1, 'HAVE_SYS_STAT_H': 1, 'HAVE_SYS_TERMIO_H': 0, 'HAVE_SYS_TIMES_H': 1, 'HAVE_SYS_TIME_H': 1, 'HAVE_SYS_TYPES_H': 1, 'HAVE_SYS_UN_H': 1, 'HAVE_SYS_UTSNAME_H': 1, 'HAVE_SYS_WAIT_H': 1, 'HAVE_TCGETPGRP': 1, 'HAVE_TCSETPGRP': 1, 'HAVE_TEMPNAM': 1, 'HAVE_TERMIOS_H': 1, 'HAVE_TERM_H': 1, 'HAVE_TGAMMA': 1, 'HAVE_THREAD_H': 0, 'HAVE_TIMEGM': 1, 'HAVE_TIMES': 1, 'HAVE_TMPFILE': 1, 'HAVE_TMPNAM': 1, 'HAVE_TMPNAM_R': 1, 'HAVE_TM_ZONE': 1, 'HAVE_TRUNCATE': 1, 'HAVE_TZNAME': 0, 'HAVE_UCS4_TCL': 0, 'HAVE_UINT32_T': 1, 'HAVE_UINT64_T': 1, 'HAVE_UINTPTR_T': 1, 'HAVE_UNAME': 1, 'HAVE_UNISTD_H': 1, 'HAVE_UNSETENV': 1, 'HAVE_USABLE_WCHAR_T': 0, 'HAVE_UTIL_H': 0, 'HAVE_UTIMES': 1, 'HAVE_UTIME_H': 1, 'HAVE_WAIT3': 1, 'HAVE_WAIT4': 1, 'HAVE_WAITPID': 1, 'HAVE_WCHAR_H': 1, 'HAVE_WCSCOLL': 1, 'HAVE_WORKING_TZSET': 1, 'HAVE_ZLIB_COPY': 1, 'HAVE__GETPTY': 0, 'HOST_GNU_TYPE': 'x86_64-pc-linux-gnu', 'HURD_C_THREADS': 0, 'INCLDIRSTOMAKE': '/opt/wsl/include /opt/wsl/include /opt/wsl/include/python2.7 /opt/wsl/include/python2.7', 'INCLUDEDIR': '/opt/wsl/include', 'INCLUDEPY': '/opt/wsl/include/python2.7', 'INSTALL': '/usr/bin/install -c', 'INSTALL_DATA': '/usr/bin/install -c -m 644', 'INSTALL_PROGRAM': '/usr/bin/install -c', 'INSTALL_SCRIPT': '/usr/bin/install -c', 'INSTALL_SHARED': '/usr/bin/install -c -m 555', 'INSTSONAME': 'libpython2.7.so.1.0', 'LDCXXSHARED': 'g++ -pthread -shared', 'LDLAST': '', 'LDLIBRARY': 'libpython2.7.so', 'LDLIBRARYDIR': '', 'LDSHARED': 'gcc -pthread -shared -Wl,-rpath=RIGIN/../lib -fprofile-generate', 'LIBC': '', 'LIBDEST': '/opt/wsl/lib/python2.7', 'LIBDIR': '/opt/wsl/lib', 'LIBFFI_INCLUDEDIR': '', 'LIBM': '-lm', 'LIBOBJDIR': 'Python/', 'LIBOBJS': '', 'LIBP': '/opt/wsl/lib/python2.7', 'LIBPC': '/opt/wsl/lib/pkgconfig', 'LIBPL': '/opt/wsl/lib/python2.7/config', 'LIBRARY': 'libpython2.7.a', 'LIBRARY_OBJS': '\\', 'LIBS': '-lpthread -ldl -lutil', 'LIBSUBDIRS': 'lib-tk lib-tk/test lib-tk/test/test_tkinter \\', 'LINKCC': 'gcc -pthread', 'LINKFORSHARED': '-Xlinker -export-dynamic', 'LLVM_PROF_ERR': 'no', 'LLVM_PROF_FILE': '', 'LLVM_PROF_MERGER': 'true', 'LN': 'ln', 'LOCALMODLIBS': '', 'MACHDEP': 'linux2', 'MACHDEPPATH': ':plat-linux2', 'MACHDEPS': 'plat-linux2', 'MACHDEP_OBJS': '', 'MACHDESTLIB': '/opt/wsl/lib/python2.7', 'MACH_C_THREADS': 0, 'MACOSX_DEPLOYMENT_TARGET': '', 'MAINCC': 'gcc -pthread', 'MAJOR_IN_MKDEV': 0, 'MAJOR_IN_SYSMACROS': 0, 'MAKESETUP': './Modules/makesetup', 'MANDIR': '/opt/wsl/share/man', 'MEMTESTOPTS': '-l -x test_subprocess test_io test_lib2to3 \\ -x test_dl test___all__ test_fork1 \\', 'MKDIR_P': '/usr/bin/mkdir -p', 'MODLIBS': '', 'MODOBJS': 'Modules/threadmodule.o Modules/signalmodule.o Modules/posixmodule.o Modules/errnomodule.o Modules/pwdmodule.o Modules/_sre.o Modules/_codecsmodule.o Modules/_weakref.o Modules/zipimport.o Modules/symtablemodule.o Modules/xxsubtype.o', 'MODULE_OBJS': '\\', 'MULTIARCH': '', 'MVWDELCH_IS_EXPRESSION': 1, 'OBJECT_OBJS': '\\', 'OLDPATH': ':lib-old', 'OPT': '-DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes', 'OTHER_LIBTOOL_OPT': '', 'PACKAGE_BUGREPORT': 0, 'PACKAGE_NAME': 0, 'PACKAGE_STRING': 0, 'PACKAGE_TARNAME': 0, 'PACKAGE_URL': 0, 'PACKAGE_VERSION': 0, 'PARSER_HEADERS': '\\', 'PARSER_OBJS': '\\ Parser/myreadline.o Parser/tokenizer.o', 'PGEN': 'Parser/pgen', 'PGENOBJS': '\\ \\', 'PGENSRCS': '\\ \\', 'PGOBJS': '\\', 'PGO_PROF_GEN_FLAG': '-fprofile-generate', 'PGO_PROF_USE_FLAG': '-fprofile-use -fprofile-correction', 'PGSRCS': '\\', 'PLATDIR': 'plat-linux2', 'PLATMACDIRS': 'plat-mac plat-mac/Carbon plat-mac/lib-scriptpackages \\', 'PLATMACPATH': ':plat-mac:plat-mac/lib-scriptpackages', 'POBJS': '\\', 'POSIX_SEMAPHORES_NOT_ENABLED': 0, 'PROFILE_TASK': '-m test.regrtest --pgo -x test_asyncore test_gdb test_multiprocessing test_subprocess', 'PSRCS': '\\', 'PTHREAD_SYSTEM_SCHED_SUPPORTED': 1, 'PURIFY': '', 'PYLONG_BITS_IN_DIGIT': 0, 'PYTHON': 'python', 'PYTHONFRAMEWORK': '', 'PYTHONFRAMEWORKDIR': 'no-framework', 'PYTHONFRAMEWORKINSTALLDIR': '', 'PYTHONFRAMEWORKPREFIX': '', 'PYTHONPATH': ':plat-linux2:lib-tk:lib-old', 'PYTHON_FOR_BUILD': './python -E', 'PYTHON_FOR_REGEN': 'python2.7', 'PYTHON_HEADERS': '\\', 'PYTHON_OBJS': '\\', 'PY_CFLAGS': '-fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I. -IInclude -I./Include -fPIC -DPy_BUILD_CORE', 'PY_FORMAT_LONG_LONG': '"ll"', 'PY_FORMAT_SIZE_T': '"z"', 'PY_UNICODE_TYPE': 'unsigned short', 'Py_DEBUG': 0, 'Py_ENABLE_SHARED': 1, 'Py_UNICODE_SIZE': 2, 'Py_USING_UNICODE': 1, 'QUICKTESTOPTS': '-l -x test_subprocess test_io test_lib2to3 \\', 'RANLIB': 'ranlib', 'RESSRCDIR': 'Mac/Resources/framework', 'RETSIGTYPE': 'void', 'RUNSHARED': 'LD_LIBRARY_PATH=/opt/jenkins/workspace/Build_WSL_Python2_on_RHEL7/BUILD/Python-2.7.14', 'SCRIPTDIR': '/opt/wsl/lib', 'SETPGRP_HAVE_ARG': 0, 'SGI_ABI': '', 'SHELL': '/bin/sh', 'SHLIBS': '-lpthread -ldl -lutil', 'SHLIB_EXT': '".so"', 'SIGNAL_OBJS': '', 'SIGNED_RIGHT_SHIFT_ZERO_FILLS': 0, 'SITEPATH': '', 'SIZEOF_DOUBLE': 8, 'SIZEOF_FLOAT': 4, 'SIZEOF_FPOS_T': 16, 'SIZEOF_INT': 4, 'SIZEOF_LONG': 8, 'SIZEOF_LONG_DOUBLE': 16, 'SIZEOF_LONG_LONG': 8, 'SIZEOF_OFF_T': 8, 'SIZEOF_PID_T': 4, 'SIZEOF_PTHREAD_T': 8, 'SIZEOF_SHORT': 2, 'SIZEOF_SIZE_T': 8, 'SIZEOF_TIME_T': 8, 'SIZEOF_UINTPTR_T': 8, 'SIZEOF_VOID_P': 8, 'SIZEOF_WCHAR_T': 4, 'SIZEOF__BOOL': 1, 'SO': '.so', 'SRCDIRS': 'Parser Objects Python Modules', 'SRC_GDB_HOOKS': './Tools/gdb/libpython.py', 'STDC_HEADERS': 1, 'STRICT_SYSV_CURSES': "/* Don't use ncurses extensions */", 'STRINGLIB_HEADERS': '\\', 'SUBDIRS': '', 'SUBDIRSTOO': 'Include Lib Misc Demo', 'SYSLIBS': '-lm', 'SYS_SELECT_WITH_SYS_TIME': 1, 'TANH_PRESERVES_ZERO_SIGN': 1, 'TCLTK_INCLUDES': '', 'TCLTK_LIBS': '', 'TESTOPTS': '-l', 'TESTPATH': '', 'TESTPROG': './Lib/test/regrtest.py', 'TESTPYTHON': 'LD_LIBRARY_PATH=/opt/jenkins/workspace/Build_WSL_Python2_on_RHEL7/BUILD/Python-2.7.14 ./python -Wd -3 -E -tt', 'TESTPYTHONOPTS': '', 'THREADOBJ': 'Python/thread.o', 'TIME_WITH_SYS_TIME': 1, 'TKPATH': ':lib-tk', 'TM_IN_SYS_TIME': 0, 'UNICODE_OBJS': 'Objects/unicodeobject.o Objects/unicodectype.o', 'UNIVERSALSDK': '', 'USE_COMPUTED_GOTOS': 0, 'USE_TOOLBOX_OBJECT_GLUE': 0, 'VA_LIST_IS_ARRAY': 1, 'VERSION': '2.7', 'WANT_SIGFPE_HANDLER': 0, 'WANT_WCTYPE_FUNCTIONS': 0, 'WINDOW_HAS_FLAGS': 1, 'WITH_DOC_STRINGS': 1, 'WITH_DYLD': 0, 'WITH_LIBINTL': 0, 'WITH_NEXT_FRAMEWORK': 0, 'WITH_PYMALLOC': 1, 'WITH_THREAD': 1, 'WITH_TSC': 0, 'WITH_VALGRIND': 0, 'X87_DOUBLE_ROUNDING': 0, 'XMLLIBSUBDIRS': 'xml xml/dom xml/etree xml/parsers xml/sax', 'abs_builddir': '/opt/jenkins/workspace/Build_WSL_Python2_on_RHEL7/BUILD/Python-2.7.14', 'abs_srcdir': '/opt/jenkins/workspace/Build_WSL_Python2_on_RHEL7/BUILD/Python-2.7.14', 'build': 'x86_64-pc-linux-gnu', 'datarootdir': '/opt/wsl/share', 'exec_prefix': '/opt/wsl', 'host': 'x86_64-pc-linux-gnu', 'prefix': '/opt/wsl', 'srcdir': '.'} ---------- components: Build messages: 304025 nosy: Brian Sidebotham priority: normal severity: normal status: open title: Python 2.7.14 Fails to compile on CentOS/RHEL7 type: compile error versions: Python 2.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Oct 10 10:08:15 2017 From: report at bugs.python.org (Kay Hayen) Date: Tue, 10 Oct 2017 14:08:15 +0000 Subject: [New-bugs-announce] [issue31745] Overloading "Py_GetPath" does not work Message-ID: <1507644495.71.0.213398074469.issue31745@psf.upfronthosting.co.za> New submission from Kay Hayen : Hello, for my Python compiler Nuitka, I want to make sure that compiled binaries in standalone mode do not access the original installation, but solely the distribution folder created. I am using the new API Py_SetPath on Python3 and it works fine. The Python2.7 documentation of the C-API however says: The embedding application can steer the search by calling Py_SetProgramName(file) before calling Py_Initialize(). Note that PYTHONHOME still overrides this and PYTHONPATH is still inserted in front of the standard path. An application that requires total control has to provide its own implementation of Py_GetPath(), Py_GetPrefix(), Py_GetExecPrefix(), and Py_GetProgramFullPath() (all defined in Modules/getpath.c). My attempts at overloading this have badly failed, because of conflicting "declspec" (dllimport vs dllexport). And when defining Py_GetPath away before making the include, so the conflict is not seen by the MSVC compiler, the result is that the function is not called. Is this known to work? Can you point me to a working example, because my searches didn't reveal anything. Not normal web search, nor global code search for Py_GetPath code. I did not try on Linux yet. I am assuming it might work, I just wanted to tap on knowledge on your side. Is it feasible. For Linux? For Windows? Thanks, Kay Hayen ---------- components: Build messages: 304045 nosy: kayhayen priority: normal severity: normal status: open title: Overloading "Py_GetPath" does not work type: behavior versions: Python 2.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Oct 10 10:31:19 2017 From: report at bugs.python.org (Oren Milman) Date: Tue, 10 Oct 2017 14:31:19 +0000 Subject: [New-bugs-announce] [issue31746] crashes in sqlite3.Connection in case it is uninitialized or partially initialized Message-ID: <1507645879.63.0.213398074469.issue31746@psf.upfronthosting.co.za> New submission from Oren Milman : The following code causes a crash: import sqlite3 connection = sqlite3.Connection.__new__(sqlite3.Connection) connection.isolation_level This is because pysqlite_connection_get_isolation_level() doesn't check whether the Connection object is initialized. pysqlite_connection_close() also doesn't check that, so we would get a crash also if we replaced `connection.isolation_level` with `connection.close()`. pysqlite_connection_set_isolation_level() doesn't crash in case of an uninitialized Connection object, but it also doesn't raise an error, and IMHO it should. The following code causes a crash, too: import sqlite3 try: connection = sqlite3.Connection.__new__(sqlite3.Connection) connection.__init__('', isolation_level='invalid isolation level') except ValueError: pass connection.cursor() This is because `self->initialized` is set to 1 in the beginning of pysqlite_connection_init(), so after it fails, we are left with a partially initialized Connection object whose `self->initialized` is 1. Thus, pysqlite_connection_cursor() thinks that the Connection object is initialized. Eventually pysqlite_connection_register_cursor() is called, and it crashes while trying to append to `connection->cursors`, which is NULL. ---------- components: Extension Modules messages: 304047 nosy: Oren Milman priority: normal severity: normal status: open title: crashes in sqlite3.Connection in case it is uninitialized or partially initialized type: crash versions: Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Oct 10 11:02:04 2017 From: report at bugs.python.org (Denis Osipov) Date: Tue, 10 Oct 2017 15:02:04 +0000 Subject: [New-bugs-announce] [issue31747] fatal error LNK1120 in PCbuild\python3dll.vcxproj Message-ID: <1507647724.94.0.213398074469.issue31747@psf.upfronthosting.co.za> New submission from Denis Osipov : Since today (2017-10-10) I have compile error on Windows 10: $ PCbuild/build.bat -e -d -p x64 Using py -3.6 (found with py.exe) Fetching external libraries... bzip2-1.0.6 already exists, skipping. sqlite-3.14.2.0 already exists, skipping. xz-5.2.2 already exists, skipping. zlib-1.2.11 already exists, skipping. Fetching external binaries... openssl-bin-1.1.0f already exists, skipping. tcltk-8.6.6.0 already exists, skipping. Finished. Using "C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\\MSBuild\15.0\Bin\msbuild.exe" (found in the Visual Studio 2017 registry) D:\repos\cpython>"C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\\MSBuild\15.0\Bin\msbuild.exe" "D:\repos\cpython\PCbuild\pcbuild.proj" /t:Build /m /nologo /v:m /p:Configuration=Debug /p:Platform=x64 /p:IncludeExternals=true /p:IncludeSSL=true /p:IncludeTkinter=true /p:UseTestMarker= /p:GIT="C:\Program Files\Git\mingw64\bin\git.exe" Killing any running python_d.exe instances... Getting build info from "C:\Program Files\Git\mingw64\bin\git.exe" Building heads/master:a997c7b434 master pythoncore.vcxproj -> D:\repos\cpython\PCbuild\amd64\python37_d.dll pythoncore.vcxproj -> D:\repos\cpython\PCbuild\amd64\python37_d.pdb (Full PDB) _ctypes_test.vcxproj -> D:\repos\cpython\PCbuild\amd64\_ctypes_test_d.pyd _ctypes_test.vcxproj -> D:\repos\cpython\PCbuild\amd64\_ctypes_test_d.pdb (Full PDB) _testbuffer.vcxproj -> D:\repos\cpython\PCbuild\amd64\_testbuffer_d.pyd _testbuffer.vcxproj -> D:\repos\cpython\PCbuild\amd64\_testbuffer_d.pdb (Full PDB) _testcapi.vcxproj -> D:\repos\cpython\PCbuild\amd64\_testcapi_d.pyd _testcapi.vcxproj -> D:\repos\cpython\PCbuild\amd64\_testcapi_d.pdb (Full PDB) _testembed.vcxproj -> D:\repos\cpython\PCbuild\amd64\_testembed_d.exe _testembed.vcxproj -> D:\repos\cpython\PCbuild\amd64\_testembed_d.pdb (Full PDB) _testimportmultiple.vcxproj -> D:\repos\cpython\PCbuild\amd64\_testimportmultiple_d.pyd _testimportmultiple.vcxproj -> D:\repos\cpython\PCbuild\amd64\_testimportmultiple_d.pdb (Full PDB) _testmultiphase.vcxproj -> D:\repos\cpython\PCbuild\amd64\_testmultiphase_d.pyd _testmultiphase.vcxproj -> D:\repos\cpython\PCbuild\amd64\_testmultiphase_d.pdb (Full PDB) _testconsole.vcxproj -> D:\repos\cpython\PCbuild\amd64\_testconsole_d.pyd _testconsole.vcxproj -> D:\repos\cpython\PCbuild\amd64\_testconsole_d.pdb (Full PDB) pylauncher.vcxproj -> D:\repos\cpython\PCbuild\amd64\py_d.exe pylauncher.vcxproj -> D:\repos\cpython\PCbuild\amd64\py_d.pdb (Full PDB) pywlauncher.vcxproj -> D:\repos\cpython\PCbuild\amd64\pyw_d.exe pywlauncher.vcxproj -> D:\repos\cpython\PCbuild\amd64\pyw_d.pdb (Full PDB) pyshellext.vcxproj -> D:\repos\cpython\PCbuild\amd64\pyshellext_d.dll pyshellext.vcxproj -> D:\repos\cpython\PCbuild\amd64\pyshellext_d.pdb (Full PDB) ????????? ?????????? D:\repos\cpython\PCbuild\amd64\python3_dstub.lib ? ?????? D:\repos\cpython\PCbuild\amd64\python3_dstub.exp _asyncio.vcxproj -> D:\repos\cpython\PCbuild\amd64\_asyncio_d.pyd _asyncio.vcxproj -> D:\repos\cpython\PCbuild\amd64\_asyncio_d.pdb (Full PDB) _ctypes.vcxproj -> D:\repos\cpython\PCbuild\amd64\_ctypes_d.pyd _ctypes.vcxproj -> D:\repos\cpython\PCbuild\amd64\_ctypes_d.pdb (Full PDB) _decimal.vcxproj -> D:\repos\cpython\PCbuild\amd64\_decimal_d.pyd _decimal.vcxproj -> D:\repos\cpython\PCbuild\amd64\_decimal_d.pdb (Full PDB) LINK : ?? ?????? ??? ?? ????????? ?????? D:\repos\cpython\PCbuild\amd64\python3_d.dll ??? ????????? ???????????? ??????????; ??????????? ?????? ?????????? _elementtree.vcxproj -> D:\repos\cpython\PCbuild\amd64\_elementtree_d.pyd _elementtree.vcxproj -> D:\repos\cpython\PCbuild\amd64\_elementtree_d.pdb (Full PDB) python3_d.def : error LNK2001: ????????????? ??????? ?????? "PyThread_tss_alloc" [D:\repos\cpython\PCbuild\python3dll.vcxproj] python3_d.def : error LNK2001: ????????????? ??????? ?????? "PyThread_tss_create" [D:\repos\cpython\PCbuild\python3dll.vcxproj] python3_d.def : error LNK2001: ????????????? ??????? ?????? "PyThread_tss_delete" [D:\repos\cpython\PCbuild\python3dll.vcxproj] python3_d.def : error LNK2001: ????????????? ??????? ?????? "PyThread_tss_free" [D:\repos\cpython\PCbuild\python3dll.vcxproj] python3_d.def : error LNK2001: ????????????? ??????? ?????? "PyThread_tss_get" [D:\repos\cpython\PCbuild\python3dll.vcxproj] python3_d.def : error LNK2001: ????????????? ??????? ?????? "PyThread_tss_is_created" [D:\repos\cpython\PCbuild\python3dll.vcxproj] python3_d.def : error LNK2001: ????????????? ??????? ?????? "PyThread_tss_set" [D:\repos\cpython\PCbuild\python3dll.vcxproj] D:\repos\cpython\PCbuild\amd64\python3_d.lib : fatal error LNK1120: ????????????? ??????? ?????????: 7 [D:\repos\cpython\PCbuild\python3dll.vcxproj] _multiprocessing.vcxproj -> D:\repos\cpython\PCbuild\amd64\_multiprocessing_d.pyd _multiprocessing.vcxproj -> D:\repos\cpython\PCbuild\amd64\_multiprocessing_d.pdb (Full PDB) _msi.vcxproj -> D:\repos\cpython\PCbuild\amd64\_msi_d.pyd _msi.vcxproj -> D:\repos\cpython\PCbuild\amd64\_msi_d.pdb (Full PDB) pyexpat.vcxproj -> D:\repos\cpython\PCbuild\amd64\pyexpat_d.pyd pyexpat.vcxproj -> D:\repos\cpython\PCbuild\amd64\pyexpat_d.pdb (Full PDB) select.vcxproj -> D:\repos\cpython\PCbuild\amd64\select_d.pyd select.vcxproj -> D:\repos\cpython\PCbuild\amd64\select_d.pdb (Full PDB) unicodedata.vcxproj -> D:\repos\cpython\PCbuild\amd64\unicodedata_d.pyd unicodedata.vcxproj -> D:\repos\cpython\PCbuild\amd64\unicodedata_d.pdb (Full PDB) winsound.vcxproj -> D:\repos\cpython\PCbuild\amd64\winsound_d.pyd winsound.vcxproj -> D:\repos\cpython\PCbuild\amd64\winsound_d.pdb (Full PDB) _bz2.vcxproj -> D:\repos\cpython\PCbuild\amd64\_bz2_d.pyd _bz2.vcxproj -> D:\repos\cpython\PCbuild\amd64\_bz2_d.pdb (Full PDB) _overlapped.vcxproj -> D:\repos\cpython\PCbuild\amd64\_overlapped_d.pyd _overlapped.vcxproj -> D:\repos\cpython\PCbuild\amd64\_overlapped_d.pdb (Full PDB) liblzma.vcxproj -> D:\repos\cpython\PCbuild\amd64\liblzma_d.lib sqlite3.vcxproj -> D:\repos\cpython\PCbuild\amd64\sqlite3_d.dll sqlite3.vcxproj -> D:\repos\cpython\PCbuild\amd64\sqlite3_d.pdb (Full PDB) _socket.vcxproj -> D:\repos\cpython\PCbuild\amd64\_socket_d.pyd _socket.vcxproj -> D:\repos\cpython\PCbuild\amd64\_socket_d.pdb (Full PDB) _hashlib.vcxproj -> D:\repos\cpython\PCbuild\amd64\_hashlib_d.pyd _hashlib.vcxproj -> D:\repos\cpython\PCbuild\amd64\_hashlib_d.pdb (Full PDB) _sqlite3.vcxproj -> D:\repos\cpython\PCbuild\amd64\_sqlite3_d.pyd _sqlite3.vcxproj -> D:\repos\cpython\PCbuild\amd64\_sqlite3_d.pdb (Full PDB) _lzma.vcxproj -> D:\repos\cpython\PCbuild\amd64\_lzma_d.pyd _lzma.vcxproj -> D:\repos\cpython\PCbuild\amd64\_lzma_d.pdb (Full PDB) _ssl.vcxproj -> D:\repos\cpython\PCbuild\amd64\_ssl_d.pyd _ssl.vcxproj -> D:\repos\cpython\PCbuild\amd64\_ssl_d.pdb (Full PDB) _tkinter.vcxproj -> D:\repos\cpython\PCbuild\amd64\_tkinter_d.pyd _tkinter.vcxproj -> D:\repos\cpython\PCbuild\amd64\_tkinter_d.pdb (Full PDB) ---------- components: Build messages: 304050 nosy: denis-osipov priority: normal severity: normal status: open title: fatal error LNK1120 in PCbuild\python3dll.vcxproj type: compile error versions: Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Oct 10 13:05:50 2017 From: report at bugs.python.org (=?utf-8?b?0JTQuNC70Y/QvSDQn9Cw0LvQsNGD0LfQvtCy?=) Date: Tue, 10 Oct 2017 17:05:50 +0000 Subject: [New-bugs-announce] [issue31748] Modules/posixmodule.c: skip compiling jka4NaPmmQ37 and posix_fildes_fd Message-ID: <1507655150.66.0.213398074469.issue31748@psf.upfronthosting.co.za> New submission from ????? ???????? : These are needed only sometimes. diff --git a/Modules/posixmodule.c b/Modules/posixmodule.c --- a/Modules/posixmodule.c +++ b/Modules/posixmodule.c @@ -1466,6 +1466,7 @@ path_error2(path_t *path, path_t *path2) /* POSIX generic methods */ +#if defined(HAVE_FSYNC) || defined(HAVE_FDATASYNC) || defined(HAVE_FCHDIR) static int fildes_converter(PyObject *o, void *p) { @@ -1495,6 +1496,7 @@ posix_fildes_fd(int fd, int (*func)(int)) return (!async_err) ? posix_error() : NULL; Py_RETURN_NONE; } +#endif #ifdef MS_WINDOWS ---------- components: Build messages: 304059 nosy: dilyan.palauzov priority: normal severity: normal status: open title: Modules/posixmodule.c: skip compiling jka4NaPmmQ37 and posix_fildes_fd type: enhancement versions: Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Oct 10 13:10:53 2017 From: report at bugs.python.org (Rich) Date: Tue, 10 Oct 2017 17:10:53 +0000 Subject: [New-bugs-announce] [issue31749] Request: Human readable byte amounts in the standard library Message-ID: <1507655453.9.0.213398074469.issue31749@psf.upfronthosting.co.za> New submission from Rich : This problem is an _extremely_ common one, a problem that almost any Python project of a reasonable size will encounter. Given a number of bytes, say 123901842, format this as '123.9MB'. The reason I strongly think that this should be included in the standard library is that if you look for how to do this through a Google search, there are an incredible amount of different solutions on StackOverflow, blog posts, forum posts, and various different libraries which provide this functionality - with varying levels of functionality and safety. You can also find different implementations of solutions to this problem inside of pretty much every major Python project (Django, etc.). In fact, I don't think I can think of any other function that gets copy-pasted into a project's 'util.py' file more commonly. I think this should functionality should be provided in the standard math package, with arguments which allow to specific SI/NIST formatting and the number of significant digits to display. Implementing this would strongly cut down on the amount of cargo-cult Python programming in the world. I'm willing to implement this if there's a consensus that it should be included. Thanks!, Rich Jones ---------- messages: 304061 nosy: miserlou2 priority: normal severity: normal status: open title: Request: Human readable byte amounts in the standard library type: enhancement versions: Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Oct 10 13:46:02 2017 From: report at bugs.python.org (Devin Bayer) Date: Tue, 10 Oct 2017 17:46:02 +0000 Subject: [New-bugs-announce] [issue31750] expose PyCell_Type in types module Message-ID: <1507657562.56.0.213398074469.issue31750@psf.upfronthosting.co.za> New submission from Devin Bayer : The type of PyCell_Type is not available in the types module or anywhere else. Since this type is exposed to users it seems like it's a rather odd omission. Related issues: - https://bugs.python.org/issue2408 - https://bugs.python.org/issue1605 ---------- components: Library (Lib) messages: 304062 nosy: akvadrako priority: normal severity: normal status: open title: expose PyCell_Type in types module type: enhancement versions: Python 3.6 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Oct 10 14:51:16 2017 From: report at bugs.python.org (steven Michalske) Date: Tue, 10 Oct 2017 18:51:16 +0000 Subject: [New-bugs-announce] [issue31751] Support for C++ 11 and/or C++ 14 in python.org installer Message-ID: <1507661476.93.0.213398074469.issue31751@psf.upfronthosting.co.za> New submission from steven Michalske : We are using some compiled cython modules with c++ 11 features. This prohibits us from instructing users to install python from the python.org download. Please consider using clang with support for C++ 11 and higher. This would move the minimum OS X version away from 10.6 but as 10.6 is 8 years old and phased out 6 years ago it should be rare that non technical users would be using this OS version. ---------- components: Build, Installation, macOS messages: 304068 nosy: hardkrash, ned.deily, ronaldoussoren priority: normal severity: normal status: open title: Support for C++ 11 and/or C++ 14 in python.org installer type: enhancement versions: Python 3.6, Python 3.7, Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Oct 10 17:29:28 2017 From: report at bugs.python.org (Serhiy Storchaka) Date: Tue, 10 Oct 2017 21:29:28 +0000 Subject: [New-bugs-announce] [issue31752] Assertion failure in timedelta() in case of bad __divmod__ Message-ID: <1507670968.8.0.213398074469.issue31752@psf.upfronthosting.co.za> New submission from Serhiy Storchaka : The following code causes an assertion error in timedelta constructor. from datetime import timedelta class BadInt(int): def __mul__(self, other): return Prod() class Prod: def __radd__(self, other): return Sum() class Sum: def __divmod__(self, other): return (0, -1) timedelta(hours=BadInt(1)) Result: python: /home/serhiy/py/cpython/Modules/_datetimemodule.c:1573: microseconds_to_delta_ex: Assertion `0 <= temp && temp < 1000000' failed. Aborted (core dumped) ---------- components: Extension Modules messages: 304083 nosy: Oren Milman, belopolsky, serhiy.storchaka priority: normal severity: normal stage: patch review status: open title: Assertion failure in timedelta() in case of bad __divmod__ type: crash versions: Python 2.7, Python 3.6, Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Oct 10 18:02:07 2017 From: report at bugs.python.org (Aaron Hall) Date: Tue, 10 Oct 2017 22:02:07 +0000 Subject: [New-bugs-announce] [issue31753] Unnecessary closure in ast.literal_eval Message-ID: <1507672927.75.0.213398074469.issue31753@psf.upfronthosting.co.za> New submission from Aaron Hall : Removing the closure seems to make the function about 10% faster. Original source code at: https://github.com/python/cpython/blob/3.6/Lib/ast.py#L40 Empirical evidence: astle.py import timeit from ast import literal_eval as orig_literal_eval from ast import * def new_literal_eval(node_or_string): """ Safely evaluate an expression node or a string containing a Python expression. The string or node provided may only consist of the following Python literal structures: strings, bytes, numbers, tuples, lists, dicts, sets, booleans, and None. """ if isinstance(node_or_string, str): node_or_string = parse(node_or_string, mode='eval') if isinstance(node_or_string, Expression): node_or_string = node_or_string.body node = node_or_string if isinstance(node, Constant): return node.value elif isinstance(node, (Str, Bytes)): return node.s elif isinstance(node, Num): return node.n elif isinstance(node, Tuple): return tuple(map(_convert, node.elts)) elif isinstance(node, List): return list(map(_convert, node.elts)) elif isinstance(node, Set): return set(map(_convert, node.elts)) elif isinstance(node, Dict): return dict((_convert(k), _convert(v)) for k, v in zip(node.keys, node.values)) elif isinstance(node, NameConstant): return node.value elif isinstance(node, UnaryOp) and isinstance(node.op, (UAdd, USub)): operand = _convert(node.operand) if isinstance(operand, _NUM_TYPES): if isinstance(node.op, UAdd): return + operand else: return - operand elif isinstance(node, BinOp) and isinstance(node.op, (Add, Sub)): left = _convert(node.left) right = _convert(node.right) if isinstance(left, _NUM_TYPES) and isinstance(right, _NUM_TYPES): if isinstance(node.op, Add): return left + right else: return left - right raise ValueError('malformed node or string: ' + repr(node)) def main(): print('orig first, then new') print("'1.01'") print(min(timeit.repeat(lambda: orig_literal_eval('1.01')))) print(min(timeit.repeat(lambda: new_literal_eval('1.01')))) print("""'"1.01"'""") print(min(timeit.repeat(lambda: orig_literal_eval('"1.01"')))) print(min(timeit.repeat(lambda: new_literal_eval('"1.01"')))) print("'1'") print(min(timeit.repeat(lambda: orig_literal_eval('1')))) print(min(timeit.repeat(lambda: new_literal_eval('1')))) if __name__ == '__main__': main() Shell: $ python -m astle orig first, then new '1.01' 3.518230145502848 3.274753015923377 '"1.01"' 3.189016693752965 2.906869704238048 '1' 3.40557457956146 3.157061471625788 ---------- components: Library (Lib) messages: 304089 nosy: Aaron Hall priority: normal severity: normal status: open title: Unnecessary closure in ast.literal_eval type: performance _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Oct 10 20:18:47 2017 From: report at bugs.python.org (Robert Snoeberger) Date: Wed, 11 Oct 2017 00:18:47 +0000 Subject: [New-bugs-announce] [issue31754] Documented type of parameter 'itemsize' to PyBuffer_FillContiguousStrides is incorrect. Message-ID: <1507681127.43.0.213398074469.issue31754@psf.upfronthosting.co.za> New submission from Robert Snoeberger : The signature for PyBuffer_FillContiguousStrides in the documentation shows that the type of parameter 'itemsize' is Py_ssize_t [1]. This is different from the signature in Include/abstract.h which shows that the type as int [2]. [1] https://docs.python.org/3/c-api/buffer.html#c.PyBuffer_FillContiguousStrides [2] https://github.com/python/cpython/blob/49b2734bf12dc1cda80fd73d3ec8896ae3e362f2/Include/abstract.h#L559-L563 ---------- assignee: docs at python components: Documentation messages: 304096 nosy: docs at python, snoeberger priority: normal severity: normal status: open title: Documented type of parameter 'itemsize' to PyBuffer_FillContiguousStrides is incorrect. versions: Python 3.6 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed Oct 11 03:45:29 2017 From: report at bugs.python.org (Kenji Asuka (Asuka Kenji)) Date: Wed, 11 Oct 2017 07:45:29 +0000 Subject: [New-bugs-announce] [issue31755] SetType is missing in the 'types' module Message-ID: <1507707929.59.0.213398074469.issue31755@psf.upfronthosting.co.za> New submission from Kenji Asuka (Asuka Kenji) : # Create a dict and a set d = {1: '1'} s = {1} # They have different types print(type(d)) # print(type(s)) # print(type(d) is type(s)) # False print(type(s) is type(d)) # False print(type(d) == type(s)) # False # Dictionary type is found in 'types' module print(type(d) is types.DictType) # True print(type(d) is types.DictionaryType) # True print(types.DictType == types.DictionaryType) # True # Set type is not found in 'types' module print(type(s) is types.DictType) # False print(type(s) is types.DictionaryType) # False print(type(s) is types.DictProxyType) # False print(type(s) is types.ListType) # False print(type(s) is types.SliceType) # False print(type(s) is types.TupleType) # False print(type(s) is types.NotImplementedType) # False print(type(s) is types.SetType) # AttributeError: 'module' object has no attribute 'SetType' ---------- components: Library (Lib) messages: 304112 nosy: Kenji Asuka (Asuka Kenji) priority: normal severity: normal status: open title: SetType is missing in the 'types' module type: compile error versions: Python 2.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed Oct 11 06:16:51 2017 From: report at bugs.python.org (Andrew Clegg) Date: Wed, 11 Oct 2017 10:16:51 +0000 Subject: [New-bugs-announce] [issue31756] subprocess.run should alias universal_newlines to text Message-ID: <1507717011.91.0.213398074469.issue31756@psf.upfronthosting.co.za> New submission from Andrew Clegg : Following on from https://bugs.python.org/issue6135 The subprocess module by default returns bytes from subprocess calls. It has a text mode, but this can only be accessed by slightly tangential arguments (setting encoding, errors or universal_newlines). ncoghlan notes in msg304118 that this is a similar situation to the binary/text mode settings for open(). From the docs " In text mode, if encoding is not specified the encoding used is platform dependent: locale.getpreferredencoding(False) is called to get the current locale encoding" The universal_newlines argument now effectively just turns on the text mode, however this is not an intuitively and obviously discoverable. So to improve usability, and to mirror the file opening behaviour, subprocess calls should be *explicitly* dual mode (binary/text), and have an explicitly named argument to control this. My enhancement suggestion is as follows: * Add a text=True/False argument that is an alias of universal_newlines, to improve the API. * Clearly document that this implies that the encoding will be guessed, and that an explicit encoding can be given if the guess is wrong For completeness, the following changes could also be made, although these may be controversial * Remove/deprecate the universal_newlines argument * Revert the default mode to text (as in Python 2.7), and have a binary=True argument instead ---------- components: Library (Lib) messages: 304125 nosy: andrewclegg, ncoghlan, steve.dower priority: normal severity: normal status: open title: subprocess.run should alias universal_newlines to text type: enhancement versions: Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed Oct 11 10:40:19 2017 From: report at bugs.python.org (Heinrich Schnermann) Date: Wed, 11 Oct 2017 14:40:19 +0000 Subject: [New-bugs-announce] [issue31757] Tutorial: Fibonacci numbers start with 1, 1 Message-ID: <1507732819.98.0.213398074469.issue31757@psf.upfronthosting.co.za> New submission from Heinrich Schnermann : In https://docs.python.org/3/tutorial/controlflow.html#defining-functions both examples produce fibonacci numbers starting with 0. They should start with 1, as in the example in https://docs.python.org/3/tutorial/introduction.html#first-steps-towards-programming The first example should change the lines while a < n: print(a, end=' ') to while b < n: print(b, end=' ') and the second example should change the lines while a < n: result.append(a) # see below to while b < n: result.append(b) # see below ---------- assignee: docs at python components: Documentation messages: 304144 nosy: docs at python, skyhein priority: normal severity: normal status: open title: Tutorial: Fibonacci numbers start with 1, 1 type: behavior versions: Python 3.6 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed Oct 11 10:53:39 2017 From: report at bugs.python.org (Oren Milman) Date: Wed, 11 Oct 2017 14:53:39 +0000 Subject: [New-bugs-announce] [issue31758] various refleaks in _elementtree Message-ID: <1507733619.97.0.213398074469.issue31758@psf.upfronthosting.co.za> New submission from Oren Milman : The following code results in refleaks: import sys import _elementtree builder = _elementtree.TreeBuilder() parser = _elementtree.XMLParser(target=builder) refcount_before = sys.gettotalrefcount() parser.__init__(target=builder) print(sys.gettotalrefcount() - refcount_before) # should be close to 0 This is because _elementtree_XMLParser___init___impl() (in Modules/_elementtree.c) doesn't decref before assigning to fields of `self`. The following code also results in refleaks: import sys import _elementtree elem = _elementtree.Element(42) elem.__setstate__({'tag': 42, '_children': list(range(1000))}) refcount_before = sys.gettotalrefcount() elem.__setstate__({'tag': 42, '_children': []}) print(sys.gettotalrefcount() - refcount_before) # should be close to -1000 This is because element_setstate_from_attributes() doesn't decref the old children before storing the new children. I would open a PR to fix this soon. ---------- components: XML messages: 304145 nosy: Oren Milman priority: normal severity: normal status: open title: various refleaks in _elementtree type: resource usage versions: Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed Oct 11 10:56:13 2017 From: report at bugs.python.org (=?utf-8?q?Rapha=C3=ABl_Riel?=) Date: Wed, 11 Oct 2017 14:56:13 +0000 Subject: [New-bugs-announce] [issue31759] re wont recover nor fail on runaway regular expression Message-ID: <1507733773.23.0.213398074469.issue31759@psf.upfronthosting.co.za> New submission from Rapha?l Riel : re won't raise nor return when working with Runaway Regular Expression. It will compute "almost" indefinitely. Although I'm pretty sure it *may* complete sometime, it's definetly looks like it's stuck. ``` > python -VVVV Python 3.6.2 (default, Aug 23 2017, 14:57:08) [GCC 4.2.1 Compatible Apple LLVM 8.1.0 (clang-802.0.42)] ``` Reproduce with attached file. Should there be a (configurable?) limit on the number of steps involved in the process. Or some warnings and/or hard limit that raises exception? https://pythex.org/ will fail with a HTTP502 BadGateway (server taking too long to respond) https://regex101.com/ python's tester seems to set a limit for this case. I can't say how they managed this. ---------- components: Regular Expressions files: re_backtracking.py messages: 304146 nosy: Rapha?l Riel, ezio.melotti, mrabarnett priority: normal severity: normal status: open title: re wont recover nor fail on runaway regular expression type: behavior versions: Python 3.6 Added file: https://bugs.python.org/file47212/re_backtracking.py _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed Oct 11 11:14:45 2017 From: report at bugs.python.org (=?utf-8?q?St=C3=A9phane_Wirtel?=) Date: Wed, 11 Oct 2017 15:14:45 +0000 Subject: [New-bugs-announce] [issue31760] Re-definition of _POSIX_C_SOURCE with Fedora 26. Message-ID: <1507734885.66.0.213398074469.issue31760@psf.upfronthosting.co.za> New submission from St?phane Wirtel : Hi all, Is it problematic ? if it is not the case, we can close it. In file included from /home/stephane/src/github.com/python/cpython/Modules/expat/expat_config.h:8:0, from /home/stephane/src/github.com/python/cpython/Modules/expat/xmltok.c:41: ./pyconfig.h:1457:0: warning: "_POSIX_C_SOURCE" redefined #define _POSIX_C_SOURCE 200809L In file included from /usr/include/bits/libc-header-start.h:33:0, from /usr/include/string.h:26, from /home/stephane/src/github.com/python/cpython/Modules/expat/xmltok.c:35: /usr/include/features.h:286:0: note: this is the location of the previous definition # define _POSIX_C_SOURCE 199506L St?phane ---------- messages: 304148 nosy: matrixise priority: normal severity: normal status: open title: Re-definition of _POSIX_C_SOURCE with Fedora 26. versions: Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed Oct 11 12:55:04 2017 From: report at bugs.python.org (Denis Osipov) Date: Wed, 11 Oct 2017 16:55:04 +0000 Subject: [New-bugs-announce] [issue31761] Possible error in devguide part about tests Message-ID: <1507740904.59.0.213398074469.issue31761@psf.upfronthosting.co.za> New submission from Denis Osipov : In Developer Guide says: "If you don?t have easy access to a command line, you can run the test suite from a Python or IDLE shell: >>> from test import autotest" But I can't run test from IDLE: Traceback (most recent call last): File "", line 1, in from test import autotest File "D:\repos\cpython\Lib\test\autotest.py", line 5, in main() File "D:\repos\cpython\Lib\test\libregrtest\main.py", line 585, in main Regrtest().main(tests=tests, **kwargs) File "D:\repos\cpython\Lib\test\libregrtest\main.py", line 510, in main self._main(tests, kwargs) File "D:\repos\cpython\Lib\test\libregrtest\main.py", line 524, in _main setup_tests(self.ns) File "D:\repos\cpython\Lib\test\libregrtest\setup.py", line 18, in setup_tests faulthandler.enable(all_threads=True) io.UnsupportedOperation: fileno If I understand it correct, this behavior is reasonable (issues 3003 and 25588). Maybe it's worth to remove words about running from IDLE. Or in case if it's possible to run such tests add some words about it. ---------- assignee: docs at python components: Documentation messages: 304163 nosy: denis-osipov, docs at python priority: normal severity: normal status: open title: Possible error in devguide part about tests type: behavior versions: Python 3.7, Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed Oct 11 13:00:59 2017 From: report at bugs.python.org (Nikhil) Date: Wed, 11 Oct 2017 17:00:59 +0000 Subject: [New-bugs-announce] [issue31762] Issue in login In-Reply-To: Message-ID: New submission from Nikhil : I have included the screen shot ---------- files: Screenshot_20171011_222848.png messages: 304164 nosy: Nik101 priority: normal severity: normal status: open title: Issue in login Added file: https://bugs.python.org/file47213/Screenshot_20171011_222848.png _______________________________________ Python tracker _______________________________________ -------------- next part -------------- A non-text attachment was scrubbed... Name: Screenshot_20171011_222848.png Type: image/png Size: 22194 bytes Desc: not available URL: From report at bugs.python.org Wed Oct 11 14:48:25 2017 From: report at bugs.python.org (Matthew Patton) Date: Wed, 11 Oct 2017 18:48:25 +0000 Subject: [New-bugs-announce] [issue31763] Add TRACE level to the logging module Message-ID: <1507747705.55.0.213398074469.issue31763@psf.upfronthosting.co.za> New submission from Matthew Patton : This was inspired by 31732. The logging module has 5 log levels: CRITICAL, ERROR, WARNING, INFO, DEBUG per https://docs.python.org/dev/library/logging.html#logging-levels However syslog(3) has for decades defined NOTICE and I can't think of a good reason why this level was carried through. It serves a very useful distinction from routine and not really attention-worthy messages (INFO) but don't rise to actual WARNINGs. Things like failed authentication attempts are not warnings but also not messages to casually ignore. Hence NOTICE. Individual timed out connection attempts but before all attempts exhausted, and many other examples exist. ---------- components: Library (Lib) messages: 304168 nosy: mp5023 priority: normal severity: normal status: open title: Add TRACE level to the logging module type: enhancement versions: Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed Oct 11 15:09:18 2017 From: report at bugs.python.org (Oren Milman) Date: Wed, 11 Oct 2017 19:09:18 +0000 Subject: [New-bugs-announce] [issue31764] sqlite3.Cursor.close() crashes in case the Cursor object is uninitialized Message-ID: <1507748958.42.0.213398074469.issue31764@psf.upfronthosting.co.za> New submission from Oren Milman : The following code causes a crash: import sqlite3 cursor = sqlite3.Cursor.__new__(sqlite3.Cursor) cursor.close() this is because pysqlite_cursor_close() (in Modules/_sqlite/cursor.c) assumes that `self->connection` is not NULL, and passes it to pysqlite_check_thread(), which crashes. I would open a PR to fix this soon. ---------- components: Extension Modules messages: 304172 nosy: Oren Milman priority: normal severity: normal status: open title: sqlite3.Cursor.close() crashes in case the Cursor object is uninitialized type: crash versions: Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed Oct 11 15:10:07 2017 From: report at bugs.python.org (Nikhil) Date: Wed, 11 Oct 2017 19:10:07 +0000 Subject: [New-bugs-announce] [issue31765] BUG: System deadlocks performing big loop operations in python 3.5.4, windows 10 In-Reply-To: Message-ID: New submission from Nikhil : What I did 1) root file declared where operation will start and import hashlib for getting hash signatures 2) os.walk for recursively diving into directory tree 3) open file and find sha256 digest and print it 4) close the file 5) recursively repeat all above 2 steps till we traversed directory tree Above logic python program caused sudden exponential increase in RAM usage (task manager by chance) and certainly DEADLOCK and...... Force shutdown I want to know what went wrong..... ---------- messages: 304173 nosy: Nik101 priority: normal severity: normal status: open title: BUG: System deadlocks performing big loop operations in python 3.5.4, windows 10 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed Oct 11 17:18:37 2017 From: report at bugs.python.org (Anthony Flury) Date: Wed, 11 Oct 2017 21:18:37 +0000 Subject: [New-bugs-announce] [issue31766] Python 3.5 missing from documentation Message-ID: <1507756717.96.0.213398074469.issue31766@psf.upfronthosting.co.za> New submission from Anthony Flury : In the Python version pull down list on docs.python.org, Python3.5 used to be listed, but has now been removed; the list only contains 2.7, 3.6 & 3.7. Python 3.5 is still the official Python 3.5 release in the Ubuntu repository, and still a supported release in other parts of python.org, so to see it disappearing from the drop-down was surprising. To note - if you pick a particular page - say : https://docs.python.org/3/tutorial/index.html and change the url to : https://docs.python.org/3.5/tutorial/index.html The pull down now does contain 3.5 (along side 2.7, 3.63 & 3.7) ---------- assignee: docs at python components: Documentation messages: 304181 nosy: anthony-flury, docs at python priority: normal severity: normal status: open title: Python 3.5 missing from documentation type: behavior versions: Python 3.5 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed Oct 11 17:23:15 2017 From: report at bugs.python.org (Igor Skochinsky) Date: Wed, 11 Oct 2017 21:23:15 +0000 Subject: [New-bugs-announce] [issue31767] Windows Installer fails with error 0x80091007 when trying to install debugging symbols Message-ID: <1507756995.05.0.213398074469.issue31767@psf.upfronthosting.co.za> New submission from Igor Skochinsky : Trying to install 3.6.3 on Windows 10 and getting this error. Repro steps: 1. Download python-3.6.3.exe or python-3.6.3-amd64.exe, run it 2. Select "Customize Installation" 3. Next 4. In Advanced Options, select [x] Install for all users (not sure if important) [x] Download debugging symbols change path to F:\Python36 (not sure if it matters but my system drive is F: and C: is missing except for when I plug in a USB drive) Ucheck [ ] Associate files with Python Click Install and accept elevation prompt. After a while it fails with Error 0x80091007 - Hash value not correct. Interestingly, "log file" link does not work. I had to go hunt for the log in %temp% myself. The relevant parts: Error 0x80091007: Hash mismatch for path: F:\ProgramData\Package Cache\.unverified\core_AllUsers_pdb, expected: A4B6E4A818E49F000513774F034EC98A194E3C3D, actual: 36C7B852E586D26F4D239ED72EFE5FFBEBA21825 Error 0x80091007: Failed to verify hash of payload: core_AllUsers_pdb Failed to verify payload: core_AllUsers_pdb at path: F:\ProgramData\Package Cache\.unverified\core_AllUsers_pdb, error: 0x80091007. Deleting file. Application requested retry of payload: core_AllUsers_pdb, encountered error: 0x80091007. Retrying... There are a few more retries basically with the same result. However, this one is interesting: Acquiring package: core_AllUsers_pdb, payload: core_AllUsers_pdb, copy from: F:\Users\Igor\Downloads\core_pdb.msi I don't have such a file there, just the installer .exe ---------- components: Installation messages: 304182 nosy: Igor.Skochinsky priority: normal severity: normal status: open title: Windows Installer fails with error 0x80091007 when trying to install debugging symbols type: security versions: Python 3.6 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed Oct 11 17:49:25 2017 From: report at bugs.python.org (caveman) Date: Wed, 11 Oct 2017 21:49:25 +0000 Subject: [New-bugs-announce] [issue31768] argparse drops '|'s when the arguments are too long Message-ID: <1507758565.64.0.213398074469.issue31768@psf.upfronthosting.co.za> New submission from caveman : if you execute the code below, mutually exclusive agrs are separated by '|' as expected. but if you uncomment the 1 line down there, then the args list will be too long, and as argparse tries to wrap the args around, it drops all '|'s which semantically destroys the intended syntax of the arguments. import argparse parser = argparse.ArgumentParser() group = parser.add_mutually_exclusive_group() group.add_argument("-v", "--verbose", action="store_true") group.add_argument("-q", "--quiet", action="store_true") group.add_argument("-x", metavar='X', type=str, help="the base", nargs='?') group.add_argument("-y", metavar='Y', type=str, help="the exponent", nargs='?') group.add_argument("z", metavar='Z', type=str, help="the exponent", nargs='?') group.add_argument("z", metavar='Z', type=str, help="the exponent", nargs='?') group.add_argument("z", metavar='Z', type=str, help="the exponent", nargs='?') group.add_argument("z", metavar='Z', type=str, help="the exponent", nargs='?') group.add_argument("z", metavar='Z', type=str, help="the exponent", nargs='?') group.add_argument("z", metavar='Z', type=str, help="the exponent", nargs='?') group.add_argument("z", metavar='Z', type=str, help="the exponent", nargs='?') #group.add_argument("z", metavar='Z', type=str, help="the exponent", nargs='?') args = parser.parse_args() ---------- components: Library (Lib) messages: 304184 nosy: caveman priority: normal severity: normal status: open title: argparse drops '|'s when the arguments are too long type: behavior versions: Python 2.7, Python 3.4, Python 3.5, Python 3.6, Python 3.7, Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed Oct 11 23:33:29 2017 From: report at bugs.python.org (Mike Gilbert) Date: Thu, 12 Oct 2017 03:33:29 +0000 Subject: [New-bugs-announce] [issue31769] configure includes user CFLAGS testing detecting pthreads support Message-ID: <1507779209.22.0.213398074469.issue31769@psf.upfronthosting.co.za> New submission from Mike Gilbert : When testing for ptheads support in the compiler, configure includes the CFLAGS value from the environment. If CFLAGS contains -pthread, or an option which implies -pthread (like -fopenmp), this will cause configure to not include -pthread in the CC variable, and -pthread will not be passed to the linker. This ultimately leads to a link failure when building with glibc/gcc. gcc -Xlinker -export-dynamic -o python Programs/python.o libpython3.7m.a -ldl -lutil -lm libpython3.7m.a(thread.o): In function `PyThread_start_new_thread': /home/floppym/src/cpython/Python/thread_pthread.h:191: undefined reference to `pthread_attr_setstacksize' ---------- components: Build messages: 304200 nosy: floppymaster priority: normal severity: normal status: open title: configure includes user CFLAGS testing detecting pthreads support type: compile error versions: Python 2.7, Python 3.4, Python 3.5, Python 3.6, Python 3.7, Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu Oct 12 04:18:36 2017 From: report at bugs.python.org (Oren Milman) Date: Thu, 12 Oct 2017 08:18:36 +0000 Subject: [New-bugs-announce] [issue31770] crash and refleaks when calling sqlite3.Cursor.__init__() more than once Message-ID: <1507796316.87.0.213398074469.issue31770@psf.upfronthosting.co.za> New submission from Oren Milman : The following code crashes: import sqlite3 import weakref def callback(*args): pass connection = sqlite3.connect(":memory:") cursor = sqlite3.Cursor(connection) ref = weakref.ref(cursor, callback) cursor.__init__(connection) del cursor del ref IIUC, this is because pysqlite_cursor_init() (in Modules/_sqlite/cursor.c) sets `self->in_weakreflist` to NULL, and thus corrupts the weakref list. Later, clear_weakref() (in Objects/weakrefobject.c) tries to remove a reference from the corrupted list, and crashes. In every other place (that i saw) where such a weakreflist field is used, it is set to NULL right after allocating the object (usually in __new__()), or just not set at all, e.g. in `functools.partial`. So since PyType_GenericNew() is the __new__() of sqlite3.Cursor, ISTM that the simplest solution is to not touch `self->in_weakreflist` at all in pysqlite_cursor_init(). Also, the following code results in refleaks: import sys import sqlite3 connection = sqlite3.connect(":memory:") cursor = sqlite3.Cursor(connection) refcount_before = sys.gettotalrefcount() cursor.__init__(connection) print(sys.gettotalrefcount() - refcount_before) # should be close to 0 This is because pysqlite_cursor_init() doesn't decref before assigning to fields of `self`. I would open a PR to fix this soon. ---------- components: Extension Modules messages: 304220 nosy: Oren Milman priority: normal severity: normal status: open title: crash and refleaks when calling sqlite3.Cursor.__init__() more than once type: crash versions: Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu Oct 12 09:26:16 2017 From: report at bugs.python.org (Peter J) Date: Thu, 12 Oct 2017 13:26:16 +0000 Subject: [New-bugs-announce] [issue31771] tkinter geometry string +- when window ovelaps left or top of screen Message-ID: <1507814776.62.0.213398074469.issue31771@psf.upfronthosting.co.za> New submission from Peter J : the root.geometry() top-level window method returns strings like "212x128+-9+-8" when the window is located in the top left corner. The documentation only describes geometry strings containing + or - location coordinates. If this is correct behaviour, the documentation should provide a better description of the syntax - perhaps a regular expression such as: "\d+x\d+(\+|-|\+-)\d+(\+|-|\+-)\d+" and some text indicating that a +- coordinate indicates a value to the left of or above the screen. Tested Linux Mint 18, Python 3.5.3, tkinter 8.6 Behaviour is consistent with python 2.7 Tkinter 8.6 ---------- components: Tkinter messages: 304232 nosy: 8baWnoVi priority: normal severity: normal status: open title: tkinter geometry string +- when window ovelaps left or top of screen type: behavior versions: Python 3.5 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu Oct 12 09:43:00 2017 From: report at bugs.python.org (Devin Bayer) Date: Thu, 12 Oct 2017 13:43:00 +0000 Subject: [New-bugs-announce] [issue31772] SourceLoader uses stale bytecode in case of equal mtime seconds Message-ID: <1507815780.34.0.213398074469.issue31772@psf.upfronthosting.co.za> New submission from Devin Bayer : The current import machinery will use stale cached bytecode if the source is modified more than once per second and with equal size. A straightforward fix is to require the bytecode mtime to be greater than the source file mtime. In the unusual case where the file is written twice with the precision the filesystem records, this will ignore the cache, but at least we aren't behaving incorrectly. ---------- components: Interpreter Core messages: 304236 nosy: akvadrako priority: normal severity: normal status: open title: SourceLoader uses stale bytecode in case of equal mtime seconds type: behavior versions: Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu Oct 12 10:14:14 2017 From: report at bugs.python.org (STINNER Victor) Date: Thu, 12 Oct 2017 14:14:14 +0000 Subject: [New-bugs-announce] [issue31773] Rewrite _PyTime_GetWinPerfCounter() for _PyTime_t Message-ID: <1507817654.69.0.213398074469.issue31773@psf.upfronthosting.co.za> New submission from STINNER Victor : The commit a997c7b434631f51e00191acea2ba6097691e859 of bpo-31415 moved the implementation of time.perf_counter() from Modules/timemodule.c to Python/pytime.c. The change not only moved the code, but also changed the internal type storing time from floatting point number (C double) to integer number (_PyTyime_t = int64_t). The drawback of this change is that time.perf_counter() now converts QueryPerformanceCounter() / QueryPerformanceFrequency() double into a _PyTime_t (integer) and then back to double. Two useless conversions required by the _PyTime_t format used in Python/pytime.c. These conversions introduced a loss of precision. Try attached round.py script which implements the double <=> _PyTime_t conversions and checks to check for precision loss. The script shows that we loose precision even with a single second for QueryPerformanceFrequency() == 3579545. It seems like QueryPerformanceFrequency() now returns 10 ** 7 (10_000_000, resolution of 100 ns) on Windows 8 and newer, but returns 3,579,545 (3.6 MHz, resolution of 279 ns) on Windows 7. It depends maybe on the hardware clock, I don't know. Anyway, whenever possible, we should avoid precision loss of a clock. ---------- components: Interpreter Core files: round.py messages: 304239 nosy: haypo priority: normal severity: normal status: open title: Rewrite _PyTime_GetWinPerfCounter() for _PyTime_t type: enhancement versions: Python 3.7 Added file: https://bugs.python.org/file47217/round.py _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu Oct 12 10:17:19 2017 From: report at bugs.python.org (Charalampos Stratakis) Date: Thu, 12 Oct 2017 14:17:19 +0000 Subject: [New-bugs-announce] [issue31774] tarfile.open ignores custom bufsize value when creating a new archive Message-ID: <1507817839.86.0.213398074469.issue31774@psf.upfronthosting.co.za> New submission from Charalampos Stratakis : Trying to create an archive with the tarfile module, by specifying a different blocking factor, doesn't seem to work as only the default value is being used. The issue is reproducible on all the active python branches. Attaching a script to reproduce it. Original bug report: https://bugzilla.redhat.com/show_bug.cgi?id=1492157 ---------- components: Library (Lib) files: tartest.py messages: 304241 nosy: cstratak, lars.gustaebel priority: normal severity: normal status: open title: tarfile.open ignores custom bufsize value when creating a new archive versions: Python 2.7, Python 3.6, Python 3.7 Added file: https://bugs.python.org/file47218/tartest.py _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu Oct 12 11:09:32 2017 From: report at bugs.python.org (STINNER Victor) Date: Thu, 12 Oct 2017 15:09:32 +0000 Subject: [New-bugs-announce] [issue31775] Support unbuffered TextIOWrapper Message-ID: <1507820972.65.0.213398074469.issue31775@psf.upfronthosting.co.za> New submission from STINNER Victor : It seems like that some methods of the io.TextIOWrapper class requires that its buffer object has the read1() method, whereas the constructor checks if the buffer has a read1() method and the TextIOWrapper _read_chunk() method is able to call buffer.read() if buffer doesn't have read1(). This issue may help to get fully unbuffered sys.stdin, at least when replaced manually: stdin = sys.stdin sys.stdin = open(0, "r", buffering=0, encoding=stdin.encoding, errors=stdin.errors, newline=stdin.newline) ---------- components: IO messages: 304250 nosy: haypo priority: normal severity: normal status: open title: Support unbuffered TextIOWrapper versions: Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu Oct 12 16:59:28 2017 From: report at bugs.python.org (Pablo) Date: Thu, 12 Oct 2017 20:59:28 +0000 Subject: [New-bugs-announce] [issue31776] Inconsistent exception chaining in /Lib/xml/etree/ElementPath.py Message-ID: <1507841968.12.0.213398074469.issue31776@psf.upfronthosting.co.za> New submission from Pablo : Based on bpo-29762 (https://github.com/python/cpython/commit/5affd23e6f42125998724787025080a24839266e), there is an inconsistency on one exception chain in /Lib/xml/etree/ElementPath.py: try: selector.append(ops[token[0]](next, token)) except StopIteration: raise SyntaxError("invalid path") should be raise SyntaxError("invalid path") from None ---------- components: Library (Lib) messages: 304288 nosy: pablogsal priority: normal severity: normal status: open title: Inconsistent exception chaining in /Lib/xml/etree/ElementPath.py type: behavior _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu Oct 12 18:07:23 2017 From: report at bugs.python.org (Terry J. Reedy) Date: Thu, 12 Oct 2017 22:07:23 +0000 Subject: [New-bugs-announce] [issue31777] IDLE: Let users add to font selection Message-ID: <1507846043.1.0.213398074469.issue31777@psf.upfronthosting.co.za> New submission from Terry J. Reedy : Config dialog Font tab has a sample box. It currently only contains Ascii chars. #13802 will add characters from extended latin1 and several other character sets. But we cannot show everything. Hence a spinoff proposal: let users augment the sample with things of particular interest. (There are multiple variations of this idea, which I defer to another post.) ---------- assignee: terry.reedy components: IDLE messages: 304292 nosy: terry.reedy priority: normal severity: normal stage: test needed status: open title: IDLE: Let users add to font selection type: enhancement versions: Python 3.6, Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu Oct 12 18:31:18 2017 From: report at bugs.python.org (David Bieber) Date: Thu, 12 Oct 2017 22:31:18 +0000 Subject: [New-bugs-announce] [issue31778] ast.literal_eval supports non-literals in Python 3 Message-ID: <1507847478.46.0.213398074469.issue31778@psf.upfronthosting.co.za> New submission from David Bieber : # Overview ast.literal_eval supports some non-literals in Python 3. The behavior of this function runs contrary to the documented behavior. # The Issue The [documentation](https://docs.python.org/3/library/ast.html#ast.literal_eval) says of the function "It is not capable of evaluating arbitrarily complex expressions, for example involving operators or indexing." However, literal_eval is capable of evaluating expressions with certain operators, particular the operators "+" and "-". As has been explained previously, the reason for this is to support "complex" literals such as 2+3j. However, this has unintended consequences which I believe to be indicative of a bug. Examples of the unintended consequences include `ast.literal_eval('1+1') == 2` `ast.literal_eval('2017-10-10') == 1997`. I would expect each of these calls to literal_eval to throw a ValueError, as the input string is not a literal. Instead, literal_eval successfully evaluates the input string, in the latter case giving an unexpected result (since the intent of the string is to represent a 21st century date.) Since issue arose as a [Python Fire issue](https://github.com/google/python-fire/issues/97), where the behavior of Python Fire was unexpected for inputs such as those described above (1+1 and 2017-10-10) only in Python 3. For context, Python Fire is a CLI library which uses literal_eval as part of its command line argument parsing procedure. I think the resolution to this issue is having literal_eval raise a ValueError if the ast of the input represents anything other than a Python literal, as described in the documentation. That is, "The string or node provided may only consist of the following Python literal structures: strings, bytes, numbers, tuples, lists, dicts, sets, booleans, and None." Additional operations, such as the binary operations "+" and "-", unless they explicitly create a complex number, should produce a ValueError. If that resolution is not the direction we take, I also would appreciate knowing if there is another built in approach for determining if a string or ast node represents a literal. # Reproducing The following code snippets produce different behaviors in Python 2 and Python 3. ```python import ast ast.literal_eval('1+1') ``` ```python import ast ast.literal_eval('2017-10-10') ``` # References - The Python Fire issue is here: https://github.com/google/python-fire/issues/97 - Prior discussion of a similar issue: https://bugs.python.org/issue22525 - I think is where complex number support was originally added: https://bugs.python.org/issue4907 - In this thread, https://bugs.python.org/issue24146, one commenter explains that literal_eval's support for "2+3" is an unintentional side effect of supporting complex literals. ---------- messages: 304294 nosy: David Bieber priority: normal severity: normal status: open title: ast.literal_eval supports non-literals in Python 3 versions: Python 3.4, Python 3.5, Python 3.6, Python 3.7, Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Oct 13 06:54:20 2017 From: report at bugs.python.org (Oren Milman) Date: Fri, 13 Oct 2017 10:54:20 +0000 Subject: [New-bugs-announce] [issue31779] assertion failures and a crash when using an uninitialized struct.Struct object Message-ID: <1507892060.42.0.213398074469.issue31779@psf.upfronthosting.co.za> New submission from Oren Milman : The following code causes an assertion failure: import _struct struct_obj = _struct.Struct.__new__(_struct.Struct) struct_obj.iter_unpack(b'foo') This is because Struct_iter_unpack() (in Modules/_struct.c) assumes that Struct.__init__() was called, and so it does `assert(self->s_codes != NULL);`. The same happens in (almost) every method of Struct, and in s_get_format(), so in all them, too, we would get an assertion failure in case of an uninitialized Struct object. The exception is __sizeof__(), which doesn't have an `assert`, and simply crashes while trying to iterate over `self->s_codes`. I would open a PR to fix this soon. ---------- components: Extension Modules messages: 304328 nosy: Oren Milman priority: normal severity: normal status: open title: assertion failures and a crash when using an uninitialized struct.Struct object type: crash versions: Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Oct 13 07:36:55 2017 From: report at bugs.python.org (FHTMitchell) Date: Fri, 13 Oct 2017 11:36:55 +0000 Subject: [New-bugs-announce] [issue31780] Using format spec ', x' displays incorrect error message Message-ID: <1507894615.97.0.213398074469.issue31780@psf.upfronthosting.co.za> New submission from FHTMitchell : Minor issue. Using the ',b', ',o' or ',x' raises the error ValueError("Cannot specify ',' or '_' with 'x'.",) (or equivalently for 'b' and 'o'). However, it is possible to use the format specs '_b', '_o' and '_x' in Python 3.6 due to PEP 515. The following test demonstrates this: >>> i = 10000 >>> for base in 'box': ... for sep in ',_': ... try: ... print(f'{i:{sep}{base}}') ... except ValueError as err: ... print(repr(err)) ValueError("Cannot specify ',' or '_' with 'b'.",) 1_1000_0110_1010_0000 ValueError("Cannot specify ',' or '_' with 'o'.",) 30_3240 ValueError("Cannot specify ',' or '_' with 'x'.",) 1_86a0 ---------- messages: 304330 nosy: FHTMitchell priority: normal severity: normal status: open title: Using format spec ',x' displays incorrect error message type: behavior versions: Python 3.6 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Oct 13 12:56:47 2017 From: report at bugs.python.org (Oren Milman) Date: Fri, 13 Oct 2017 16:56:47 +0000 Subject: [New-bugs-announce] [issue31781] crashes when calling methods of an uninitialized zipimport.zipimporter object Message-ID: <1507913807.6.0.213398074469.issue31781@psf.upfronthosting.co.za> New submission from Oren Milman : The following code crashes: import zipimport zi = zipimport.zipimporter.__new__(zipimport.zipimporter) zi.find_module('foo') This is because get_module_info() (in Modules/zipimport.c) assumes that the zipimporter object is initialized, so it assumes that `self->prefix` is not NULL, and passes it to make_filename(), which crashes. get_module_code() makes the same assumption, and zipimport_zipimporter_get_data_impl() assumes that `self->archive` is not NULL, and passes it to PyUnicode_GET_LENGTH(), which crashes. Thus, every method of an uninitialized zipimporter object might crash. I would open a PR to fix this soon. ---------- components: Extension Modules messages: 304346 nosy: Oren Milman priority: normal severity: normal status: open title: crashes when calling methods of an uninitialized zipimport.zipimporter object type: crash versions: Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Oct 13 15:23:42 2017 From: report at bugs.python.org (Will Starms) Date: Fri, 13 Oct 2017 19:23:42 +0000 Subject: [New-bugs-announce] [issue31782] Add a timeout to multiprocessing's Pool.join Message-ID: <1507922622.85.0.213398074469.issue31782@psf.upfronthosting.co.za> New submission from Will Starms : Pool's join function currently (3.6.3) lacks a timeout, which can cause the managing thread to sleep indefinitely when a pool worker hangs or starts misbehaving. Adding a timeout allows the owning thread to attempt a join and, after the timeout, return to other tasks, such as monitoring worker health. In my specific situation, I have a Pool running a task on a large set of files. If any single task fails, the whole operation is ruined and the pool should be terminated. A task can communicate with the main thread through error_callback, but if the thread has already called join, it can't check until join returns, after the Pool has finished all processing. Attached is an incredibly simple patch to the current (3.6) cpython implementation that emulates threading.thread.join's behavior. ---------- components: Library (Lib) files: cpython_timeout.patch keywords: patch messages: 304350 nosy: Will Starms priority: normal severity: normal status: open title: Add a timeout to multiprocessing's Pool.join type: enhancement versions: Python 3.6 Added file: https://bugs.python.org/file47219/cpython_timeout.patch _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Oct 13 17:27:15 2017 From: report at bugs.python.org (Steven Barker) Date: Fri, 13 Oct 2017 21:27:15 +0000 Subject: [New-bugs-announce] [issue31783] Race condition in ThreadPoolExecutor when scheduling new jobs while the interpreter shuts down Message-ID: <1507930035.71.0.213398074469.issue31783@psf.upfronthosting.co.za> New submission from Steven Barker : While investigating a Stack Overflow question (here: https://stackoverflow.com/q/46529767/1405065), I found that there may be a race condition in the cleanup code for concurrent.futures.ThreadPoolIterator. The behavior in normal situations is fairly benign (the executor may run a few more jobs than you'd expect, but exits cleanly), but in rare situations it might lose track of a running thread and allow the interpreter to shut down while the thread is still trying to do work. Here's some example that concisely demonstrates the situation where the issue can come up (it doesn't actually cause the race to go the wrong way on my system, but sets up the possibility for it to occur): from threading import current_thread from concurrent.futures import ThreadPoolExecutor from time import sleep pool = ThreadPoolExecutor(4) def f(_): print(current_thread().name) future = pool.submit(sleep, 0.1) future.add_done_callback(f) f(None) The callback from completion of one job schedules another job, indefinitely. When run in an interactive session, this code will print thread names forever. You should get "MainThread" once, followed by a bunch of "ThreadPoolExecutor-X_Y" names (often the same name will be repeated most of the time, due to the GIL I think, but in theory the work could rotate between threads). The main thread will return to the interactive REPL right away, so you can type in other stuff while the executor's worker threads are printing stuff the background (I suggest running pool.shutdown() to make them stop). This is fine. But if the example code is run as a script, you'll usually get "MainThread", followed by exactly four repeats of "ThreadPoolExecutor-0_0" (or fewer in the unlikely case that the race condition strikes you). That's the number of threads the ThreadPoolExecutor was limited to, but note that the thread name that gets printed will usually end with 0 every time (you don't get one output from each worker thread, just the same number of outputs as there are threads, all from the first thread). Why you get that number of outputs (instead of zero or one or an infinite number) was one part of the Stack Overflow question. The answer turned out to be that after the main thread has queued up the first job in the ThreadPoolExecutor, it runs off the end of the script's code, so it starts shutting down the interpreter. The cleanup function _python_exit (in Lib/concurrent/futures/thread.py) gets run since it is registered with atexit, and it tries to signal the worker threads to shut down cleanly. However, the shutdown logic interacts oddly with an executor that's still spinning up its threads. It will only signal and join the threads that existed when it started running, not any new threads. As it turns out, usually the newly spawned threads will shut themselves down immediately after they spawn, but as a side effect, the first worker thread carries on longer than expected, doing one additional job for each new thread that gets spawned and exiting itself only when the executor has a full set. This is why there are four outputs from the worker thread instead of some other number. But the exact behavior is dependent on the thread scheduling order, so there is a race condition. You can demonstrate a change in behavior from different timing by putting a call to time.sleep at the very top of the _worker() function (delaying how quickly the new threads can get to the work queue). You should see the program behavior change to only print "ThreadPoolExecutor-0_0" once before exiting. Lets go through the steps of the process: 1. The main thread runs f() and schedules a job (which adds a work item to the executor's work queue). The first worker thread is spawned by the executor to run the job, since it doesn't have any threads yet. The main thread also sets a callback on the future to run f() again. 2. The main thread exits f() and reaches the end of the script, so it begins the interpreter shutdown process, including calling atexit functions. One of those is _python_exit, which adds a reference to None to the executor's work queue. Note that the None is added *after* the job item from step 1 (since they're both done by the same main thread). It then calls join() on the worker thread spawned in step 1, waiting for it to exit. It won't try to join any other threads that spawn later, since they don't exist yet. 3. The first worker thread spawned by the executor in step 1 begins running and pops an item off the work queue. The first item is a real job, so it runs it. (The first parts of this step may be running in parallel with step 2, but completing job will take much longer than step 2, so the rest of this step runs by itself after step 2 has finished.) Eventually the job ends and the callback function on the Future is called, which schedules another job (putting a job item in the queue after the None), and spawning a second worker thread (since the executor doesn't have enough yet). 4. The race condition occurs here. Usually the new worker thread (created in step 3) starts running next and pops the None off of the work queue (leaving the a real work item still in the queue). It checks and finds the the global _shutdown flag is set, so it adds another None to the job queue (at the end again) and quits. 5. The other half of the race is here. The first worker finishes the callback and is done with the first job, so it goes back to the work queue for another one. It usually finds the real job it scheduled in step 3 (since the None was already taken by the other thread in step 4). From then on, the code repeats the behavior from step 3 (doing the job, calling the callback, queuing a new job, and spawning a new thread since the executor still isn't at full capacity). 6. Steps 4 and 5 will repeat a few more times until the executor has as many threads as it wants. If no new thread is spawned at the end of step 5, the first worker thread finally gets to pop a None from the queue instead of a job, so it will shut down. This lets the the main thread, which has been blocked since step 2, finally finish it's join() and shut down the rest of the interpreter. The race condition occurs between steps 4 and 5. If the first worker thread (that usually runs step 5) reaches the work queue before the other worker thread (which usually runs step 4), the first worker thread will get the None instead of the new thread. Thus the first worker will shut down earlier that in the usual scenario described above. The second thread (or third or fourth, depending on which cycle of steps 4 and 5 we're on) could get the job off the queue and start working on it while the first thread is exiting. That would be fine, but when the first thread exits, it will unblock the main thread and the interpreter will continue shutting down. This could cut the ground out from under the code running in the remaining worker thread. One solution would be to avoid creating any new threads when the interpreter is in the process of shutting down. We can check for the global _shutdown variable inside ThreadPoolExecutor._adjust_thread_count, though I think it needs a lock to avoid another race condition (where _shutdown and the contents of _thread_queues are accessed out of sync, e.g. a race between steps 2 and 3 above that could only occur if the jobs were exceedingly fast to complete). There are other options though. We could make it an error to queue up new work to an executor when the interpreter is in the process of shutting down (just change the "if self._shutdown" test in ThreadPoolExecutor.submit to also look at the global _shutdown variable, and the worker thread will crash with a RuntimeError just before things shut down). Or we could change the behavior of the workers to shut down as soon as possible rather than finishing all queued work items (remove the continue from the inner block in the loop in _worker so that it always checks the global _shutdown after completing each job). Or another option might be to change the cleanup logic in _python_exit to double check for additional threads to join() after it finishes waiting on its first set. ---------- components: Library (Lib) messages: 304364 nosy: Steven.Barker priority: normal severity: normal status: open title: Race condition in ThreadPoolExecutor when scheduling new jobs while the interpreter shuts down type: behavior versions: Python 3.6, Python 3.7, Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Oct 13 17:34:15 2017 From: report at bugs.python.org (STINNER Victor) Date: Fri, 13 Oct 2017 21:34:15 +0000 Subject: [New-bugs-announce] [issue31784] Add time.time_ns(): get time with nanosecond resolution Message-ID: <1507930455.75.0.213398074469.issue31784@psf.upfronthosting.co.za> New submission from STINNER Victor : time.time() returns time as a float, but the conversion to float lose precision at the nanosecond resolution. I propose to add a new time.time_ns() function which returns time as an integer number of nanoseconds since epoch. It's similar to the st_mtime_ns field of os.stat_result which extended the old st_mtime field. For the full rationale, see my thread on python-ideas: [Python-ideas] Add time.time_ns(): system clock with nanosecond resolution https://mail.python.org/pipermail/python-ideas/2017-October/047318.html ---------- components: Library (Lib) messages: 304365 nosy: haypo priority: normal severity: normal status: open title: Add time.time_ns(): get time with nanosecond resolution type: enhancement versions: Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Oct 13 19:58:24 2017 From: report at bugs.python.org (pdox) Date: Fri, 13 Oct 2017 23:58:24 +0000 Subject: [New-bugs-announce] [issue31785] Move instruction code blocks to separate file Message-ID: <1507939104.79.0.213398074469.issue31785@psf.upfronthosting.co.za> New submission from pdox : I'd like to move all instruction code (the code inside a TARGET(NAME) block) from Python/ceval.c to a new file, Python/instructions.h. The new file will contain only instruction bodies (prefixed by related helper functions/macros). Eval-context macros (e.g. TARGET, DISPATCH, TOP, PUSH, POP, etc) will not be moved to instructions.h, but will be expected to be available (defined by the #includer). ceval.c will define the eval-context macros in the same way, and #include "instructions.h", inside the body of _PyEval_EvalFrameDefault. The code emitted should remain exactly the same. The benefit of this change, is that it becomes easy to produce alternative implementations of EvalFrame which reuse the same instruction code, but with changes to the evaluation context or dispatch mechanism. In particular, after this change, I hope to experiment with adding a cross-platform subroutine-threading code evaluator. (for performance testing) ---------- components: Interpreter Core messages: 304370 nosy: pdox priority: normal severity: normal status: open title: Move instruction code blocks to separate file type: enhancement versions: Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sat Oct 14 04:07:08 2017 From: report at bugs.python.org (Serhiy Storchaka) Date: Sat, 14 Oct 2017 08:07:08 +0000 Subject: [New-bugs-announce] [issue31786] In select.poll.poll() ms can be 0 if timeout < 0 Message-ID: <1507968428.08.0.213398074469.issue31786@psf.upfronthosting.co.za> New submission from Serhiy Storchaka : According to the documentation select.poll.poll() is blocked for infinite timeout if the timeout argument is negative. But due to rounding it is possible that timeout < 0, but an integer ms passed to the poll() syscall is 0, and poll() is not blocked. ---------- components: Extension Modules messages: 304386 nosy: haypo, serhiy.storchaka priority: normal severity: normal status: open title: In select.poll.poll() ms can be 0 if timeout < 0 type: behavior versions: Python 3.6, Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sat Oct 14 05:16:28 2017 From: report at bugs.python.org (Oren Milman) Date: Sat, 14 Oct 2017 09:16:28 +0000 Subject: [New-bugs-announce] [issue31787] various refleaks when calling the __init__() method of an object more than once Message-ID: <1507972588.41.0.213398074469.issue31787@psf.upfronthosting.co.za> New submission from Oren Milman : Various __init__() methods don't decref (if needed) before assigning to fields of the object's struct (i.e. assigning to `self->some_field`): - _asyncio_Task___init___impl() (in Modules/_asynciomodule.c) - _lzma_LZMADecompressor___init___impl() (in Modules/_lzmamodule.c) - _bz2_BZ2Decompressor___init___impl() (in Modules/_bz2module.c) - EVP_tp_init() (in Modules/_hashopenssl.c) - property_init_impl() (in Objects/descrobject.c) - cm_init() (in Objects/funcobject.c) - sm_init() (in Objects/funcobject.c) For example, _asyncio_Task___init___impl() does `self->task_coro = coro;` instead of `Py_XSETREF(self->task_coro, coro);`. Thus, the following code would result in at least one refleak: import _asyncio task = _asyncio.Task('foo') task.__init__('foo') I would open a PR to fix this soon. ---------- components: Extension Modules messages: 304389 nosy: Oren Milman priority: normal severity: normal status: open title: various refleaks when calling the __init__() method of an object more than once type: resource usage versions: Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sat Oct 14 19:55:59 2017 From: report at bugs.python.org (Semen Zhydenko) Date: Sat, 14 Oct 2017 23:55:59 +0000 Subject: [New-bugs-announce] [issue31788] Typo in comments Modules/_ssl.c Message-ID: <1508025359.26.0.213398074469.issue31788@psf.upfronthosting.co.za> New submission from Semen Zhydenko : completly must be changed to completely ---------- assignee: docs at python components: Documentation messages: 304405 nosy: Semen Zhydenko, docs at python priority: normal severity: normal status: open title: Typo in comments Modules/_ssl.c versions: Python 2.7, Python 3.4, Python 3.5, Python 3.6, Python 3.7, Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sat Oct 14 20:34:04 2017 From: report at bugs.python.org (Dan Snider) Date: Sun, 15 Oct 2017 00:34:04 +0000 Subject: [New-bugs-announce] [issue31789] Better error message when failing to overload metaclass.mro Message-ID: <1508027644.3.0.213398074469.issue31789@psf.upfronthosting.co.za> New submission from Dan Snider : class meta(type): mros = (object,) def __new__(metacls, name, bases, namespace, fake_bases=()): print('entering __new__') metacls.fake_bases = fake_bases cls = type.__new__(metacls, name, bases, namespace) print('exiting __new__') return cls @staticmethod def mro(*args): print('entering mro') return meta.fake_bases + (object,) class a(metaclass=meta, fake_bases=()): pass That puts out the error message: entering meta.__new__ entering meta.mro Traceback (most recent call last): File "", line 1, in exec(code, u) File "", line 14, in File "", line 6, in __new__ TypeError: super(type, obj): obj must be an instance or subtype of type That doesn't at all explain why it doesn't work because super was never explicitly called. If the argument fake_bases isn't empty, or if mro isn't made a staticmethod/classmethod, it returns an appropriate message TypeError: mro() returned base with unsuitable layout ('tuple') ---------- components: Interpreter Core messages: 304406 nosy: bup priority: normal severity: normal status: open title: Better error message when failing to overload metaclass.mro type: enhancement versions: Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun Oct 15 02:00:32 2017 From: report at bugs.python.org (Zoltan Krajcsovics) Date: Sun, 15 Oct 2017 06:00:32 +0000 Subject: [New-bugs-announce] [issue31790] double free or corruption (while using smem) Message-ID: <1508047232.57.0.213398074469.issue31790@psf.upfronthosting.co.za> New submission from Zoltan Krajcsovics : Command executed when the crash happened: /usr/bin/python /usr/bin/smem -t -c command pss -P /usr/lib/chromium/chromium The attached error.log contains the backtrace and memory map printed. This is a sporadic error, the command is called every second 0-24, and I got one such error after half a day of running. I probably had another after a few days, but I failed to save that one, so I am not sure, if they are really the same. It would seem that using a more overloaded / older machine triggers the error more often, the machine I got it on was a Hewlett-Packard HP Compaq dc7900 Small Form Factor/3031h, BIOS 786G1 v01.27 10/22/2015 The problem seems to have emerged after updating to Debian Stretch. python version: Python 2.7.13 smem version: pool/main/s/smem/smem_1.4-2_all.deb (could not get an internal smem version, only the debian package) ---------- components: Library (Lib) messages: 304418 nosy: zulthan priority: normal severity: normal status: open title: double free or corruption (while using smem) type: crash versions: Python 2.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun Oct 15 02:22:29 2017 From: report at bugs.python.org (pdox) Date: Sun, 15 Oct 2017 06:22:29 +0000 Subject: [New-bugs-announce] [issue31791] Ensure that all PyTypeObject fields are set to non-NULL defaults Message-ID: <1508048549.34.0.213398074469.issue31791@psf.upfronthosting.co.za> New submission from pdox : Ensure that every function pointer in every PyTypeObject is set to a non-NULL default, to spare the cost of checking for NULL with every use. As a basic example, consider PyNumber_Negative: PyObject * PyNumber_Negative(PyObject *o) { PyNumberMethods *m; if (o == NULL) { return null_error(); } m = o->ob_type->tp_as_number; if (m && m->nb_negative) return (*m->nb_negative)(o); return type_error("bad operand type for unary -: '%.200s'", o); } If "tp_as_number" and "nb_negative" were always guaranteed non-NULL, then the function could omit the second if statement, and invoke the function pointer directly. To maintain the existing behavior, the default nb_negative function would be set to the following: PyObject* nb_negative_default(PyObject *o) { return type_error("bad operand type for unary -: '%.200s'", o); } This removes two NULL-checks from the PyNumber_Negative. Many other operators and builtins would be able to benefit in the same way. ---------- components: Interpreter Core messages: 304420 nosy: pdox priority: normal severity: normal status: open title: Ensure that all PyTypeObject fields are set to non-NULL defaults type: performance versions: Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun Oct 15 05:35:57 2017 From: report at bugs.python.org (Serhiy Storchaka) Date: Sun, 15 Oct 2017 09:35:57 +0000 Subject: [New-bugs-announce] [issue31792] test_buffer altered the execution environment Message-ID: <1508060157.98.0.213398074469.issue31792@psf.upfronthosting.co.za> New submission from Serhiy Storchaka : test_buffer adds two environment variables: 'OPENBLAS_MAIN_FREE': '1', 'GOTOBLAS_MAIN_FREE': '1'. $ ./python -m test -v test_buffer ... Warning -- os.environ was modified by test_buffer Before: (140174267697792, environ({...}) After: (140174267697792, environ({..., 'OPENBLAS_MAIN_FREE': '1', 'GOTOBLAS_MAIN_FREE': '1'}) test_buffer failed (env changed) 1 test altered the execution environment: test_buffer Total duration: 8 sec Tests result: SUCCESS ---------- components: Tests messages: 304426 nosy: ezio.melotti, haypo, michael.foord, serhiy.storchaka, skrah priority: normal severity: normal status: open title: test_buffer altered the execution environment type: resource usage versions: Python 3.6, Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun Oct 15 05:49:49 2017 From: report at bugs.python.org (Julien Palard) Date: Sun, 15 Oct 2017 09:49:49 +0000 Subject: [New-bugs-announce] [issue31793] Allow to specialize smart quotes in documentation translations Message-ID: <1508060988.99.0.213398074469.issue31793@psf.upfronthosting.co.za> New submission from Julien Palard : It would be great to allow each translation to specify their smart-quotes configuration, typically Japanese prefer no smart-quotes: - https://github.com/python/docsbuild-scripts/issues/32 It's easily done by adding a Doc/docutils.conf file, I'm opening a PR. ---------- assignee: docs at python components: Documentation messages: 304429 nosy: docs at python, mdk priority: normal severity: normal status: open title: Allow to specialize smart quotes in documentation translations versions: Python 3.6, Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun Oct 15 07:39:04 2017 From: report at bugs.python.org (Serhiy Storchaka) Date: Sun, 15 Oct 2017 11:39:04 +0000 Subject: [New-bugs-announce] [issue31794] Issues with test.autotest Message-ID: <1508067544.81.0.213398074469.issue31794@psf.upfronthosting.co.za> New submission from Serhiy Storchaka : There are issues with running tests by importing test.autotest. 1. test_builtin failed. 0:00:00 load avg: 0.70 [ 4/407] test_builtin test test_builtin failed -- Traceback (most recent call last): File "/home/serhiy/py/cpython/Lib/test/test_builtin.py", line 1735, in test_input_no_stdout_fileno lines = self.run_child(child, b"quux\r") File "/home/serhiy/py/cpython/Lib/test/test_builtin.py", line 1680, in run_child % (len(lines), child_output)) AssertionError: got 1 lines in pipe but expected 2, child output was: quux Traceback (most recent call last): File "/home/serhiy/py/cpython/Lib/test/test_builtin.py", line 1646, in run_child child(wpipe) File "/home/serhiy/py/cpython/Lib/test/test_builtin.py", line 1733, in child input("prompt") EOFError: EOF when reading a line 0:00:00 load avg: 0.70 [ 5/407/1] test_exceptions -- test_builtin failed test test_builtin failed -- Traceback (most recent call last): File "/home/serhiy/py/cpython/Lib/test/test_builtin.py", line 1735, in test_input_no_stdout_fileno lines = self.run_child(child, b"quux\r") File "/home/serhiy/py/cpython/Lib/test/test_builtin.py", line 1680, in run_child % (len(lines), child_output)) AssertionError: got 1 lines in pipe but expected 2, child output was: quux Traceback (most recent call last): File "/home/serhiy/py/cpython/Lib/test/test_builtin.py", line 1646, in run_child child(wpipe) File "/home/serhiy/py/cpython/Lib/test/test_builtin.py", line 1733, in child input("prompt") EOFError: EOF when reading a line 0:00:01 load avg: 0.70 [ 5/407/1] test_exceptions -- test_builtin failed 0:00:01 load avg: 0.70 [ 4/407] test_builtin Warning -- reap_children() reaped child process 4752 0:00:01 load avg: 0.70 [ 4/407] test_builtin 0:00:01 load avg: 0.70 [ 6/407/1] test_types test test_builtin failed -- Traceback (most recent call last): File "/home/serhiy/py/cpython/Lib/test/test_builtin.py", line 1735, in test_input_no_stdout_fileno lines = self.run_child(child, b"quux\r") File "/home/serhiy/py/cpython/Lib/test/test_builtin.py", line 1680, in run_child % (len(lines), child_output)) AssertionError: got 1 lines in pipe but expected 2, child output was: quux Traceback (most recent call last): File "/home/serhiy/py/cpython/Lib/test/test_builtin.py", line 1646, in run_child child(wpipe) File "/home/serhiy/py/cpython/Lib/test/test_builtin.py", line 1733, in child input("prompt") EOFError: EOF when reading a line 0:00:01 load avg: 0.70 [ 5/407/1] test_exceptions -- test_builtin failed Warning -- reap_children() reaped child process 4755 0:00:01 load avg: 0.70 [ 6/407/1] test_types 0:00:01 load avg: 0.70 [ 7/407/1] test_unittest Warning -- reap_children() reaped child process 4758 0:00:01 load avg: 1.12 [ 6/407/1] test_types 0:00:01 load avg: 1.12 [ 7/407/1] test_unittest 0:00:01 load avg: 1.12 [ 7/407/1] test_unittest test test_builtin failed -- Traceback (most recent call last): File "/home/serhiy/py/cpython/Lib/test/test_builtin.py", line 1735, in test_input_no_stdout_fileno lines = self.run_child(child, b"quux\r") File "/home/serhiy/py/cpython/Lib/test/test_builtin.py", line 1680, in run_child % (len(lines), child_output)) AssertionError: got 1 lines in pipe but expected 2, child output was: quux Traceback (most recent call last): File "/home/serhiy/py/cpython/Lib/test/test_builtin.py", line 1646, in run_child child(wpipe) File "/home/serhiy/py/cpython/Lib/test/test_builtin.py", line 1733, in child input("prompt") EOFError: EOF when reading a line 0:00:01 load avg: 1.12 [ 5/407/1] test_exceptions -- test_builtin failed test test_builtin failed -- Traceback (most recent call last): File "/home/serhiy/py/cpython/Lib/test/test_builtin.py", line 1735, in test_input_no_stdout_fileno lines = self.run_child(child, b"quux\r") File "/home/serhiy/py/cpython/Lib/test/test_builtin.py", line 1680, in run_child % (len(lines), child_output)) AssertionError: got 1 lines in pipe but expected 2, child output was: quux Traceback (most recent call last): File "/home/serhiy/py/cpython/Lib/test/test_builtin.py", line 1646, in run_child child(wpipe) File "/home/serhiy/py/cpython/Lib/test/test_builtin.py", line 1733, in child input("prompt") EOFError: EOF when reading a line 0:00:01 load avg: 1.12 [ 5/407/1] test_exceptions -- test_builtin failed Warning -- reap_children() reaped child process 4761 0:00:01 load avg: 1.12 [ 6/407/1] test_types Warning -- reap_children() reaped child process 4763 Note that warnings and errors are emitted even after reporting that test_builtin failed. 2. Tests are ran multiple times. 0:02:32 load avg: 4.82 [ 77/407/1] test_concurrent_futures 0:02:35 load avg: 4.82 [ 76/407/1] test_complex 0:02:35 load avg: 4.82 [ 77/407/1] test_concurrent_futures test test_concurrent_futures failed -- multiple errors occurred; run in verbose mode for details 0:03:33 load avg: 1.77 [ 78/407/2] test_configparser -- test_concurrent_futures failed in 1 min 5 sec test test_concurrent_futures failed -- multiple errors occurred; run in verbose mode for details 0:03:33 load avg: 1.77 [ 78/407/2] test_configparser -- test_concurrent_futures failed in 1 min 5 sec 0:03:34 load avg: 1.77 [ 79/407/2] test_contains 0:03:34 load avg: 1.77 [ 79/407/2] test_contains 0:03:34 load avg: 1.77 [ 80/407/2] test_contextlib 0:03:34 load avg: 1.77 [ 80/407/2] test_contextlib 0:03:34 load avg: 1.77 [ 81/407/2] test_contextlib_async 0:03:34 load avg: 1.77 [ 81/407/2] test_contextlib_async 0:03:34 load avg: 1.77 [ 82/407/2] test_copy 0:03:34 load avg: 1.77 [ 82/407/2] test_copy 0:03:34 load avg: 1.77 [ 83/407/2] test_copyreg 0:03:34 load avg: 1.77 [ 83/407/2] test_copyreg 0:03:34 load avg: 1.77 [ 84/407/2] test_coroutines test test_concurrent_futures failed -- multiple errors occurred; run in verbose mode for details 0:03:35 load avg: 1.77 [ 84/407/2] test_coroutines 0:03:35 load avg: 1.77 [ 78/407/2] test_configparser -- test_concurrent_futures failed in 1 min 5 sec 0:03:35 load avg: 1.77 [ 85/407/2] test_cprofile 0:03:35 load avg: 1.77 [ 85/407/2] test_cprofile 0:03:35 load avg: 1.77 [ 79/407/2] test_contains 0:03:35 load avg: 1.77 [ 86/407/2] test_crashers 0:03:36 load avg: 1.77 [ 80/407/2] test_contextlib 0:03:36 load avg: 1.77 [ 86/407/2] test_crashers 0:03:36 load avg: 1.77 [ 87/407/2] test_crypt 0:03:36 load avg: 1.77 [ 87/407/2] test_crypt 0:03:36 load avg: 1.77 [ 88/407/2] test_csv 0:03:36 load avg: 1.77 [ 81/407/2] test_contextlib_async 0:03:36 load avg: 1.95 [ 88/407/2] test_csv 0:03:36 load avg: 1.95 [ 82/407/2] test_copy 0:03:36 load avg: 1.95 [ 83/407/2] test_copyreg 0:03:36 load avg: 1.95 [ 84/407/2] test_coroutines 0:03:37 load avg: 1.95 [ 85/407/2] test_cprofile test test_concurrent_futures failed -- multiple errors occurred; run in verbose mode for details 0:03:37 load avg: 1.95 [ 78/407/2] test_configparser -- test_concurrent_futures failed in 1 min 6 sec 3. test_import changed the environment. Warning -- files was modified by test_import Before: [] After: ['@test_4738_tmp.pyc'] 0:06:18 load avg: 3.72 [178/407/3] test_importlib -- test_import failed (env changed) 4. test_logging failed. 0:07:43 load avg: 3.39 [200/407/3] test_logging test test_logging failed -- Traceback (most recent call last): File "/home/serhiy/py/cpython/Lib/test/test_logging.py", line 3667, in test_multiprocessing self.assertEqual(r.processName, 'MainProcess') AssertionError: 'ForkServerProcess-192' != 'MainProcess' - ForkServerProcess-192 + MainProcess 5. test_mailbox changed the environment. 0:08:10 load avg: 3.13 [205/407/4] test_mailbox Warning -- files was modified by test_mailbox Before: ['@test_4736_tmp.pyc'] After: [] 0:08:11 load avg: 3.36 [206/407/5] test_mailcap -- test_mailbox failed (env changed) 6. test_multiprocessing_fork, test_multiprocessing_forkserver, test_multiprocessing_spawn failed. 7. test_os failed. test test_os failed -- Traceback (most recent call last): File "/home/serhiy/py/cpython/Lib/test/test_os.py", line 2923, in test_stty_match actual = os.get_terminal_size(sys.__stdin__.fileno()) ValueError: I/O operation on closed file 0:11:07 load avg: 2.82 [234/407/9] test_ossaudiodev -- test_os failed 8. The following traceback is output after reporting results. Exception in thread Thread-162: Traceback (most recent call last): File "/home/serhiy/py/cpython/Lib/threading.py", line 917, in _bootstrap_inner self.run() File "/home/serhiy/py/cpython/Lib/threading.py", line 865, in run self._target(*self._args, **self._kwargs) File "/home/serhiy/py/cpython/Lib/concurrent/futures/process.py", line 298, in _queue_management_worker shutdown_worker() File "/home/serhiy/py/cpython/Lib/concurrent/futures/process.py", line 256, in shutdown_worker call_queue.put_nowait(None) File "/home/serhiy/py/cpython/Lib/multiprocessing/queues.py", line 129, in put_nowait return self.put(obj, False) File "/home/serhiy/py/cpython/Lib/multiprocessing/queues.py", line 83, in put raise Full queue.Full 9. After running all tests they are ran again from the start. ---------- components: Tests messages: 304433 nosy: serhiy.storchaka priority: normal severity: normal status: open title: Issues with test.autotest type: behavior versions: Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun Oct 15 18:45:56 2017 From: report at bugs.python.org (Nitish) Date: Sun, 15 Oct 2017 22:45:56 +0000 Subject: [New-bugs-announce] [issue31795] Slicings documentation doesn't mention Ellipsis Message-ID: <1508107556.27.0.213398074469.issue31795@psf.upfronthosting.co.za> New submission from Nitish : Ellipsis object documentation[1] specifies that Ellipsis is mainly used by Slicings. But Slicing's documentation[2] doesn't mention anything about Ellipsis. Consequently - it is not immediately clear from Ellipsis documentation what it's used for. [1] https://docs.python.org/3/library/stdtypes.html?highlight=ellipsis#the-ellipsis-object [2] https://docs.python.org/3/reference/expressions.html#slicings ---------- assignee: docs at python components: Documentation messages: 304449 nosy: docs at python, nitishch priority: normal severity: normal status: open title: Slicings documentation doesn't mention Ellipsis type: enhancement _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun Oct 15 21:07:22 2017 From: report at bugs.python.org (fuqianz) Date: Mon, 16 Oct 2017 01:07:22 +0000 Subject: [New-bugs-announce] [issue31796] The mccabe complexity output in module flake8. Message-ID: New submission from fuqianz : Dear, I have found that if a function name appears more than one time in a file, the output of McCabe complexity for this function name only appear once. I think it should output all of them. So, I send this email for inquiring about this confusing problems. Thanks, Sincerely, fuqianz ---------- messages: 304450 nosy: fqbrighter priority: normal severity: normal status: open title: The mccabe complexity output in module flake8. _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Oct 16 06:14:43 2017 From: report at bugs.python.org (Francesco Mantovani) Date: Mon, 16 Oct 2017 10:14:43 +0000 Subject: [New-bugs-announce] [issue31797] Python 3.6.3: JSON loop fails using elif Message-ID: <1508148883.41.0.213398074469.issue31797@psf.upfronthosting.co.za> New submission from Francesco Mantovani : I attach here a file with 3 different way to parse Google Places API. Please put your Google API key token into the variable [PutHereYourGoogleAPIKey] to make the script work. The script contains 3 loops: - the first loop works and that's how I fixed the problem - the second loop fail because of the `elif` - the third loop works because all `elif` were removed ---------- components: Interpreter Core files: Test_log_bug.py messages: 304465 nosy: Francesco Mantovani priority: normal severity: normal status: open title: Python 3.6.3: JSON loop fails using elif type: behavior versions: Python 3.6 Added file: https://bugs.python.org/file47222/Test_log_bug.py _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Oct 16 06:55:47 2017 From: report at bugs.python.org (Alexander McFarlane) Date: Mon, 16 Oct 2017 10:55:47 +0000 Subject: [New-bugs-announce] [issue31798] abs__file__ in "Lib/site.py" Message-ID: <1508151347.82.0.213398074469.issue31798@psf.upfronthosting.co.za> Change by Alexander McFarlane : ---------- nosy: Alexander McFarlane priority: normal severity: normal status: open title: abs__file__ in "Lib/site.py" _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Oct 16 10:16:27 2017 From: report at bugs.python.org (Barry A. Warsaw) Date: Mon, 16 Oct 2017 14:16:27 +0000 Subject: [New-bugs-announce] [issue31799] Improve __spec__ discoverability Message-ID: <1508163387.44.0.213398074469.issue31799@psf.upfronthosting.co.za> New submission from Barry A. Warsaw : __spec__ is defined in PEP 451. If you search for "__spec__" in the docs, you get a number of hits. https://docs.python.org/3/search.html?q=__spec__&check_keywords=yes&area=default Click on the first link: https://docs.python.org/3/reference/import.html?highlight=__spec__#__spec__ but that still leaves you scratching your head as to what exactly is in __spec__. If you happen to scroll up a little bit though, you end up here: https://docs.python.org/3/reference/import.html?highlight=__spec__#module-spec and then if you follow the link to ModuleSpec, you finally get to here: https://docs.python.org/3/library/importlib.html#importlib.machinery.ModuleSpec and *that's* where the contents of __spec__ are defined. Not very discoverable. I propose just a couple of small documentation fixes to add "__spec__" in both of those locations so that a search lands you in a useful place. ---------- assignee: barry components: Documentation messages: 304474 nosy: barry priority: normal severity: normal status: open title: Improve __spec__ discoverability versions: Python 3.4, Python 3.5, Python 3.6, Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Oct 16 18:51:49 2017 From: report at bugs.python.org (Mario Corchero) Date: Mon, 16 Oct 2017 22:51:49 +0000 Subject: [New-bugs-announce] [issue31800] datetime.strptime: Support for parsing offsets with a colon Message-ID: <1508194309.26.0.213398074469.issue31800@psf.upfronthosting.co.za> New submission from Mario Corchero : Currently, datetime.strptime does not support parsing utc offsets that include a colon. "+0000" is parsed without issues whilst it fails with "+00:00". "+NN:NN" is not only ISO8601 valid but also the way the offset is presented to the user when using .isoformat on a datetime with a timezone/offset. This lead to the users needing to go to external libraries like dateutil or iso8601 just to be able to parse the datetime encoded in strings that "datetime" produces. Even if a long-term goal would be to provide a way to parse any isoformatted string this issue just aims to address the problem that the %z parsing presents. This already unblocks users from parsing datetime object serialized with isoformat. With this change, the following will just work: >>> import datetime as dt >>> iso_fmt = '%Y-%m-%dT%H:%M:%S%z' >>> d = dt.datetime.strptime('2004-01-01T10:10:10+05:00', iso_fmt) *'2004-01-01T10:10:10+05:00' is a sample string generated via datetime.isoformat() Other options like having a new %:z was proposed but having just %z seems much simpler for the user. Note: There has been already conversations about adding support on datetime to parse any ISO-formatted string. This is a more simplistic approach. We might be able to get to that situation after this patch, but this aims just to unblock us. Related: http://www.loc.gov/standards/datetime/iso-tc154-wg5_n0039_iso_wd_8601-2_2016-02-16.pdf https://mail.python.org/pipermail/python-ideas/2014-March/027018.html https://bugs.python.org/issue15873 ---------- components: Library (Lib) messages: 304486 nosy: mariocj89 priority: normal severity: normal status: open title: datetime.strptime: Support for parsing offsets with a colon type: enhancement versions: Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Oct 16 20:21:55 2017 From: report at bugs.python.org (Ethan Furman) Date: Tue, 17 Oct 2017 00:21:55 +0000 Subject: [New-bugs-announce] [issue31801] vars() manipulation encounters problems with Enum Message-ID: <1508199715.37.0.213398074469.issue31801@psf.upfronthosting.co.za> New submission from Ethan Furman : The following code should work (from https://stackoverflow.com/a/46780742/208880): class BaudRate(Enum): cls = vars() regexp = r"(?:^|,)B(?P\d+)" rates = sorted(map(int, re.findall(regexp, ",".join(dir(termios))))) for value in rates: cls['B%d' % value] = value @classmethod def valid_rate(cls, value): return (any(value == item.value for item in cls)) This doesn't work because EnumMeta does not allow names to be reassigned nor deleted. aenum gets around this by allowing an _ignore_ attribute which contains the names that should not be tracked (and, in fact, those names are removed from the final class). Unless a better idea is put forward, I will add _ignore_ support to Enum. ---------- assignee: ethan.furman messages: 304487 nosy: barry, eli.bendersky, ethan.furman priority: normal severity: normal stage: test needed status: open title: vars() manipulation encounters problems with Enum type: behavior versions: Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Oct 17 05:45:21 2017 From: report at bugs.python.org (Mark Shannon) Date: Tue, 17 Oct 2017 09:45:21 +0000 Subject: [New-bugs-announce] [issue31802] 'import posixpath' fails if 'os.path' has not be imported already. Message-ID: <1508233521.36.0.213398074469.issue31802@psf.upfronthosting.co.za> New submission from Mark Shannon : $ python3.7 -S >>> import posixpath Traceback (most recent call last): File "", line 1, in File "--/Lib/posixpath.py", line 13, in import os File "--/Lib/os.py", line 92, in from os.path import (curdir, pardir, sep, pathsep, defpath, extsep, altsep, ImportError: cannot import name 'curdir' from 'posixpath' (--/Lib/posixpath.py) Whether this counts as a bug or not is debatable. It could be argued that you shouldn't be importing 'posixpath' directly, in which case it ought to be called '_posixpath', but I guess it is too late to be changing the name. ---------- messages: 304494 nosy: Mark.Shannon priority: normal severity: normal stage: needs patch status: open title: 'import posixpath' fails if 'os.path' has not be imported already. type: crash _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Oct 17 09:00:45 2017 From: report at bugs.python.org (STINNER Victor) Date: Tue, 17 Oct 2017 13:00:45 +0000 Subject: [New-bugs-announce] [issue31803] Remove not portable time.clock(), replaced by time.perf_counter() and time.process_time() Message-ID: <1508245245.67.0.213398074469.issue31803@psf.upfronthosting.co.za> New submission from STINNER Victor : The behaviour of the time.clock() function is not portable: on Windows it mesures wall-clock, whereas it measures CPU time on Unix. Python has time.process_time() and time.perf_counter(), since Python 3.3, which are portable, well defined and have a better resolution. time.clock() was deprecated in Python 3.3 documentation, but calling time.clock() didn't raise a DeprecationWarning. I proposed to remove it from Python 3.7. Sometimes, a deprecated function raises a DeprecationWarning in Python version N, before removing it from Python version N. time.clock() doesn't emit such warning, but I consider that it is possible to remove it anyway. While it can be annoying to have to patch code to no more use time.clock(), I consider that it's worth it for portability and better clock resolution. Attached PR removes time.clock() and time.get_clock_info() doens't accept 'clock' anymore. Current time.clock() documentation: ---------------------------- .. function:: clock() .. index:: single: CPU time single: processor time single: benchmarking On Unix, return the current processor time as a floating point number expressed in seconds. The precision, and in fact the very definition of the meaning of "processor time", depends on that of the C function of the same name. On Windows, this function returns wall-clock seconds elapsed since the first call to this function, as a floating point number, based on the Win32 function :c:func:`QueryPerformanceCounter`. The resolution is typically better than one microsecond. .. deprecated:: 3.3 The behaviour of this function depends on the platform: use :func:`perf_counter` or :func:`process_time` instead, depending on your requirements, to have a well defined behaviour. ---------------------------- ---------- components: Library (Lib) messages: 304502 nosy: haypo priority: normal severity: normal status: open versions: Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Oct 17 10:59:34 2017 From: report at bugs.python.org (Pox TheGreat) Date: Tue, 17 Oct 2017 14:59:34 +0000 Subject: [New-bugs-announce] [issue31804] multiprocessing calls flush on sys.stdout at exit even if it is None (pythonw) Message-ID: <1508252374.96.0.213398074469.issue31804@psf.upfronthosting.co.za> New submission from Pox TheGreat : If you start Python by pythonw then sys.stdout and sys.stderr are set to None. If you also use multiprocessing then when the child process finishes BaseProcess._bootstrap calls sys.stdout.flush() and sys.stderr.flush() finally. This causes the process return code to be not zero (it is 1). ---------- components: Library (Lib) files: process.py.patch keywords: patch messages: 304512 nosy: Pox TheGreat priority: normal severity: normal status: open title: multiprocessing calls flush on sys.stdout at exit even if it is None (pythonw) type: crash versions: Python 3.5 Added file: https://bugs.python.org/file47224/process.py.patch _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Oct 17 15:47:15 2017 From: report at bugs.python.org (STINNER Victor) Date: Tue, 17 Oct 2017 19:47:15 +0000 Subject: [New-bugs-announce] [issue31805] support._is_gui_available() hangs on x86-64 Sierra 3.6 buildbot Message-ID: <1508269635.57.0.213398074469.issue31805@psf.upfronthosting.co.za> New submission from STINNER Victor : http://buildbot.python.org/all/#/builders/20/builds/11 Re-running test 'test_ttk_guionly' in verbose mode Timeout (0:15:00)! Thread 0x00007fffc0cd73c0 (most recent call first): File "/Users/buildbot/buildarea/3.6.billenstein-sierra/build/Lib/test/support/__init__.py", line 485 in _is_gui_available File "/Users/buildbot/buildarea/3.6.billenstein-sierra/build/Lib/test/support/__init__.py", line 531 in requires File "/Users/buildbot/buildarea/3.6.billenstein-sierra/build/Lib/test/test_ttk_guionly.py", line 8 in File "", line 219 in _call_with_frames_removed File "", line 678 in exec_module File "", line 665 in _load_unlocked File "", line 955 in _find_and_load_unlocked File "", line 971 in _find_and_load File "", line 994 in _gcd_import File "/Users/buildbot/buildarea/3.6.billenstein-sierra/build/Lib/importlib/__init__.py", line 126 in import_module File "/Users/buildbot/buildarea/3.6.billenstein-sierra/build/Lib/test/libregrtest/runtest.py", line 160 in runtest_inner File "/Users/buildbot/buildarea/3.6.billenstein-sierra/build/Lib/test/libregrtest/runtest.py", line 137 in runtest File "/Users/buildbot/buildarea/3.6.billenstein-sierra/build/Lib/test/libregrtest/main.py", line 290 in rerun_failed_tests File "/Users/buildbot/buildarea/3.6.billenstein-sierra/build/Lib/test/libregrtest/main.py", line 539 in _main File "/Users/buildbot/buildarea/3.6.billenstein-sierra/build/Lib/test/libregrtest/main.py", line 509 in main File "/Users/buildbot/buildarea/3.6.billenstein-sierra/build/Lib/test/libregrtest/main.py", line 584 in main File "/Users/buildbot/buildarea/3.6.billenstein-sierra/build/Lib/test/__main__.py", line 2 in File "/Users/buildbot/buildarea/3.6.billenstein-sierra/build/Lib/runpy.py", line 85 in _run_code File "/Users/buildbot/buildarea/3.6.billenstein-sierra/build/Lib/runpy.py", line 193 in _run_module_as_main make: *** [buildbottest] Error 1 ---------- components: Tests, macOS keywords: buildbot messages: 304518 nosy: haypo, ned.deily, ronaldoussoren priority: normal severity: normal status: open title: support._is_gui_available() hangs on x86-64 Sierra 3.6 buildbot _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Oct 17 18:21:32 2017 From: report at bugs.python.org (Pablo Galindo Salgado) Date: Tue, 17 Oct 2017 22:21:32 +0000 Subject: [New-bugs-announce] [issue31806] Use _PyTime_ROUND_TIMEOUT in _threadmodule.c, timemodule.c and socketmodule.c Message-ID: <1508278892.53.0.213398074469.issue31806@psf.upfronthosting.co.za> New submission from Pablo Galindo Salgado : Following PR for https://bugs.python.org/issue31786 time_sleep, lock_acquire_parse_args and socket_parse_timeout should use _PyTime_ROUND_TIMEOUT instead of _PyTime_ROUND_CEILING. ---------- components: Library (Lib) messages: 304540 nosy: pablogsal priority: normal severity: normal status: open title: Use _PyTime_ROUND_TIMEOUT in _threadmodule.c, timemodule.c and socketmodule.c versions: Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Oct 17 20:48:15 2017 From: report at bugs.python.org (John Villalovos) Date: Wed, 18 Oct 2017 00:48:15 +0000 Subject: [New-bugs-announce] [issue31807] Using autospec=True conflicts with 'wraps' Message-ID: <1508287695.22.0.213398074469.issue31807@psf.upfronthosting.co.za> New submission from John Villalovos : If have autospec=True, then no ValueError is raised. If autospec=False or not defined, then the ValueError is raised. import sys from unittest import mock def wrapped_func(value): raise ValueError(value) @mock.patch('__main__.wrapped_func', autospec=True, wraps=wrapped_func) def main(mock_wrap): wrapped_func("testing") if '__main__' == __name__: sys.exit(main()) ---------- components: Library (Lib) messages: 304549 nosy: John Villalovos priority: normal severity: normal status: open title: Using autospec=True conflicts with 'wraps' type: behavior versions: Python 3.6 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed Oct 18 04:29:35 2017 From: report at bugs.python.org (Frederic Beister) Date: Wed, 18 Oct 2017 08:29:35 +0000 Subject: [New-bugs-announce] [issue31808] tarfile.extractall fails to overwrite symlinks Message-ID: <1508315375.63.0.213398074469.issue31808@psf.upfronthosting.co.za> New submission from Frederic Beister : The fix for https://bugs.python.org/issue10761 somehow didn't make it into 3.5 and later - I have the same behavior as described there in 3.5.1 and 3.6.1 (tested on Ubuntu 16.04) ---------- components: Library (Lib) messages: 304563 nosy: Frederic Beister priority: normal severity: normal status: open title: tarfile.extractall fails to overwrite symlinks type: behavior versions: Python 3.5, Python 3.6 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed Oct 18 09:15:40 2017 From: report at bugs.python.org (A) Date: Wed, 18 Oct 2017 13:15:40 +0000 Subject: [New-bugs-announce] [issue31809] ssl module unnecessarily pins the client curve when using ECDH Message-ID: <1508332540.28.0.213398074469.issue31809@psf.upfronthosting.co.za> New submission from A : When using elliptic curves in combination with the ssl module to wrap a socket, the only curve the client accepts is prime256v1. Expected behavior is to accept all curves that are on the default list and supported by the server. We noticed the issue when connecting to a server that is configured with a non standard curve, we get [SSL: WRONG_CURVE] wrong curve (_ssl.c:661) unless we explicitly specify to use the correct curve for the context using context.set_ecdh_curve("secp521r1") The bug happens both when using a context or ssl.wrap_socket directly. The bug is that cpython calls OpenSSL (or whatever lib is linked) server methods even when the context is purely for client communications. The flow is: - A ssl.SSLContext gets instantiated that will later be used in SSLContext.wrap_socket with server_side either True or False (bug happens when this is client side, so server_side=False). - In the context's constructor, where it's not yet clear if the context is for a client or a server socket, there is this code: https://github.com/python/cpython/blob/b9a860f3bf80b0d4a6c25d0f2f6ef849d9bf3594/Modules/_ssl.c#L2180 That code calls (in many cases, depending on the Python and OpenSSL versions involved) server side only methods, SSL_CTX_set_ecdh_auto or SSL_CTX_set_tmp_ecdh. Those methods should in theory not influence the client side of the context, there is even a comment in the docs for SSLContext.set_ecdh_curve which does similar things that says "This setting doesn?t apply to client sockets". However, this being OpenSSL, this is not true. The methods actually set the list of acceptable curves that is used for both, server and client side connections, to exactly a single curve, prime256v1. This happens for both, SSL_CTX_set_tmp_ecdh and SSL_CTX_set_ecdh_auto. Versions affected: OpenSSL has changed the API twice here. Before 1.0.2, SSL_CTX_set_tmp_ecdh was mandatory, 1.0.2 - <1.1.0 used SSL_CTX_set_ecdh_auto and 1.1.0+ doesn't need this setting at all. Python 2.7.14+ added a check for 1.1.0+ in https://github.com/python/cpython/commit/f1a696efd6ca674579e25de29ec4053ff5a5ade1 so starting from that version the issue only happens when using OpenSSL <1.1.0. I suspect this is the case for many machines still, including all Macs (haven't confirmed the actual bug there). For LibreSSL the server side methods will be called too, I can't confirm if that combination is vulnerable too. Python 3.5.3 gives me the same error, no reason to believe that higher versions are not affected. ---------- assignee: christian.heimes components: SSL messages: 304577 nosy: christian.heimes, grrrrrrrrr priority: normal severity: normal status: open title: ssl module unnecessarily pins the client curve when using ECDH type: behavior versions: Python 2.7, Python 3.5 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed Oct 18 11:22:59 2017 From: report at bugs.python.org (STINNER Victor) Date: Wed, 18 Oct 2017 15:22:59 +0000 Subject: [New-bugs-announce] [issue31810] Travis CI, buildbots: run "make smelly" to check if CPython leaks symbols Message-ID: <1508340179.09.0.213398074469.issue31810@psf.upfronthosting.co.za> New submission from STINNER Victor : Recently, a new "cell_set_contents()" public symbol was added by mistake: see bpo-30486. It was quickly noticed by doko, and fixed by me (commit 0ad05c32cc41d4c21bfd78b9ffead519ead475a2). The problem is that we already have a "make smelly" command to check if Python "leaks" symbols, but this check is not run on our CIs (Travis CI or buildbots). It would be nice to run automatically this test. ---------- components: Tests keywords: buildbot messages: 304582 nosy: haypo, zach.ware priority: normal severity: normal status: open title: Travis CI, buildbots: run "make smelly" to check if CPython leaks symbols versions: Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed Oct 18 11:32:03 2017 From: report at bugs.python.org (Colin Dunklau) Date: Wed, 18 Oct 2017 15:32:03 +0000 Subject: [New-bugs-announce] [issue31811] async and await missing from keyword list in lexical analysis doc Message-ID: <1508340723.02.0.213398074469.issue31811@psf.upfronthosting.co.za> New submission from Colin Dunklau : I see that code making async/await real keywords has been merged, but it looks like Doc/reference/lexical_analysis.rst doesn't have those added https://github.com/python/cpython/blob/4a2d00c/Doc/reference/lexical_analysis.rst#keywords Is that list autogenerated somehow or was it just overlooked? ---------- assignee: docs at python components: Documentation messages: 304583 nosy: Colin Dunklau, docs at python priority: normal severity: normal status: open title: async and await missing from keyword list in lexical analysis doc type: enhancement versions: Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed Oct 18 11:42:17 2017 From: report at bugs.python.org (STINNER Victor) Date: Wed, 18 Oct 2017 15:42:17 +0000 Subject: [New-bugs-announce] [issue31812] Document PEP 545 (documentation translation) in What's New in Python 3.7 Message-ID: <1508341337.17.0.213398074469.issue31812@psf.upfronthosting.co.za> New submission from STINNER Victor : The PEP 545 should be documented in What's New in Python 3.7: mention that the Python documentation is now *officially* translated to french and japanese. ---------- assignee: docs at python components: Documentation messages: 304584 nosy: docs at python, haypo, inada.naoki, mdk priority: normal severity: normal status: open title: Document PEP 545 (documentation translation) in What's New in Python 3.7 versions: Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed Oct 18 11:42:58 2017 From: report at bugs.python.org (Serhy Pyton) Date: Wed, 18 Oct 2017 15:42:58 +0000 Subject: [New-bugs-announce] [issue31813] python -m enshure pip stucks Message-ID: <1508341378.28.0.213398074469.issue31813@psf.upfronthosting.co.za> New submission from Serhy Pyton : pip stucks, during installing python 3.6.3 in termux and not complete it. Posibly it's could be a problem only on meizu m2. Also it produse load in such state. ---------- components: Installation files: S71018-182104.jpg messages: 304585 nosy: Serhy Pyton priority: normal severity: normal status: open title: python -m enshure pip stucks type: behavior versions: Python 3.6 Added file: https://bugs.python.org/file47226/S71018-182104.jpg _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed Oct 18 12:24:24 2017 From: report at bugs.python.org (Albert Zeyer) Date: Wed, 18 Oct 2017 16:24:24 +0000 Subject: [New-bugs-announce] [issue31814] subprocess_fork_exec more stable with vfork Message-ID: <1508343864.17.0.213398074469.issue31814@psf.upfronthosting.co.za> New submission from Albert Zeyer : subprocess_fork_exec currently calls fork(). I propose to use vfork() or posix_spawn() or syscall(SYS_clone, SIGCHLD, 0) instead if possible and if there is no preexec_fn. The difference would be that fork() will call any atfork handlers (registered via pthread_atfork()), while the suggested calls would not. There are cases where atfork handlers are registered which are not save to be called e.g. in multi-threaded environments. In the case of subprocess_fork_exec without preexec_fn, there is no need to call those atfork handlers, so avoiding this could avoid potential problems. It's maybe acceptable if a pure fork() without exec() doesn't work in this case anymore, but there is no reason that a fork()+exec() should not work in any such cases. This is fixed by my proposed solution. An example case is OpenBlas and OpenMP, which registers an atfork handler which is safe to be called if there are other threads running. See here: https://github.com/tensorflow/tensorflow/issues/13802 https://github.com/xianyi/OpenBLAS/issues/240 https://trac.sagemath.org/ticket/22021 About fork+exec without the atfork handlers, see here for alternatives (like vfork): https://stackoverflow.com/questions/46810597/forkexec-without-atfork-handlers/ ---------- components: Interpreter Core messages: 304587 nosy: Albert.Zeyer priority: normal severity: normal status: open title: subprocess_fork_exec more stable with vfork type: behavior versions: Python 2.7, Python 3.4, Python 3.5, Python 3.6, Python 3.7, Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed Oct 18 12:43:01 2017 From: report at bugs.python.org (Serhiy Storchaka) Date: Wed, 18 Oct 2017 16:43:01 +0000 Subject: [New-bugs-announce] [issue31815] Make itertools iterators interrable Message-ID: <1508344981.58.0.213398074469.issue31815@psf.upfronthosting.co.za> New submission from Serhiy Storchaka : Proposed PR makes tight C loops with itertools iterators interruptible with Ctrl-C. It adds checks for keyboard interrupt in iterators that can produce long sequences without advancing other iterators. For performance checks are performed only for every 0x1000-th item. If for generating new value other iterator should be advanced, the responsibility for checking for keyboard interrupt is attributed to that iterator. This would solve the problem discussed on Python-ideas: https://mail.python.org/pipermail/python-ideas/2017-October/047412.html http://permalink.gmane.org/gmane.comp.python.ideas/47429 Example: >>> import itertools >>> it = itertools.count() >>> it in it ^CTraceback (most recent call last): File "", line 1, in KeyboardInterrupt >>> ---------- components: Extension Modules messages: 304588 nosy: ncoghlan, rhettinger, serhiy.storchaka priority: normal severity: normal stage: patch review status: open title: Make itertools iterators interrable type: enhancement versions: Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu Oct 19 03:48:26 2017 From: report at bugs.python.org (Nils Diefenbach) Date: Thu, 19 Oct 2017 07:48:26 +0000 Subject: [New-bugs-announce] [issue31816] Unexpected behaviour of `dir()` after implementation of __dir__ Message-ID: <1508399306.09.0.213398074469.issue31816@psf.upfronthosting.co.za> New submission from Nils Diefenbach <23okrs20+python at mykolab.com>: When defining a custom __dir__ method in a class, calling dir() on said class still sorts the output. This is, imo, unexpected behaviour, especially if the order of output was specified in __dir__ and is somehow relevant. Example and more detail here https://stackoverflow.com/questions/46824459/custom-dir-returns-list-of-attributes-sorted-alphabetically ---------- messages: 304606 nosy: nlsdfnbch priority: normal severity: normal status: open title: Unexpected behaviour of `dir()` after implementation of __dir__ type: behavior versions: Python 3.5 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu Oct 19 06:07:54 2017 From: report at bugs.python.org (Josh Cullum) Date: Thu, 19 Oct 2017 10:07:54 +0000 Subject: [New-bugs-announce] [issue31817] Compilation Error with Python 3.6.1/3.6.3 with Tkinter Message-ID: <1508407674.96.0.213398074469.issue31817@psf.upfronthosting.co.za> New submission from Josh Cullum : Hi Guys, I'm trying to build Python 3.6.1 and 3.6.3, with both, .configure / make / make install work correctly as they should, however, trying to import _tkinter doesn't work. Going back to the compilation, I get the following error in the make log: *** WARNING: renaming "_tkinter" since importing it failed: build/lib.linux-x86_64-3.6/_tkinter.cpython-36m-x86_64-linux-gnu.so: undefined symbol: Tcl_GetCharLength Following modules built successfully but were removed because they could not be imported: _tkinter Any Idea's on how to fix the undefined symbol error? This happens with both Tcl/Tk 8.6.6 and 8.6.7. The following is the configure options: --prefix=/tools/apps/python/3.6.1 --with-tcltk-includes='-I/tools/apps/Tcl/8.6.7/include -I/tools/apps/Tk/8.6.7/include' --with-tcltk-libs='-L/tools/apps/Tcl/8.6.7/lib -L/tools/apps/Tk/8.6.7/lib' --enable-optimizations --enable-shared ---------- components: Tkinter messages: 304612 nosy: jpc2350 priority: normal severity: normal status: open title: Compilation Error with Python 3.6.1/3.6.3 with Tkinter type: compile error versions: Python 3.6 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu Oct 19 06:30:35 2017 From: report at bugs.python.org (Mirko Friedenhagen) Date: Thu, 19 Oct 2017 10:30:35 +0000 Subject: [New-bugs-announce] [issue31818] macOS HighSierra final - Python Crash because of _scproxy Message-ID: <1508409035.64.0.213398074469.issue31818@psf.upfronthosting.co.za> New submission from Mirko Friedenhagen : The same bug which shows up in https://bugs.python.org/issue30837 is in both the System python provided by Apple (2.7.10) as well as the one coming via Homebrew (2.7.14) (See https://github.com/Homebrew/homebrew-core/blob/master/Formula/python.rb) for build instructions. The culprit is the same as before, `_scproxy`. Setting the environment variable `no_proxy` did the trick for me as well. I attached the crash report ---------- components: macOS files: python2.7_2017-10-18-092216-1_lmka-2hpphfdty3.crash messages: 304614 nosy: Mirko Friedenhagen, ned.deily, ronaldoussoren priority: normal severity: normal status: open title: macOS HighSierra final - Python Crash because of _scproxy type: crash versions: Python 2.7 Added file: https://bugs.python.org/file47227/python2.7_2017-10-18-092216-1_lmka-2hpphfdty3.crash _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu Oct 19 13:37:49 2017 From: report at bugs.python.org (Antoine Pitrou) Date: Thu, 19 Oct 2017 17:37:49 +0000 Subject: [New-bugs-announce] [issue31819] Add sock_recv_into to AbstractEventLoop Message-ID: <1508434669.81.0.213398074469.issue31819@psf.upfronthosting.co.za> New submission from Antoine Pitrou : As sock_recv() is already exposed, it is a no-brainer to add sock_recv_into(), allowing finer buffer management when people are willing to handle a socket object. ---------- components: Library (Lib), asyncio messages: 304623 nosy: giampaolo.rodola, haypo, pitrou, yselivanov priority: normal severity: normal status: open title: Add sock_recv_into to AbstractEventLoop type: enhancement versions: Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu Oct 19 13:57:14 2017 From: report at bugs.python.org (Zirak) Date: Thu, 19 Oct 2017 17:57:14 +0000 Subject: [New-bugs-announce] [issue31820] Calling email.message.set_payload twice produces an invalid eml Message-ID: <1508435834.65.0.213398074469.issue31820@psf.upfronthosting.co.za> New submission from Zirak : Example: In [52]: import email.message In [53]: m = email.message.Message() In [54]: m.set_payload('abc', 'utf8') In [55]: m.get_payload() # correctly encoded Out[55]: 'YWJj\n' In [56]: m.set_payload('abc', 'utf8') In [57]: m.get_payload() # no more encoding? Out[57]: 'abc' In [58]: m.get_payload(decode=True) # wut? Out[58]: b'i\xb7' In [59]: print(str(m)) MIME-Version: 1.0 Content-Type: text/plain; charset="utf8" Content-Transfer-Encoding: base64 abc While the first `set_payload` correctly encodes and sets the message's Content-Transfer-Encoding, the second call doesn't properly encode the payload according to its existing Content-Transfer-Encoding. Tested on 3.6, 3.5 and 2.7. `email.message.set_payload` does not directly encode the payload, instead `email.message.set_charset` does, around line 353: https://github.com/python/cpython/blob/b067c8fdd1e205bd0411417b6d5e4b832c3773fc/Lib/email/message.py#L353-L368 In both invocations of `set_payload`, the payload is not encoded according to the encoding. On the first invocation, the `CTE` header is correctly set according to `charset.get_body_encoding` (#354 and #368) and the payload is encoded (#356 or #367, the latter in this case). On the second invocation, the `CTE` header is already set, so the payload is never encoded. This is especially dangerous when passing `decode=True` to `get_payload` after the 2nd `set_payload`, as that may throw an error in some cases (trying to base64 decode a string which makes no sense to it. that's how I arrived on this bug, but I can't for the life of me replicate an exception). This is a bit finicky to fix. If we change `set_charset` to always encode the current payload, we risk double-encoding when `set_charset` is not called through `set_payload`. However if `set_charset` tries to decode the payload, it will produce incorrect results when *it is* called through `set_payload`. urgh. We can move the actual encoding code away from `set_charset`, either into `set_payload` or a third function, but that'll break any code calling `set_payload` without a charset and then calling `set_charset`. urgh. One possible solution is for both `set_charset` and `set_payload` to call a third function, e.g. `_encode_payload`. Perhaps something like (pseudocode): def set_payload(self, payload, charset): # ... if 'Content-Transfer-Encoding' in self: self._payload = self._encode_payload(payload) self.set_charset(charset) # ... def set_charset(self, charset): # ... if 'Content-Transfer-Encoding' not in self: self._payload = self._encode_payload() self.add_header(...) def _encode_payload(self, payload): # code in lines 353-366 This way, `set_charset` handles the cases where CTE was never defined, and `set_payload` makes sure to encode the payload when a CTE is present. It may work, but something about this gives me unrest. For example, if you `set_charset` once and it encodes your payload as base64, and you `set_charset` again with a charset whose `get_body_encoding` returns qp, the payload would still be base64 even though it *should be* qp. urgh. Is this a big enough concern? Is there a superior approach I'm missing? Thanks in advance! ---------- components: email messages: 304628 nosy: Zirak Ertan, barry, r.david.murray priority: normal severity: normal status: open title: Calling email.message.set_payload twice produces an invalid eml type: behavior versions: Python 2.7, Python 3.5, Python 3.6 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu Oct 19 14:49:03 2017 From: report at bugs.python.org (Antoine Pitrou) Date: Thu, 19 Oct 2017 18:49:03 +0000 Subject: [New-bugs-announce] [issue31821] pause_reading() doesn't work from connection_made() Message-ID: <1508438943.14.0.213398074469.issue31821@psf.upfronthosting.co.za> New submission from Antoine Pitrou : At least in SelectorEventLoop, as add_reader() is called inconditionally after connection_made() returns. ---------- components: Library (Lib), asyncio messages: 304636 nosy: giampaolo.rodola, haypo, pitrou, yselivanov priority: normal severity: normal stage: needs patch status: open title: pause_reading() doesn't work from connection_made() type: behavior versions: Python 3.6, Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu Oct 19 14:54:33 2017 From: report at bugs.python.org (Allen Li) Date: Thu, 19 Oct 2017 18:54:33 +0000 Subject: [New-bugs-announce] [issue31822] Document that urllib.parse.{Defrag, Split, Parse}Result are namedtuples Message-ID: <1508439273.05.0.213398074469.issue31822@psf.upfronthosting.co.za> New submission from Allen Li : It would be useful to document that urllib.parse.{Defrag,Split,Parse}Result are namedtuples, and make that API officially public if it was not otherwise. These classes are implemented as namedtuples in Python 2 and 3, and I am not aware of a reason that that would need to change in the future. In particular, the namedtuple _replace() method is very useful for modifying parts of a URL, a common use case. u = urllib.parse.urlsplit(some_url) u = u._replace(netloc=other_netloc) urllib.parse.urlunsplit(u) # Alternatives not depending on namedtuple API parts = list(u) parts[1] = other_netloc # Using a magic index urllib.parse.urlunsplit(u) u = urllib.parse.SplitResult( # Very ugly scheme=u.scheme, netloc=other_netloc, path=u.path, query=u.query, fragment=u.fragment) ---------- assignee: docs at python components: Documentation messages: 304637 nosy: Allen Li, docs at python priority: normal severity: normal status: open title: Document that urllib.parse.{Defrag,Split,Parse}Result are namedtuples type: enhancement versions: Python 2.7, Python 3.6 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu Oct 19 16:34:16 2017 From: report at bugs.python.org (=?utf-8?b?0JDQu9C10LrRgdC10Lkg0JDQstC10YDRh9C10L3QutC+?=) Date: Thu, 19 Oct 2017 20:34:16 +0000 Subject: [New-bugs-announce] [issue31823] Opaque default value for close_fds argument in Popen.__init__ Message-ID: <1508445256.4.0.213398074469.issue31823@psf.upfronthosting.co.za> New submission from ??????? ????????? : [11:30:03 ~]$ python Python 3.6.1 |Anaconda 4.4.0 (x86_64)| (default, May 11 2017, 13:04:09) [GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.57)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> import subprocess >>> help(subprocess.Popen.__init__) Help on function __init__ in module subprocess: __init__(self, args, bufsize=-1, executable=None, stdin=None, stdout=None, stderr=None, preexec_fn=None, close_fds=, shell=False, cwd=None, env=None, universal_newlines=False, startupinfo=None, creationflags=0, restore_signals=True, start_new_session=False, pass_fds=(), *, encoding=None, errors=None) Create new Popen instance. ---------- components: Library (Lib) messages: 304642 nosy: ??????? ????????? priority: normal severity: normal status: open title: Opaque default value for close_fds argument in Popen.__init__ versions: Python 3.6 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu Oct 19 19:17:42 2017 From: report at bugs.python.org (Peter Lovett) Date: Thu, 19 Oct 2017 23:17:42 +0000 Subject: [New-bugs-announce] [issue31824] Missing default argument detail in documentation of StreamReaderWriter Message-ID: <1508455062.42.0.213398074469.issue31824@psf.upfronthosting.co.za> New submission from Peter Lovett : Documentation of StreamReaderWriter at https://docs.python.org/3/library/codecs.html#codecs.StreamReaderWriter section 7.2.1.4.3 is missing the default value on the errors argument. Should change from: class codecs.StreamReaderWriter(stream, Reader, Writer, errors) to be: class codecs.StreamReaderWriter(stream, Reader, Writer, errors='strict') ---------- assignee: docs at python components: Documentation messages: 304646 nosy: PeterLovett, docs at python priority: normal severity: normal status: open title: Missing default argument detail in documentation of StreamReaderWriter type: behavior versions: Python 3.6 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Oct 20 05:58:20 2017 From: report at bugs.python.org (Sebastian Kacprzak) Date: Fri, 20 Oct 2017 09:58:20 +0000 Subject: [New-bugs-announce] [issue31825] bytes decode raises OverflowError desipte errors='ignore' Message-ID: <1508493500.2.0.213398074469.issue31825@psf.upfronthosting.co.za> New submission from Sebastian Kacprzak : In Python 3.5 and earlier (tested 3.5.3 and 2.7.13) b'\\\xfa'.decode('unicode-escape', errors='ignore') did return '\\?' but in Python 3.6 it raises OverflowError: decoding with 'unicode-escape' codec failed (OverflowError: character argument not in range(0x110000)) Python 3.6.1 (default, Sep 7 2017, 16:36:03) [GCC 6.3.0 20170406] on linux ---------- messages: 304651 nosy: nait priority: normal severity: normal status: open title: bytes decode raises OverflowError desipte errors='ignore' versions: Python 3.6 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Oct 20 09:21:04 2017 From: report at bugs.python.org (Jakub Mateusz Dzik) Date: Fri, 20 Oct 2017 13:21:04 +0000 Subject: [New-bugs-announce] [issue31826] Misleading __version__ attribute of modules in standard library Message-ID: <1508505664.08.0.213398074469.issue31826@psf.upfronthosting.co.za> New submission from Jakub Mateusz Dzik : Several modules of the standard library (at least `re` and `csv`) have `__version__` strings. The string is the same for Python 2.7-3.6: >>> import re, csv; print(re.__version__, csv.__version__) 2.2.1 1.0 while documentation indicates changes in the modules. Semantic versioning (recommended by PEP 440) suggests that at least minor version should change in such case. I suspect it to be a "code fossil" which may be removed according to PEP 396. ---------- components: Library (Lib) messages: 304654 nosy: abukaj priority: normal severity: normal status: open title: Misleading __version__ attribute of modules in standard library type: behavior versions: Python 2.7, Python 3.4, Python 3.5, Python 3.6 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Oct 20 10:03:40 2017 From: report at bugs.python.org (STINNER Victor) Date: Fri, 20 Oct 2017 14:03:40 +0000 Subject: [New-bugs-announce] [issue31827] Remove os.stat_float_times() Message-ID: <1508508220.14.0.213398074469.issue31827@psf.upfronthosting.co.za> New submission from STINNER Victor : os.stat_float_times() was introduced in Python 2.3 to get file modification times with sub-second resolution. The default remains to get time as seconds (integer). See commit f607bdaa77475ec8c94614414dc2cecf8fd1ca0a. The function was introduced to get a smooth transition to time as floating point number, to keep the backward compatibility with Python 2.2. In Python 2.5, os.stat() returns time as float by default: commit fe33d0ba87f5468b50f939724b303969711f3be5. Python 2.5 was released 11 years ago. I consider that people had enough time to migrate their code to float time :-) I modified os.stat_float_times() to emit a DeprecationWarning in Python 3.1: commit 034d0aa2171688c40cee1a723ddcdb85bbce31e8 (bpo-14711). ---------- messages: 304655 nosy: haypo priority: normal severity: normal status: open title: Remove os.stat_float_times() versions: Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Oct 20 12:07:56 2017 From: report at bugs.python.org (Stefan Behnel) Date: Fri, 20 Oct 2017 16:07:56 +0000 Subject: [New-bugs-announce] [issue31828] Support Py_tss_NEEDS_INIT outside of static initialisation Message-ID: <1508515676.46.0.213398074469.issue31828@psf.upfronthosting.co.za> New submission from Stefan Behnel : Following up on issue 25658, it was found that the current definition of Py_tss_NEEDS_INIT restricts its use to initialisers in C and cannot be used for arbitrary assignments. It is currently declared as follows: #define Py_tss_NEEDS_INIT {0} which results in a C compiler error for assignments like "x = Py_tss_NEEDS_INIT". I proposed to change this to #define Py_tss_NEEDS_INIT ((Py_tss_t) {0}) in compliance with GCC and C99, but that fails to compile in MSVC and probably other old C++-ish compilers. I'm not sure how to improve this declaration, but given that it's a public header file, restricting its applicability seems really unfortunate. ---------- components: Extension Modules, Interpreter Core messages: 304661 nosy: masamoto, ncoghlan, scoder priority: normal pull_requests: 4031 severity: normal status: open title: Support Py_tss_NEEDS_INIT outside of static initialisation type: enhancement versions: Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Oct 20 13:41:29 2017 From: report at bugs.python.org (Serhiy Storchaka) Date: Fri, 20 Oct 2017 17:41:29 +0000 Subject: [New-bugs-announce] [issue31829] Portability issues with pickle Message-ID: <1508521289.37.0.213398074469.issue31829@psf.upfronthosting.co.za> New submission from Serhiy Storchaka : After reading numerous pickle-related issues on GitHab, I have found that the most common issue with pickle in Python 2 is using it with files opened with text mode. with open(file_name, "w") as f: pickle.dump(data, f) Initially pickle was a text protocol. But since implementing more efficient binary opcodes it is a binary protocol. Even the default protocol 0 is not completely text-safe. If save and load data containing Unicode strings with "text" protocol 0 using different text/binary modes or using text mode on different platforms, you can get an error or incorrect data. I propose to add more defensive checks for pickle. 1. When save a pickle with protocol 0 (default) to a file opened in text mode (default) emit a Py3k warning. 2. When save a pickle with binary protocols (must be specified explicitly) to a file opened in text mode raise a ValueError on Windows and Mac Classic (resulting data is likely corrupted) and emit a warning on Unix and Linux. What the type of of warnings is more appropriate? DeprecationWarning, DeprecationWarning in py3k mode, RuntimeWarning, or UnicodeWarning? 3. Escape \r and \x1a (end-of-file in MS DOS) when pickle Unicode strings with protocol 0. 4. Detect the most common errors (e.g. module name ending with \r when load on Linux a pickle saved with text mode on Windows) and raise more informative error message. 5. Emit a warning when load an Unicode string ending with \r. This is likely an error (if the pickle was saved with text mode on Windows), but this can a correct data if the saved Unicode string actually did end with \r. This is the most dubious proposition. On one hand, it is better to warn than silently return an incorrect result. On other hand, the correct result shouldn't provoke a warning. ---------- components: Library (Lib) messages: 304667 nosy: alexandre.vassalotti, benjamin.peterson, pitrou, serhiy.storchaka priority: normal severity: normal status: open title: Portability issues with pickle type: enhancement versions: Python 2.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Oct 20 16:59:08 2017 From: report at bugs.python.org (Roel van der Goot) Date: Fri, 20 Oct 2017 20:59:08 +0000 Subject: [New-bugs-announce] [issue31830] asyncio.create_subprocess_exec doesn't capture all stdout output Message-ID: <1508533148.51.0.213398074469.issue31830@psf.upfronthosting.co.za> Change by Roel van der Goot : ---------- components: asyncio files: show.py nosy: cannedrag, yselivanov priority: normal severity: normal status: open title: asyncio.create_subprocess_exec doesn't capture all stdout output type: behavior versions: Python 3.6 Added file: https://bugs.python.org/file47228/show.py _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Oct 20 18:22:56 2017 From: report at bugs.python.org (calimeroteknik) Date: Fri, 20 Oct 2017 22:22:56 +0000 Subject: [New-bugs-announce] =?utf-8?q?=5Bissue31831=5D_EmailMessage=2Ead?= =?utf-8?q?d=5Fattachment=28filename=3D=22long_or_sp=C3=A9cial=22=29_crash?= =?utf-8?q?es_or_produces_invalid_output?= Message-ID: <1508538176.94.0.213398074469.issue31831@psf.upfronthosting.co.za> New submission from calimeroteknik : The following code excerpt demonstrates a crash: import email.message mail = email.message.EmailMessage() mail.add_attachment( b"test", maintype = "text", subtype = "plain", filename = "I thought I could put a few words in the filename but apparently it does not go so well.txt" ) print(mail) Output on python 3.7.0a1: https://gist.github.com/altendky/33c235e8a693235acd0551affee0a4f6 Output on python 3.6.2: https://oremilac.tk/paste/python-rfc2231-oops.log Additionally, a behavioral issue is demonstrated by replacing in the above: filename = "What happens if we try French in here? touch?!.txt" Which results in the following output (headers): Content-Type: text/plain Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename*=utf-8''What%20happens%20if%20we%20try%20French%20in%20here%3F%20touch%C3%A9%21.txt MIME-Version: 1.0 Instead of, for example, this correct output (by Mozilla Thunderbird here): Content-Type: text/plain; charset=UTF-8; name="=?UTF-8?Q?What_happens_if_we_try_French_in_here=3f_touch=c3=a9!.txt?=" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename*0*=utf-8''%57%68%61%74%20%68%61%70%70%65%6E%73%20%69%66%20%77%65; filename*1*=%20%74%72%79%20%46%72%65%6E%63%68%20%69%6E%20%68%65%72%65%3F; filename*2*=%20%74%6F%75%63%68%C3%A9%21%2E%74%78%74 Issues to note here: -the "filename" parameter is not indented, mail clients ignore it -the "filename" parameter is not split according to RFC 2231 The relevant standard is exemplified in section 4.1 of https://tools.ietf.org/html/rfc2231#page-5 Python 3.4.6 and 3.5.4 simply do not wrap anything, which works with but is not conformant to standards. Solving all of the above would imply correctly splitting any header. Function "set_param" in /usr/lib/python*/email/message.py looked like a place to look. Unfortunately I do not understand what's going on there very well. As yet an additional misbehaviour to note, try to repeat the above print statement twice. The result is not identical, and the second time you get the following output: Content-Type: text/plain Content-Transfer-Encoding: base64 Content-Disposition: attachment;*=utf-8''What%20happens%20if%20we%20try%20French%20in%20here%3F%20touch%C3%A9%21.txt MIME-Version: 1.0 It would appear that "filename" has disappeared. The issue does not reveal itself with simple values for the 'filename' argument, e.g. "test.txt". PS: The above output also illustrates this (way more minor) issue: https://bugs.python.org/issue25235 ---------- components: email messages: 304684 nosy: barry, calimeroteknik, r.david.murray priority: normal severity: normal status: open title: EmailMessage.add_attachment(filename="long or sp?cial") crashes or produces invalid output type: crash versions: Python 3.6, Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Oct 20 19:08:38 2017 From: report at bugs.python.org (Roel van der Goot) Date: Fri, 20 Oct 2017 23:08:38 +0000 Subject: [New-bugs-announce] [issue31832] Async list comprehension (PEP 530) causes SyntaxError in Python 3.6.3 Message-ID: <1508540918.29.0.213398074469.issue31832@psf.upfronthosting.co.za> New submission from Roel van der Goot : $ python3 Python 3.6.3 (default, Oct 3 2017, 21:45:48) [GCC 7.2.0] on linux Type "help", "copyright", "credits" or "license" for more information. >>> async def arange(n): ... for i in range(n): ... yield i ... >>> [i async for i in arange(10)] File "", line 1 [i async for i in arange(10)] ^ SyntaxError: invalid syntax >>> ---------- components: asyncio messages: 304688 nosy: cannedrag, yselivanov priority: normal severity: normal status: open title: Async list comprehension (PEP 530) causes SyntaxError in Python 3.6.3 versions: Python 3.6 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sat Oct 21 00:32:22 2017 From: report at bugs.python.org (emtone) Date: Sat, 21 Oct 2017 04:32:22 +0000 Subject: [New-bugs-announce] [issue31833] Compile fail on gentoo for MIPS CPU of loongson 2f Message-ID: <1508560342.69.0.213398074469.issue31833@psf.upfronthosting.co.za> New submission from emtone : Here is the error message: building '_multiprocessing' extension creating build/temp.linux-mips64-2.7/var/tmp/portage/dev-lang/python-2.7.12/work/Python-2.7.12/Modules/_multiprocessing mips64el-unknown-linux-gnu-gcc -pthread -fPIC -fno-strict-aliasing -O2 -march=loongson2f -Wa,-mfix-loongson2f-nop -pipe -fwrapv -DNDEBUG -IModules/_multiprocessing -I. -IInclude -I/usr/local/include -I/var/tmp/portage/dev-lang/python-2.7.12/work/Python-2.7.12/Include -I/var/tmp/portage/dev-lang/python-2.7.12/work/mips64el-unknown-linux-gnu -c /var/tmp/portage/dev-lang/python-2.7.12/work/Python-2.7.12/Modules/_multiprocessing/multiprocessing.c -o build/temp.linux-mips64-2.7/var/tmp/portage/dev-lang/python-2.7.12/work/Python-2.7.12/Modules/_multiprocessing/multiprocessing.o mips64el-unknown-linux-gnu-gcc -pthread -fPIC -fno-strict-aliasing -O2 -march=loongson2f -Wa,-mfix-loongson2f-nop -pipe -fwrapv -DNDEBUG -IModules/_multiprocessing -I. -IInclude -I/usr/local/include -I/var/tmp/portage/dev-lang/python-2.7.12/work/Python-2.7.12/Include -I/var/tmp/portage/dev-lang/python-2.7.12/work/mips64el-unknown-linux-gnu -c /var/tmp/portage/dev-lang/python-2.7.12/work/Python-2.7.12/Modules/_multiprocessing/socket_connection.c -o build/temp.linux-mips64-2.7/var/tmp/portage/dev-lang/python-2.7.12/work/Python-2.7.12/Modules/_multiprocessing/socket_connection.o mips64el-unknown-linux-gnu-gcc -pthread -fPIC -fno-strict-aliasing -O2 -march=loongson2f -Wa,-mfix-loongson2f-nop -pipe -fwrapv -DNDEBUG -IModules/_multiprocessing -I. -IInclude -I/usr/local/include -I/var/tmp/portage/dev-lang/python-2.7.12/work/Python-2.7.12/Include -I/var/tmp/portage/dev-lang/python-2.7.12/work/mips64el-unknown-linux-gnu -c /var/tmp/portage/dev-lang/python-2.7.12/work/Python-2.7.12/Modules/_multiprocessing/semaphore.c -o build/temp.linux-mips64-2.7/var/tmp/portage/dev-lang/python-2.7.12/work/Python-2.7.12/Modules/_multiprocessing/semaphore.o mips64el-unknown-linux-gnu-gcc -pthread -shared -Wl,-O1 -Wl,--as-needed -L. -Wl,-O1 -Wl,--as-needed -L. -fno-strict-aliasing -O2 -march=loongson2f -Wa,-mfix-loongson2f-nop -pipe -fwrapv -DNDEBUG -I. -IInclude -I/var/tmp/portage/dev-lang/python-2.7.12/work/Python-2.7.12/Include build/temp.linux-mips64-2.7/var/tmp/portage/dev-lang/python-2.7.12/work/Python-2.7.12/Modules/_multiprocessing/multiprocessing.o build/temp.linux-mips64-2.7/var/tmp/portage/dev-lang/python-2.7.12/work/Python-2.7.12/Modules/_multiprocessing/socket_connection.o build/temp.linux-mips64-2.7/var/tmp/portage/dev-lang/python-2.7.12/work/Python-2.7.12/Modules/_multiprocessing/semaphore.o -L. -lpython2.7 -o build/lib.linux-mips64-2.7/_multiprocessing.so *** WARNING: importing extension "_multiprocessing" failed with : [Errno 89] Function not implemented ---------- components: Extension Modules messages: 304695 nosy: emtone priority: normal severity: normal status: open title: Compile fail on gentoo for MIPS CPU of loongson 2f type: compile error versions: Python 2.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sat Oct 21 03:57:11 2017 From: report at bugs.python.org (=?utf-8?b?TWljaGHFgiBHw7Nybnk=?=) Date: Sat, 21 Oct 2017 07:57:11 +0000 Subject: [New-bugs-announce] [issue31834] BLAKE2: the (pure) SSE2 impl forced on x86_64 is slower than reference Message-ID: <1508572631.92.0.213398074469.issue31834@psf.upfronthosting.co.za> New submission from Micha? G?rny : The setup.py file for Python states: if (not cross_compiling and os.uname().machine == "x86_64" and sys.maxsize > 2**32): # Every x86_64 machine has at least SSE2. Check for sys.maxsize # in case that kernel is 64-bit but userspace is 32-bit. blake2_macros.append(('BLAKE2_USE_SSE', '1')) While the assertion about having SSE2 is true, it doesn't mean that it's worthwhile to use. I've tested pure (i.e. without SSSE3 and so on) on three different machines, getting the following results: Athlon64 X2 (SSE2 is the best supported variant), 540 MiB of data: SSE2: [5.189988004000043, 5.070812243997352] ref: [2.0161159170020255, 2.0475422790041193] Core i3, same data file: SSE2: [1.924425926999902, 1.92461746999993, 1.9298037500000191] ref: [1.7940209749999667, 1.7900855569999976, 1.7835538760000418] Xeon E5630 server, 230 MiB data file: SSE2: [0.7671358410007088, 0.7797677099879365, 0.7648976119962754] ref: [0.5784736709902063, 0.5717909929953748, 0.5717219939979259] So in all the tested cases, pure SSE2 implementation is *slower* than the reference implementation. SSSE3 and other variants are faster and AFAIU they are enabled automatically based on CFLAGS, so it doesn't matter for most of the systems. However, for old CPUs that do not support SSSE3, the choice of SSE2 makes the algorithm prohibitively slow -- it's 2.5 times slower than the reference implementation! ---------- components: Extension Modules messages: 304696 nosy: mgorny priority: normal severity: normal status: open title: BLAKE2: the (pure) SSE2 impl forced on x86_64 is slower than reference type: performance versions: Python 3.6 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sat Oct 21 07:14:04 2017 From: report at bugs.python.org (Anselm Kruis) Date: Sat, 21 Oct 2017 11:14:04 +0000 Subject: [New-bugs-announce] [issue31835] _PyFunction_FastCallDict and _PyFunction_FastCallKeywords: fast path not used Message-ID: <1508584444.52.0.213398074469.issue31835@psf.upfronthosting.co.za> New submission from Anselm Kruis : Just a minor performance issue. The C functions _PyFunction_FastCallDict() and _PyFunction_FastCallKeywords() (branch 'master', Objects/call.c) and their predecessors fast_function() and _PyFunction_FastCallDict() in Python/ceval.c all contain the following sub-expression in the "if"-statement for the fast-path. For instance Objects/call.c:318 co->co_flags == (CO_OPTIMIZED | CO_NEWLOCALS | CO_NOFREE) Now, if co_flags has any of the CO_FUTURE_... bits set, the expression is always False and the fast path is not used. Currently this affects only Python 3.6 and Python 2.7, because other Python versions do not use the __future__ mechanism. The fix is simple. Replace the faulty sub-expression by (co->co_flags & (~PyCF_MASK)) == (CO_OPTIMIZED | CO_NEWLOCALS | CO_NOFREE)) I discovered this issue while debugging reference leaks in Stackless Python a few month ago. It is hard to write a test case, but one can compare C call stacks using a debugger. $ ulimit -c unlimited # enable core dumps $ python3.6 -c 'from __future__ import generator_stop; import os; (lambda: os.abort())()' $ gdb -batch -ex bt python3.6 core > trace_with_future $ python3.6 -c 'import os; (lambda: os.abort())()' $ gdb -batch -ex bt python3.6 core > trace_without_future If you compare the traces, the difference is in stack frame #9. Same for python2.7. ---------- components: Interpreter Core messages: 304702 nosy: anselm.kruis priority: normal severity: normal status: open title: _PyFunction_FastCallDict and _PyFunction_FastCallKeywords: fast path not used type: performance versions: Python 2.7, Python 3.6 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sat Oct 21 12:03:19 2017 From: report at bugs.python.org (Serhiy Storchaka) Date: Sat, 21 Oct 2017 16:03:19 +0000 Subject: [New-bugs-announce] [issue31836] test_code_module fails after test_idle Message-ID: <1508601799.24.0.213398074469.issue31836@psf.upfronthosting.co.za> New submission from Serhiy Storchaka : $ ./python -m test -uall test_idle test_code_module Run tests sequentially 0:00:00 load avg: 0.42 [1/2] test_idle 0:00:02 load avg: 0.42 [2/2] test_code_module test test_code_module failed -- Traceback (most recent call last): File "/home/serhiy/py/cpython/Lib/test/test_code_module.py", line 35, in test_ps1 self.assertEqual(self.sysmod.ps1, '>>> ') AssertionError: != '>>> ' test_code_module failed 1 test OK. 1 test failed: test_code_module Total duration: 2 sec Tests result: FAILURE ---------- assignee: terry.reedy components: IDLE, Tests messages: 304709 nosy: serhiy.storchaka, terry.reedy priority: normal severity: normal status: open title: test_code_module fails after test_idle type: behavior versions: Python 3.6, Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sat Oct 21 13:08:02 2017 From: report at bugs.python.org (Serhiy Storchaka) Date: Sat, 21 Oct 2017 17:08:02 +0000 Subject: [New-bugs-announce] [issue31837] ParseError in test_all_project_files() Message-ID: <1508605682.35.0.213398074469.issue31837@psf.upfronthosting.co.za> New submission from Serhiy Storchaka : test_all_project_files() in test_lib2to3 emits warnings in verbose mode. /home/serhiy/py/cpython/Lib/lib2to3/tests/test_parser.py:415: UserWarning: ParseError on file /home/serhiy/py/cpython/Lib/lib2to3/main.py (bad input: type=22, value='=', context=('', (130, 38))) warnings.warn('ParseError on file %s (%s)' % (filepath, err)) /home/serhiy/py/cpython/Lib/lib2to3/tests/test_parser.py:415: UserWarning: ParseError on file /home/serhiy/py/cpython/Lib/lib2to3/tests/pytree_idempotency.py (bad input: type=22, value='=', context=('', (49, 33))) warnings.warn('ParseError on file %s (%s)' % (filepath, err)) ParseError is raised by lines like: print("Use --help to show usage.", file=sys.stderr) Seems "from __future__ import print_function" doesn't work. ---------- components: 2to3 (2.x to 3.x conversion tool), Tests messages: 304715 nosy: benjamin.peterson, serhiy.storchaka priority: normal severity: normal status: open title: ParseError in test_all_project_files() type: behavior _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sat Oct 21 19:25:15 2017 From: report at bugs.python.org (Mauro) Date: Sat, 21 Oct 2017 23:25:15 +0000 Subject: [New-bugs-announce] [issue31838] Python 3.4 supported SSL version Message-ID: <1508628315.52.0.213398074469.issue31838@psf.upfronthosting.co.za> New submission from Mauro : Hello and sorry to bother. This is my first message to the list. I'm trying to build python 3.4.7 downloaded from python.org (released in August this year) and while following the exact same steps detailed in https://bugs.python.org/issue29027, I get getting the exact same error he does. Doesn't Python 3.4 support OpenSSL v 1.1.0? Is this is the case, I guess it will never do, right? More importantly, is there a place to get build dependencies (with appropriate versions) for each CPython version? I scrawled the source and the GitHub repositories without luck. I guess this has been asked before, but after several hours, I couldn't find anything. Cheers, Mauro ---------- assignee: christian.heimes components: SSL messages: 304725 nosy: christian.heimes, sabian2008 priority: normal severity: normal status: open title: Python 3.4 supported SSL version versions: Python 3.4 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun Oct 22 00:56:08 2017 From: report at bugs.python.org (Oren Tirosh) Date: Sun, 22 Oct 2017 04:56:08 +0000 Subject: [New-bugs-announce] [issue31839] datetime: add method to parse isoformat() output Message-ID: <1508648168.49.0.213398074469.issue31839@psf.upfronthosting.co.za> New submission from Oren Tirosh : The object types in the datetime module can produce a standard string representation (rfc3339, a subset of iso8601) but they do not provide a method to parse it. Proposed method names: isoparse or fromisoformat. In addition, a constructor call with a single string argument may also be supported. This would make the behavior of datetime/date/time more similar to other value types such as int, str or float and maintain the same invariant of x == type(x)(str(x)) Requirements: support lossless round-tripping of all valid datetime object values to str and back, with the exception of objects having a custom tzinfo (not None or an instance of datetime.timezone). The _strptime format of '%Y-%m-%d %H:%M:S.%f%z' comes close, but fails to meet these requirements in the following ways: 1. %z matches +HHMM, not +HH:MM (addressed by issue 31800, currently discussed on datetime-sig) 2. %z does not match the empty string, indicating a naive datetime object (tzinfo=None) 3. .%f requires a fraction part, while isoformat drops it when the timestamp is an exact second (microsecond=0). ---------- messages: 304728 nosy: orent priority: normal severity: normal status: open title: datetime: add method to parse isoformat() output type: enhancement _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun Oct 22 02:06:57 2017 From: report at bugs.python.org (=?utf-8?b?SGlyb2tpIOKAnGgy4oCdIEhvcml1Y2hp?=) Date: Sun, 22 Oct 2017 06:06:57 +0000 Subject: [New-bugs-announce] [issue31840] `ImportError` is` raised` only when `python -m unittest discover` is given Message-ID: <1508652417.45.0.213398074469.issue31840@psf.upfronthosting.co.za> New submission from Hiroki ?h2? Horiuchi : Create a package `pkg`. Put `test.py` there. Write `none = None` in `pkg/__init__.py`. In the directory containing `pkg/`, `python [23] -m unittest pkg.test` will terminate normally, but `python [23] -m unittest discover pkg` will raise `ImportError`. The phenomenon is the same in macOS Sierra, Debian 9.0, Windows 7. I do not think this is a very kind behavior. Thank you. ---------- components: Library (Lib), Tests, Windows, macOS files: test.py messages: 304729 nosy: Hiroki ?h2? Horiuchi, ned.deily, paul.moore, ronaldoussoren, steve.dower, tim.golden, zach.ware priority: normal severity: normal status: open title: `ImportError` is` raised` only when `python -m unittest discover` is given type: behavior versions: Python 2.7, Python 3.6 Added file: https://bugs.python.org/file47230/test.py _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun Oct 22 15:46:35 2017 From: report at bugs.python.org (Dmitry Kazakov) Date: Sun, 22 Oct 2017 19:46:35 +0000 Subject: [New-bugs-announce] [issue31841] Several methods of collections.UserString do not return instances of UserString or its subclasses Message-ID: <1508701595.06.0.213398074469.issue31841@psf.upfronthosting.co.za> New submission from Dmitry Kazakov : A few of the methods of collections.UserString return objects of type str when one would expect instances of UserString or a subclass. Here's an example for UserString.join: >>> s = UserString(', ').join(['a', 'b', 'c']); print(repr(s), type(s)) 'a, b, c' This *looks* like a bug to me, but since I was unable to find similar bug reports, and this behaviour has existed for years, I'm not too sure. Same holds for UserString.format and UserString.format_map, which were added recently (issue 22189): >>> s = UserString('{}').format(1); print(repr(s), type(s)) '1' At least, this output is inconsistent with %-based formatting: >>> s = UserString('%d') % 1; print(repr(s), type(s)) '1' ---------- components: Library (Lib) messages: 304761 nosy: vaultah priority: normal severity: normal status: open title: Several methods of collections.UserString do not return instances of UserString or its subclasses type: behavior _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun Oct 22 17:31:13 2017 From: report at bugs.python.org (Erik Aronesty) Date: Sun, 22 Oct 2017 21:31:13 +0000 Subject: [New-bugs-announce] [issue31842] pathlib: "Incorrect function" during resolve() Message-ID: <1508707873.05.0.213398074469.issue31842@psf.upfronthosting.co.za> New submission from Erik Aronesty : When strict is "false", pathlib should not fail if the network share is inaccessible. It should, as documented, catch the exception and continue with as much of the path as it has. >>> pathlib.Path("v:").resolve() Traceback (most recent call last): File "", line 1, in File "C:\Users\erik\AppData\Local\Programs\Python\Python36\lib\pathlib.py", line 1119, in resolve s = self._flavour.resolve(self, strict=strict) File "C:\Users\erik\AppData\Local\Programs\Python\Python36\lib\pathlib.py", line 193, in resolve s = self._ext_to_normal(_getfinalpathname(s)) OSError: [WinError 1] Incorrect function: 'v:' ---------- messages: 304767 nosy: earonesty2 priority: normal severity: normal status: open title: pathlib: "Incorrect function" during resolve() versions: Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun Oct 22 20:13:45 2017 From: report at bugs.python.org (Allen Li) Date: Mon, 23 Oct 2017 00:13:45 +0000 Subject: [New-bugs-announce] [issue31843] sqlite3.connect() should accept PathLike objects Message-ID: <1508717625.1.0.213398074469.issue31843@psf.upfronthosting.co.za> New submission from Allen Li : sqlite3.connect() should accept PathLike objects (objects that implement __fspath__) ---------- messages: 304773 nosy: Allen Li priority: normal severity: normal status: open title: sqlite3.connect() should accept PathLike objects versions: Python 3.6 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Oct 23 04:27:23 2017 From: report at bugs.python.org (Sebastian Rittau) Date: Mon, 23 Oct 2017 08:27:23 +0000 Subject: [New-bugs-announce] [issue31844] HTMLParser: undocumented not implemented method Message-ID: <1508747243.44.0.213398074469.issue31844@psf.upfronthosting.co.za> New submission from Sebastian Rittau : HTMLParser derives from _markupbase.ParserBase, which has the following method: class HTMLParser: ... def error(self, message): raise NotImplementedError( "subclasses of ParserBase must override error()") HTMLParser does not implement this method and the documentation for HTMLParser (https://docs.python.org/3.6/library/html.parser.html) does not mention that its sub-classes need to override it. I am not sure whether this is a documentation omission, whether HTMLParser should provide an (empty?) implementation, or whether ParserBase should not raise a NotImplementedError (to make linters happy). ---------- messages: 304782 nosy: srittau priority: normal severity: normal status: open title: HTMLParser: undocumented not implemented method versions: Python 3.6 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Oct 23 07:25:30 2017 From: report at bugs.python.org (Sviatoslav Abakumov) Date: Mon, 23 Oct 2017 11:25:30 +0000 Subject: [New-bugs-announce] [issue31845] Envvar PYTHONDONTWRITEBYTECODE is not respected Message-ID: <1508757930.52.0.213398074469.issue31845@psf.upfronthosting.co.za> New submission from Sviatoslav Abakumov : Setting PYTHONDONTWRITEBYTECODE doesn't seem to have an effect in Python 3.7.0a2: $ python -V Python 3.7.0a2 $ env PYTHONDONTWRITEBYTECODE=1 python -c 'import sys; print(sys.dont_write_bytecode)' False ---------- components: Interpreter Core messages: 304794 nosy: Perlence priority: normal severity: normal status: open title: Envvar PYTHONDONTWRITEBYTECODE is not respected versions: Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Oct 23 08:46:04 2017 From: report at bugs.python.org (Nathan Henrie) Date: Mon, 23 Oct 2017 12:46:04 +0000 Subject: [New-bugs-announce] [issue31846] Error in 3.6.3 epub docs Message-ID: <1508762764.69.0.213398074469.issue31846@psf.upfronthosting.co.za> New submission from Nathan Henrie : I routinely download the epub version of the docs to my computer and mobile devices as an offline copy. The 3.6.3 version reports a big error on the first (and many other pages): > This page contains the following errors: error on line 5176 at column 11: Entity 'copy' not defined Below is a rendering of the page up to the first error. Numerous similar errors reporting `Entity 'copy' not defined` scattered throughout the epub. Wonder if this was introduced by the change to `conf.py` that fixed the recent problem of the 404s with the HTML docs. ---------- messages: 304799 nosy: n8henrie priority: normal severity: normal status: open title: Error in 3.6.3 epub docs versions: Python 3.6 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Oct 23 08:52:13 2017 From: report at bugs.python.org (Serhiy Storchaka) Date: Mon, 23 Oct 2017 12:52:13 +0000 Subject: [New-bugs-announce] [issue31847] Fix commented out tests in test_syntax Message-ID: <1508763133.89.0.213398074469.issue31847@psf.upfronthosting.co.za> New submission from Serhiy Storchaka : Two doctest tests in test_syntax are commented out because SyntaxWarning was expected, but doctests couldn't be used for testing them. But now SyntaxError is raised in these cases instead of SyntaxWarning. The proposed PR restores and fixes tests. ---------- assignee: serhiy.storchaka components: Tests messages: 304800 nosy: serhiy.storchaka priority: normal severity: normal stage: patch review status: open title: Fix commented out tests in test_syntax type: enhancement versions: Python 3.6, Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Oct 23 09:33:01 2017 From: report at bugs.python.org (Stephen Paul Chappell) Date: Mon, 23 Oct 2017 13:33:01 +0000 Subject: [New-bugs-announce] [issue31848] "aifc" module does not always initialize "Aifc_read._ssnd_chunk" Message-ID: <1508765581.06.0.213398074469.issue31848@psf.upfronthosting.co.za> New submission from Stephen Paul Chappell : When Aifc_read runs initfp, it conditionally sets self._ssnd_chunk and is not guaranteed to do so. At the bottom of the method, a check is made to see if the attribute has a false value; and if so, an error is supposed to be raised. If a SSND chunk is never found, checking self._ssnd_chunk causes an attribute error to be raised similar to this: AttributeError: 'Aifc_read' object has no attribute '_ssnd_chunk' The error was found on the following distribution: Python 3.6.2 (v3.6.2:5fd33b5, Jul 8 2017, 04:57:36) [MSC v.1900 64 bit (AMD64)] on win32 ---------- components: Library (Lib), Tests messages: 304804 nosy: Zero priority: normal severity: normal status: open title: "aifc" module does not always initialize "Aifc_read._ssnd_chunk" type: crash versions: Python 3.6 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Oct 23 12:01:16 2017 From: report at bugs.python.org (Xavier de Gaye) Date: Mon, 23 Oct 2017 16:01:16 +0000 Subject: [New-bugs-announce] [issue31849] Python/pyhash.c warning: comparison of integers of different signs Message-ID: <1508774476.87.0.213398074469.issue31849@psf.upfronthosting.co.za> New submission from Xavier de Gaye : When built with: Android clang version 3.8.275480 (based on LLVM 3.8.275480) The following warning is emitted: ccache /pathto/android/android-ndk/toolchains/llvm/prebuilt/linux-x86_64/bin/clang -target x86_64-none-linux-androideabi -gcc-toolchain /pathto/android/android-ndk/toolchains/x86_64-4.9/prebuilt/linux-x86_64 -c -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes --sysroot=/pathto/android/android-ndk/sysroot -D__ANDROID_API__=21 -isystem /pathto/android/android-ndk/sysroot/usr/include/x86_64-linux-android -Wno-unused-value -Wno-empty-body -Qunused-arguments -Wno-nullability-completeness -Wno-parentheses-equality -std=c99 -Wextra -Wno-unused-result -Wno-unused-parameter -Wno-missing-field-initializers -Werror=implicit-function-declaration -IObjects -IInclude -IPython -I. -I/pathto/src/python/master/Include -I/pathto/tmp/android-makesetup/build/python3.7-extlibs-android-21-x86_64//data/local/tmp/python/include --sysroot=/pathto/android/android-ndk/sysroot -D__ANDROID_API__=21 -isystem /pathto/android/android-ndk/sysroot/usr/include/x86_64-linux-android -DPy_BUILD_CORE -o Python/pyhash.o /pathto/src/python/master/Python/pyhash.c /pathto/src/python/master/Python/pyhash.c:275:11: warning: comparison of integers of different signs: 'Py_uhash_t' (aka 'unsigned long') and 'int' [-Wsign-compare] if (x == -1) { ~ ^ ~~ 1 warning generated. ---------- components: Interpreter Core messages: 304816 nosy: christian.heimes, xdegaye priority: normal severity: normal status: open title: Python/pyhash.c warning: comparison of integers of different signs type: compile error versions: Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Oct 23 13:45:15 2017 From: report at bugs.python.org (STINNER Victor) Date: Mon, 23 Oct 2017 17:45:15 +0000 Subject: [New-bugs-announce] [issue31850] test_nntplib failed with "nntplib.NNTPDataError: line too long" Message-ID: <1508780715.86.0.213398074469.issue31850@psf.upfronthosting.co.za> New submission from STINNER Victor : test_nntplib failed on many buildbots yesterday with the "line too long" error. It may be related to bpo-28971. Example: http://buildbot.python.org/all/#/builders/12/builds/52 (...) test_with_statement (test.test_nntplib.NetworkedNNTP_SSLTests) ... ok test_xhdr (test.test_nntplib.NetworkedNNTP_SSLTests) ... ok test_xover (test.test_nntplib.NetworkedNNTP_SSLTests) ... ERROR test_zlogin (test.test_nntplib.NetworkedNNTP_SSLTests) ... ok test_zzquit (test.test_nntplib.NetworkedNNTP_SSLTests) ... ok test_module_all_attribute (test.test_nntplib.PublicAPITests) ... ok test test_nntplib failed test_we_are_in_reader_mode_after_connect (test.test_nntplib.SendReaderNNTPv2Tests) ... ok ====================================================================== ERROR: test_xover (test.test_nntplib.NetworkedNNTP_SSLTests) ---------------------------------------------------------------------- Traceback (most recent call last): File "D:\buildarea\3.x.ware-win81-release\build\lib\test\test_nntplib.py", line 241, in wrapped meth(self) File "D:\buildarea\3.x.ware-win81-release\build\lib\test\test_nntplib.py", line 125, in test_xover resp, lines = self.server.xover(last - 5, last) File "D:\buildarea\3.x.ware-win81-release\build\lib\nntplib.py", line 804, in xover file) File "D:\buildarea\3.x.ware-win81-release\build\lib\nntplib.py", line 525, in _longcmdstring resp, list = self._getlongresp(file) File "D:\buildarea\3.x.ware-win81-release\build\lib\nntplib.py", line 494, in _getlongresp line = self._getline() File "D:\buildarea\3.x.ware-win81-release\build\lib\nntplib.py", line 434, in _getline raise NNTPDataError('line too long') nntplib.NNTPDataError: line too long ---------- components: Tests keywords: buildbot messages: 304829 nosy: haypo priority: normal severity: normal status: open title: test_nntplib failed with "nntplib.NNTPDataError: line too long" versions: Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Oct 23 13:54:49 2017 From: report at bugs.python.org (STINNER Victor) Date: Mon, 23 Oct 2017 17:54:49 +0000 Subject: [New-bugs-announce] [issue31851] test_subprocess hangs randomly on x86 Windows7 3.x Message-ID: <1508781289.15.0.213398074469.issue31851@psf.upfronthosting.co.za> New submission from STINNER Victor : test_subprocess hanged on the build 46 of x86 Windows7 3.x: http://buildbot.python.org/all/#/builders/58/builds/46 (...) 1:29:55 [405/407] test_cmath passed -- running: test_subprocess (1249 sec) 1:29:58 [406/407] test_pstats passed -- running: test_subprocess (1252 sec) The buildbot is slow which might have caused a timing issue. ---------- components: Tests keywords: buildbot messages: 304831 nosy: haypo priority: normal severity: normal status: open title: test_subprocess hangs randomly on x86 Windows7 3.x versions: Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Oct 23 15:22:17 2017 From: report at bugs.python.org (Alexandre Hamelin) Date: Mon, 23 Oct 2017 19:22:17 +0000 Subject: [New-bugs-announce] [issue31852] Crashes with lines of the form "async \" Message-ID: <1508786537.77.0.213398074469.issue31852@psf.upfronthosting.co.za> New submission from Alexandre Hamelin : Hi. Python 3.6.2 crashes when interpreting lines with the text "async \" (future keyword 'async' and ending with a backslash). Tested in a docker environment (debian jessie). (see github.com/0xquad/docker-python36 if needed) Examples: $ docker run -ti --rm python36 root at 4c09392f83c8:/# python3.6 Python 3.6.2 (default, Aug 4 2017, 14:35:04) [GCC 6.4.0 20170724] on linux Type "help", "copyright", "credits" or "license" for more information. >>> async \ ... File "", line 1 \ufffd\ufffdF\ufffd\ufffd ^ SyntaxError: invalid syntax >>> async \ Segmentation fault root at 4c09392f83c8:/# Also, ----- file: test.py #/usr/bin/python3.6 async \ ----- $ ./test.py Segmentation fault $ Haven't taken the time to produce a backtrace or investigate with latest the dev versions or any further. Let me know if I can assist in any way. ---------- components: Interpreter Core messages: 304835 nosy: Alexandre Hamelin priority: normal severity: normal status: open title: Crashes with lines of the form "async \" type: crash versions: Python 3.6 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Oct 23 16:26:17 2017 From: report at bugs.python.org (Erik Aronesty) Date: Mon, 23 Oct 2017 20:26:17 +0000 Subject: [New-bugs-announce] [issue31853] Use super().method instead of socket.method in SSLSocket Message-ID: <1508790377.43.0.213398074469.issue31853@psf.upfronthosting.co.za> New submission from Erik Aronesty : I asked on #python-dev and was told that it's most likely due to legacy reasons that the class has things like `socket.__init__` instead of `super().__init__` ---------- assignee: christian.heimes components: SSL messages: 304838 nosy: christian.heimes, earonesty priority: normal pull_requests: 4063 severity: normal status: open title: Use super().method instead of socket.method in SSLSocket type: enhancement versions: Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Oct 23 16:26:44 2017 From: report at bugs.python.org (Justus Schwabedal) Date: Mon, 23 Oct 2017 20:26:44 +0000 Subject: [New-bugs-announce] [issue31854] Add mmap.ACCESS_DEFAULT to namespace Message-ID: <1508790404.89.0.213398074469.issue31854@psf.upfronthosting.co.za> New submission from Justus Schwabedal : I propose to add mmap.ACCESS_DEFAULT into the namespace. Accessing the default value might be necessary, if one needs to overwrite an `ACCESS` keyword argument. ---------- components: Extension Modules messages: 304839 nosy: Justus Schwabedal priority: normal severity: normal status: open title: Add mmap.ACCESS_DEFAULT to namespace type: enhancement versions: Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Oct 23 17:03:10 2017 From: report at bugs.python.org (Ron Rothman) Date: Mon, 23 Oct 2017 21:03:10 +0000 Subject: [New-bugs-announce] [issue31855] mock_open is not compatible with read(n) (and pickle.load) Message-ID: <1508792590.83.0.213398074469.issue31855@psf.upfronthosting.co.za> New submission from Ron Rothman : mock.mock_open works as expected when reading the entire file (read()) or when reading a single line (readline()), but it seems to not support reading a number of bytes (read(n)). These work as expected: from mock import mock_open, patch # works: consume entire "file" with patch('__main__.open', mock_open(read_data='bibble')) as m: with open('foo') as h: result = h.read() assert result == 'bibble' # ok # works: consume one line with patch('__main__.open', mock_open(read_data='bibble\nbobble')) as m: with open('foo') as h: result = h.readline() assert result == 'bibble\n' # ok But trying to read only a few bytes fails--mock_open returns the entire read_data instead: # consume first 3 bytes of the "file" with patch('__main__.open', mock_open(read_data='bibble')) as m: with open('foo') as h: result = h.read(3) assert result == 'bib', 'result of read: {}'.format(result) # fails Output: Traceback (most recent call last): File "/tmp/t.py", line 25, in assert result == 'bib', 'result of read: {}'.format(result) AssertionError: result of read: bibble The unfortunate effect of this is that mock_open cannot be used with pickle.load. with open('/path/to/file.pkl', 'rb') as f: x = pickle.load(f) # this requires f.read(1) to work ---------- components: Library (Lib) messages: 304841 nosy: ron.rothman priority: normal severity: normal status: open title: mock_open is not compatible with read(n) (and pickle.load) type: behavior versions: Python 2.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Oct 23 19:25:05 2017 From: report at bugs.python.org (Bob Kline) Date: Mon, 23 Oct 2017 23:25:05 +0000 Subject: [New-bugs-announce] [issue31856] Unexpected behavior of re module when VERBOSE flag is set Message-ID: <1508801105.35.0.213398074469.issue31856@psf.upfronthosting.co.za> New submission from Bob Kline : According to the documentation of the re module, "When this flag [re.VERBOSE] has been specified, whitespace within the RE string is ignored, except when the whitespace is in a character class or preceded by an unescaped backslash; this lets you organize and indent the RE more clearly. This flag also lets you put comments within a RE that will be ignored by the engine; comments are marked by a '#' that?s neither in a character class [n]or preceded by an unescaped backslash." (I'm quoting from the 3.6.3 documentation, but I've tested with several versions of Python, as indicated in the issue's `Versions` field, all with the same results.) Given this description, I would have expected the output for each of the pairs of calls to findall() in the attached repro code to be the same, but that is not what's happening. In the case of the first pair of calls, for example, the non-verbose version finds two more matches than the verbose version, even though the regular expression is identical for the two calls, ignoring whitespace and comments in the expression string. Similar problems appear with the other two pairs of calls. Here's the output from the attached code: ['&', '(', '/Term/SemanticType/@cdr:ref', '=='] ['/Term/SemanticType/@cdr:ref', '=='] [' XXX '] [] [' XXX '] [] It would seem that at least one of the following is true: 1. the module is not behaving as it should 2. the documentation is wrong 3. I have not understood the documentation correctly I'm happy for it to be #3, as long as someone can explain what I have not understood. ---------- components: Library (Lib), Regular Expressions files: regex-repro.py messages: 304849 nosy: bkline, ezio.melotti, mrabarnett priority: normal severity: normal status: open title: Unexpected behavior of re module when VERBOSE flag is set type: behavior versions: Python 2.7, Python 3.5, Python 3.6 Added file: https://bugs.python.org/file47232/regex-repro.py _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Oct 23 21:09:22 2017 From: report at bugs.python.org (pdox) Date: Tue, 24 Oct 2017 01:09:22 +0000 Subject: [New-bugs-announce] [issue31857] Make the behavior of USE_STACKCHECK deterministic Message-ID: <1508807362.1.0.213398074469.issue31857@psf.upfronthosting.co.za> New submission from pdox : USE_STACKCHECK is a Windows-only feature which provides additional safety against C stack overflow by periodically calling PyOS_CheckStack to determine whether the current thread is too close to the end of the stack. The way USE_STACKCHECK ensures that PyOS_CheckStack is called frequently is surprising. It does this by artificially decrementing _Py_CheckRecursionLimit with every call to Py_EnterRecursiveCall: #ifdef USE_STACKCHECK /* With USE_STACKCHECK, we artificially decrement the recursion limit in order to trigger regular stack checks in _Py_CheckRecursiveCall(), except if the "overflowed" flag is set, in which case we need the true value of _Py_CheckRecursionLimit for _Py_MakeEndRecCheck() to function properly. */ # define _Py_MakeRecCheck(x) \ (++(x) > (_Py_CheckRecursionLimit += PyThreadState_GET()->overflowed - 1)) #else # define _Py_MakeRecCheck(x) (++(x) > _Py_CheckRecursionLimit) #endif _Py_CheckRecursionLimit defaults to 1000, and is constant when USE_STACKCHECK is off. (except when changed by Py_SetRecursionLimit) With a single thread, each call to Py_EnterRecursiveCall causes _Py_CheckRecursionLimit to decrement by 1 until it equals the current recursion depth, at which point _Py_CheckRecursiveCall is triggered, resetting _Py_CheckRecursionLimit back to the default. This could be anywhere from 500 to 1000 calls depending on the recursion depth. With multiple threads, the behavior may be more chaotic, as each thread will be decrementing _Py_CheckRecursionLimit independently. I propose that instead, the call to PyOS_CheckStack is triggered in a predictable fashion, using a separate counter for each thread, so that it occurs every N'th call to Py_EnterRecursiveCall. (N = 64 seems reasonable, as PyOS_CheckStack guarantees 2048 * sizeof(void*) bytes remaining on the C stack). ---------- components: Windows messages: 304854 nosy: paul.moore, pdox, steve.dower, tim.golden, zach.ware priority: normal severity: normal status: open title: Make the behavior of USE_STACKCHECK deterministic type: behavior versions: Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Oct 24 01:58:28 2017 From: report at bugs.python.org (Terry J. Reedy) Date: Tue, 24 Oct 2017 05:58:28 +0000 Subject: [New-bugs-announce] [issue31858] IDLE: cleanup use of sys.ps1 and never set it. Message-ID: <1508824708.42.0.213398074469.issue31858@psf.upfronthosting.co.za> New submission from Terry J. Reedy : This issue is about cleaning up IDLE's use of sys.ps1 for its prompt (sys.ps2 is not used). A. This will make for cleaner code and fix some bugs. B. This will be better for testing. (Some possible changes to pyshell might make sys.ps1 irrelevant, but that is for the future.) Part1. editor.EditorWindow.__init__ sets sys.ps1 to '>>> ' if not set. try: sys.ps1 except AttributeError: sys.ps1 = '>>> ' IDLE initially respects a user setting of sys.ps1 in a startup file. pyshell.PyShell.open_debugger hasand .close_debugger has these lines sys.ps1 = "[DEBUG ON]\n>>> " sys.ps1 = ">>> " These overwrite any user setting of sys.ps1. As long as IDLE pays attention to the initial value of sys.ps1, I consider this a bug. pyshell.PyShell.show_prompt starts with try: s = str(sys.ps1) except: s = "" self.console.write(s) I suspect that this is a holdover from when IDLE executed user code in its own process, as it still does with the deprecated '-n' option. However, if a -n user deletes sys.ps1, the replacement should be '>>> ', not '' (bug 2). In the current default subprocess mode, users cannot change the IDLE process sys module (see #13657), so rereading sys.ps1 for every editor window and prompt is nonsensical. Patch 1: replace the EditorWindow.__init__ code with module level code sys_ps1 = sys.ps1 if hasattr(sys, 'ps1') else '>>> ' prompt = sys_ps1 Fix pyshell to import editor rather than two of its objects. In its debugger methods, set editor.prompt, using editor.sys_ps1, thus preserving any user setting. In print_prompt, print the current editor.prompt. (This will disable users resetting ps1 in deprecated -n mode, but I consider that okay as it matches the normal mode.) Part 2. The reason the prompt is set in EditorWindow, instead of where is it obviously needed, the PyShell subclass of the OutputWindow subclass of EditorWindow, is that it is currently used in EditorWindow.smart_backspace_event and .newline_and_indent_event, while pyshell imports editor (and must), and not the other way around. Treating the prompt text as special in an editor lead to a bug in smart_backspace that was fixed in #13039 by guarding it use with self.context_use_ps1. There is still a nearly inconsequential bug in the newline method where the prompt use is not guarded. (Hitting newline with the cursor before the 't' in '>>> test' leaves the preceding space instead of deleting it.) Patch 2: Only the last line of the prompt is relevant in either method. I believe that replacing self.context_use_ps1 = False in an editor, = True in Shell with self.last_prompt_line = '' in an editor, = whatever it is in Shell, will allow moving sys_ps1 and prompt to pyshell. This would simplify patch 1. ---------- assignee: terry.reedy components: IDLE messages: 304860 nosy: terry.reedy priority: normal severity: normal stage: test needed status: open title: IDLE: cleanup use of sys.ps1 and never set it. type: behavior versions: Python 3.6, Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Oct 24 08:13:37 2017 From: report at bugs.python.org (Tim) Date: Tue, 24 Oct 2017 12:13:37 +0000 Subject: [New-bugs-announce] [issue31859] sharedctypes.RawArray initialization Message-ID: <1508847217.79.0.213398074469.issue31859@psf.upfronthosting.co.za> New submission from Tim : In the initialization of a new `RawArray` the `size_or_initializer` parameter is tested for being an instance of `int`. When passing something like numpy.int64 here, the code crashes, because it does not recognize this as an integer. The workaround is to cast to int(). Wouldn't it be nicer to compare to types.IntType to allow for custom integer types? def RawArray(typecode_or_type, size_or_initializer): ''' Returns a ctypes array allocated from shared memory ''' type_ = typecode_to_type.get(typecode_or_type, typecode_or_type) - if isinstance(size_or_initializer, int): + if size_or_initializer is IntType: ---------- messages: 304902 nosy: meetaig priority: normal severity: normal status: open title: sharedctypes.RawArray initialization type: behavior versions: Python 3.5, Python 3.6 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Oct 24 09:20:40 2017 From: report at bugs.python.org (Serhiy Storchaka) Date: Tue, 24 Oct 2017 13:20:40 +0000 Subject: [New-bugs-announce] [issue31860] IDLE: Make font sample editable Message-ID: <1508851240.71.0.213398074469.issue31860@psf.upfronthosting.co.za> New submission from Serhiy Storchaka : The proposed patch makes the font sample in IDLE font configuration dialog editable. This allows users to test fonts with arbitrary samples. ---------- assignee: terry.reedy components: IDLE messages: 304909 nosy: serhiy.storchaka, terry.reedy priority: normal severity: normal status: open title: IDLE: Make font sample editable type: enhancement versions: Python 3.6, Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Oct 24 09:26:15 2017 From: report at bugs.python.org (Davide Rizzo) Date: Tue, 24 Oct 2017 13:26:15 +0000 Subject: [New-bugs-announce] [issue31861] aiter() and anext() built-in functions Message-ID: <1508851575.6.0.213398074469.issue31861@psf.upfronthosting.co.za> New submission from Davide Rizzo : PEP 525 suggested that adding aiter() and anext() would need to wait until async __aiter__ is dropped in 3.7. Issue 31709 solved that, so now it would be possible to add them. ---------- components: Library (Lib) messages: 304910 nosy: davide.rizzo priority: normal severity: normal status: open title: aiter() and anext() built-in functions versions: Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Oct 24 11:08:56 2017 From: report at bugs.python.org (Marcel Plch) Date: Tue, 24 Oct 2017 15:08:56 +0000 Subject: [New-bugs-announce] [issue31862] Port the standard library to PEP 489 multiphase initialization Message-ID: <1508857736.11.0.213398074469.issue31862@psf.upfronthosting.co.za> New submission from Marcel Plch : PEP 489 introduced multiphase initialization of extension and built-in modules. Now, almost no module in the standard library supports this feature. This should be improved to prepare Python for better testing of subinterpreters. Many benefits of PEP 489 don't apply to stdlib modules. However, the PEP effectively says that by using multi-phase init, the module author "promises" that the module is "subinterpreter-friendly" [0]. So, when porting, each module should be checked that it e.g. doesn't use mutable process-global state. I'd like to port stdlib to multi-phase init, starting with the easier modules, to: - get familiar with contributing to CPython, - check and track which modules are already "subinterpreter-friendly", and - figure out how and where PEP 489 is lacking (beyond what is discussed in the PEP itself). [0]: https://www.python.org/dev/peps/pep-0489/#subinterpreters-and-interpreter-reloading ---------- components: Extension Modules messages: 304914 nosy: Dormouse759, encukou, ncoghlan priority: normal severity: normal status: open title: Port the standard library to PEP 489 multiphase initialization type: enhancement versions: Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Oct 24 11:46:53 2017 From: report at bugs.python.org (Akos Kiss) Date: Tue, 24 Oct 2017 15:46:53 +0000 Subject: [New-bugs-announce] [issue31863] Inconsistent returncode/exitcode for terminated child processes on Windows Message-ID: <1508860013.85.0.213398074469.issue31863@psf.upfronthosting.co.za> New submission from Akos Kiss : I've been working with various approaches for running and terminating subprocesses on Windows and I've obtained surprisingly different results if I used different modules and ways of termination. Here is the script I wrote, it uses the `subprocess` and the `multiprocessing` modules for starting new subprocesses, and process termination is performed either by the modules' own `terminate` functions or by `os.kill`. ```py import multiprocessing import os import signal import subprocess import sys import time def kill_with_os_kill(proc): print('kill with os.kill(pid,SIGTERM)') os.kill(proc.pid, signal.SIGTERM) def kill_with_terminate(proc): print('kill child with proc.terminate()') proc.terminate() def run_and_kill_subprocess(killfn, procarg): print('run subprocess child with %s' % procarg) with subprocess.Popen([sys.executable, __file__, procarg]) as proc: time.sleep(1) killfn(proc) proc.wait() print('child terminated with %s' % proc.returncode) def run_and_kill_multiprocessing(killfn, procarg): print('run multiprocessing child with %s' % procarg) proc = multiprocessing.Process(target=childmain, args=(procarg,)) proc.start() time.sleep(1) killfn(proc) proc.join() print('child terminated with %s' % proc.exitcode) def childmain(arg): print('child process started with %s' % arg) while True: pass if __name__ == '__main__': if len(sys.argv) < 2: print('parent process started') run_and_kill_subprocess(kill_with_os_kill, 'subprocess-oskill') run_and_kill_subprocess(kill_with_terminate, 'subprocess-terminate') run_and_kill_multiprocessing(kill_with_os_kill, 'multiprocessing-oskill') run_and_kill_multiprocessing(kill_with_terminate, 'multiprocessing-terminate') else: childmain(sys.argv[1]) ``` On macOS, everything works as expected (and I think that Linux will behave alike): ``` $ python3 killtest.py parent process started run subprocess child with subprocess-oskill child process started with subprocess-oskill kill with os.kill(pid,SIGTERM) child terminated with -15 run subprocess child with subprocess-terminate child process started with subprocess-terminate kill child with proc.terminate() child terminated with -15 run multiprocessing child with multiprocessing-oskill child process started with multiprocessing-oskill kill with os.kill(pid,SIGTERM) child terminated with -15 run multiprocessing child with multiprocessing-terminate child process started with multiprocessing-terminate kill child with proc.terminate() child terminated with -15 ``` But on Windows, I got: ``` >py -3 killtest.py parent process started run subprocess child with subprocess-oskill child process started with subprocess-oskill kill with os.kill(pid,SIGTERM) child terminated with 15 run subprocess child with subprocess-terminate child process started with subprocess-terminate kill child with proc.terminate() child terminated with 1 run multiprocessing child with multiprocessing-oskill child process started with multiprocessing-oskill kill with os.kill(pid,SIGTERM) child terminated with 15 run multiprocessing child with multiprocessing-terminate child process started with multiprocessing-terminate kill child with proc.terminate() child terminated with -15 ``` Notes: - On Windows with `os.kill(pid, sig)`, "sig will cause the process to be unconditionally killed by the TerminateProcess API, and the exit code will be set to sig." I.e., it is not possible to detect on Windows whether a process was terminated by a signal or it exited properly, because `kill` does not actually raise a signal and no Windows API allows to differentiate between proper or forced termination. - The `multiprocessing` module has a workaround for this by terminating the process with a designated exit code (`TERMINATE = 0x10000`) and checking for that value afterwards, rewriting it to `-SIGTERM` if found. The related documentation is a bit misleading, as `exitcode` is meant to have "negative value -N [which] indicates that the child was terminated by signal N" -- however, if the process was indeed killed with `SIGTERM` (and not via `terminate`), then `exitcode` will be `SIGTERM` and not `-SIGTERM` (see above). (The documentation of `terminate` does not clarify the situation much by stating that "on Windows TerminateProcess() is used", since it does not mention the special exit code -- and well, it's not even a signal after all, so it's not obvious whether negative or positive exit code is to be expected.) - The `subprocess` module choses the quite arbitrary exit code of 1 and documents that "negative value -N indicates that the child was terminated by signal N" is POSIX only, not mentioning anything about what to expect on Windows. Long story short: on Windows, the observable exit code of a forcibly terminated child process is quite inconsistent even across standard modules and termination methods, unlike on other (Linux/macOS) platforms. I think that having results consistent with `os.kill(,SIGTERM)` would be desirable even if that means non-negative values. I'm willing to post a PR if the issue is deemed to be valid. ---------- components: Library (Lib), Windows messages: 304920 nosy: Akos Kiss, paul.moore, steve.dower, tim.golden, zach.ware priority: normal severity: normal status: open title: Inconsistent returncode/exitcode for terminated child processes on Windows type: behavior _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Oct 24 18:56:14 2017 From: report at bugs.python.org (Geoff Kuenning) Date: Tue, 24 Oct 2017 22:56:14 +0000 Subject: [New-bugs-announce] [issue31864] datetime violates Postel's law Message-ID: <1508885774.85.0.213398074469.issue31864@psf.upfronthosting.co.za> New submission from Geoff Kuenning : The datetime package is too eager to reject nonconforming VCALENDAR-format files. The particular issue I encountered is related to time zones. RFC 5545 clearly states that the DTSTART field is required. However, there are situations where DTSTART isn't strictly necessary because the zone in question doesn't observe daylight-savings time and never has. For example, I have a VCALENDAR file that was generated by a program that omits DTSTART for such zones. Here's an example: BEGIN:VTIMEZONE TZID:America/Phoenix X-LIC-LOCATION:America/Phoenix BEGIN:DAYLIGHT TZOFFSETFROM:-0700 TZOFFSETTO:-0700 TZNAME:Ariz END:DAYLIGHT BEGIN:STANDARD TZOFFSETFROM:-0700 TZOFFSETTO:-0700 TZNAME:Ariz END:STANDARD END:VTIMEZONE Clearly, this file violates RFC 5445, and I have reported that fact to the program's maintainer (who will fix the problem soon). Nevertheless, feeding an ICS file to datetime._parse_rfc with this error causes a ValueError exception, which makes the VCALENDAR file unreadable. In keeping with Postel's law ("Be conservative in what you do, be liberal in what you accept from others"), _parse_rfc should attempt to accept VCALENDAR files whenever it is possible to make sense of them. Thus, for example: if not founddtstart: rrulelines.append('DTSTART:19000101T020000') The above could be improved a bit, for example by still rejecting entries in which the standard and daylight sections had different offsets (although even then it seems valid to assume a DTSTART in the distant past). Although the dtstart issue is the one that prompted this report, I also noticed the following in _parse_rfc: if name == "BEGIN": if value in ("STANDARD", "DAYLIGHT"): # Process component pass else: raise ValueError("unknown component: "+value) Again, there's an opportunity to be more robust here. One could issue a warning message, but then ignore the unknown component. In both cases (and I suspect numerous others), halting parsing is an extreme response to various errors, since it leaves the user of the package with no way to process a nonconforming file. That's especially problematic since VCALENDAR files are generated by so many different programs, many of which are written by programmers who haven't bothered to read RFC 5445--or who have read it but then made some minor mistake that produces broken files. ---------- messages: 304944 nosy: gkuenning priority: normal severity: normal status: open title: datetime violates Postel's law _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Oct 24 22:46:46 2017 From: report at bugs.python.org (Jun Hui Lee) Date: Wed, 25 Oct 2017 02:46:46 +0000 Subject: [New-bugs-announce] [issue31865] html.unescape does not work as per documentation Message-ID: <1508899606.23.0.213398074469.issue31865@psf.upfronthosting.co.za> New submission from Jun Hui Lee : html.unescape(s) Convert all named and numeric character references (e.g. >, >, &x3e;) But running this gives: >>> html.unescape('>, >, &x3e;') '>, >, &x3e;' ---------- assignee: docs at python components: Documentation messages: 304957 nosy: Jun Hui Lee, docs at python priority: normal severity: normal status: open title: html.unescape does not work as per documentation type: behavior versions: Python 3.6 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed Oct 25 02:20:23 2017 From: report at bugs.python.org (Benjamin Peterson) Date: Wed, 25 Oct 2017 06:20:23 +0000 Subject: [New-bugs-announce] [issue31866] clean out some more AtheOS code Message-ID: <1508912423.08.0.213398074469.issue31866@psf.upfronthosting.co.za> New submission from Benjamin Peterson : We stopped support AtheOS in 2007. But, there are still references in the code: $ git grep -i AtheOS Doc/whatsnew/2.3.rst:Other new platforms now supported by Python include AtheOS Doc/whatsnew/2.3.rst:(http://atheos.cx/), GNU/Hurd, and OpenVMS. Lib/distutils/command/build_ext.py: # for extensions under Cygwin and AtheOS Python's library directory must be Lib/distutils/command/build_ext.py: if sys.platform[:6] == 'cygwin' or sys.platform[:6] == 'atheos': Lib/distutils/command/build_ext.py: elif sys.platform[:6] == "atheos": Lib/test/test_os.py: # On AtheOS, glibc always returns ENOSYS Lib/test/test_os.py: # On AtheOS, glibc always returns ENOSYS Misc/HISTORY:- Support for AtheOS has been completely removed from the code base. It was Misc/HISTORY:- Support for BeOS and AtheOS was removed (according to PEP 11). Misc/HISTORY:- AtheOS is now supported. Modules/_cryptmodule.c: /* On some platforms (AtheOS) crypt returns NULL for an invalid config.guess: i*86:atheos:*:*) config.guess: echo ${UNAME_MACHINE}-unknown-atheos config.sub: -atheos*) config.sub: os=-atheos configure:atheos*|Linux*/1*) configure.ac:atheos*|Linux*/1*) ---------- messages: 304963 nosy: benjamin.peterson priority: normal severity: normal status: open title: clean out some more AtheOS code versions: Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed Oct 25 06:14:00 2017 From: report at bugs.python.org (Mark Shannon) Date: Wed, 25 Oct 2017 10:14:00 +0000 Subject: [New-bugs-announce] [issue31867] Duplicated keys, but with different values in dictionary literals Message-ID: <1508926440.94.0.213398074469.issue31867@psf.upfronthosting.co.za> New submission from Mark Shannon : Here https://github.com/python/cpython/blob/master/Lib/mimetypes.py#L416 and here https://github.com/python/cpython/blob/master/Lib/mimetypes.py#L526 I have no idea which is the correct value for either, but the code is misleading at best. ---------- components: Library (Lib) messages: 304973 nosy: Mark.Shannon priority: normal severity: normal status: open title: Duplicated keys, but with different values in dictionary literals versions: Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed Oct 25 06:25:18 2017 From: report at bugs.python.org (=?utf-8?q?Tomasz_Mi=C4=85sko?=) Date: Wed, 25 Oct 2017 10:25:18 +0000 Subject: [New-bugs-announce] [issue31868] Null pointer dereference in ndb.ndbm get when used with a default value. Message-ID: <1508927118.96.0.213398074469.issue31868@psf.upfronthosting.co.za> New submission from Tomasz Mi?sko : Using ndb.ndbm get when key is missing and default value has to be returned results in invocation of Py_INCREF on null pointer. Test case to reproduce the issue: ``` import dbm.ndbm with dbm.ndbm.open('db', 'n') as db: print(db.get('missing-key')) ``` ---------- files: 0001-Fix-null-pointer-dereference-in-dbm.ndbm-get.patch keywords: patch messages: 304977 nosy: tmiasko priority: normal severity: normal status: open title: Null pointer dereference in ndb.ndbm get when used with a default value. type: crash versions: Python 3.5, Python 3.6, Python 3.7, Python 3.8 Added file: https://bugs.python.org/file47236/0001-Fix-null-pointer-dereference-in-dbm.ndbm-get.patch _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed Oct 25 12:01:35 2017 From: report at bugs.python.org (J Sloot) Date: Wed, 25 Oct 2017 16:01:35 +0000 Subject: [New-bugs-announce] [issue31869] commentary on ssl.PROTOCOL_TLS Message-ID: <1508947295.4.0.213398074469.issue31869@psf.upfronthosting.co.za> New submission from J Sloot : on https://docs.python.org/2/library/ssl.html#ssl.PROTOCOL_TLS for ssl.PROTOCOL_TLS When the commentary from ssl.PRTOCOL_TLS moved from PROTOCOL_SSLv23 to PROTOCOL_TLS the note {Despite the name, this option can select "TLS" protocols as well as "SSL"} didn't change. Suggested new wording {Despite the name, this option can select "SSL" protocols as well as "TLS"} ---------- assignee: docs at python components: Documentation messages: 304996 nosy: J Sloot, docs at python priority: normal severity: normal status: open title: commentary on ssl.PROTOCOL_TLS versions: Python 2.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu Oct 26 00:11:25 2017 From: report at bugs.python.org (Vex Woo) Date: Thu, 26 Oct 2017 04:11:25 +0000 Subject: [New-bugs-announce] [issue31870] add timeout parameter for get_server_certificate in ssl.py Message-ID: <1508991085.56.0.213398074469.issue31870@psf.upfronthosting.co.za> New submission from Vex Woo : The original get_server_certificate in ssl.py does not support socket timeout, def get_server_certificate(addr, ssl_version=PROTOCOL_TLS, ca_certs=None): """Retrieve the certificate from the server at the specified address, and return it as a PEM-encoded string. If 'ca_certs' is specified, validate the server cert against it. If 'ssl_version' is specified, use it in the connection attempt.""" host, port = addr if ca_certs is not None: cert_reqs = CERT_REQUIRED else: cert_reqs = CERT_NONE context = _create_stdlib_context(ssl_version, cert_reqs=cert_reqs, cafile=ca_certs) with create_connection(addr) as sock: with context.wrap_socket(sock) as sslsock: dercert = sslsock.getpeercert(True) return DER_cert_to_PEM_cert(dercert) If a timeout parameter, a sample demo can be here: >>> import ssl >>> ssl.get_server_certificate(("www.qq.com", 443), timeout=6) Traceback (most recent call last): File "", line 1, in File "/usr/lib/python2.7/ssl.py", line 1017, in get_server_certificate with closing(create_connection(addr, timeout)) as sock: File "/usr/lib/python2.7/socket.py", line 575, in create_connection raise err socket.error: [Errno 101] Network is unreachable ---------- components: Library (Lib) files: ssl.py messages: 305021 nosy: Nixawk priority: normal pull_requests: 4092 severity: normal status: open title: add timeout parameter for get_server_certificate in ssl.py type: enhancement Added file: https://bugs.python.org/file47238/ssl.py _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu Oct 26 01:33:13 2017 From: report at bugs.python.org (Mateusz Kurek) Date: Thu, 26 Oct 2017 05:33:13 +0000 Subject: [New-bugs-announce] [issue31871] Support for file descriptor params in os.path Message-ID: <1508995993.41.0.213398074469.issue31871@psf.upfronthosting.co.za> New submission from Mateusz Kurek : Since Python 3.3, some os module functions, like os.stat (https://docs.python.org/3/library/os.html#os.stat), support passing file descriptor instead of a path. os.path functions, on the other hand (like os.path.exists - https://docs.python.org/3/library/os.path.html#os.path.exists - or os.path.samefile - https://docs.python.org/3/library/os.path.html#os.path.samefile) mention using os.stat underneath, but according to documentation, does not support passing file descriptor instead of a path (at least it's not mentioned in the docs that this feature is supported). Is this intentional (os.path functions should not take file descriptor params) or this feature is officialy supported, but it's ommited in the docs? ---------- assignee: docs at python components: Documentation messages: 305023 nosy: Mateusz Kurek, docs at python priority: normal severity: normal status: open title: Support for file descriptor params in os.path type: enhancement versions: Python 3.4, Python 3.5, Python 3.6, Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu Oct 26 06:09:19 2017 From: report at bugs.python.org (Andrew Svetlov) Date: Thu, 26 Oct 2017 10:09:19 +0000 Subject: [New-bugs-announce] [issue31872] SSL BIO is broken for internationalized domains Message-ID: <1509012559.64.0.213398074469.issue31872@psf.upfronthosting.co.za> New submission from Andrew Svetlov : `SSLContext.wrap_bio` creates a new `SSLObject` instance with passed `server_hostname`. The name becomes IDNA-decoded: `'xn--2qq421aovb6v1e3pu.xn--j6w193g'` is converted to `'?????.??'` by `SSLObject` constructor. Than on SSL handshake `ssl.match_hostname()` is called with `sslobject.server_hostname` parameter (`'?????.??'` in my example). But certificate for the site is contains IDNA-encoded DNS names: ``` {'OCSP': ('http://ocsp.comodoca4.com',), 'caIssuers': ('http://crt.comodoca4.com/COMODOECCDomainValidationSecureServerCA2.crt',), 'crlDistributionPoints': ('http://crl.comodoca4.com/COMODOECCDomainValidationSecureServerCA2.crl',), 'issuer': ((('countryName', 'GB'),), (('stateOrProvinceName', 'Greater Manchester'),), (('localityName', 'Salford'),), (('organizationName', 'COMODO CA Limited'),), (('commonName', 'COMODO ECC Domain Validation Secure Server CA 2'),)), 'notAfter': 'Mar 28 23:59:59 2018 GMT', 'notBefore': 'Sep 19 00:00:00 2017 GMT', 'serialNumber': 'FBFE0BF7CACA6DDC15968410BAA1908D', 'subject': ((('organizationalUnitName', 'Domain Control Validated'),), (('organizationalUnitName', 'PositiveSSL Multi-Domain'),), (('commonName', 'sni38752.cloudflaressl.com'),)), 'subjectAltName': (('DNS', 'sni38752.cloudflaressl.com'), ('DNS', '*.1km.hk'), ('DNS', '*.acg-cafe.com'), ('DNS', '*.acgapp.moe'), ('DNS', '*.acgapp.net'), ('DNS', '*.cosmatch.org'), ('DNS', '*.dirimusik.com'), ('DNS', '*.dirimusik.info'), ('DNS', '*.downloadlagi.club'), ('DNS', '*.downloadlaguaz.info'), ('DNS', '*.farmprecision.com'), ('DNS', '*.glowecommercialphotography.co.uk'), ('DNS', '*.hypertechglobal.com'), ('DNS', '*.hypertechglobal.hk'), ('DNS', '*.infoku.download'), ('DNS', '*.inimp3.com'), ('DNS', '*.luciafitness.com.au'), ('DNS', '*.merdeka.news'), ('DNS', '*.promisecos.com'), ('DNS', '*.promisecos.hk'), ('DNS', '*.ps9architects.com'), ('DNS', '*.rubaxeu.gq'), ('DNS', '*.ruth-fox.com'), ('DNS', '*.simmit.net.au'), ('DNS', '*.startss.today'), ('DNS', '*.xn--2qq421aovb6v1e3pu.xn--j6w193g'), ('DNS', '*.xn--hhrw16aw6jizf.xn--j6w193g'), ('DNS', '1km.hk'), ('DNS', 'acg-cafe.com'), ('DNS', 'acgapp.moe'), ('DNS', 'acgapp.net'), ('DNS', 'cosmatch.org'), ('DNS', 'dirimusik.com'), ('DNS', 'dirimusik.info'), ('DNS', 'downloadlagi.club'), ('DNS', 'downloadlaguaz.info'), ('DNS', 'farmprecision.com'), ('DNS', 'glowecommercialphotography.co.uk'), ('DNS', 'hypertechglobal.com'), ('DNS', 'hypertechglobal.hk'), ('DNS', 'infoku.download'), ('DNS', 'inimp3.com'), ('DNS', 'luciafitness.com.au'), ('DNS', 'merdeka.news'), ('DNS', 'promisecos.com'), ('DNS', 'promisecos.hk'), ('DNS', 'ps9architects.com'), ('DNS', 'rubaxeu.gq'), ('DNS', 'ruth-fox.com'), ('DNS', 'simmit.net.au'), ('DNS', 'startss.today'), ('DNS', 'xn--2qq421aovb6v1e3pu.xn--j6w193g'), ('DNS', 'xn--hhrw16aw6jizf.xn--j6w193g')), 'version': 3} ``` Match `'?????.??'` to `('DNS', 'xn--2qq421aovb6v1e3pu.xn--j6w193g')` obviously fails. I see two possible solutions: 1. Always do IDNA encoding for `server_hostname` stored in ssl object. 2. Do two checks for both IDNA and original server hostname values. I don't sure if certificates always use IDNA-encoded DNS names only. The fix is trivial, I'll make a Pull Request after choosing what option we want to support. Personally I'm inclined to second one. P.S. `requests` library is not affected because it uses `ssl.wrap_socket`. The bug is reproducible for `asyncio` only (and maybe Tornado with `asyncio` `IOLoop`). ---------- assignee: christian.heimes components: SSL messages: 305042 nosy: asvetlov, christian.heimes, pitrou priority: normal severity: normal status: open title: SSL BIO is broken for internationalized domains versions: Python 3.5, Python 3.6, Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu Oct 26 09:15:32 2017 From: report at bugs.python.org (David Antonini) Date: Thu, 26 Oct 2017 13:15:32 +0000 Subject: [New-bugs-announce] [issue31873] Inconsistent capitalization of proper noun - Unicode. Message-ID: <1509023732.92.0.213398074469.issue31873@psf.upfronthosting.co.za> New submission from David Antonini : Make 'unicode'/'Unicode' capitalization consistent. 'Unicode' is a proper noun, and as such should be capitalized. I submitted a pull request correcting the inconsistent capitalization in the Unicode page of the Documentation - Doc/c-api/unicode.rst - capitalizing 12 errant instances to reflect the correct capitalization in most of the document. I was then requested to open an issue here for discussion. ---------- assignee: docs at python components: Documentation messages: 305050 nosy: docs at python, toonarmycaptain priority: normal pull_requests: 4096 severity: normal status: open title: Inconsistent capitalization of proper noun - Unicode. type: enhancement _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu Oct 26 10:01:14 2017 From: report at bugs.python.org (Jason R. Coombs) Date: Thu, 26 Oct 2017 14:01:14 +0000 Subject: [New-bugs-announce] [issue31874] [feature] runpy.run_module should mimic script launch behavior for sys.path Message-ID: <1509026474.72.0.213398074469.issue31874@psf.upfronthosting.co.za> New submission from Jason R. Coombs : In [this comment](https://bugs.python.org/issue16737#msg282872) I describe an issue whereby launching an application via runpy.run_module (aka python -m) produces different and unexpected behavior than running the same app via an entry script. In [this followup comment](https://bugs.python.org/issue16737#msg304662), I provide more detail on why a user might expect for run_module to mimic the behavior of launching a script. [Nick suggests](https://bugs.python.org/issue16737#msg304707): > Update sys.path[0] based on __main__.__spec__.origin after [resolving] __main__.__spec__. That way it will only stay as the current directory if the module being executed is in a subdirectory of the current directory, and will otherwise be updated appropriately for wherever we actually found the main module. ---------- messages: 305054 nosy: jason.coombs priority: normal severity: normal status: open title: [feature] runpy.run_module should mimic script launch behavior for sys.path versions: Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu Oct 26 10:16:27 2017 From: report at bugs.python.org (Gareth Moger) Date: Thu, 26 Oct 2017 14:16:27 +0000 Subject: [New-bugs-announce] [issue31875] Error 0x80070642: Failed to install MSI package. Message-ID: <1509027387.81.0.213398074469.issue31875@psf.upfronthosting.co.za> New submission from Gareth Moger : I had my work computer re-imaged this week but Python 3.6.1 is gone and I am unable to install. I have tried to completely remove it with CCleaner and any other references I found but it still will not install. I am installing the same version as before. Following the advice from another post, I installed on another machine and then copied the folder onto my work machine but was unable to uninstall/repair as described. Attached is the log file. ---------- components: Installation files: Python 3.6.1 (32-bit)_20171026151143.log messages: 305058 nosy: Gareth Moger priority: normal severity: normal status: open title: Error 0x80070642: Failed to install MSI package. versions: Python 3.6 Added file: https://bugs.python.org/file47240/Python 3.6.1 (32-bit)_20171026151143.log _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu Oct 26 10:21:36 2017 From: report at bugs.python.org (Hery) Date: Thu, 26 Oct 2017 14:21:36 +0000 Subject: [New-bugs-announce] [issue31876] python363.chm includes gibberish Message-ID: <1509027696.53.0.213398074469.issue31876@psf.upfronthosting.co.za> New submission from Hery : Just Open https://www.python.org/ftp/python/3.6.3/python363.chm And click the first page, it says "What? New in Python". And most of pages the chm file include some gibberish. ---------- assignee: docs at python components: Documentation messages: 305059 nosy: Nim, docs at python priority: normal severity: normal status: open title: python363.chm includes gibberish type: behavior versions: Python 3.6 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu Oct 26 10:54:59 2017 From: report at bugs.python.org (Erik Bray) Date: Thu, 26 Oct 2017 14:54:59 +0000 Subject: [New-bugs-announce] [issue31877] Build fails on Cython since issue28180 Message-ID: <1509029699.29.0.213398074469.issue31877@psf.upfronthosting.co.za> New submission from Erik Bray : I'm trying once again to get the test suite up to snuff on Cygwin so I can start running a buildbot (particularly now that PEP 539 is done). However, as of issue28180 the build fails with: gcc -o python.exe Programs/python.o libpython3.7m.dll.a -lintl -ldl -lm Programs/python.o: In function `main': ./Programs/python.c:81: undefined reference to `_Py_LegacyLocaleDetected' ./Programs/python.c:81:(.text.startup+0x86): relocation truncated to fit: R_X86_64_PC32 against undefined symbol `_Py_LegacyLocaleDetected' ./Programs/python.c:82: undefined reference to `_Py_CoerceLegacyLocale' ./Programs/python.c:82:(.text.startup+0x19f): relocation truncated to fit: R_X86_64_PC32 against undefined symbol `_Py_CoerceLegacyLocale' collect2: error: ld returned 1 exit status It seems _Py_LegacyLocaleDetected and _PyCoerceLegacyLocale are missing the necessary PyAPI_FUNC declarations in pylifecycle.h. ---------- messages: 305061 nosy: erik.bray priority: normal severity: normal status: open title: Build fails on Cython since issue28180 type: compile error _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu Oct 26 11:42:21 2017 From: report at bugs.python.org (Erik Bray) Date: Thu, 26 Oct 2017 15:42:21 +0000 Subject: [New-bugs-announce] [issue31878] Cygwin: _socket module does not compile due to missing ioctl declaration Message-ID: <1509032541.6.0.213398074469.issue31878@psf.upfronthosting.co.za> New submission from Erik Bray : On Cygwin, ioctl() is found in sys/ioctl.h (as on Darwin). Without adding something to the effect of #ifdef __CYGWIN__ # include #endif the _socket module cannot compile on Cygwin. A fix was this was included in the (rejected) https://bugs.python.org/issue29718; this issue is just as a reminder that it remains an issue and to have a bug report to attach a more focused PR to. ---------- messages: 305065 nosy: erik.bray priority: normal severity: normal status: open title: Cygwin: _socket module does not compile due to missing ioctl declaration type: compile error _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu Oct 26 13:01:08 2017 From: report at bugs.python.org (Robert) Date: Thu, 26 Oct 2017 17:01:08 +0000 Subject: [New-bugs-announce] [issue31879] Launcher fails on custom command starting with "python" Message-ID: <1509037268.43.0.213398074469.issue31879@psf.upfronthosting.co.za> New submission from Robert : In the "py.ini" file it is possible to specifiy customized commands (see https://www.python.org/dev/peps/pep-0397/#customized-commands). Unfortunately it is not possible to specify custom commands beginning with one of the buildin names (i.e. "python" or "/usr/bin/python"). This means that if I want to provide a virtual command like "python-xyz" I get an error message "Invalid version specification: '-xyz'" instead of running the executable assigned to python-xyz. I would provide a Pullrequest if you accept this change in behaviour. ---------- components: Windows messages: 305072 nosy: mrh1997, paul.moore, steve.dower, tim.golden, zach.ware priority: normal severity: normal status: open title: Launcher fails on custom command starting with "python" type: behavior versions: Python 2.7, Python 3.4, Python 3.5, Python 3.6, Python 3.7, Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu Oct 26 13:23:38 2017 From: report at bugs.python.org (John Brearley) Date: Thu, 26 Oct 2017 17:23:38 +0000 Subject: [New-bugs-announce] [issue31880] subprocess process interaction with IDLEX GUI causes pygnuplot silent failures Message-ID: <1509038618.41.0.213398074469.issue31880@psf.upfronthosting.co.za> New submission from John Brearley : There is an interesting interaction between the IDLEX GUI and subprocess module that causes pygnuplot silent failures. The example.py script below works fine when run from the WinPython Command Prompt.exe terminal window. The script will popup a new window from gnuplot with the desired graph, and in the same directory save the generated raw data as example.out and the generated graph as example.pdf. If you erase the generated files and then run the same script via the IDLEX GUI, there is no graph window created and only the raw data example.out file is created. There are no error messages. The PyGnuplot module sets up a subprocess.Popen pipe as persistant connection to the gnuplot.exe utility. This allows the top level script to send multiple commands to gnuplot to compose the desired graph, and then finally save the file to disk. This all works fine when run from the WinPython Command Prompt.exe terminal window. However something subtle is breaking when the same script is run from the IDLEX GUI environment. It is not at all obvious if the subprocess module is not as forgiving as it needs to be of the IDLEX GUI input or if the IDLEX GUI is breaking the rules somewhere. I will start by asking the subprocess module team to comment. I did try adding some trace code to the PyGnuplot.py module. In particular I turned on stdout=subprocess.PIPE and stderr=subprocess.PIPE. I did proc.poll() before/after the command is sent to gnuplot. Interestingly, when I did proc.communicate(timeout=0) to do an immediate read for any stdout/err data, this seems to cause the subsequent write to the pipe.stdin to fail, saying the file is already closed. In another learning exercise script for subprocess, the communicate() method does not seem to interfere with the pipe behavior. This issue is quite repeatable on demand. I set up a second PC and reproduced the issue on demand on the second PC. I am using WinPython 3.6.1 on Windows 7 with gnuplot 5.2.0 on both PC. Here are the links to the various components needed to setup the environment to reproduce this issue: 1) gnuplot 5.2.0 https://sourceforge.net/projects/gnuplot/files/gnuplot/5.2.0/ 2) pygnuplot 0.10.0 https://pypi.python.org/pypi/PyGnuplot/0.10.0 3) There is a one-line fix to PyGnuplot.py needed https://github.com/benschneider/PyGnuplot/blob/master/PyGnuplot.py 4) example.py script https://github.com/benschneider/PyGnuplot/blob/master/example.py WinPython 3.6.1 installer comes with subprocess.py as part of the package, no version info that I can find. When installing gnuplot there is an advanced option to check on that will automatically update your PATH variable. If you dont do this, then you must manully add "C:\gnuplot\bin" (or whatever the directory is) to your PATH variable. To check if gnuplot is correctly installed, from a DOS prompt terminal window, type "gnuplot -V". You should get a one line response "gnuplot 5.2 patchlevel 0". ---------- components: Interpreter Core, Windows messages: 305073 nosy: jbrearley, paul.moore, steve.dower, tim.golden, zach.ware priority: normal severity: normal status: open title: subprocess process interaction with IDLEX GUI causes pygnuplot silent failures type: behavior versions: Python 3.6 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu Oct 26 20:20:33 2017 From: report at bugs.python.org (Nick) Date: Fri, 27 Oct 2017 00:20:33 +0000 Subject: [New-bugs-announce] [issue31881] subprocess.returncode not set depending on arguments to subprocess.run Message-ID: <1509063633.12.0.213398074469.issue31881@psf.upfronthosting.co.za> New submission from Nick : If subprocess.run is called with a single string, say: completed_ps = subprocess.run('mpirun -np 4 myexe.x moreargs', shell=True) and 'myexe.x moreargs' fails with a returncode of 1, then 'completed_ps.returncode' is None. However, if we split the args with shlex, we obtain the desired result, which is a returncode of 1: import shlex args = shlex.split('mpirun -np 4 myexe.x moreargs') completed_ps = subprocess.run(args) # now completed_ps.returncode = 1 if myexe.x moreargs fails. Reproduced on Mac, Ubuntu 17.04, Python 3.6.1 and Python 3.6.3. ---------- messages: 305094 nosy: nthompson priority: normal severity: normal status: open title: subprocess.returncode not set depending on arguments to subprocess.run versions: Python 3.6 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Oct 27 09:08:09 2017 From: report at bugs.python.org (Erik Bray) Date: Fri, 27 Oct 2017 13:08:09 +0000 Subject: [New-bugs-announce] [issue31882] Cygwin: asyncio and asyncore test suites hang indefinitely due to bug in Cygwin Message-ID: <1509109689.46.0.213398074469.issue31882@psf.upfronthosting.co.za> New submission from Erik Bray : Some of the tests for asyncio and asyncore block forever on Cygwin, due to a known (and seemingly difficult to fix) bug [1] in Cygwin involving SO_PEERCRED on UNIX sockets. SO_PEERCRED is a socket option that can be used to exchange file ownership info of the socket at the time the connection was established (specifically on UNIX sockets). This feature is technically supported on Cygwin, but the effect of the bug is that if two sockets are opened on the same process (even without using socketpair()), the credential exchange protocol can cause connect() on the "client" socket to block unless the "server" socket is already listen()-ing. This situation is not all that common in practice (it is not a problem if the "client" and "server" are separate processes). But it does show up in the test suite in a number of places, since both sockets belong to the same process. I have a patch to work around this and will post a PR shortly. [1] https://cygwin.com/ml/cygwin/2017-01/msg00054.html ---------- messages: 305118 nosy: erik.bray priority: normal severity: normal status: open title: Cygwin: asyncio and asyncore test suites hang indefinitely due to bug in Cygwin type: crash _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Oct 27 09:42:11 2017 From: report at bugs.python.org (Erik Bray) Date: Fri, 27 Oct 2017 13:42:11 +0000 Subject: [New-bugs-announce] [issue31883] Cygwin: heap corruption bug in wcsxfrm Message-ID: <1509111731.98.0.213398074469.issue31883@psf.upfronthosting.co.za> New submission from Erik Bray : There is an acknowledged bug in Cygwin's implementation of wcsxfrm() [1] that can cause heap corruption in certain cases. This bug has since been fixed in Cygwin 2.8.1-1 [2] and all current and future releases. However, that was relatively recent (July 2017) so it may still crop up. I also have a workaround for this from the Python side, but rather than clutter the code with workarounds for platform-specific bugs I think it suffices just to skip the test in this case. [1] https://cygwin.com/ml/cygwin/2017-05/msg00149.html [2] https://cygwin.com/ml/cygwin-announce/2017-07/msg00002.html ---------- messages: 305120 nosy: erik.bray priority: normal severity: normal status: open title: Cygwin: heap corruption bug in wcsxfrm _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Oct 27 10:02:33 2017 From: report at bugs.python.org (Mr JG Kent) Date: Fri, 27 Oct 2017 14:02:33 +0000 Subject: [New-bugs-announce] [issue31884] subprocess set priority on windows Message-ID: <1509112953.47.0.213398074469.issue31884@psf.upfronthosting.co.za> Change by Mr JG Kent : ---------- components: Library (Lib) nosy: JamesGKent priority: normal severity: normal status: open title: subprocess set priority on windows type: enhancement _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Oct 27 10:56:03 2017 From: report at bugs.python.org (Erik Bray) Date: Fri, 27 Oct 2017 14:56:03 +0000 Subject: [New-bugs-announce] [issue31885] Cygwin: socket test suites hang indefinitely due to bug in Cygwin Message-ID: <1509116163.97.0.213398074469.issue31885@psf.upfronthosting.co.za> New submission from Erik Bray : Like issue31882, this is to mention an upstream bug in Cygwin that causes one of the tests in the test_socket test suite to hang indefinitely. That bug is fixed upstream [1], but for now it would still be better to skip the test on Cygwin. The bug is that in some cases a blocking send() (particularly for a large amount data) cannot be interrupted by a signal even if SA_RESTART is not set. Fixes to this issue, along with issue31882, issue31883, and issue31878 provide the bare minimum for Cygwin to at least compile (not necessarily all optional extension modules) and run the test suite from start to finish (though there may be other tests that cause the interpreter to lock up, but that are currently masked by other bugs). [1] https://cygwin.com/ml/cygwin-patches/2017-q2/msg00037.html ---------- messages: 305123 nosy: erik.bray priority: normal severity: normal status: open title: Cygwin: socket test suites hang indefinitely due to bug in Cygwin type: crash _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Oct 27 11:10:27 2017 From: report at bugs.python.org (olarn) Date: Fri, 27 Oct 2017 15:10:27 +0000 Subject: [New-bugs-announce] [issue31886] Multiprocessing.Pool hangs after re-spawning several worker process. Message-ID: <1509117027.21.0.213398074469.issue31886@psf.upfronthosting.co.za> New submission from olarn : Multiprocessing's pool apparently attempts to repopulate the pool in an event of sub-process worker crash. However the pool seems to hangs after about ~ 4*(number of worker) process re-spawns. I've tracked the issue down to queue.get() stalling at multiprocessing.pool, line 102 Is this a known issue? Are there any known workaround? To reproduce this issue: import multiprocessing import multiprocessing.util import logging multiprocessing.util._logger = multiprocessing.util.log_to_stderr(logging.DEBUG) import time import ctypes def crash_py_interpreter(): print("attempting to crash the interpreter in ", multiprocessing.current_process()) i = ctypes.c_char('a'.encode()) j = ctypes.pointer(i) c = 0 while True: j[c] = 'a'.encode() c += 1 j def test_fn(x): print("test_fn in ", multiprocessing.current_process().name, x) exit(0) time.sleep(0.1) if __name__ == '__main__': # pool = multiprocessing.Pool(processes=multiprocessing.cpu_count()) pool = multiprocessing.Pool(processes=1) args_queue = [n for n in range(20)] # subprocess quits pool.map(test_fn, args_queue) # subprocess crashes # pool.map(test_fn,queue) ---------- components: Library (Lib) messages: 305124 nosy: olarn priority: normal severity: normal status: open title: Multiprocessing.Pool hangs after re-spawning several worker process. type: behavior versions: Python 2.7, Python 3.6 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Oct 27 13:23:47 2017 From: report at bugs.python.org (Peter Wullinger) Date: Fri, 27 Oct 2017 17:23:47 +0000 Subject: [New-bugs-announce] [issue31887] docs for email.generator are missing a comment on special multipart/signed handling Message-ID: <1509125027.89.0.213398074469.issue31887@psf.upfronthosting.co.za> New submission from Peter Wullinger : The documentation for email.generator.{Bytes,}Generator do not mention that Generator._handle_multipart_signed() exists and disables header folding for subparts of multipart/signed. This may cause some frustration, since, as implemented, rendering a message part enclosed in multipart/signed causes headers to fold differently than rendering the respective message part individually: >>> text = MIMEText('', 'plain', 'utf-8') >>> text['X-X'] = 'a b c d e f g h i j k l m n o p q r s t u v w x y z a b c d e f g h i j k l m n o p' >>> print(text.as_string()) Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: base64 X-X: a b c d e f g h i j k l m n o p q r s t u v w x y z a b c d e f g h i j k l m n o p >>> signed = MIMEMultipart('signed', filename='smime.p7s') >>> signed.attach(text) >>> print(signed.as_string()) Content-Type: multipart/signed; filename="smime.p7s"; boundary="===============8046244967080646804==" MIME-Version: 1.0 --===============8046244967080646804== Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: base64 X-X: a b c d e f g h i j k l m n o p q r s t u v w x y z a b c d e f g h i j k l m n o p --===============8046244967080646804==-- This breaks SMIME signatures when using the partial render to generate the signature. A possible workaround is to set maxheaderlen=0 for the partial render, which is the same as what Generator._handle_multipart_signed() does, but to do that one needs to know about the special processing. Attached is a suggestion at a documentation fix that remedies this problem at least for the online docs. ---------- components: email files: email-generator-multipart-signed-docs.patch keywords: patch messages: 305127 nosy: barry, dhke, r.david.murray priority: normal severity: normal status: open title: docs for email.generator are missing a comment on special multipart/signed handling type: enhancement versions: Python 2.7, Python 3.4, Python 3.5, Python 3.6, Python 3.7, Python 3.8 Added file: https://bugs.python.org/file47243/email-generator-multipart-signed-docs.patch _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sat Oct 28 01:40:10 2017 From: report at bugs.python.org (Tilman Krummeck) Date: Sat, 28 Oct 2017 05:40:10 +0000 Subject: [New-bugs-announce] [issue31888] Creating a UUID with a list throws bad exception Message-ID: <1509169210.77.0.213398074469.issue31888@psf.upfronthosting.co.za> New submission from Tilman Krummeck : I found a problem by accident on Python 3.5.3 with the uuid library. Running this: from uuid import UUID UUID(["string"]) This throws an AttributeError: Traceback (most recent call last): File "", line 1, in File "C:\Users\Tilman Krummeck\AppData\Local\Programs\Python\Python35-32\lib\uuid.py", line 137, in __init__ hex = hex.replace('urn:', '').replace('uuid:', '') AttributeError: 'list' object has no attribute 'replace' This is for sure not intended to work, but should throw a type error in my opinion even before trying to create that UUID object. ---------- components: Library (Lib) messages: 305148 nosy: TilmanKrummeck priority: normal severity: normal status: open title: Creating a UUID with a list throws bad exception type: behavior versions: Python 3.5 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sat Oct 28 06:03:33 2017 From: report at bugs.python.org (Simon Descarpentries) Date: Sat, 28 Oct 2017 10:03:33 +0000 Subject: [New-bugs-announce] [issue31889] difflib SequenceMatcher ratio() still have unpredictable behavior Message-ID: <1509185013.27.0.213398074469.issue31889@psf.upfronthosting.co.za> New submission from Simon Descarpentries : I, it's my 1st post here. I'm a French computer-science engineer with 10 years XP and manager at Acoeuro.com SSLL compagny. I suggested a better regexp integration on python-ideas a few months ago failing to convince getting things done. Despite issue https://bugs.python.org/issue25391 closed in 2010, nothing seems visible on https://docs.python.org/3/library/difflib.html to help users anticipating that a string greatly matching at 199 characters length, won't at all at the 200th one. It's an inconsistent behavior that looks like a bug. #!/usr/bin/env python3 from difflib import SequenceMatcher a = 'ab'*400 b = 'ba'*400 for i in range(1, 400): diff_ratio = SequenceMatcher(None, a=a[:i], b=b[:i]).ratio() print('%3.i %.2f' % (i, diff_ratio), end=' ') not i % 10 and print('') EOF At 199c I have a 0.99 ratio, and 0.00 at 200c. The results are nearly the same with strings made of emails like in https://github.com/Siltaar/drop_alternatives especially comparing strings like : "performantsetdvelopperducontenusimilairepourvosprochainsTweets.Suiveznous at TwitterBusinesspourdautresinfosetactus.TesterlesPublicitsTwitterbusiness.twitter.com|@TwitterBusiness|Aide|SedsinscrireLemailsadresse at gggTwitter,Inc.MarketStreet,SuiteSanFrancisco,CA" "rducontenusimilairepourvosprochainsTweets.Suiveznous at TwitterBusinesspourprofiterdautresinfosetactus.TesterlesPublicit?sTwitterbusiness.twitter.com at TwitterBusinessAideSed?sinscrireTwitterInternationalCompanyOneCumberlandPlace,FenianStreetDublin,DAXIRELAND" Fortunately, I didn't experienced the problem using quick_ratio(). The documentation is not clear about ratio / quick_ratio / real_quick_ratio ; but looks like unstable. With in addition an inconsistent behavior it looks like worthing some fix. ---------- components: Library (Lib) messages: 305153 nosy: Siltaar priority: normal severity: normal status: open title: difflib SequenceMatcher ratio() still have unpredictable behavior type: behavior versions: Python 2.7, Python 3.5, Python 3.6 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sat Oct 28 13:06:54 2017 From: report at bugs.python.org (Anselm Kruis) Date: Sat, 28 Oct 2017 17:06:54 +0000 Subject: [New-bugs-announce] [issue31890] Please define the flag METH_STACKLESS for Stackless Python Message-ID: <1509210414.45.0.213398074469.issue31890@psf.upfronthosting.co.za> New submission from Anselm Kruis : The header Include/methodobject.h defines ml_flags METH_xxx. Stackless Python adds the flag METH_STACKLESS. Traditionally Stackless used bit 0x0080 for METH_STACKLESS, but starting with C-Python 3.6 bit 0x0080 is used for METH_FASTCALL. In order to prevent future conflicts, I propose to add METH_STACKLESS to methodobject.h. #ifdef STACKLESS #define METH_STACKLESS 0x0100 #else #define METH_STACKLESS 0x0000 #endif Include/object.h already contains a similar definition. ---------- components: Interpreter Core messages: 305164 nosy: anselm.kruis priority: normal severity: normal status: open title: Please define the flag METH_STACKLESS for Stackless Python type: enhancement versions: Python 3.7, Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun Oct 29 05:21:28 2017 From: report at bugs.python.org (Serhiy Storchaka) Date: Sun, 29 Oct 2017 09:21:28 +0000 Subject: [New-bugs-announce] [issue31891] Make curses compiling on NetBSD 7.1 and tests passing Message-ID: <1509268888.25.0.213398074469.issue31891@psf.upfronthosting.co.za> New submission from Serhiy Storchaka : NetBSD perhaps is the last open OS that don't use ncurses. For now the _curses module is not compiled on NetBSD. Its curses implementation doesn't contain several ncurses functions. included for non-ncurses implementations defines a couple of macros that conflict with local variable names. The hardcoded signature of setupterm differs from the signature in . The fallback implementation of has_key() depends on KEY_* constants that are absent in NetBSD curses too. It seems to me that curses is broken on NetBSD for very long time. The proposed PR fixes building of the _curses module and makes curses tests passing. This of course doesn't guaranties that curses works correctly on NetBSD, our curses tests are rudimentary. There were other NetBSD specific guards in the code. They are no longer needed because the guarded functions already are supported in NetBSD for long time. There are issue9667 and issue21457 for removing unneeded conditional compilation. ---------- assignee: serhiy.storchaka components: Extension Modules messages: 305180 nosy: serhiy.storchaka priority: normal severity: normal status: open title: Make curses compiling on NetBSD 7.1 and tests passing type: compile error versions: Python 2.7, Python 3.6, Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun Oct 29 05:59:06 2017 From: report at bugs.python.org (Hanno Boeck) Date: Sun, 29 Oct 2017 09:59:06 +0000 Subject: [New-bugs-announce] [issue31892] ssl.get_server_certificate should allow specifying certificate / key type Message-ID: <1509271146.03.0.213398074469.issue31892@psf.upfronthosting.co.za> New submission from Hanno Boeck : The function ssl.get_server_certificate() from the ssl module is supposed to allow fetching the certificate of a TLS server. However in its current form it provides no way to specify a key type. Many popular hosts (e.g. facebook, google) support both ECDSA and RSA these days, depending on the cipher suites one uses to try to connect to them. If one wants to fetch the RSA certificate of e.g. facbeook this is not possible with the current python ssl module, as it will always output the ECDSA certificate. One can create a connection with an SSLContext that has only RSA ciphers set, but it's not possible to get the certificate out of an SSLContext. And the get_server_certificate function provides neither a way to bind it to a context nor a way to specify ciphers or key types. I think there should be an optional parameter to get_server_certificate that allows asking for a specific key type. ---------- assignee: christian.heimes components: SSL messages: 305182 nosy: christian.heimes, hanno priority: normal severity: normal status: open title: ssl.get_server_certificate should allow specifying certificate / key type type: enhancement _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun Oct 29 07:44:41 2017 From: report at bugs.python.org (Serhiy Storchaka) Date: Sun, 29 Oct 2017 11:44:41 +0000 Subject: [New-bugs-announce] [issue31893] Issues with kqueue Message-ID: <1509277481.96.0.213398074469.issue31893@psf.upfronthosting.co.za> New submission from Serhiy Storchaka : In Modules/selectmodule.c it is assumed that the kevent structure is defined on FreeBSD and NetBSD as: struct kevent { uintptr_t ident; short filter; u_short flags; u_int fflags; intptr_t data; uintptr_t udata; }; and on OpenBSD as: struct kevent { u_int ident; short filter; u_short flags; u_int fflags; intptr_t data; int udata; }; Actually it is defined on FreeBSD as: struct kevent { uintptr_t ident; short filter; u_short flags; u_int fflags; intptr_t data; void *udata; }; On OpenBSD as: struct kevent { uintptr_t ident; short filter; u_short flags; u_int fflags; int64_t data; void *udata; }; And on NetBSD as: struct kevent { uintptr_t ident; uint32_t filter; uint32_t flags; uint32_t fflags; int64_t data; intptr_t udata; }; Other issues are related to rich comparison. Due to integer overflows the ordering is not transitive. The rich comparison protocol is not properly supported, comparing a kevent_event object with a non-kevent_event object don't falls back to the rich comparison methods of the right arguments. ---------- assignee: serhiy.storchaka components: Extension Modules, FreeBSD messages: 305187 nosy: koobs, serhiy.storchaka priority: normal severity: normal status: open title: Issues with kqueue type: behavior versions: Python 2.7, Python 3.6, Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun Oct 29 17:10:45 2017 From: report at bugs.python.org (Serhiy Storchaka) Date: Sun, 29 Oct 2017 21:10:45 +0000 Subject: [New-bugs-announce] [issue31894] test_timestamp_naive failed on NetBSD Message-ID: <1509311445.47.0.213398074469.issue31894@psf.upfronthosting.co.za> New submission from Serhiy Storchaka : $ ./python -m test -vuall test_datetime ... ====================================================================== FAIL: test_timestamp_naive (test.datetimetester.TestDateTime_Pure) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/serhiy/py/cpython3.7/Lib/test/support/__init__.py", line 1622, in inner return func(*args, **kwds) File "/home/serhiy/py/cpython3.7/Lib/test/datetimetester.py", line 1977, in test_timestamp_naive self.assertEqual(t.timestamp(), 18000.0) AssertionError: 14400.0 != 18000.0 ... Simple reproducer: $ TZ=EST+05EDT,M3.2.0,M11.1.0 ./python -c 'import datetime; print(datetime.datetime(1970, 1, 1).timestamp())' 14400.0 It should print 18000.0 (5 hours) instead of 14400.0 (4 hours). ---------- components: Library (Lib), Tests messages: 305192 nosy: belopolsky, serhiy.storchaka priority: normal severity: normal status: open title: test_timestamp_naive failed on NetBSD type: behavior versions: Python 3.6, Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun Oct 29 20:57:41 2017 From: report at bugs.python.org (Haneef) Date: Mon, 30 Oct 2017 00:57:41 +0000 Subject: [New-bugs-announce] [issue31895] Native hijri calendar support Message-ID: <1509325061.68.0.213398074469.issue31895@psf.upfronthosting.co.za> New submission from Haneef : There should be native support for converting between Hijri (Umm al-Qura), Gregorian, Julian and UnixTime calendars. Many big players have included native support for Hijri calendar in their SDKs and software. Below are some: (Java Hijri calendar support)[https://docs.oracle.com/javase/8/docs/api/java/time/chrono/HijrahChronology.html] (Apple supports the Hijri calendar in 5 of their SDKs (Software Development Kits))[https://developer.apple.com/documentation/foundation/calendar.identifier] (Microsoft Windows supports Hijri calendar naively)[https://www.microsoft.com/middleeast/msdn/ArabicCalendar.aspx] (Microsoft Office has native support)[https://blogs.technet.microsoft.com/office_global_experience/2010/01/13/um-al-qura-calendar-support-in-office-2010/] (Android Hijri calendar support)[https://developer.android.com/reference/android/icu/util/IslamicCalendar.html] (Google Calendar allows Hijri calendar as an alternate calendar)[https://www.maketecheasier.com/display-alternate-calendar-google-calendar/] ---------- messages: 305199 nosy: haneef95 priority: normal severity: normal status: open title: Native hijri calendar support type: enhancement versions: Python 2.7, Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun Oct 29 23:53:39 2017 From: report at bugs.python.org (Yang Big) Date: Mon, 30 Oct 2017 03:53:39 +0000 Subject: [New-bugs-announce] [issue31896] In function define class inherit ctypes.structure, and using ctypes.pointer leak memory Message-ID: <1509335619.79.0.213398074469.issue31896@psf.upfronthosting.co.za> New submission from Yang Big : Here my code: 2 import sys 3 import ctypes 4 5 6 def hi(): 7 class timespec(ctypes.Structure): 8 ? """Time specification, as described in clock_gettime(3).""" 9 ? _fields_ = (('tv_sec', ctypes.c_long), 10 ? ? ? ? ('tv_nsec', ctypes.c_long)) 11 12 13 ts = timespec() 14 po = ctypes.pointer(ts) 15 16 17 def main(): 18 while True: 19 ? test = hi() 20 21 if __name__ == "__main__": 22 sys 2 import sys 3 import ctypes 4 5 6 def hi(): 7 class timespec(ctypes.Structure): 8 ? """Time specification, as described in clock_gettime(3).""" 9 ? _fields_ = (('tv_sec', ctypes.c_long), 10 ? ? ? ? ('tv_nsec', ctypes.c_long)) 11 12 13 ts = timespec() 14 po = ctypes.pointer(ts) 15 16 17 def main(): 18 while True: 19 ? test = hi() 20 21 if __name__ == "__main__": 22 sys.exit(main()) run this code, i find that the program occupy more and more VmRss my python is 2.7.11, in python3 i also find this problem. Is this normal?I am a newbie with python, not very familiar with some python's internal principle. Tanks. ---------- components: ctypes messages: 305204 nosy: Yang Big, amaury.forgeotdarc, belopolsky, meador.inge priority: normal severity: normal status: open title: In function define class inherit ctypes.structure, and using ctypes.pointer leak memory type: resource usage versions: Python 2.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Oct 30 00:04:14 2017 From: report at bugs.python.org (Ned Williamson) Date: Mon, 30 Oct 2017 04:04:14 +0000 Subject: [New-bugs-announce] [issue31897] RecursionError in plistlib.loads Message-ID: <1509336254.22.0.213398074469.issue31897@psf.upfronthosting.co.za> New submission from Ned Williamson : Hi, The following program crashes for me using the current Python3.7 master: ``` import plistlib plistlib.loads(b'\xdd\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00' b'\xda\x0cw\xb7\x00\x00\x00\x00\x00\x00\x00\xc7\x00' b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\xd6\xd5\x00' b'\x00\x00\x00\x00\x00\x00\x04\x00\x00\x00\x00\x00' b'\x00\x00\x00\x00\x10\x00\x00\x00\x00\x00\x00', fmt=plistlib.FMT_BINARY) ``` The last few lines look like ``` File "/usr/lib/python3.5/plistlib.py", line 728, in _read_object ] = self._read_object(self._object_offsets[o]) File "/usr/lib/python3.5/plistlib.py", line 728, in _read_object ] = self._read_object(self._object_offsets[o]) File "/usr/lib/python3.5/plistlib.py", line 723, in _read_object key_refs = self._read_refs(s) File "/usr/lib/python3.5/plistlib.py", line 647, in _read_refs return self._read_ints(n, self._ref_size) File "/usr/lib/python3.5/plistlib.py", line 644, in _read_ints for i in range(0, size * n, size)) RecursionError: maximum recursion depth exceeded in comparison ``` This bug was found using an alpha version of python-fuzz. ---------- messages: 305205 nosy: Ned Williamson priority: normal severity: normal status: open title: RecursionError in plistlib.loads type: crash versions: Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Oct 30 01:17:27 2017 From: report at bugs.python.org (Nick Coghlan) Date: Mon, 30 Oct 2017 05:17:27 +0000 Subject: [New-bugs-announce] [issue31898] Add `recommended-packages.txt` to `venv` documentation Message-ID: <1509340647.27.0.213398074469.issue31898@psf.upfronthosting.co.za> New submission from Nick Coghlan : As per the python-ideas thread starting at https://mail.python.org/pipermail/python-ideas/2017-October/047508.html, this is proposal to add a `pip -r` compatible `recommended-packages.txt` file to the `venv` documentation that summarises replacement module recommendations in other parts of the standard library documentation. Suggested entries are: requests # Recommendation in urllib.request docs regex # Recommendation in re docs cffi # More typesafe alternative to ctypes six # Python 2/3 compatibility library setuptools # Alternative to distutils that supports modern packaging standards I'm also tempted to suggest pytz as the preferred source of up-to-date named timezones, but I'm not sure how often that's really needed in situations where you can't just depend on it explicitly instead. ---------- assignee: ncoghlan components: Documentation messages: 305210 nosy: ncoghlan priority: normal severity: normal status: open title: Add `recommended-packages.txt` to `venv` documentation versions: Python 3.7, Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Oct 30 01:23:17 2017 From: report at bugs.python.org (Nick Coghlan) Date: Mon, 30 Oct 2017 05:23:17 +0000 Subject: [New-bugs-announce] [issue31899] Ensure backwards compatibility with recommended packages Message-ID: <1509340997.49.0.213398074469.issue31899@psf.upfronthosting.co.za> New submission from Nick Coghlan : This is a potential follow-up to issue 31898, whereby the recommended packages list could be taken into account as part of CPython's own testing regime. The gist of the idea would be to: 1. Update the recommended-packages list to nominate particular *versions* of those dependencies (the latest available version as of each CPython maintenance release) 2. Add a new "-uthird-party" resource definition to the regression test suite 3. Add a new regression test that installed the recommended packages into a virtual environment and ran their tests when that resource was enabled It's a separate issue, since this would be more work to maintain than the simple "we recommend these packages" list proposed in issue 31898. ---------- assignee: ncoghlan components: Tests messages: 305211 nosy: ncoghlan priority: low severity: normal stage: test needed status: open title: Ensure backwards compatibility with recommended packages type: enhancement versions: Python 3.7, Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Oct 30 09:41:05 2017 From: report at bugs.python.org (Charalampos Stratakis) Date: Mon, 30 Oct 2017 13:41:05 +0000 Subject: [New-bugs-announce] [issue31900] UnicodeDecodeError in localeconv() makes test_float fail with glibc 2.26.90 Message-ID: <1509370865.74.0.213398074469.issue31900@psf.upfronthosting.co.za> New submission from Charalampos Stratakis : Original bug report: https://bugzilla.redhat.com/show_bug.cgi?id=1484497 It seems that on the development branch of Fedora, when we updated glibc from 2.26 to 2.26.90, test_float_with_comma started failing. Details from the original bug report: Under certain circumstances, when LC_NUMERIC is fr_FR.ISO8859-1 but LC_ALL is C.UTF-8, locale.localeconv() fails with UnicodeDecodeError: 'locale' codec can't decode byte 0xa0 in position 0: Invalid or incomplete multibyte or wide character Apparently, the thousands separator (or something else) in the lconv is "\xa0" (unbreakable space in fr_FR.ISO8859-1), and it's being decoded with UTF-8. This is tripped by Python's test suite, namely test_float.GeneralFloatCases.test_float_with_comma ---------- components: Tests messages: 305227 nosy: cstratak priority: normal severity: normal status: open title: UnicodeDecodeError in localeconv() makes test_float fail with glibc 2.26.90 versions: Python 3.5, Python 3.6, Python 3.7, Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Oct 30 11:44:43 2017 From: report at bugs.python.org (Marcel Plch) Date: Mon, 30 Oct 2017 15:44:43 +0000 Subject: [New-bugs-announce] [issue31901] atexit callbacks only called for current subinterpreter Message-ID: <1509378283.18.0.213398074469.issue31901@psf.upfronthosting.co.za> New submission from Marcel Plch : `Py_FinalizeEx()` only executes callbacks from the subinterpreter from which Py_FinalizeEx is called. What is the expected behavior here? Should the callbacks be specific to individual subinterpreters? ---------- components: Extension Modules messages: 305233 nosy: Dormouse759, encukou, ncoghlan priority: normal severity: normal status: open title: atexit callbacks only called for current subinterpreter type: behavior versions: Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Oct 30 12:14:48 2017 From: report at bugs.python.org (Guo Ci Teo) Date: Mon, 30 Oct 2017 16:14:48 +0000 Subject: [New-bugs-announce] [issue31902] Fix col_offset for ast nodes: AsyncFor, AsyncFunctionDef, AsyncWith Message-ID: <1509380088.03.0.213398074469.issue31902@psf.upfronthosting.co.za> Change by Guo Ci Teo : ---------- components: Library (Lib) nosy: guoci priority: normal severity: normal status: open title: Fix col_offset for ast nodes: AsyncFor, AsyncFunctionDef, AsyncWith versions: Python 3.5, Python 3.6, Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Oct 30 15:38:53 2017 From: report at bugs.python.org (Maxime Belanger) Date: Mon, 30 Oct 2017 19:38:53 +0000 Subject: [New-bugs-announce] [issue31903] `_scproxy` calls SystemConfiguration functions in a way that can cause deadlocks Message-ID: <1509392333.58.0.213398074469.issue31903@psf.upfronthosting.co.za> New submission from Maxime Belanger : Through the use of various Python packages (such as `pyobjc`), it is possible for a deadlock to occur due to how `_scproxy.c` calls `SCDynamicStoreCopyProxies`. In more recent versions of macOS (10.7+), this function can block on acquiring a lock deep inside `NSUserPreferences`. As `pyobjc` allows Python-wrapped `NSString`s to be stored in `CFPreferences`, it is thus possible for one thread to hold the `CFPreferences` lock and block on the GIL while another thread holds the GIL and blocks on the `CFPreferences` lock. We've observed this on a significant number of macOS devices before fixing ourselves by wrapping the calls to `SCDynamicStoreCopyProxies` with `Py_BEGIN/END_ALLOW_THREADS`. ---------- components: macOS messages: 305247 nosy: Maxime Belanger, ned.deily, ronaldoussoren priority: normal severity: normal status: open title: `_scproxy` calls SystemConfiguration functions in a way that can cause deadlocks type: crash versions: Python 2.7, Python 3.5, Python 3.6, Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Oct 30 15:56:33 2017 From: report at bugs.python.org (Brian Kuhl) Date: Mon, 30 Oct 2017 19:56:33 +0000 Subject: [New-bugs-announce] [issue31904] Python should support VxWorks RTOS Message-ID: <1509393393.78.0.213398074469.issue31904@psf.upfronthosting.co.za> New submission from Brian Kuhl : With the trend to use REST APIs between the cloud and the IoT there is increasing interest in Python on embedded devices. Cloud developer?s typical release a reference REST implementation of JSON and/or Python on Linux and leave it to the device developer to adapt it to their platform. While many devices use eLinux, others with IP and/or hard real-time constraints need a commercial RTOS platform. Currently the automake configure explicitly prevents configuration of VxWorks as a build target. I'll provide a pull request referencing this issue with the required changes. ---------- components: Cross-Build messages: 305249 nosy: Alex.Willmer, Brian Kuhl priority: normal severity: normal status: open title: Python should support VxWorks RTOS type: enhancement versions: Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Oct 30 16:56:08 2017 From: report at bugs.python.org (Gerard Weatherby) Date: Mon, 30 Oct 2017 20:56:08 +0000 Subject: [New-bugs-announce] [issue31905] IPv4Networkcontains raises exception on None Message-ID: <1509396968.75.0.213398074469.issue31905@psf.upfronthosting.co.za> New submission from Gerard Weatherby : Given a IPvNetwork network if x in network: ... raises an AttributeError instead of returning False. ---------- components: Library (Lib) files: ne.py messages: 305250 nosy: Gerard Weatherby priority: normal severity: normal status: open title: IPv4Networkcontains raises exception on None type: behavior versions: Python 3.6 Added file: https://bugs.python.org/file47247/ne.py _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Oct 30 17:30:45 2017 From: report at bugs.python.org (Sam Lobel) Date: Mon, 30 Oct 2017 21:30:45 +0000 Subject: [New-bugs-announce] [issue31906] String literals next to each other does not cause error Message-ID: <1509399045.99.0.213398074469.issue31906@psf.upfronthosting.co.za> New submission from Sam Lobel : This seems too obvious to have been missed, but also too strange behaviour to be on purpose. The following works for some reason (note there's no + between the words) >>> variable = "first" "second" >>> print(variable) "firstsecond" In a file, if you're missing a comma between two string literals, it combines them into one string (instead of throwing a syntax error). E.G: >>> a = ["first", ... "second" ... "third"] >>> print(a) ["first" "secondthird"] BUT, the same thing with variables (thankfully) does not work. >>> a = "first" >>> b = "second" >>> c = a b Throws a syntax error. The same sort of thing also breaks for integers. >>> a = 4 7 throws a syntax error. This just seems wrong to me. Is it? Has this been discussed a million times before? ---------- messages: 305252 nosy: Sam Lobel2 priority: normal severity: normal status: open title: String literals next to each other does not cause error versions: Python 2.7, Python 3.4, Python 3.5, Python 3.6 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Oct 30 20:35:20 2017 From: report at bugs.python.org (mickey695) Date: Tue, 31 Oct 2017 00:35:20 +0000 Subject: [New-bugs-announce] [issue31907] Clarify error message when attempting to call function via str.format() Message-ID: <1509410120.18.0.213398074469.issue31907@psf.upfronthosting.co.za> New submission from mickey695 : PEP 3101 states that format strings may only use the "."(getattr) or the "[]" (getitem) operators to address either attributes or items of parameters. Should a programmer attempt to, for example, call a function of a parameter as follows: >>> d = datetime.datetime(2017, 10, 31) >>> "{0.ctime()}".format(d) they will receive an error message such as: AttributeError: 'datetime.datetime' object has no attribute 'ctime()' Proposal: Raise an error stating that cannot embed arbitrary expressions in str.format() format strings ---------- components: Interpreter Core messages: 305263 nosy: mickey695 priority: normal severity: normal status: open title: Clarify error message when attempting to call function via str.format() type: enhancement versions: Python 2.7, Python 3.4, Python 3.5, Python 3.6, Python 3.7, Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Oct 30 22:55:06 2017 From: report at bugs.python.org (Michael Selik) Date: Tue, 31 Oct 2017 02:55:06 +0000 Subject: [New-bugs-announce] [issue31908] trace module cli does not write cover files Message-ID: <1509418506.94.0.213398074469.issue31908@psf.upfronthosting.co.za> New submission from Michael Selik : The trace module command-line utility doesn't write cover files. I've noticed this issue for some years now. It works fine in Python 2. When using Python 3, no ".cover" files are written, regardless of how "--coverdir" is specified. mike on macbook in ~/ $ echo 'print("hello")' > foo.py mike on macbook in ~/ $ python -m trace --count foo.py hello mike on macbook in ~/ $ ls *.cover ls: *.cover: No such file or directory My apologies if this is a duplicate bug. I searched the tracker and Google for a while, but couldn't find a relevant issue. ---------- messages: 305268 nosy: Michael Selik priority: normal severity: normal status: open title: trace module cli does not write cover files type: behavior versions: Python 3.6 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Oct 31 02:29:04 2017 From: report at bugs.python.org (Ilya Kulakov) Date: Tue, 31 Oct 2017 06:29:04 +0000 Subject: [New-bugs-announce] [issue31909] Missing definition of HAVE_SYSCALL_GETRANDOM Message-ID: <1509431344.9.0.213398074469.issue31909@psf.upfronthosting.co.za> New submission from Ilya Kulakov : Python 3.5 and 3.6 in their corresponding configure.ac try to detect presence of sys_getrandom. The result is written into the `HAVE_GETRANDOM_SYSCALL` definition. libexpact checks for `HAVE_SYSCALL_GETRANDOM` and since it's not defined, does not use it. This fails compilation with up to date libexpact due to a new condition [1]. [1] https://github.com/python/cpython/blob/master/Modules/expat/xmlparse.c#L87-L91 ---------- components: Build messages: 305270 nosy: Ilya.Kulakov priority: normal severity: normal status: open title: Missing definition of HAVE_SYSCALL_GETRANDOM versions: Python 3.4, Python 3.5, Python 3.6, Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Oct 31 13:05:22 2017 From: report at bugs.python.org (STINNER Victor) Date: Tue, 31 Oct 2017 17:05:22 +0000 Subject: [New-bugs-announce] [issue31910] test_socket.test_create_connection() failed with EADDRNOTAVAIL (err 99) Message-ID: <1509469522.6.0.213398074469.issue31910@psf.upfronthosting.co.za> New submission from STINNER Victor : On my PR 4200 which is unrelated to networking, the following test failed once. FAIL: test_create_connection (test.test_socket.NetworkConnectionNoServer) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/travis/build/python/cpython/Lib/test/test_socket.py", line 4537, in test_create_connection self.assertIn(cm.exception.errno, expected_errnos) AssertionError: 99 not found in [111, 101] With error codes: * 99: EADDRNOTAVAIL: Cannot assign requested address * 111: ECONNREFUSED: Connection refused * 101: ENETUNREACH: Network is unreachable Maybe we shold include EADDRNOTAVAIL in the expected error codes? ---------- components: Tests messages: 305316 nosy: haypo, martin.panter, serhiy.storchaka priority: normal severity: normal status: open title: test_socket.test_create_connection() failed with EADDRNOTAVAIL (err 99) versions: Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Oct 31 13:57:44 2017 From: report at bugs.python.org (STINNER Victor) Date: Tue, 31 Oct 2017 17:57:44 +0000 Subject: [New-bugs-announce] [issue31911] Use malloc_usable_size() is pymalloc for realloc Message-ID: <1509472664.46.0.213398074469.issue31911@psf.upfronthosting.co.za> New submission from STINNER Victor : Objects/obmalloc.c contains an interesting comment: if (!address_in_range(p, pool)) { /* pymalloc is not managing this block. If nbytes <= SMALL_REQUEST_THRESHOLD, it's tempting to try to take over this block. However, if we do, we need to copy the valid data from the C-managed block to one of our blocks, and there's no portable way to know how much of the memory space starting at p is valid. As bug 1185883 pointed out the hard way, it's possible that the C-managed block is "at the end" of allocated VM space, so that a memory fault can occur if we try to copy nbytes bytes starting at p. Instead we punt: let C continue to manage this block. */ return 0; } See also bpo-1185883. We don't have to guess, it's possible to get the size of a memory block allocated by malloc() with: * malloc_usable_size(ptr): available at least on Linux * _msize(ptr): Windows Maybe we could add a "msize" field to PyMemAllocatorEx? See also bpo-18835 whic adds PyMem_AlignedAlloc() and so already modify (and rename PyMemAllocatorEx). See also bpo-31626, but I'm not sure that it would help for _PyMem_DebugRawRealloc(). ---------- components: Interpreter Core messages: 305319 nosy: haypo, serhiy.storchaka, skrah priority: normal severity: normal status: open title: Use malloc_usable_size() is pymalloc for realloc type: performance versions: Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Oct 31 14:01:35 2017 From: report at bugs.python.org (Stefan Krah) Date: Tue, 31 Oct 2017 18:01:35 +0000 Subject: [New-bugs-announce] [issue31912] PyMem_Malloc() should guarantee alignof(max_align_t) Message-ID: <1509472895.95.0.213398074469.issue31912@psf.upfronthosting.co.za> New submission from Stefan Krah : This is related to #27987 and #20064 and perhaps the pymalloc patch from #18835. I think PyMem_Malloc() should guarantee alignof(max_align_t). It actually did before the "#define PYMEM_FUNCS PYOBJ_FUNCS" optimization, so we have a sort of regression here. ---------- components: Interpreter Core messages: 305320 nosy: skrah priority: normal severity: normal stage: needs patch status: open title: PyMem_Malloc() should guarantee alignof(max_align_t) type: behavior versions: Python 3.4, Python 3.5, Python 3.6, Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Oct 31 14:12:26 2017 From: report at bugs.python.org (Antoine Pitrou) Date: Tue, 31 Oct 2017 18:12:26 +0000 Subject: [New-bugs-announce] [issue31913] forkserver could warn if several threads are running Message-ID: <1509473546.78.0.213398074469.issue31913@psf.upfronthosting.co.za> New submission from Antoine Pitrou : I'm not sure this is worth handling, but I had an interaction with a user who had weird deadlock problems in a glibc function (getaddrinfo) in worker processes launched with the forkserver method. The explanation turned out to be that a sitecustomize.py did some stuff that eventually launched a helper thread, and that made the forkserver process's forking fundamentally unsafe (fork() is guaranteed to be safe if there's only one thread running in the parent process). It would be easy to check that threading.enumerate() returns only one thread, and otherwise warn the user about it. Note this only handles Python threads and not C threads invisible to Python... (a more complete solution would involve psutil :-)). ---------- components: Library (Lib) messages: 305323 nosy: davin, gregory.p.smith, pitrou priority: low severity: normal status: open title: forkserver could warn if several threads are running type: enhancement versions: Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Oct 31 16:45:13 2017 From: report at bugs.python.org (=?utf-8?b?0JTQuNC70Y/QvSDQn9Cw0LvQsNGD0LfQvtCy?=) Date: Tue, 31 Oct 2017 20:45:13 +0000 Subject: [New-bugs-announce] [issue31914] Document Pool.(star)map return type Message-ID: <1509482713.68.0.213398074469.issue31914@psf.upfronthosting.co.za> New submission from ????? ???????? : https://docs.python.org/3/library/multiprocessing.html#multiprocessing.pool.Pool.starmap says: starmap(func, iterable[, chunksize]) Like map() except that the elements of the iterable are expected to be iterables that are unpacked as arguments. Hence an iterable of [(1,2), (3, 4)] results in [func(1,2), func(3,4)]. If it was like map() then it would have returned an iterator. Please clarify, that Pool.map and Pool.starmap return list. ---------- assignee: docs at python components: Documentation messages: 305337 nosy: dilyan.palauzov, docs at python priority: normal severity: normal status: open title: Document Pool.(star)map return type type: enhancement versions: Python 3.6 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Oct 31 21:30:20 2017 From: report at bugs.python.org (Chance Parsons) Date: Wed, 01 Nov 2017 01:30:20 +0000 Subject: [New-bugs-announce] [issue31915] (list).insert() not working Message-ID: <1509499820.34.0.213398074469.issue31915@psf.upfronthosting.co.za> Change by Chance Parsons : ---------- files: Times Tables.py nosy: Chance Parsons priority: normal severity: normal status: open title: (list).insert() not working type: behavior versions: Python 3.6 Added file: https://bugs.python.org/file47250/Times Tables.py _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Oct 31 22:08:45 2017 From: report at bugs.python.org (multimiler) Date: Wed, 01 Nov 2017 02:08:45 +0000 Subject: [New-bugs-announce] [issue31916] ensurepip not honoring value of $(DESTDIR) - pip not installed Message-ID: <1509502125.3.0.213398074469.issue31916@psf.upfronthosting.co.za> New submission from multimiler : I am building python from source for installation at /opt/python-3.6.3. The result of the build will be installed in /somewhere/deb-pkg-build-dir. I configure, build, and install, the package as follows: ./configure --prefix=/opt/python-$VER --enable-optimizations --with-ensurepip=install make make install DESTDIR=/somewhere/deb-pkg-build-dir At the very end of the build log I find: Requirement already satisfied: setuptools in /opt/python-3.6.3/lib/python3.6/site-packages As a result PIP is not installed. This is an error. The pip installation process should be looking in $(DESTDIR)/opt/python-3.6.3. Instead it is looking at /opt/python-3.6.3/... which is the python installation on the current build host -- NOT INSTALLED BY THIS BUILD. In the top-level Makefile.pre I find the following that runs ensurepip to install pip: install: commoninstall bininstall maninstall if test "x$(ENSUREPIP)" != "xno" ; then \ case $(ENSUREPIP) in \ upgrade) ensurepip="--upgrade" ;; \ install|*) ensurepip="" ;; \ esac; \ $(RUNSHARED) $(PYTHON_FOR_BUILD) -m ensurepip \ $$ensurepip --root=$(DESTDIR)/ ; \ fi As you can see $(DESTDIR) is passed into ensurepip, but it is never used when checking for pip existence. ---------- components: Installation messages: 305351 nosy: multimiler priority: normal severity: normal status: open title: ensurepip not honoring value of $(DESTDIR) - pip not installed type: behavior versions: Python 3.6 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Oct 31 22:27:00 2017 From: report at bugs.python.org (STINNER Victor) Date: Wed, 01 Nov 2017 02:27:00 +0000 Subject: [New-bugs-announce] [issue31917] Add time.CLOCK_PROF constant Message-ID: <1509503220.89.0.213398074469.issue31917@psf.upfronthosting.co.za> New submission from STINNER Victor : The CLOCK_PROF clock is used internally by time.process_time(). It may be interesting to expose this constant if available. ---------- components: FreeBSD, Library (Lib) messages: 305352 nosy: haypo, koobs priority: normal severity: normal status: open title: Add time.CLOCK_PROF constant type: enhancement versions: Python 3.7 _______________________________________ Python tracker _______________________________________