From report at bugs.python.org  Tue Aug  1 00:45:33 2017
From: report at bugs.python.org (Terry J. Reedy)
Date: Tue, 01 Aug 2017 04:45:33 +0000
Subject: [New-bugs-announce] [issue31093] IDLE: Add tests for configdialog
 extensions tab
Message-ID: <1501562733.48.0.832995547822.issue31093@psf.upfronthosting.co.za>


Changes by Terry J. Reedy <tjreedy at udel.edu>:


----------
assignee: terry.reedy
components: IDLE
nosy: terry.reedy
priority: normal
severity: normal
stage: needs patch
status: open
title: IDLE: Add tests for configdialog extensions tab
type: enhancement
versions: Python 3.6, Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31093>
_______________________________________

From report at bugs.python.org  Tue Aug  1 01:07:44 2017
From: report at bugs.python.org (Dmitry Vinokurov)
Date: Tue, 01 Aug 2017 05:07:44 +0000
Subject: [New-bugs-announce] [issue31094] asyncio: get list of connected
 clients
Message-ID: <1501564064.1.0.785973244818.issue31094@psf.upfronthosting.co.za>


New submission from Dmitry Vinokurov:

Hi!

I'm using python-opcua and wonder if it is possible to get list of clients connected to running server? I've asked such question in python-opcua repo https://github.com/FreeOpcUa/python-opcua/issues/480 and developers redirected me here.

Thanks in advance

----------
components: asyncio
messages: 299598
nosy: Dmitry Vinokurov, yselivanov
priority: normal
severity: normal
status: open
title: asyncio: get list of connected clients
type: enhancement
versions: Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31094>
_______________________________________

From report at bugs.python.org  Tue Aug  1 01:49:49 2017
From: report at bugs.python.org (INADA Naoki)
Date: Tue, 01 Aug 2017 05:49:49 +0000
Subject: [New-bugs-announce] [issue31095] Checking all tp_dealloc with
 Py_TPFLAGS_HAVE_GC
Message-ID: <1501566589.91.0.719711304494.issue31095@psf.upfronthosting.co.za>


New submission from INADA Naoki:

like GH-2966, most types with Py_TPFLAGS_HAVE_GC
should call PyObject_GC_UnTrack() at top of the tp_dealloc.

For example, I found lru_cache doesn't call it and I can produce
segmentation fault.

I'll check other types too.

----------
components: Extension Modules, Interpreter Core
files: lru_cache_segv.py
messages: 299600
nosy: inada.naoki
priority: normal
severity: normal
status: open
title: Checking all tp_dealloc with Py_TPFLAGS_HAVE_GC
versions: Python 3.6, Python 3.7
Added file: http://bugs.python.org/file47055/lru_cache_segv.py

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31095>
_______________________________________

From report at bugs.python.org  Tue Aug  1 03:41:06 2017
From: report at bugs.python.org (Dmitry Malinovsky)
Date: Tue, 01 Aug 2017 07:41:06 +0000
Subject: [New-bugs-announce] [issue31096]
 asyncio.stream.FlowControlMixin._drain_helper may lead to a blocking
 behavior
Message-ID: <1501573266.21.0.530489547005.issue31096@psf.upfronthosting.co.za>


New submission from Dmitry Malinovsky:

When there is a huge amount of `writer.write` calls followed by `await writer.drain()` on a non-paused channel, and there are no other coroutine switches, `await writer.drain()` immediately returns without a switch. This is because `asyncio.stream.FlowControlMixin._drain_helper` do not `yield` or `yield from` on a non-paused stream.

Use-case: AMQP basic.publish method, for which the broker (rabbitmq) do not send any replies back. Trying to publish 4k messages results in the following warnings (PYTHONASYNCIODEBUG env variable is set):
`Executing <Handle <TaskWakeupMethWrapper object at 0x1106fde28>(<Future finis...events.py:275>) created at /Users/malinoff/Projects/ideas/amqproto/amqproto/channel.py:85> took 2.371 seconds`
2.371 seconds is the time spent on 4k `basic_publish` calls.

You can find the test itself on github: https://github.com/malinoff/amqproto/blob/master/tests/stress/test_4k_msgs.py#L11-L12

An easy fix would be to replace return (https://github.com/python/cpython/blob/master/Lib/asyncio/streams.py#L206) with yield (and but the code below under the else clause; I'm willing to prepare a pull request), but maybe I'm missing something and such behavior is intentional?

----------
components: asyncio
messages: 299610
nosy: Dmitry Malinovsky, yselivanov
priority: normal
severity: normal
status: open
title: asyncio.stream.FlowControlMixin._drain_helper may lead to a blocking behavior
versions: Python 3.5, Python 3.6, Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31096>
_______________________________________

From report at bugs.python.org  Tue Aug  1 09:23:06 2017
From: report at bugs.python.org (LB)
Date: Tue, 01 Aug 2017 13:23:06 +0000
Subject: [New-bugs-announce] [issue31097] itertools.islice not accepting
 np.int64
Message-ID: <1501593786.72.0.0101517996884.issue31097@psf.upfronthosting.co.za>


New submission from LB:

The error arises when this code is used:
itertools.islice( [3, 4, 5], np.int64(2) )

but not with this
itertools.islice([3,4,5], 2)

----------
components: Library (Lib)
messages: 299617
nosy: braaannigan
priority: normal
severity: normal
status: open
title: itertools.islice not accepting np.int64
type: crash
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31097>
_______________________________________

From report at bugs.python.org  Tue Aug  1 14:57:38 2017
From: report at bugs.python.org (Arfrever Frehtes Taifersar Arahesis)
Date: Tue, 01 Aug 2017 18:57:38 +0000
Subject: [New-bugs-announce] [issue31098] test target of Makefile always run
 tests in parallel mode
Message-ID: <1501613858.46.0.0685506045585.issue31098@psf.upfronthosting.co.za>


New submission from Arfrever Frehtes Taifersar Arahesis:

In CPython >=3.3, "test" target of Makefile calls Tools/scripts/run_tests.py script.
This script contains:

def is_multiprocess_flag(arg):
    return arg.startswith('-j') or arg.startswith('--multiprocess')
...
def main(regrtest_args):
...
    if threading and not any(is_multiprocess_flag(arg) for arg in regrtest_args):
        args.extend(['-j', '0'])  # Use all CPU cores


In order to run tests sequentially, in CPython 3.5 and older branches, it is possible to set EXTRATESTOPTS with -j1:
    make test EXTRATESTOPTS="-j1"


In CPython >=3.6, this workaround no longer works, because this code (Lib/test/regrtest.py in 3.5):

    if ns.use_mp is not None:
        if ns.use_mp <= 0:
            # Use all cores + extras for tests that like to sleep
            ns.use_mp = 2 + (os.cpu_count() or 1)
        if ns.use_mp == 1:
            ns.use_mp = None

Was changed into (Lib/test/libregrtest/cmdline.py in 3.6 and 3.7):

    if ns.use_mp is not None:
        if ns.use_mp <= 0:
            # Use all cores + extras for tests that like to sleep
            ns.use_mp = 2 + (os.cpu_count() or 1)


Currently the only remaining ways to run tests sequentially are to locally edit Tools/scripts/run_tests.py or to not  use "test" target of Makefile and to run Lib/test/regrtest.py directly with appropriate options.

----------
assignee: haypo
components: Tests
messages: 299629
nosy: Arfrever, haypo
priority: normal
severity: normal
status: open
title: test target of Makefile always run tests in parallel mode
versions: Python 3.6, Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31098>
_______________________________________

From report at bugs.python.org  Tue Aug  1 16:50:29 2017
From: report at bugs.python.org (Randy Henderson)
Date: Tue, 01 Aug 2017 20:50:29 +0000
Subject: [New-bugs-announce] [issue31099] Timer error pop from empty
 deque/list
Message-ID: <1501620629.5.0.924905930294.issue31099@psf.upfronthosting.co.za>


New submission from Randy Henderson:

Even when a deque is populated in the Timer setup parameter, the interpreter gives "IndexError: pop from an empty deque".  IndexError is generated for lists as well.

>>> Timer(stmt='d.popleft()',setup='d=deque([1,2,3])',globals=globals()).timeit()
Traceback (most recent call last):
  File "<pyshell#75>", line 1, in <module>
 Timer(stmt='d.popleft()',setup='d=deque([1,2,3])',globals=globals()).timeit()
  File "C:\Program Files\Python\Python36\lib\timeit.py", line 178, in timeit
    timing = self.inner(it, self.timer)
  File "<timeit-src>", line 6, in inner
IndexError: pop from an empty deque

----------
components: Library (Lib)
messages: 299634
nosy: Randy Henderson
priority: normal
severity: normal
status: open
title: Timer error pop from empty deque/list
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31099>
_______________________________________

From report at bugs.python.org  Tue Aug  1 21:07:09 2017
From: report at bugs.python.org (Kenneth Griffin)
Date: Wed, 02 Aug 2017 01:07:09 +0000
Subject: [New-bugs-announce] [issue31100] unable to open python
Message-ID: <1501636029.03.0.741480027722.issue31100@psf.upfronthosting.co.za>


New submission from Kenneth Griffin:

tried installing multiple versions of 3.6 on windows. the installation works. However, when trying to put python --version in the command line of powershell, i get a pop up that says python has stopped working.

----------
components: Installation
messages: 299640
nosy: Kenneth Griffin
priority: normal
severity: normal
status: open
title: unable to open python
type: crash
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31100>
_______________________________________

From report at bugs.python.org  Wed Aug  2 03:25:45 2017
From: report at bugs.python.org (Re-ax)
Date: Wed, 02 Aug 2017 07:25:45 +0000
Subject: [New-bugs-announce] [issue31101] os.remove() auto add \ in
 Windows2012R2
Message-ID: <1501658745.81.0.821961109576.issue31101@psf.upfronthosting.co.za>


New submission from Re-ax:

I need to remove a file in python at Windows2012R2, but, os.remove()auto add \ in each seq.

code:
#coding=utf-8
import os
dir_path='d:\c\d\e\t\c\t.xf'
os.remove(dir_path)


result:
Python 2.7.13 (v2.7.13:a06454b1afa1, Dec 17 2016, 20:53:40) [MSC v.1500 64 bit (AMD64)] on win32
Type "copyright", "credits" or "license()" for more information.
>>> ================================ RESTART ================================
>>> 

Traceback (most recent call last):
  File "D:\test\script\test.py", line 4, in <module>
    os.remove(dir_path)
WindowsError: [Error 3] The system cannot find the path specified: 'd:\\c\\d\\e\t\\c\t.xf'
>>>

----------
components: Windows
messages: 299645
nosy: 007hengxyx, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: os.remove() auto add \ in Windows2012R2
type: resource usage
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31101>
_______________________________________

From report at bugs.python.org  Wed Aug  2 05:51:51 2017
From: report at bugs.python.org (=?utf-8?b?0JTQuNC70Y/QvSDQn9Cw0LvQsNGD0LfQvtCy?=)
Date: Wed, 02 Aug 2017 09:51:51 +0000
Subject: [New-bugs-announce] [issue31102] deheader: double #incude of the
 same file
Message-ID: <1501667511.46.0.397475744495.issue31102@psf.upfronthosting.co.za>


New submission from ????? ????????:

For cpython 3.6 deheader (http://www.catb.org/esr/deheader/) reports:

Mac/Tools/pythonw.c has more than one inclusion of <Python.h>
Modules/_sre.c has more than one inclusion of "sre_lib.h"
Modules/_testcapimodule.c has more than one inclusion of "testcapi_long.h"
Modules/_threadmodule.c has more than one inclusion of "structmember.h"
Modules/expat/xmltok.c has more than one inclusion of "asciitab.h"
Modules/expat/xmltok.c has more than one inclusion of "latin1tab.h"
Modules/expat/xmltok.c has more than one inclusion of "utf8tab.h"
Modules/faulthandler.c has more than one inclusion of <signal.h>
Objects/unicodeobject.c has more than one inclusion of "stringlib/codecs.h"
Objects/unicodeobject.c has more than one inclusion of "stringlib/undef.h"
Objects/unicodeobject.c has more than one inclusion of "stringlib/fastsearch.h"
Objects/unicodeobject.c has more than one inclusion of "stringlib/split.h"
Objects/unicodeobject.c has more than one inclusion of "stringlib/ucs2lib.h"
Objects/unicodeobject.c has more than one inclusion of "stringlib/ucs1lib.h"
Objects/unicodeobject.c has more than one inclusion of "stringlib/partition.h"
Objects/unicodeobject.c has more than one inclusion of "stringlib/find_max_char.h"
Objects/unicodeobject.c has more than one inclusion of "stringlib/ucs4lib.h"
Objects/unicodeobject.c has more than one inclusion of "stringlib/asciilib.h"
Objects/unicodeobject.c has more than one inclusion of "stringlib/find.h"
Objects/unicodeobject.c has more than one inclusion of "stringlib/replace.h"
Objects/unicodeobject.c has more than one inclusion of "stringlib/localeutil.h"
Objects/unicodeobject.c has more than one inclusion of "stringlib/count.h"

----------
components: Build
messages: 299650
nosy: dilyan.palauzov
priority: normal
severity: normal
status: open
title: deheader: double #incude of the same file
type: enhancement
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31102>
_______________________________________

From report at bugs.python.org  Wed Aug  2 06:04:48 2017
From: report at bugs.python.org (Markus Kramer)
Date: Wed, 02 Aug 2017 10:04:48 +0000
Subject: [New-bugs-announce] [issue31103] Windows Installer Product Version
 3.6.2150.0 Offset By 0.0.150.0
Message-ID: <1501668288.41.0.228516476421.issue31103@psf.upfronthosting.co.za>


New submission from Markus Kramer:

Each Windows installation has a ?product version?.

The Windows installer python-3.6.2.exe has product version "3.6.2150.0"  (accessible with context menu / Properties / Details).

This causes at least 2 problems:
 - Automated software inventory relies on product version and therefore does not detect version 3.6.2
 - Microsoft installation guidelines require the first three fields to be smaller than 256.

Proposed alternatives for the value of product version:
- "3.6.2.0" to indicate the final release build.
- "3.6.2.150" to indicate the build number. The build number may be higher than 256, but this is unusual for a final release.



Side note: 
This is a sub-problem of http://bugs.python.org/issue31077

----------
components: Windows
messages: 299651
nosy: Damon Atkins, Markus Kramer, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: Windows Installer Product Version 3.6.2150.0 Offset By 0.0.150.0
type: behavior
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31103>
_______________________________________

From report at bugs.python.org  Wed Aug  2 07:14:07 2017
From: report at bugs.python.org (Govind S Menokee)
Date: Wed, 02 Aug 2017 11:14:07 +0000
Subject: [New-bugs-announce] [issue31104] posixpath.normpath truncating
 forward slashes from URL
Message-ID: <1501672447.79.0.933201717254.issue31104@psf.upfronthosting.co.za>


Changes by Govind S Menokee <govindsmenokee at gmail.com>:


----------
components: Library (Lib)
nosy: govindsmenokee
priority: normal
severity: normal
status: open
title: posixpath.normpath truncating forward slashes from URL
type: behavior
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31104>
_______________________________________

From report at bugs.python.org  Wed Aug  2 08:22:29 2017
From: report at bugs.python.org (Armin Rigo)
Date: Wed, 02 Aug 2017 12:22:29 +0000
Subject: [New-bugs-announce] [issue31105] Cyclic GC threshold may need tweaks
Message-ID: <1501676549.78.0.999656095107.issue31105@psf.upfronthosting.co.za>


New submission from Armin Rigo:

The cyclic GC uses a simple and somewhat naive policy to know when it must run.  It is based on counting "+1" for every call to _PyObject_GC_Alloc().  Explicit calls to PyObject_GC_Del() are counted as "-1".  The cyclic GC will only be executed after the count reaches 700.  There is then a scheme with multiple generations, but the point is that nothing is done at all before _PyObject_GC_Alloc() has been called 700 times.

The problem is that each of these _PyObject_GC_Alloc() can be directly or indirectly responsible for a large quantity of memory.  Take this example:

    while True:
        l = [None] * 10000000    # 80 MB, on 64-bit
        l[-1] = l
        del l

This loop actually consumes 700 times 80 MB, which is unexpected to say the least, and looks like a very fast memory leak.  The same program on 32-bit architectures simply runs out of virtual address space and fails with a MemoryError---even if we lower the length of the list to 10**9/700 = 1428571.

The same problem exists whenever a single object is "large", we allocate and forget many such objects in sequence, and they are kept alive by a cycle.  This includes the case where the large object is not part of a cycle, but merely referenced from a cycle.  For examples of "large" objects with potentially low lifetimes, maybe more natural than large lists, would include bz2 objects (17MB each) or Numpy arrays.

To fix it, the basic idea would be to have the "large" allocations count for more than "+1" in _PyObject_GC_Alloc().  Maybe they would also need to decrease the count by the same amount in PyObject_GC_Del(), though that may be less important.  Still, I am unsure about how it could be implemented.  Maybe a new C API is needed, which could then be used by a few built-in types (lists, bz2 objects, numpy arrays...) where the bulk of the memory allocation is not actually done by _PyObject_GC_Alloc() but by a separate call.  I am thinking about something like PyMem_AddPressure(size), which would simply increase the count by a number based on 'size'.

----------
components: Interpreter Core
messages: 299656
nosy: arigo
priority: normal
severity: normal
status: open
title: Cyclic GC threshold may need tweaks
versions: Python 2.7, Python 3.3, Python 3.4, Python 3.5, Python 3.6, Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31105>
_______________________________________

From report at bugs.python.org  Wed Aug  2 11:03:06 2017
From: report at bugs.python.org (=?utf-8?b?0JzQsNGA0Log0JrQvtGA0LXQvdCx0LXRgNCz?=)
Date: Wed, 02 Aug 2017 15:03:06 +0000
Subject: [New-bugs-announce] [issue31106] os.posix_fallocate() generate
 exception with errno 0
Message-ID: <1501686186.83.0.555586832167.issue31106@psf.upfronthosting.co.za>


New submission from ???? ?????????:

===
os.posix_fallocate(os.open('qwe.qwe', os.O_RDONLY|os.O_CREAT), 0, 1024*1024)
===

generates OSError with errno 0. Suppose this happen due to O_RDONLY flag.

strace :

====
open("qwe.qwe", O_RDONLY|O_CREAT|O_CLOEXEC, 0777) = 3
fallocate(3, 0, 0, 1048576)             = -1 EBADF (Bad file descriptor)
====


Python 3.5.3, Ubuntu 64-bit.

----------
components: Library (Lib)
messages: 299660
nosy: socketpair
priority: normal
severity: normal
status: open
title: os.posix_fallocate() generate exception with errno 0
type: behavior
versions: Python 3.5, Python 3.6, Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31106>
_______________________________________

From report at bugs.python.org  Wed Aug  2 14:53:14 2017
From: report at bugs.python.org (Shane Harvey)
Date: Wed, 02 Aug 2017 18:53:14 +0000
Subject: [New-bugs-announce] [issue31107] copyreg does not properly mangle
 __slots__ names
Message-ID: <1501699994.79.0.0201468057837.issue31107@psf.upfronthosting.co.za>


New submission from Shane Harvey:

This line in copyreg._slotnames does not properly calculate mangled attribute names:
https://github.com/python/cpython/blob/v3.6.2/Lib/copyreg.py#L131

The problem is that it does not strip leading underscores from the class name:

>>> class _LeadingUnderscoreClassName(object):
...     __slots__ = ("__bar",)
...
>>> import copy_reg
>>> copy_reg._slotnames(_LeadingUnderscoreClassName)
['__LeadingUnderscoreClassName__bar']

The result is that copy, pickle, and anything else that relies on _slotnames() do not work on classes with leading underscores and private __slots__. This bug is present in all versions of Python.

----------
components: Library (Lib)
messages: 299665
nosy: Shane Harvey
priority: normal
severity: normal
status: open
title: copyreg does not properly mangle __slots__ names
type: behavior
versions: Python 2.7, Python 3.3, Python 3.4, Python 3.5, Python 3.6, Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31107>
_______________________________________

From report at bugs.python.org  Wed Aug  2 14:53:46 2017
From: report at bugs.python.org (Sergey Fedoseev)
Date: Wed, 02 Aug 2017 18:53:46 +0000
Subject: [New-bugs-announce] [issue31108] add __contains__ for list_iterator
 (and others) for better performance
Message-ID: <1501700026.5.0.776654640231.issue31108@psf.upfronthosting.co.za>


New submission from Sergey Fedoseev:

> python -mtimeit -s "l = list(range(100000))" "l[-1] in l"
1000 loops, best of 3: 1.34 msec per loop
> python -mtimeit -s "l = list(range(100000))" "l[-1] in iter(l)"                                
1000 loops, best of 3: 1.59 msec per loop

----------
messages: 299666
nosy: sir-sigurd
priority: normal
severity: normal
status: open
title: add __contains__ for list_iterator (and others) for better performance
type: performance

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31108>
_______________________________________

From report at bugs.python.org  Wed Aug  2 15:05:24 2017
From: report at bugs.python.org (Yaron de Leeuw)
Date: Wed, 02 Aug 2017 19:05:24 +0000
Subject: [New-bugs-announce] [issue31109] zipimport argument clinic
 conversion
Message-ID: <1501700724.15.0.635937814263.issue31109@psf.upfronthosting.co.za>


New submission from Yaron de Leeuw:

Convert zipimport to use the argument clinic.
I will submit the PR shortly.

----------
components: Argument Clinic, Library (Lib)
messages: 299667
nosy: jarondl, larry, twouters
priority: normal
severity: normal
status: open
title: zipimport argument clinic conversion
type: enhancement
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31109>
_______________________________________

From report at bugs.python.org  Wed Aug  2 15:06:01 2017
From: report at bugs.python.org (cwee)
Date: Wed, 02 Aug 2017 19:06:01 +0000
Subject: [New-bugs-announce] [issue31110] Small typo in plistlib docs
Message-ID: <1501700761.19.0.284106835354.issue31110@psf.upfronthosting.co.za>


New submission from cwee:

Starting with 3.4, the docs for plistlib have had a typo:
https://docs.python.org/3.4/library/plistlib.html#plistlib.readPlist
https://docs.python.org/3.5/library/plistlib.html#plistlib.readPlist
https://docs.python.org/3.6/library/plistlib.html#plistlib.readPlist
https://docs.python.org/3.7/library/plistlib.html#plistlib.readPlist

Functions `readPlist` and `readPlistFromBytes` reference the `__getitem_` method, which should be `__getitem__`.

I'd be happy to submit a PR if there's a repo for the site's documentation.

----------
assignee: docs at python
components: Documentation
messages: 299668
nosy: cwee, docs at python
priority: normal
severity: normal
status: open
title: Small typo in plistlib docs
type: enhancement
versions: Python 3.4, Python 3.5, Python 3.6, Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31110>
_______________________________________

From report at bugs.python.org  Wed Aug  2 15:57:43 2017
From: report at bugs.python.org (Charles Ferguson)
Date: Wed, 02 Aug 2017 19:57:43 +0000
Subject: [New-bugs-announce] [issue31111] Python 3.6 has an inaccessible
 attribute on FileNotFoundError
Message-ID: <1501703863.51.0.936615948464.issue31111@psf.upfronthosting.co.za>


New submission from Charles Ferguson:

Whilst debugging a problem in some of my code (which turned out to be a misspelt filename), I found that I could not access one of the properties of the FileNotFoundError object.

Essentially, if you get a 'FileNotFoundError' for opening a file that does not exist, you expect to be able to enumerate the attributes on that object. And if you try to access them, that they be accessible.
However, there is an attribute - 'characters_written' - which claims to be present according to 'dir()' but not according to 'hasattr()' and trying to access it with 'getattr()' confirms that it's not really there.

Looking at the documentation at https://docs.python.org/3/library/exceptions.html#OSError I see that it is a subclass of OSError(), and that the BlockingIOError can have this attribute. But none of the other OSError subclasses are documented as having the attribute.

It is still reasonable that any attribute access could generate another exception (including an AttributeError), as their implementation may have other issues, but I do not feel that this applies here, as this is an internal exception object that probably ought not to have an issue.

Since 'characters_written' doesn't seem to have any meaning in the context of 'FileNotFound', it seems like it's an oversight from the other exception subclass.

What I conclude from this is that the documentation, hasattr() and getattr() are correct, but dir() is acting wrongly. Principle of least surprise also suggests that having inconsistent returns from these functions probably isn't correct. I guess it could be the other way around, and the documentation, hasattr and getattr could be returning incorrectly, but that feels like something of a stretch.

I would wonder if the other OSError subclasses also suffer from this extraneous attribute name, and that it's been implemented at the wrong level, but I have no evidence to suggest that's the case (just that that's something I'd probably look at if I had the time).

Reproduction code:

----
#!/usr/bin/python3.6
##
# Demonstrate oddity of FileNotFoundError.
#

try:
    fh = open('/nothing/at/all', 'r')
except Exception as ex:
    print("Exception: {}".format(ex))
    for attrname in dir(ex):
        if attrname.startswith('__'):
            # Ignore dunders for the sake of brevity
            continue

        print("  Attribute name: {}".format(attrname))
        if not hasattr(ex, attrname):
            print("         hasattr: False - surprising!")
        print("           value: {}".format(getattr(ex, attrname)))
----

On 3.6 this generates:

----
Charles at charlesmbp:~/Coding/terraspock-develop (develop)$ python python3.6.2-filenotfound.py 
Exception: [Errno 2] No such file or directory: '/nothing/at/all'
  Attribute name: args
           value: (2, 'No such file or directory')
  Attribute name: characters_written
         hasattr: False - surprising!
Traceback (most recent call last):
  File "python3.6.2-filenotfound.py", line 7, in <module>
    fh = open('/nothing/at/all', 'r')
FileNotFoundError: [Errno 2] No such file or directory: '/nothing/at/all'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "python3.6.2-filenotfound.py", line 18, in <module>
    print("           value: {}".format(getattr(ex, attrname)))
AttributeError: characters_written
----

On 2.7 this works fine, but I've not tested the other 3.x series versions as I don't have them to hand.

Testing performed on macOS using Python 3.6.2.

I find it hard to think that this is intended behaviour, but whether it's something that debugging tools (and users) would expect or find useful I don't know.

----------
components: Library (Lib)
messages: 299669
nosy: gerph
priority: normal
severity: normal
status: open
title: Python 3.6 has an inaccessible attribute on FileNotFoundError
type: behavior
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31111>
_______________________________________

From report at bugs.python.org  Wed Aug  2 23:11:54 2017
From: report at bugs.python.org (ericshenjs)
Date: Thu, 03 Aug 2017 03:11:54 +0000
Subject: [New-bugs-announce] [issue31112] cannot run module with double
 quotes
Message-ID: <1501729914.71.0.88312721886.issue31112@psf.upfronthosting.co.za>


New submission from ericshenjs:

(vPy3X64) test01 at NJNU f:\Src\localCode\python\py3_eric
> python "-m locust.main -V"
f:\SDKs\Python\vPy3X64\Scripts\python.exe: Error while finding module specification for ' locust.main -V' (ModuleNotFoundError: No module named ' locust')

(vPy3X64) test01 at NJNU f:\Src\localCode\python\py3_eric
> python -m locust.main -V
[2017-08-03 10:53:19,111] NJNU/INFO/stdout: Locust 0.8a2
[2017-08-03 10:53:19,111] NJNU/INFO/stdout:

(vPy3X64) test01 at NJNU f:\Src\localCode\python\py3_eric
> python -V
Python 3.6.1 :: Anaconda 4.4.0 (64-bit)

----------
components: Windows
messages: 299681
nosy: ericshenjs, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: cannot run module with double quotes
type: resource usage
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31112>
_______________________________________

From report at bugs.python.org  Thu Aug  3 04:10:51 2017
From: report at bugs.python.org (Manuel Krebber)
Date: Thu, 03 Aug 2017 08:10:51 +0000
Subject: [New-bugs-announce] [issue31113] Stack overflow with large program
Message-ID: <1501747851.67.0.0193426592051.issue31113@psf.upfronthosting.co.za>


New submission from Manuel Krebber:

With a pattern matching library I am generating some Python code that matches patterns. For a very big pattern set I generate a Python file which is about 20MB and has ~300K LOC. When I try to execute the file with Python 3.6.2 on Windows 10 (64bit), the interpreter crashes. I do not have the Python source locally, but I could get it if necessary. The crash is because of a stack overflow when calling dfs() in compile.c. I can attach you the program, but it needs some dependencies which currently are only availiable via some Github repos.

I will try to split the ig file into multiple smaller ones, but I thought you might want to know about an interpreter crash.

----------
components: Interpreter Core, Windows
messages: 299689
nosy: Wheerd, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: Stack overflow with large program
type: crash
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31113>
_______________________________________

From report at bugs.python.org  Thu Aug  3 13:40:40 2017
From: report at bugs.python.org (Xavier de Gaye)
Date: Thu, 03 Aug 2017 17:40:40 +0000
Subject: [New-bugs-announce] [issue31114] 'make install' fails when
 exec_prefix is '/' and DESTDIR not empty
Message-ID: <1501782040.16.0.9530512469.issue31114@psf.upfronthosting.co.za>


New submission from Xavier de Gaye:

The error messages:
    running install_lib
    creating /lib/python3.7
    error: could not create '/lib/python3.7': Permission denied
    make[1]: *** [Makefile:1449: sharedinstall] Error 1

The command that triggers this failure:
    _PYTHON_PROJECT_BASE=/tmp/android-makesetup/build/python3.7-android-21-armv7 _PYTHON_HOST_PLATFORM=linux-arm PYTHONPATH=/tmp/android-makesetup/build/python3.7-android-21-armv7/build/lib.linux-arm-3.7:/src/python/master/Lib _PYTHON_SYSCONFIGDATA_NAME=_sysconfigdata_m_linux_ python /src/python/master/setup.py install \
      --prefix=/ \
      --install-scripts=//bin \
      --install-platlib=//lib/python3.7/lib-dynload \
      --root=/tmp/android-makesetup/build/python3.7-install-android-21-armv7/

Note how some paths in the setup.py options start with two slashes.

The problem is that change_root() in Lib/distutils/util.py cannot concatenate 'root' with 'pathname' as 'pathname' starts with two slashes and therefore cannot be made relative by just removing one slash.  The behavior of change_root() is correct (although it could be made more robust by calling os.path.normpath() first before removing the first slash) because according to POSIX a pathname starting with two slashes is implementation defined (but not when starting with three or more slashes).  And normpath respects this rule:
    >>> from os.path import normpath
    >>> normpath('//tmp')
    '//tmp'
    >>> normpath('///tmp')
    '/tmp'
However for most posix systems (possibly except Cygwin ?), a path starting with two slashes is the same path as the path starting with one slash.

The problem lies with the Makefile. A workaround for the user is to set the --exec-prefix option to '/./' instead of '/' when running configure and this fixes the problem, the installation is successfull. So a fix could be to swap '/' for '/./' in configure itself.
Maybe there is a better solution.

References:
issue 1676135:
    The changes made by this issue have excluded the case of '/' as there was a concern at that time that the sys.prefix value would be wrong.  I have checked that sys.prefix is correct when '/' is changed to '/./' (on configure command line). This is because the determination of sys.prefix made by Lib/site.py is quite different now from what it was 10 years ago.

issue 9674:
    This is exactly the same issue as this one.  I have preferred to open a new issue here because the discussion on issue 9674 is focused on making changes to distutils while the problem is actually in the posixly incorrect use of paths starting with double slashes in the Makefile.

----------
components: Build
messages: 299718
nosy: xdegaye
priority: normal
severity: normal
status: open
title: 'make install' fails when exec_prefix is '/' and DESTDIR not empty
type: behavior
versions: Python 2.7, Python 3.5, Python 3.6, Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31114>
_______________________________________

From report at bugs.python.org  Thu Aug  3 17:04:49 2017
From: report at bugs.python.org (Robert McLeod)
Date: Thu, 03 Aug 2017 21:04:49 +0000
Subject: [New-bugs-announce] [issue31115] Py3.6 threading/reference counting
 issues with `numexpr`
Message-ID: <1501794289.11.0.944319221721.issue31115@psf.upfronthosting.co.za>


New submission from Robert McLeod:

I'm working on the development branch of the `numexpr` module and I've run into some problems on Python 3.6, where I seem to get a variety of errors relating to threading or reference counting errors. This module is commonly used for accelerating NumPy code, as it can parse an expression into a C-extension virtual machine program and break the calculations down into blocks which are then dispatched over multiple threads. I get similar errors on both Ubuntu 16.04 with GCC5.4 and Win7 with MSVC2015.  I created an issue here where I provide some of the `gcc` error outputs.  

https://github.com/pydata/numexpr/issues/252

Typically I'm getting a different error on every run. 

We use a wrapper for Windows threads that emulated pthreads, and I found on Windows crashes occurred on calling the Windows system function `WaitForSingleObject(cond->sema, INFINITE);` in the file numexpr3/win32/pthread.c.  

I cannot replicate this problem in Python 2.7/3.4/3.5 on Windows or Linux. I'm using Anaconda in both instances, with nomkl NumPy on Linux and mkl NumPy on Windows.  

I tried valgrinding with Python 3.5 and 3.6, and I get numerous errors coming from places like pickle and ast (which the new NumExpr uses) in 3.6 and it's basically clean in 3.5.  The logs are attached to the issue linked above.

----------
components: Extension Modules, Library (Lib)
messages: 299722
nosy: Robert McLeod
priority: normal
severity: normal
status: open
title: Py3.6 threading/reference counting issues with `numexpr`
type: crash
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31115>
_______________________________________

From report at bugs.python.org  Thu Aug  3 18:32:22 2017
From: report at bugs.python.org (Derek Wilson)
Date: Thu, 03 Aug 2017 22:32:22 +0000
Subject: [New-bugs-announce] [issue31116] base85 z85 variant encoding
Message-ID: <1501799542.97.0.652442678927.issue31116@psf.upfronthosting.co.za>


New submission from Derek Wilson:

Issue #17618 introduced base85 options into the base64 module including b85 and a85/adobe variants.

There has since been introduced a variant that is easier to work with specifically as it avoids " ' \ characters which avoids quoting/escaping issues when working with base85 literals in python, json, xml, etc.

https://rfc.zeromq.org/spec:32/Z85/

there is a reference implementation in c for which an extension could easily be built.

alternately it would be very easy to add a _z85alphabet etc. to the base64 module to mirror what has been done with a/b85.

1) is there any interest in this being in the language
2) if so do we want python impl or c extension around ref impl?

----------
components: Library (Lib)
messages: 299724
nosy: underrun
priority: normal
severity: normal
status: open
title: base85 z85 variant encoding
type: enhancement
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31116>
_______________________________________

From report at bugs.python.org  Thu Aug  3 18:37:21 2017
From: report at bugs.python.org (Benjamin Swan)
Date: Thu, 03 Aug 2017 22:37:21 +0000
Subject: [New-bugs-announce] [issue31117] Intersphinx file objects.inv
 missing 'list' from py:class
Message-ID: <1501799841.44.0.68321420837.issue31117@psf.upfronthosting.co.za>


New submission from Benjamin Swan:

The objects.inv file my project pulls in for python 2 (https://docs.python.org/2/objects.inv) no longer contains an entry of list under py:class. This is causing sphinx to output

WARNING: py:obj reference target not found: list

for intershpinx links to the list class.

----------
messages: 299725
nosy: Benjamin Swan
priority: normal
severity: normal
status: open
title: Intersphinx file objects.inv missing 'list' from py:class
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31117>
_______________________________________

From report at bugs.python.org  Fri Aug  4 02:42:55 2017
From: report at bugs.python.org (Ashwini Chaudhary)
Date: Fri, 04 Aug 2017 06:42:55 +0000
Subject: [New-bugs-announce] [issue31118] Make super() work with
 staticmethod by using __class__ for both arguments to super()
Message-ID: <1501828975.1.0.737825658034.issue31118@psf.upfronthosting.co.za>


New submission from Ashwini Chaudhary:

>From this question I noticed that super() doesn't work with static methods with no arguments: https://stackoverflow.com/q/45498675/846892

My question is what's the issue with using __class__ for both arguments to super() in case of static method? Or was it just an oversight.

----------
messages: 299733
nosy: ashwch
priority: normal
severity: normal
status: open
title: Make super() work with staticmethod by using __class__ for both arguments to super()
type: enhancement
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31118>
_______________________________________

From report at bugs.python.org  Fri Aug  4 04:36:33 2017
From: report at bugs.python.org (Nathaniel Smith)
Date: Fri, 04 Aug 2017 08:36:33 +0000
Subject: [New-bugs-announce] [issue31119] Signal tripped flags need memory
 barriers
Message-ID: <1501835793.8.0.586485802562.issue31119@psf.upfronthosting.co.za>


New submission from Nathaniel Smith:

Sometimes, CPython's signal handler (signalmodule.c:signal_handler) runs in a different thread from the main thread. On Unix this is rare but it does happen (esp. for SIGCHLD), and on Windows it happens 100% of the time. It turns out that there is a subtle race condition that means such signals can be lost.

To notify the main thread that a signal has been received, and which signal it is, CPython's signal handler sets two global variables:

  Handlers[sig_num].tripped = 1;
  is_tripped = 1

And then PyErr_CheckSignals does:

  if (is_tripped) {
      is_tripped = 0;
      for (i = 1; i < NSIG; i++) {
          if (Handlers[i].is_tripped) {
              ... run signal handler ...
          }
      }
  }

As some comments in the source note, this logic depends strongly on the assumption that when the signal handler sets the two flags, they are immediately visible to PyErr_CheckSignals, and that the two assignments happen in the given order. For example, suppose that is_tripped were set before Handlers[i].is_tripped. Then we could have the following sequence:

1. Thread A sets is_tripped = 1
2. Thread B runs PyErr_CheckSignals
3. Thread B notices is_tripped == 1, and sets it to 0
4. Thread B scans through all the Handlers, sees that none are set, and carries on
5. Thread A sets Handlers[i].is_tripped = 1

In this case the signal is lost until another signal arrives, which may not happen until an arbitrary amount of time has passed.

Also, the fix for bpo-30038 (making sure that we set the flags before writing to the wakeup fd) assumes that setting the flags before writing to the wakeup fd means that, well, the flags will be written before the fd.

However, when you have code running in separate threads, none of this is actually guaranteed. In general, the compiler is free to re-order writes, and more perniciously, the hardware memory hierarchy is also free to arbitrarily delay when writes to RAM made by one CPU become visible to another CPU, which can also create arbitrary reorderings.

In an attempt to avoid these issues, signalmodule.c declares the signal flags as 'volatile sig_atomic_t'. However, the best one can hope for from this is that it'll stop the *compiler* from reordering writes. It does nothing to stop the *hardware* from reordering writes. Using volatile like this might be sufficient if the signal handler runs in the main thread, but not if it runs in another thread.

The correct way to handle this is to use the primitives in pyatomic.h to get/set the tripped flags.

----

I noticed this because of the same test case that was failing due to bpo-30038... this thing has been frustrating me for months because it's intermittent and never seems to happen on my local VM. But specifically what I'm seeing at this point is:

Thread A (the thread where the signal handler runs):
A1. takes a lock.
A2. with the lock held, calls the C function raise(SIGINT), which directly calls the CPython C-level signal handler. (Verified by consulting the Windows C runtime source.)
A3. writes to the wakeup fd & sets both tripped flags, in whatever order
A4. releases the lock

Simultaneously, in thread B (the main thread):
B1. observes that the wakeup fd has been written to
B2. takes the lock
B3. releases the lock
B4. calls repr(), which unconditionally calls PyObject_Repr, which unconditionally calls PyErr_CheckSignals

I expect the call in B4 to run the Python-level signal handler, but this does not happen. Instead it runs some time later, causing an unexpected KeyboardInterrupt.

My reasoning is: the lock guarantees that no matter what the internal details of the C-level signal handler actually are, they have to all be complete before my code checks for signals. In excruciating detail:

- the write to the wakeup fd must happen-after the lock is acquired (A1)
- therefore, step B1 happens-after step A1
- B2 happens-after B1, so B2 happens-after A1.
- B2 takes the same lock as A1, so if B2 happens-after A1, it must also happen-after A4, when the lock is released.
- step B4 happens-after B2, therefore it happens-after A4 and A3. Therefore the tripped flags should be visible and the Python-level signal handler should run.

Yet this does not happen, so *something* really weird is going on. The hypothesis that it's the lack of memory barriers both explains how this could happen -- specifically I think thread B may be running PyErr_CheckSignals as part of the wakeup-fd reading logic, and it's clearing is_tripped before Handlers[i].tripped becomes visible, like in my code above. And it also explains why it happens in like 1/50 runs in Appveyor, but never on my local Windows 10 VM that only has 1 virtual CPU.

But regardless of whether this is the cause of my particular weird bug, it clearly *is* a bug and should be fixed.

----------
messages: 299736
nosy: haypo, njs
priority: normal
severity: normal
status: open
title: Signal tripped flags need memory barriers
type: behavior
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31119>
_______________________________________

From report at bugs.python.org  Fri Aug  4 09:02:50 2017
From: report at bugs.python.org (Hiren Vadalia)
Date: Fri, 04 Aug 2017 13:02:50 +0000
Subject: [New-bugs-announce] [issue31120] [2.7] Python 64 bit _ssl compile
 fails due missing buildinf_amd64.h
Message-ID: <1501851770.24.0.114310480088.issue31120@psf.upfronthosting.co.za>


New submission from Hiren Vadalia:

I am trying to compile Python 2.7.13 (https://github.com/python/cpython/tree/9c1426de7521299f70eb5483e7e25d1c2a73dbbd) in 64 bit but I am getting following error:
Build started: Project: _ssl, Configuration: Release|x64
Performing Pre-Build Event...
Found a working perl at 'c:\perl\bin\perl.exe'
Traceback (most recent call last):
  File "build_ssl.py", line 248, in <module>
    main()
  File "build_ssl.py", line 231, in main
    shutil.copy(r"crypto\buildinf_%s.h" % arch, r"crypto\buildinf.h")
  File "C:\projects\windows-work\cpython-2.7.13\lib\shutil.py", line 119, in copy
    copyfile(src, dst)
  File "C:\projects\windows-work\cpython-2.7.13\lib\shutil.py", line 82, in copyfile
    with open(src, 'rb') as fsrc:
IOError: [Errno 2] No such file or directory: 'crypto\\buildinf_amd64.h'
Project : error PRJ0019: A tool returned an error code from "Performing Pre-Build Event..."
Build log was saved at "file://C:\projects\windows-work\cpython-2.7.13\PC\VS9.0\x64-temp-Release\_ssl\BuildLog.htm"
_ssl - 1 error(s), 0 warning(s)
Build started: Project: _hashlib, Configuration: Release|x64
Performing Pre-Build Event...
Found a working perl at 'c:\perl\bin\perl.exe'
Traceback (most recent call last):
  File "build_ssl.py", line 248, in <module>
    main()
  File "build_ssl.py", line 231, in main
    shutil.copy(r"crypto\buildinf_%s.h" % arch, r"crypto\buildinf.h")
  File "C:\projects\windows-work\cpython-2.7.13\lib\shutil.py", line 119, in copy
    copyfile(src, dst)
  File "C:\projects\windows-work\cpython-2.7.13\lib\shutil.py", line 82, in copyfile
    with open(src, 'rb') as fsrc:
IOError: [Errno 2] No such file or directory: 'crypto\\buildinf_amd64.h'
Project : error PRJ0019: A tool returned an error code from "Performing Pre-Build Event..."
Build log was saved at "file://C:\projects\windows-work\cpython-2.7.13\PC\VS9.0\x64-temp-Release\_hashlib\BuildLog.htm"
_hashlib - 1 error(s), 0 warning(s)
Build complete: 23 Projects succeeded, 2 Projects failed, 1 Projects skipped


Then I did some search here + buildbots and I found there is already bug for it: https://bugs.python.org/issue30368 which seems to be fixed for upcoming Python 2.7.14 release.
Also I found that for commit 9c1426de7521299f70eb5483e7e25d1c2a73dbbd AMD64 buildbot has failed build (http://buildbot.python.org/all/builders/AMD64%20Windows7%20SP1%20VS9.0%202.7/builds/23/)

----------
assignee: christian.heimes
components: Build, SSL
messages: 299743
nosy: Hiren Vadalia, christian.heimes
priority: normal
severity: normal
status: open
title: [2.7] Python 64 bit _ssl compile fails due missing buildinf_amd64.h
type: compile error
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31120>
_______________________________________

From report at bugs.python.org  Fri Aug  4 10:03:56 2017
From: report at bugs.python.org (Jason R. Coombs)
Date: Fri, 04 Aug 2017 14:03:56 +0000
Subject: [New-bugs-announce] [issue31121] Unable to exit pdb when script
 becomes invalid
Message-ID: <1501855436.72.0.136626793025.issue31121@psf.upfronthosting.co.za>


New submission from Jason R. Coombs:

Similar to #16180 and possible a duplicate of #14743, if the script being run under pdb becomes invalid such as through a chdir operation, pdb will get trapped in a loop, catching its own exception.

$ mkdir foo
$ cat > foo/script.py
import os
os.chdir('foo')
$ python -m pdb foo/script.py
> /Users/jaraco/Dropbox/code/privacymate/harvester/foo/script.py(1)<module>()
-> import os
(Pdb) c
The program finished and will be restarted
Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/pdb.py", line 1667, in main
    pdb._runscript(mainpyfile)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/pdb.py", line 1545, in _runscript
    with open(filename, "rb") as fp:
FileNotFoundError: [Errno 2] No such file or directory: 'foo/script.py'
Uncaught exception. Entering post mortem debugging
Running 'cont' or 'step' will restart the program
> /Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/pdb.py(1545)_runscript()
-> with open(filename, "rb") as fp:
(Pdb) q
Post mortem debugger finished. The foo/script.py will be restarted
Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/pdb.py", line 1667, in main
    pdb._runscript(mainpyfile)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/pdb.py", line 1545, in _runscript
    with open(filename, "rb") as fp:
FileNotFoundError: [Errno 2] No such file or directory: 'foo/script.py'
Uncaught exception. Entering post mortem debugging
Running 'cont' or 'step' will restart the program
> /Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/pdb.py(1545)_runscript()
-> with open(filename, "rb") as fp:
(Pdb) q
Post mortem debugger finished. The foo/script.py will be restarted
Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/pdb.py", line 1667, in main
    pdb._runscript(mainpyfile)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/pdb.py", line 1545, in _runscript
    with open(filename, "rb") as fp:
FileNotFoundError: [Errno 2] No such file or directory: 'foo/script.py'
Uncaught exception. Entering post mortem debugging
Running 'cont' or 'step' will restart the program
> /Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/pdb.py(1545)_runscript()
-> with open(filename, "rb") as fp:
(Pdb)

I suggest PDB should always pass exceptions when the stack trace terminates in the pdb module.

----------
components: Library (Lib)
messages: 299748
nosy: jason.coombs
priority: normal
severity: normal
status: open
title: Unable to exit pdb when script becomes invalid
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31121>
_______________________________________

From report at bugs.python.org  Fri Aug  4 15:16:41 2017
From: report at bugs.python.org (Nikolaus Rath)
Date: Fri, 04 Aug 2017 19:16:41 +0000
Subject: [New-bugs-announce] [issue31122] SSLContext.wrap_socket() throws
 OSError with errno == 0
Message-ID: <1501874201.48.0.0126792321555.issue31122@psf.upfronthosting.co.za>


New submission from Nikolaus Rath:

With a particularly atrocious network connection, I often get the following exception:

  File "/usr/lib/python3/dist-packages/dugong/__init__.py", line 503, in connect
    self._sock = self.ssl_context.wrap_socket(self._sock, server_hostname=server_hostname)
  File "/usr/lib/python3.5/ssl.py", line 385, in wrap_socket
    _context=self)
  File "/usr/lib/python3.5/ssl.py", line 760, in __init__
    self.do_handshake()
  File "/usr/lib/python3.5/ssl.py", line 996, in do_handshake
    self._sslobj.do_handshake()
  File "/usr/lib/python3.5/ssl.py", line 641, in do_handshake
    self._sslobj.do_handshake()
OSError: [Errno 0] Error

I don't think an error with errno == 0 should ever be raised by Python.

----------
assignee: christian.heimes
components: SSL
messages: 299759
nosy: christian.heimes, nikratio
priority: normal
severity: normal
status: open
title: SSLContext.wrap_socket() throws OSError with errno == 0
type: behavior
versions: Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31122>
_______________________________________

From report at bugs.python.org  Fri Aug  4 19:22:19 2017
From: report at bugs.python.org (Varun Agrawal)
Date: Fri, 04 Aug 2017 23:22:19 +0000
Subject: [New-bugs-announce] [issue31123] fnmatch does not follow Unix
 fnmatch functionality
Message-ID: <1501888939.57.0.889177550309.issue31123@psf.upfronthosting.co.za>


New submission from Varun Agrawal:

Currently, the fnmatch module in Python does not operate like the Unix equivalent command. This is especially the case with patterns such as `dir/` which does not match directory path returned by `os` (which would be `dir`) and neither does the pattern match the underlying files and sub-directories. Adding these functionalities so that fnmatch matches it's Unix equivalent would make it much more powerful to use.

Glob is not useful since glob is a special case that matches only with the current directory paths rather than the a general purpose filename pattern matcher.

----------
components: Library (Lib)
messages: 299763
nosy: Varun Agrawal
priority: normal
severity: normal
status: open
title: fnmatch does not follow Unix fnmatch functionality
type: behavior
versions: Python 2.7, Python 3.4, Python 3.5, Python 3.6, Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31123>
_______________________________________

From report at bugs.python.org  Sat Aug  5 09:41:21 2017
From: report at bugs.python.org (=?utf-8?q?Jos=C3=A9_Leite?=)
Date: Sat, 05 Aug 2017 13:41:21 +0000
Subject: [New-bugs-announce] [issue31124] ImportError: cannot import name
 HAS_ALPN
Message-ID: <1501940481.62.0.440341735277.issue31124@psf.upfronthosting.co.za>


New submission from Jos? Leite:

When Running Web2py with Python 2.7.11 or 2.7.13 in windows 10 I get the following error:

Traceback (most recent call last):
File "C:\web2py\web2py.py", line 21, in 
import gluon.widget
File "C:\web2py\gluon\widget.py", line 26, in 
from gluon import main, newcron
File "C:\web2py\gluon\main.py", line 28, in 
import ssl
File "C:\Python27\lib\ssl.py", line 126, in 
from _ssl import HAS_SNI, HAS_ECDH, HAS_NPN, HAS_ALPN
ImportError: cannot import name HAS_ALPN

----------------------------------------------------
It seem like a windows Python bug in _ssl.py

----------
assignee: christian.heimes
components: SSL, Windows
messages: 299781
nosy: Jhleite, christian.heimes, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: ImportError: cannot import name HAS_ALPN
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31124>
_______________________________________

From report at bugs.python.org  Sat Aug  5 12:26:59 2017
From: report at bugs.python.org (Stephen Larroque)
Date: Sat, 05 Aug 2017 16:26:59 +0000
Subject: [New-bugs-announce] [issue31125] shelve.open of temporary file
 fails with error "anydbm.error: db type could not be determined"
Message-ID: <1501950419.2.0.502891257534.issue31125@psf.upfronthosting.co.za>


New submission from Stephen Larroque:

This is a followup of issue 23174.

When using `shelve.open(tempfile.mkstemp()[1])`, the shelve (or anydbm or dumbdbm) fail with "anydbm.error: db type could not be determined".

Instead, the module could detect the file is empty and use it as if it were not existing before ('c' or 'n' flag). For the moment, there is no way around except creating the temporary file and then delete it.

----------
components: IO
messages: 299790
nosy: Stephen Larroque
priority: normal
severity: normal
status: open
title: shelve.open of temporary file fails with error "anydbm.error: db type could not be determined"
type: behavior
versions: Python 2.7, Python 3.3, Python 3.4, Python 3.5, Python 3.6, Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31125>
_______________________________________

From report at bugs.python.org  Sun Aug  6 09:17:16 2017
From: report at bugs.python.org (ksqsf)
Date: Sun, 06 Aug 2017 13:17:16 +0000
Subject: [New-bugs-announce] [issue31126] dict comprehension shouldn't raise
 UnboundLocalError
Message-ID: <1502025436.65.0.0753322940142.issue31126@psf.upfronthosting.co.za>


New submission from ksqsf:

The code
    key = ["a", "b"]
    val = [1, 2]
    dic = {key:val for key in key for val in val}
will raise UnboundLocalError in Python 3.6.2 and 2.7.13.

Intuitively, the element 'key' and the list 'key' are not the same, so generally the expected result is {"a": 1, "b": 2}.

There are similar cases for listcomps, setcomps and genexprs:
    l = [1, 2, 3]
    {l for l in l}   # => {1, 2, 3}
    [l for l in l]   # => [1, 2, 3]
    for l in (l for l in l):
        print(l, end=' ')
                     # => 1 2 3 
All of them do as what is expected. 

For consistency and intuitiveness, the behavior of distcomps should be modified so that no UnboundLocalError is raised.

----------
components: Interpreter Core
messages: 299799
nosy: ksqsf
priority: normal
severity: normal
status: open
title: dict comprehension shouldn't raise UnboundLocalError
type: behavior
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31126>
_______________________________________

From report at bugs.python.org  Sun Aug  6 15:00:42 2017
From: report at bugs.python.org (Kevin Shweh)
Date: Sun, 06 Aug 2017 19:00:42 +0000
Subject: [New-bugs-announce] [issue31127] Abstract classes derived from
 built-in classes don't block instance creation
Message-ID: <1502046042.43.0.230510870536.issue31127@psf.upfronthosting.co.za>


New submission from Kevin Shweh:

The only check that prevents instantiating abstract classes is in object.__new__, but most built-in classes never actually call object.__new__. That means you can do stuff like

import abc

class Foo(list, metaclass=abc.ABCMeta):
	@abc.abstractmethod
	def abstract(self):
		pass

Foo()

and the Foo() call will silently succeed.

Ideally, the Foo() call should fail. Other options include having the Foo class definition itself fail, or just making a note in the documentation describing the limitation. (As far as I can see, this is currently undocumented.)

----------
assignee: docs at python
components: Documentation, Library (Lib)
messages: 299810
nosy: Kevin Shweh, docs at python
priority: normal
severity: normal
status: open
title: Abstract classes derived from built-in classes don't block instance creation
type: behavior
versions: Python 2.7, Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31127>
_______________________________________

From report at bugs.python.org  Sun Aug  6 18:55:50 2017
From: report at bugs.python.org (Feanil Patel)
Date: Sun, 06 Aug 2017 22:55:50 +0000
Subject: [New-bugs-announce] [issue31128] Allow pydoc to run with an
 arbitrary hostname
Message-ID: <1502060150.4.0.12208327143.issue31128@psf.upfronthosting.co.za>


New submission from Feanil Patel:

I want to run the pydoc server while doing development inside a container.  However, the pydoc http server only binds to localhost with no option to override this.  This means that if I'm trying to read the docs from a remote machine,  I have to do some network Jiu Jitsu.

Acceptance Criteria:
- Pydoc takes a new argument that overrides the hostname the server binds to.
- Pydoc continues to default to localhost

----------
assignee: docs at python
components: Documentation
messages: 299817
nosy: Feanil Patel, docs at python
priority: normal
severity: normal
status: open
title: Allow pydoc to run with an arbitrary hostname
type: enhancement
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31128>
_______________________________________

From report at bugs.python.org  Sun Aug  6 23:28:30 2017
From: report at bugs.python.org (Daniel Watkins)
Date: Mon, 07 Aug 2017 03:28:30 +0000
Subject: [New-bugs-announce] [issue31129] RawConfigParser.items() is unusual
 in taking arguments
Message-ID: <1502076510.54.0.648446505741.issue31129@psf.upfronthosting.co.za>


New submission from Daniel Watkins:

A grep through the codebase shows that RawConfigParser.items() is the only .items() method in the stdlib which accepts arguments.

This is annoying as a stdlib user because when I see the arguments being passed to RawConfigParser.items(), I have _no idea_ what they do.  Instinctively, I do not expect .items() to take arguments, and I have to go and work out what it does each time.

I think it would be both easier to understand, and more consistent with general Pythonic practice, if the two codepaths in RawConfigParser.items() were split in to separate methods; one which takes no arguments (which will continue to behave as it does today when called that way) and one which is named in a way that makes it clearer what it does (and takes arguments).

----------
components: Library (Lib)
messages: 299821
nosy: odd_bloke
priority: normal
severity: normal
status: open
title: RawConfigParser.items() is unusual in taking arguments
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31129>
_______________________________________

From report at bugs.python.org  Mon Aug  7 05:24:47 2017
From: report at bugs.python.org (STINNER Victor)
Date: Mon, 07 Aug 2017 09:24:47 +0000
Subject: [New-bugs-announce] [issue31130] test_idle: idlelib.configdialog
 leaks references
Message-ID: <1502097887.03.0.529765316852.issue31130@psf.upfronthosting.co.za>


New submission from STINNER Victor:

Starting at the following build, test_idle started to leak references:
http://buildbot.python.org/all/builders/AMD64%20Windows8.1%20Refleaks%203.x/builds/60/steps/test/logs/stdio

Example of leak:

$ ./python -m test -R 3:3 -u all test_idle -m idlelib.idle_test.test_configdialog.FontPageTest.test_set_samples
(...)
test_idle leaked [2168, 2168, 2168] references, sum=6504
test_idle leaked [1152, 1154, 1155] memory blocks, sum=3461
(...)

A bisection identified this commit:

commit 9397e2a87ed6e0e724ad71a0c751553620cb775e
Author: csabella <cheryl.sabella at gmail.com>
Date:   Sun Jul 30 13:34:25 2017 -0400

    bpo-31004: IDLE: Factor out FontPage class from configdialog (step 1) (#2905)
    
    The slightly modified tests continue to pass. The General test
    broken by the switch to Notebook is fixed.
    Patch mostly by Cheryl Sabella.

----------
messages: 299834
nosy: haypo
priority: normal
severity: normal
status: open
title: test_idle: idlelib.configdialog leaks references
type: resource usage
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31130>
_______________________________________

From report at bugs.python.org  Mon Aug  7 07:06:35 2017
From: report at bugs.python.org (Chris Jerdonek)
Date: Mon, 07 Aug 2017 11:06:35 +0000
Subject: [New-bugs-announce] [issue31131] asyncio.wait_for() TimeoutError
 doesn't provide full traceback
Message-ID: <1502103995.24.0.900898482206.issue31131@psf.upfronthosting.co.za>


New submission from Chris Jerdonek:

In Python 3.6.1, if asyncio.wait_for() times out with a TimeoutError, the traceback doesn't show what line the code was waiting on when the timeout occurred. This makes it more difficult to diagnose the cause of a timeout.

To reproduce, you can use the following code:

    import asyncio

    async def run():
        print('running...')
        await asyncio.sleep(1000000)

    def main(coro):
        loop = asyncio.new_event_loop()
        future = asyncio.wait_for(coro, timeout=1)
        loop.run_until_complete(future)

    main(run())

It gives the following output (notice that the sleep() line is missing):

    $ python test-timeouterror.py 
    running...
    Traceback (most recent call last):
      File "test-timeouterror.py", line 12, in <module>
        main(run())
      File "test-timeouterror.py", line 10, in main
        loop.run_until_complete(future)
      File "/Users/.../python3.6/asyncio/base_events.py", line 466,
         in run_until_complete
        return future.result()
      File "/Users/.../python3.6/asyncio/tasks.py", line 356, in wait_for
        raise futures.TimeoutError()
    concurrent.futures._base.TimeoutError

It seems like it should be possible to show the full traceback because, for example, if I register a signal handler with loop.add_signal_handler() as described here:
https://mail.python.org/pipermail/async-sig/2017-August/000374.html
and press Control-C before the timeout occurs, I do get a full traceback showing the line:

    await asyncio.sleep(1000000)

----------
components: Library (Lib), asyncio
messages: 299845
nosy: chris.jerdonek, yselivanov
priority: normal
severity: normal
status: open
title: asyncio.wait_for() TimeoutError doesn't provide full traceback
type: behavior
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31131>
_______________________________________

From report at bugs.python.org  Mon Aug  7 12:43:22 2017
From: report at bugs.python.org (Charalampos Stratakis)
Date: Mon, 07 Aug 2017 16:43:22 +0000
Subject: [New-bugs-announce] [issue31132] test_prlimit from test_resource
 fails when building python3 inside systemd-nspawn environment
Message-ID: <1502124202.05.0.637493609291.issue31132@psf.upfronthosting.co.za>


New submission from Charalampos Stratakis:

FAIL: test_prlimit (test.test_resource.ResourceTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/builddir/build/BUILD/Python-3.6.2/Lib/test/support/__init__.py", line 556, in wrapper
    return func(*args, **kw)
  File "/builddir/build/BUILD/Python-3.6.2/Lib/test/test_resource.py", line 153, in test_prlimit
    1, resource.RLIMIT_AS)
AssertionError: PermissionError not raised by prlimit


I observed this issue when mock [0] the underlying build system we use in Fedora started using the systemd-nspawn container technology instead of chroot [1][2] in order to create a minimal build environment.

[0] https://github.com/rpm-software-management/mock
[1] https://github.com/rpm-software-management/mock/wiki/Release-Notes-1.4.1
[2] https://www.freedesktop.org/software/systemd/man/systemd-nspawn.html

----------
messages: 299852
nosy: cstratak
priority: normal
severity: normal
status: open
title: test_prlimit from test_resource fails when building python3 inside systemd-nspawn environment
versions: Python 3.6, Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31132>
_______________________________________

From report at bugs.python.org  Mon Aug  7 13:25:50 2017
From: report at bugs.python.org (STINNER Victor)
Date: Mon, 07 Aug 2017 17:25:50 +0000
Subject: [New-bugs-announce] [issue31133] [2.7] PCbuild/pcbuild.sln cannot
 be open by Visual Studio 2010
Message-ID: <1502126750.5.0.768313426275.issue31133@psf.upfronthosting.co.za>


New submission from STINNER Victor:

The devguide says that Python 2.7 can be build using VS 2010 but it's wrong.

PCbuild/pcbuild.sln starts with:

Microsoft Visual Studio Solution File, Format Version 12.00

Where 12.00 means Visual Studio 2013.

Replacing 12.00 with 11.00 allows me to open the solution in Visual Studio 2010. So I suggest to make this change.

Note: By the way, why VS wants to build in 32-bit mode by default? It would be nice to switch to 64-bit mode by default!

----------
components: Build, Windows
messages: 299854
nosy: haypo, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: [2.7] PCbuild/pcbuild.sln cannot be open by Visual Studio 2010
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31133>
_______________________________________

From report at bugs.python.org  Mon Aug  7 16:46:32 2017
From: report at bugs.python.org (AW)
Date: Mon, 07 Aug 2017 20:46:32 +0000
Subject: [New-bugs-announce] [issue31134] python not added to path
Message-ID: <1502138792.87.0.0405556214712.issue31134@psf.upfronthosting.co.za>


New submission from AW:

I selected "Add Python 3.6 to PATH" when I installed the program. 

I am able to launch Python from the start button. 

It isn't recognized on the command line.

See screen shots in the attached Word file.

----------
files: PythonError.docx
messages: 299868
nosy: AW
priority: normal
severity: normal
status: open
title: python not added to path
type: behavior
Added file: http://bugs.python.org/file47060/PythonError.docx

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31134>
_______________________________________

From report at bugs.python.org  Mon Aug  7 20:46:39 2017
From: report at bugs.python.org (STINNER Victor)
Date: Tue, 08 Aug 2017 00:46:39 +0000
Subject: [New-bugs-announce] [issue31135] test_ttk_guionly doesn't destroy
 all widgets
Message-ID: <1502153199.22.0.664616149528.issue31135@psf.upfronthosting.co.za>


New submission from STINNER Victor:

While working on bpo-31068, I wrote attached widget_instance.patch to check if test_ttk_guionly really removes all widgets. Bad news: it doesn't. Example:

haypo at selma$ ./python -m test  -u all test_ttk_guionly
(...)
test test_ttk_guionly crashed -- <type 'exceptions.Exception'>: leaking 69 widgets
(...)

Attached fix_destroy.patch is my attempt to fix this issue.

----------
components: Tkinter
files: widget_instance.patch
keywords: patch
messages: 299880
nosy: haypo
priority: normal
severity: normal
status: open
title: test_ttk_guionly doesn't destroy all widgets
type: resource usage
versions: Python 2.7
Added file: http://bugs.python.org/file47061/widget_instance.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31135>
_______________________________________

From report at bugs.python.org  Mon Aug  7 20:50:49 2017
From: report at bugs.python.org (Gregory P. Smith)
Date: Tue, 08 Aug 2017 00:50:49 +0000
Subject: [New-bugs-announce] [issue31136] raw strings cannot end with a
 backslash character r'\'
Message-ID: <1502153449.52.0.763252325075.issue31136@psf.upfronthosting.co.za>


New submission from Gregory P. Smith:

A raw string literal cannot end in a backslash.  There is no friendly way to write a string that ends in a backslash character.

In particular I want to put the following into a Python string: \\?\

'\\\\?\\' works but is escaping hell so I wanted to suggest to the author to use r'\\?\' but that leads to:
SyntaxError: EOL while scanning string literal

Tested in a random 3.7.0a0 build as well as older 2.7 and 3.x stable versions.

r'\' is the easiest way to reproduce this.  (which could be written using the same number of bytes as '\\'... the use case above where a string containing a lot of \s that also ends in a \ is where it matters more from a code beauty point of view)

Can we update the parser to allow this?

----------
components: Interpreter Core
messages: 299883
nosy: gregory.p.smith
priority: normal
severity: normal
status: open
title: raw strings cannot end with a backslash character r'\'
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31136>
_______________________________________

From report at bugs.python.org  Mon Aug  7 20:59:25 2017
From: report at bugs.python.org (Wilfredo Sanchez)
Date: Tue, 08 Aug 2017 00:59:25 +0000
Subject: [New-bugs-announce] [issue31137] Add a path attribute to
 NamedTemporaryFile
Message-ID: <1502153965.8.0.297181946686.issue31137@psf.upfronthosting.co.za>


New submission from Wilfredo Sanchez:

Add a path attribute to NamedTemporaryFile which provides a pathlib.Path object pointing to the file on disk.

----------
components: Library (Lib)
messages: 299884
nosy: wsanchez
priority: normal
severity: normal
status: open
title: Add a path attribute to NamedTemporaryFile
type: enhancement
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31137>
_______________________________________

From report at bugs.python.org  Mon Aug  7 21:19:17 2017
From: report at bugs.python.org (Wilfredo Sanchez)
Date: Tue, 08 Aug 2017 01:19:17 +0000
Subject: [New-bugs-announce] [issue31138] dir argument NamedTemporaryFile
 should accept a pathlib.Path
Message-ID: <1502155157.73.0.00743957621165.issue31138@psf.upfronthosting.co.za>


New submission from Wilfredo Sanchez:

Or, better, any object which conforms to the file system path protocol (__fspath__).

----------
components: Library (Lib)
messages: 299886
nosy: wsanchez
priority: normal
severity: normal
status: open
title: dir argument NamedTemporaryFile should accept a pathlib.Path
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31138>
_______________________________________

From report at bugs.python.org  Mon Aug  7 22:26:08 2017
From: report at bugs.python.org (Louie Lu)
Date: Tue, 08 Aug 2017 02:26:08 +0000
Subject: [New-bugs-announce] [issue31139] Improve tracemalloc demo code in
 docs
Message-ID: <1502159168.93.0.181996633352.issue31139@psf.upfronthosting.co.za>


New submission from Louie Lu:

Current tutorial of tracemalloc:
https://docs.python.org/3/library/tracemalloc.html

Current doc using a un-reproducible output, that will confuse the reader. Propose to use the test code `allocate_bytes` to control the memory usage output, provide more meaningful output to reader.

----------
assignee: docs at python
components: Documentation
messages: 299892
nosy: docs at python, haypo, louielu
priority: normal
severity: normal
status: open
title: Improve tracemalloc demo code in docs
type: enhancement
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31139>
_______________________________________

From report at bugs.python.org  Tue Aug  8 04:55:58 2017
From: report at bugs.python.org (chris.259263)
Date: Tue, 08 Aug 2017 08:55:58 +0000
Subject: [New-bugs-announce] [issue31140] Insufficient error message with
 incorrect formated string literal
Message-ID: <1502182558.08.0.700048111188.issue31140@psf.upfronthosting.co.za>


New submission from chris.259263:

Trying to run a script with a bug in a formated string literal (in the example the ":" is missing) throws an "invalid syntax" error at the first expression of the script.

Example:

import math
a = "a"
b = f"{a>2s}"   # forgotten ":", should be f"{a:>2s}"

Running this script throws an "invalid syntax" error in the first line "import math".

In a large program this behavior makes it nearly impossible to find the real bug in the program as the error message does not give any hint about the line or the type of the error in the program.

Using Python 3.6.2, IDLE 3.6.2 on Mac macOS Sierra (10.12.1)

----------
messages: 299901
nosy: chris.259263
priority: normal
severity: normal
status: open
title: Insufficient error message with incorrect formated string literal
type: behavior
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31140>
_______________________________________

From report at bugs.python.org  Tue Aug  8 07:52:48 2017
From: report at bugs.python.org (Mark Bell)
Date: Tue, 08 Aug 2017 11:52:48 +0000
Subject: [New-bugs-announce] [issue31141] Start should be a keyword argument
 of the built-in sum
Message-ID: <1502193168.16.0.095124886528.issue31141@psf.upfronthosting.co.za>


New submission from Mark Bell:

The built-in function sum takes an optional argument "start" to specify what value to start adding from (defaults to 0). This argument should be a keyword argument in order to match the other built-in functions such as:

    enumerate(range(10), start=5)

This patch allows users to write:

    sum(range(10), start=5)

which previously raised "TypeError: sum() takes no keyword arguments". Since the only change is making an optional positional argument into a keyword argument, this has no effect on any existing code using the current convention of:

    sum(range(10), 5)

----------
components: ctypes
messages: 299908
nosy: Mark.Bell
priority: normal
severity: normal
status: open
title: Start should be a keyword argument of the built-in sum
versions: Python 2.7, Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31141>
_______________________________________

From report at bugs.python.org  Tue Aug  8 08:09:52 2017
From: report at bugs.python.org (py78py90py)
Date: Tue, 08 Aug 2017 12:09:52 +0000
Subject: [New-bugs-announce] [issue31142] python shell crashed while input
 quotes in print()
Message-ID: <1502194192.96.0.619398073227.issue31142@psf.upfronthosting.co.za>


New submission from py78py90py:

I have downloaded python 3.6 into MacOSX system.
Python 3.6.2 (v3.6.2:5fd33b5926, Jul 16 2017, 20:11:06) 
[GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] on darwin

Then I opened the IDLE and try to input some sentences.
I inputed "print()", then input single quote in print.
But the IDLE crashed.

----------
components: Demos and Tools
messages: 299909
nosy: py78py90py
priority: normal
severity: normal
status: open
title: python shell crashed while input quotes in print()
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31142>
_______________________________________

From report at bugs.python.org  Tue Aug  8 09:43:40 2017
From: report at bugs.python.org (Todd Schiller)
Date: Tue, 08 Aug 2017 13:43:40 +0000
Subject: [New-bugs-announce] [issue31143] lib2to3 requires source files for
 fixes
Message-ID: <1502199820.45.0.266789530898.issue31143@psf.upfronthosting.co.za>


New submission from Todd Schiller:

The lib2to3 library doesn't detect/apply any fixes if the source files aren't included with the distribution. See get_all_fix_names at https://github.com/python/cpython/blob/master/Lib/lib2to3/refactor.py#L30

This affects Azure's official Python site extensions [1], which only include bytecode pyc's for the fixes [2].

This results in bad installs when using setuptools to install packages that leverage the use_2to3 flag during the install [3]

[1] https://github.com/Azure/azure-python-siteextensions
[2] https://github.com/Azure/azure-python-siteextensions/issues/14
[3] https://github.com/pypa/setuptools/issues/1120#issuecomment-320769664

----------
components: 2to3 (2.x to 3.x conversion tool)
messages: 299920
nosy: tschiller
priority: normal
severity: normal
status: open
title: lib2to3 requires source files for fixes
type: behavior

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31143>
_______________________________________

From report at bugs.python.org  Tue Aug  8 10:02:29 2017
From: report at bugs.python.org (nvdv)
Date: Tue, 08 Aug 2017 14:02:29 +0000
Subject: [New-bugs-announce] [issue31144] add initializer to
 concurrent.futures.ProcessPoolExecutor
Message-ID: <1502200949.97.0.102357103627.issue31144@psf.upfronthosting.co.za>


New submission from nvdv:

Unfortunately concurrent.futures.ProcessPoolExecutor does not provide any means to interact with worker processes (like initializer function in multiprocessing.Pool constructor).

This problem has been mentioned at least once: 
https://mail.python.org/pipermail/python-dev/2014-March/133697.html.

It's not hard to work around this issue by subclassing ProcessPoolExecutor and adding all necessary methods, but IMO solution can be more generic.

----------
components: Library (Lib)
messages: 299921
nosy: nvdv
priority: normal
severity: normal
status: open
title: add initializer to concurrent.futures.ProcessPoolExecutor
type: enhancement
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31144>
_______________________________________

From report at bugs.python.org  Tue Aug  8 10:09:13 2017
From: report at bugs.python.org (=?utf-8?q?Miko=C5=82aj_Babiak?=)
Date: Tue, 08 Aug 2017 14:09:13 +0000
Subject: [New-bugs-announce] [issue31145] PriorityQueue.put() fails with
 TypeError if priority_number in tuples of (priority_number,
 data) are the same.
Message-ID: <1502201353.41.0.0622825133632.issue31145@psf.upfronthosting.co.za>


New submission from Miko?aj Babiak:

# list of tuples in form of (priority_number, data)
bugged = [
(-25.691, {'feedback': 13, 'sentiment': 0.309, 'support_ticket': 5}), (-25.691, {'feedback': 11, 'sentiment': 0.309, 'support_ticket': 3}), (-25.0, {'feedback': 23, 'sentiment': 0.0, 'support_ticket': 15}),
]

from queue import PriorityQueue

pq = PriorityQueue()

for item in bugged:
   pq.put(item)

# TypeError: '<' not supported between instances of 'dict' and 'dict'

It seems that if priority_numbers are equal, heapq.heapify() falls back to comparing data element from tuple (priority_number, data).
I belive this is an undesired behaviour.
It is acctually listed as one of implementation challenges on https://docs.python.org/3/library/heapq.html:
"Tuple comparison breaks for (priority, task) pairs if the priorities are equal and the tasks do not have a default comparison order."

In python 2.7 the issue in not present and PriorityQueue.put() works as expected

----------
components: Library (Lib)
messages: 299922
nosy: Miko?aj Babiak
priority: normal
severity: normal
status: open
title: PriorityQueue.put() fails with TypeError if priority_number in tuples of (priority_number, data) are the same.
type: behavior
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31145>
_______________________________________

From report at bugs.python.org  Tue Aug  8 11:29:40 2017
From: report at bugs.python.org (Julien Palard)
Date: Tue, 08 Aug 2017 15:29:40 +0000
Subject: [New-bugs-announce] [issue31146] Docs: On non-public translations,
 language picker fallback to "English"
Message-ID: <1502206180.29.0.859774070814.issue31146@psf.upfronthosting.co.za>


New submission from Julien Palard:

As, by default, "english" is selected, and as we're building translations far before rendering them public by adding them to the language switcher, there's a timespan where a human can voluntarily land on a translation with a language switcher unaware of this language.

The language switcher currently defaults to english, so english is selected. As "english" is already selected, one can't select "english" to navigate to the english version.

Solutions may be to just hide the select element on unknown translations, or display the unknown language tag. With more work and a "predicted" list of tags-names, it may be possible to also display the name of the language.


Code is here: https://github.com/python/cpython/blob/master/Doc/tools/static/switchers.js

I'll take a look, but if some want to do it, let me know.

----------
assignee: docs at python
components: Documentation
messages: 299932
nosy: docs at python, mdk
priority: normal
severity: normal
status: open
title: Docs: On non-public translations, language picker fallback to "English"
type: enhancement
versions: Python 2.7, Python 3.6, Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31146>
_______________________________________

From report at bugs.python.org  Tue Aug  8 11:30:36 2017
From: report at bugs.python.org (Oren Milman)
Date: Tue, 08 Aug 2017 15:30:36 +0000
Subject: [New-bugs-announce] [issue31147] a mostly useless check in
 list_extend()
Message-ID: <1502206236.15.0.974518538682.issue31147@psf.upfronthosting.co.za>


New submission from Oren Milman:

in listobject.c, in case list_extend() receives an 'iterable' which isn't 'self'
nor a tuple nor a list, we have the following (heavily edited for brevity):
    mn = Py_SIZE(self) + PyObject_LengthHint(iterable);
    list_resize(self, mn);

    ... // self is extended to also include the elements of 'iterable'.

    // (*)
    if (Py_SIZE(self) < self->allocated) {
        list_resize(self, Py_SIZE(self));
    }

IMHO, the condition in (*) is mostly useless, for two reasons:
1. the list_resize() which is called in (*) does nothing in case
   (Py_SIZE(self) >= (self->allocated >> 1)) is true. In particular, this call
   to list_resize() would have done nothing if it had been called while the
   condition in (*) was false.
2. the condition in (*) is false only in the following cases:
    - list_resize(self, mn) caused
      (self->allocated == Py_SIZE(self) + actual_length_of_iterable) to be
      true. e.g.:
          Py_SIZE(self) = 58 and PyObject_LengthHint(iterable) == 8 and
          actual_length_of_iterable == 22
          (because 66 + 66 // 8 + 6 == 80 == 58 + 22).
    - list_resize(self, mn) caused
      (self->allocated < Py_SIZE(self) + actual_length_of_iterable), which
      sometime later caused list_extend() to call app1(), which called
      list_resize(), which caused
      (self->allocated == Py_SIZE(self) + actual_length_of_iterable) to be true.
      e.g.:
          Py_SIZE(self) == 58 and PyObject_LengthHint(iterable) == 8 and
          actual_length_of_iterable == 39
          (because 66 + 66 // 8 + 6 == 80 and
                   81 + 81 // 8 + 6 == 97 == 58 + 39)

   so ISTM that the condition is only rarely false, especially as
   PyObject_LengthHint(iterable) >= actual_length_of_iterable is usually true
   (i guess).


Thus, i believe we should either change the condition in (*) to
(Py_SIZE(self) < (self->allocated >> 1)), or remove it (and always call
list_resize()).


(note that i ignored error cases, as ISTM they are quite irrelevant to the
issue.)

----------
components: Interpreter Core
messages: 299933
nosy: Oren Milman
priority: normal
severity: normal
status: open
title: a mostly useless check in list_extend()
type: performance
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31147>
_______________________________________

From report at bugs.python.org  Tue Aug  8 12:38:31 2017
From: report at bugs.python.org (D Gentry)
Date: Tue, 08 Aug 2017 16:38:31 +0000
Subject: [New-bugs-announce] [issue31148] Can we get an MSI installer for
 something past 3.4.4?
Message-ID: <1502210311.11.0.464984034279.issue31148@psf.upfronthosting.co.za>


New submission from D Gentry:

I've noticed that the only installers that are able to successfully complete on my windows 7 x64 system are the MSI installers.

Unfortunately, python.org hasn't released an MSI version since 3.4.4 it seems.

Is it possible to get an MSI installer for one of the more recent versions?

----------
files: Python 3.6.2 (64-bit)_20170808103648.log
messages: 299936
nosy: D Gentry
priority: normal
severity: normal
status: open
title: Can we get an MSI installer for something past 3.4.4?
type: crash
versions: Python 3.5, Python 3.6, Python 3.7
Added file: http://bugs.python.org/file47066/Python 3.6.2 (64-bit)_20170808103648.log

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31148>
_______________________________________

From report at bugs.python.org  Tue Aug  8 16:58:44 2017
From: report at bugs.python.org (Julien Palard)
Date: Tue, 08 Aug 2017 20:58:44 +0000
Subject: [New-bugs-announce] [issue31149] Add Japanese to the language
 switcher
Message-ID: <1502225924.72.0.833937750759.issue31149@psf.upfronthosting.co.za>


New submission from Julien Palard:

Japanese translation have reached the prerequisites from PEP 545, their last step is to be added to the language switcher.

Current build has been reviewed by Inada and is OK: https://docs.python.org/ja/3.7/

Note: On the current build, the /ja/ is not detected by the language switcher, as this *is* the step to add /ja/ to this same language switcher, this is the issue http://bugs.python.org/issue31146

----------
messages: 299947
nosy: inada.naoki, mdk, ned.deily
priority: normal
severity: normal
status: open
title: Add Japanese to the language switcher

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31149>
_______________________________________

From report at bugs.python.org  Tue Aug  8 20:35:38 2017
From: report at bugs.python.org (STINNER Victor)
Date: Wed, 09 Aug 2017 00:35:38 +0000
Subject: [New-bugs-announce] [issue31150] [EASY] test_thread: Warning --
 reap_children() reaped child process
Message-ID: <1502238938.02.0.869596216729.issue31150@psf.upfronthosting.co.za>


New submission from STINNER Victor:

It seems like an unit test of test_thread leaks a process. A bisection found the following 2 tests are enough to get the warning:

* test.test_thread.TestForkInThread.test_forkinthread
* test.test_thread.LockTests.test_acquire_contended

I bet that test_forkinthread() doesn't wait correctly for the child process (fork) completion using os.waitpid() or something like that.



Example 1 (from Refleak buildbot):

haypo at selma$ ./python -m test -R 3:3 test_thread
Run tests sequentially
0:00:00 load avg: 0.55 [1/1] test_thread
beginning 6 repetitions
123456
Warning -- reap_children() reaped child process 6984
.Warning -- reap_children() reaped child process 7054
.Warning -- reap_children() reaped child process 7125
.Warning -- reap_children() reaped child process 7195
.Warning -- reap_children() reaped child process 7266
.Warning -- reap_children() reaped child process 7336
.
1 test OK.

Total duration: 5 sec
Tests result: SUCCESS


Example 2 (only run the 2 methods):

haypo at selma$ ./python -m test -m test.test_thread.TestForkInThread.test_forkinthread -m test.test_thread.LockTests.test_acquire_contended -v test_thread test_thread 

0:00:00 load avg: 0.27 [1/2] test_thread
test_acquire_contended (test.test_thread.LockTests) ... ok
test_forkinthread (test.test_thread.TestForkInThread) ... ok

----------------------------------------------------------------------
Ran 2 tests in 0.037s

OK
0:00:00 load avg: 0.27 [2/2] test_thread
test_acquire_contended (test.test_thread.LockTests) ... Warning -- reap_children() reaped child process 10613
ok
test_forkinthread (test.test_thread.TestForkInThread) ... ok

----------------------------------------------------------------------
Ran 2 tests in 0.036s

OK
test_thread failed (env changed)

----------
keywords: easy
messages: 299958
nosy: haypo
priority: normal
severity: normal
status: open
title: [EASY] test_thread: Warning -- reap_children() reaped child process
versions: Python 3.6, Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31150>
_______________________________________

From report at bugs.python.org  Tue Aug  8 20:40:39 2017
From: report at bugs.python.org (STINNER Victor)
Date: Wed, 09 Aug 2017 00:40:39 +0000
Subject: [New-bugs-announce] [issue31151] [3.6] test_socketserver: Warning
 -- reap_children() reaped child process
Message-ID: <1502239239.94.0.0719119279673.issue31151@psf.upfronthosting.co.za>


New submission from STINNER Victor:

It seems like test_socketserver leaks child processes on the "x86 Gentoo Refleaks 3.6" buildbot, but I'm unable to reproduce the bug.

http://buildbot.python.org/all/builders/x86%20Gentoo%20Refleaks%203.6/builds/49/steps/test/logs/stdio

0:48:11 load avg: 3.91 [154/405/2] test_socketserver failed -- running: test_decimal (630 sec)
Warning -- reap_children() reaped child process 1044
beginning 6 repetitions
123456
Warning -- reap_children() reaped child process 1104
Warning -- reap_children() reaped child process 1115
.Warning -- reap_children() reaped child process 1170
Warning -- reap_children() reaped child process 1175
Warning -- reap_children() reaped child process 1184
.Warning -- reap_children() reaped child process 1249
.Warning -- reap_children() reaped child process 1311
Warning -- reap_children() reaped child process 1316
...
test_socketserver leaked [1, 1, 1] memory blocks, sum=3

(...)

test_ForkingUnixStreamServer (test.test_socketserver.SocketServerTest) ... creating server
ADDR = /tmp/unix_socket.hqh5x95a
CLASS = <class 'test.test_socketserver.ForkingUnixStreamServer'>
server running
test client 0
test client 1
test client 2
waiting for server
done
Warning -- reap_children() reaped child process 17938
ok

test_ForkingUnixDatagramServer (test.test_socketserver.SocketServerTest) ... creating server
ADDR = /tmp/unix_socket.gry6ulhp
CLASS = <class 'test.test_socketserver.ForkingUnixDatagramServer'>
server running
test client 0
test client 1
test client 2
waiting for server
done
Warning -- reap_children() reaped child process 18212
ok

test_ForkingUDPServer (test.test_socketserver.SocketServerTest) ... creating server
ADDR = ('127.0.0.1', 43415)
CLASS = <class 'socketserver.ForkingUDPServer'>
server running
test client 0
test client 1
test client 2
waiting for server
done
Warning -- reap_children() reaped child process 18281
ok
test_ForkingUnixDatagramServer (test.test_socketserver.SocketServerTest)

----------
components: Tests
messages: 299959
nosy: haypo
priority: normal
severity: normal
status: open
title: [3.6] test_socketserver: Warning -- reap_children() reaped child process
type: resource usage
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31151>
_______________________________________

From report at bugs.python.org  Tue Aug  8 22:37:55 2017
From: report at bugs.python.org (Lorem Ipsum)
Date: Wed, 09 Aug 2017 02:37:55 +0000
Subject: [New-bugs-announce] [issue31152] Tutorial wording implies an
 understanding of a concept prior to it being introduced
Message-ID: <1502246275.38.0.890057309211.issue31152@psf.upfronthosting.co.za>


New submission from Lorem Ipsum:

Python 3.5.4 Tutorial Section 8.5. User-defined Exceptions Paragraph 2 (https://docs.python.org/3.5/tutorial/errors.html#user-defined-exceptions) states [emphasis mine]:

"When creating a module that can raise several distinct errors, a common practice is to create a base class for exceptions defined by that module, and SUBCLASS THAT TO create specific exception classes for different error conditions:"

The use of 'subclass' as a verb when it has not been used so prior is confusing, especially to beginners.  The concept of a class is not formally covered until Section 9 and up until this point in the tutorial, 'class' has been used as a noun.  When read with 'subclass' as a noun, the sentence is nonsensical. It may also be that the comma which precedes 'and' is not proper usage. 

Suggested improvement: change "subclass that to create specific classes for..." to "then create specific exception subclasses for..."

"When creating a module that can raise several distinct errors, a common practice is to create a base class for exceptions defined by that module, and then create specific exception subclasses for different error conditions:"

----------
assignee: docs at python
components: Documentation
messages: 299962
nosy: Lorem Ipsum, docs at python
priority: normal
severity: normal
status: open
title: Tutorial wording implies an understanding of a concept prior to it being introduced
type: enhancement
versions: Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31152>
_______________________________________

From report at bugs.python.org  Wed Aug  9 03:42:39 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Wed, 09 Aug 2017 07:42:39 +0000
Subject: [New-bugs-announce] [issue31153] Update docstrings of itertools
 function
Message-ID: <1502264559.87.0.404253644107.issue31153@psf.upfronthosting.co.za>


New submission from Serhiy Storchaka:

Docstrings of some itertools functions look outdated.

1. The docstring of groupby() contains the signature "groupby(iterable[, keyfunc])". But groupby() supports keyword arguments, the name of the second parameter is "key", and its default value is None.

2. accumulate() accepts None as the second argument.

3. The equivalent code of count() has the first parameter "firstval" instead of "start". It uses the "while 1" loop instead of more idiomatic in Python 3 "while True".

4. The term "sequence" is used in the docstring of starmap(), while actually an iterable is accepted. Compare with the docstring of map(). See also issue31082.

----------
assignee: docs at python
components: Documentation
keywords: easy
messages: 299975
nosy: docs at python, rhettinger, serhiy.storchaka
priority: normal
severity: normal
status: open
title: Update docstrings of itertools function
type: enhancement
versions: Python 3.6, Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31153>
_______________________________________

From report at bugs.python.org  Wed Aug  9 04:07:01 2017
From: report at bugs.python.org (Alberto Gomez-Casado)
Date: Wed, 09 Aug 2017 08:07:01 +0000
Subject: [New-bugs-announce] [issue31154] Path.replace and Path.rename
 naming, behavior and documentation
Message-ID: <1502266021.69.0.915795011515.issue31154@psf.upfronthosting.co.za>


New submission from Alberto Gomez-Casado:

I find the naming of Path.replace quite unfortunate, coming from the os.path and basestring mindset, the expectation would be that it would change some substring by another substring in the Path instance. Instead, it _moves_ the file indicated by the Path object in the file system. Most of the methods of Path have no effect in the filesystem, and we have here one with a slightly misleading name which goes and moves your files around.

Practically the same operation is done with Path.rename, which is less surprising since it matches popular OS commands/UIs (ren, Rename).

IMHO Path.replace should not exist as is and a keyword modifier (silent?) for Path.rename toggles between the minor behavior differences of the two. 

Additionally:

Both calls lack any kind of return, which leads me to expect after a rename/replace the Path instance would be changed to the new path. This is not the case, after the call the Path instance keeps "pointing" to the previous (and quite likely now useless) path. Returning the new path would be another reasonable option. In any case, I think the documentation should mention this behavior explicitly. 

And, while we are at it, having implemented the relatively dangerous replace/rename (along with convenience shorcuts for reading, writing, opening...), why was a Path.copy not included?

----------
assignee: docs at python
components: Documentation, Library (Lib)
messages: 299976
nosy: Alberto Gomez-Casado, docs at python
priority: normal
severity: normal
status: open
title: Path.replace and Path.rename naming, behavior and documentation
type: behavior
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31154>
_______________________________________

From report at bugs.python.org  Wed Aug  9 05:23:16 2017
From: report at bugs.python.org (Javen O'Neal)
Date: Wed, 09 Aug 2017 09:23:16 +0000
Subject: [New-bugs-announce] [issue31155] Encode set, frozenset, bytearray,
 and iterators as json arrays
Message-ID: <1502270596.69.0.819927554445.issue31155@psf.upfronthosting.co.za>


New submission from Javen O'Neal:

Currently json.dump, json.dumps, and json.JSONEncoder do not support writing iterator objects. Users can choose to pass the iterator to a tuple and write that as a json array, but this is wasteful if the iterator is large, requiring a large amount of memory to be temporarily allocated for the tuple prior to json encoding.

The json module also does not support writing sets, frozensets, or bytearrays.

----------
components: Library (Lib)
messages: 299981
nosy: javenoneal
priority: normal
pull_requests: 3069
severity: normal
status: open
title: Encode set, frozenset, bytearray, and iterators as json arrays
type: enhancement
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31155>
_______________________________________

From report at bugs.python.org  Wed Aug  9 05:49:19 2017
From: report at bugs.python.org (=?utf-8?q?G=C3=BCnter_Rote?=)
Date: Wed, 09 Aug 2017 09:49:19 +0000
Subject: [New-bugs-announce] [issue31156] Stopiteration terminates while-loop
Message-ID: <1502272159.56.0.437301068833.issue31156@psf.upfronthosting.co.za>


New submission from G?nter Rote:

It should be mentioned in the documentation that 

A StopIteration exception raised in the body of a while loop will terminate (and is caught by) the while-loop, thus leading to graceful termination.

A good place would be here:
1) https://docs.python.org/3/reference/compound_stmts.html#the-while-statement

I don't know how such a StopIteration termination of a while loop
affects the else-clause. This should be clarified.

Here:
2) https://docs.python.org/3/library/exceptions.html#StopIteration
it would be good to explicitly state:

An enclosing while-loop or for-loop acts like an implicit catch for StopIteration. The StopIteration exception will terminate the loop.

(I guess, a for-loop is also just terminated when the StopIteration originates in the BODY of the loop, although this is not the typical case.)

----------
assignee: docs at python
components: Documentation
messages: 299982
nosy: G?nter Rote, docs at python
priority: normal
severity: normal
status: open
title: Stopiteration terminates while-loop
type: enhancement
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31156>
_______________________________________

From report at bugs.python.org  Wed Aug  9 05:54:19 2017
From: report at bugs.python.org (Mick Press)
Date: Wed, 09 Aug 2017 09:54:19 +0000
Subject: [New-bugs-announce] [issue31157] math.sqrt function wrong
Message-ID: <1502272459.7.0.676857812876.issue31157@psf.upfronthosting.co.za>


New submission from Mick Press:

math.sqrt for very large numbers returns wrong value. Attached is screen shot showing my sqrt function result and math.sqrt function result.

----------
components: Windows
files: PyCuda - [C__Users_mick_PyScripts_PyCuda] - C__Users_mick_.PyCharmCE2017.2_config_scratches_scratch_5.py - PyCharm Community Edition 2017.2 09_08_2017 10_44_31.png
messages: 299983
nosy: Mick Press, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: math.sqrt function wrong
type: performance
versions: Python 3.5
Added file: http://bugs.python.org/file47067/PyCuda - [C__Users_mick_PyScripts_PyCuda] - C__Users_mick_.PyCharmCE2017.2_config_scratches_scratch_5.py - PyCharm Community Edition 2017.2 09_08_2017 10_44_31.png

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31157>
_______________________________________

From report at bugs.python.org  Wed Aug  9 06:52:13 2017
From: report at bugs.python.org (STINNER Victor)
Date: Wed, 09 Aug 2017 10:52:13 +0000
Subject: [New-bugs-announce] [issue31158] test_pty: test_basic() fails
 randomly on Travis CI
Message-ID: <1502275933.96.0.349321322315.issue31158@psf.upfronthosting.co.za>


New submission from STINNER Victor:

https://travis-ci.org/python/cpython/jobs/262606831

0:13:26 load avg: 134.81 [256/403] test_pty
test test_pty failed -- Traceback (most recent call last):
  File "/home/travis/build/python/cpython/Lib/test/test_pty.py", line 109, in test_basic
    self.assertEqual(b'For my pet fish, Eric.\n', normalize_output(s2))
AssertionError: b'For my pet fish, Eric.\n' != b'For m'

----------
components: Tests
messages: 299987
nosy: haypo
priority: normal
severity: normal
status: open
title: test_pty: test_basic() fails randomly on Travis CI
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31158>
_______________________________________

From report at bugs.python.org  Wed Aug  9 07:38:46 2017
From: report at bugs.python.org (Julien Palard)
Date: Wed, 09 Aug 2017 11:38:46 +0000
Subject: [New-bugs-announce] [issue31159] Doc: Language switch can't switch
 on specific cases
Message-ID: <1502278726.47.0.617689885912.issue31159@psf.upfronthosting.co.za>


New submission from Julien Palard:

Since ja has been added to the language switch, a bug appeared and in some case the switch won't ... switch.

Due to a regex bug in switchers.js, a needed trailing slash is sometimes not matched. It has not been detected previously as it can happen only when there more than two languages in the picker which is now the case.

Bug is in:  var language_regexp = '\.org/(' + Object.keys(all_languages).join('|') + '/)';

It only matches the trailing slash for the last language, currently the / is for ja, so fr does no longer match the needed slash.

Fix may consists in adding a non-matching group around the join.

But there's another issue when a user goes to a version in which there's a language picker but the current langage is not known by the picker, typically if a user is on /ja/2.7/ but the language picker with Japanese has only been merged on 3.6 and 3.7: the regex on /ja/2.7 will not contain "ja", so it will not be able to match it.


So we should write this regex otherwise, I grepped on the server logs and checked on the filesystem, this one should work better and still be safe:


    var language_regexp = '\.org/([a-z]{2}(?:-[a-z]{2})?/)';

----------
messages: 299990
nosy: mdk
priority: normal
severity: normal
status: open
title: Doc: Language switch can't switch on specific cases

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31159>
_______________________________________

From report at bugs.python.org  Wed Aug  9 08:50:03 2017
From: report at bugs.python.org (STINNER Victor)
Date: Wed, 09 Aug 2017 12:50:03 +0000
Subject: [New-bugs-announce] [issue31160] Enhance support.reap_children()
Message-ID: <1502283003.36.0.514915057261.issue31160@psf.upfronthosting.co.za>


New submission from STINNER Victor:

Attached PR enhances the support.reap_children() function:

* reap_children() now sets environment_altered to True to detect bugs using python3 -m test --fail-env-changed
* Replace bare "except:" with "except OSError:" in reap_children()
* Write an unit test for reap_children() using a timeout of 60 seconds

----------
messages: 299996
nosy: haypo
priority: normal
pull_requests: 3073
severity: normal
status: open
title: Enhance support.reap_children()

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31160>
_______________________________________

From report at bugs.python.org  Wed Aug  9 09:31:18 2017
From: report at bugs.python.org (Martijn Pieters)
Date: Wed, 09 Aug 2017 13:31:18 +0000
Subject: [New-bugs-announce] [issue31161] Only check for print and exec
 parentheses cases for SyntaxError, not subclasses
Message-ID: <1502285478.81.0.436457469881.issue31161@psf.upfronthosting.co.za>


New submission from Martijn Pieters:

SyntaxError.__init__() checks for the `print` and `exec` error cases where the user forgot to use parentheses:

>>> exec 1
  File "<stdin>", line 1
    exec 1
         ^
SyntaxError: Missing parentheses in call to 'exec'

>>> print 1
  File "<stdin>", line 1
    print 1
          ^
SyntaxError: Missing parentheses in call to 'print'

However, this check is also applied to *subclasses* of SyntaxError:

>>> if True:
... print "Look ma, no parens!"
  File "<stdin>", line 2
    print "Look ma, no parens!"
        ^
IndentationError: Missing parentheses in call to 'print'

and

>>> compile('if 1:\n    1\n\tprint "Look ma, tabs!"', '', 'single')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "", line 3
    print "Look ma, tabs!"
                         ^
TabError: Missing parentheses in call to 'print'

Perhaps the check needs to be limited to just the exact type.

----------
messages: 300002
nosy: mjpieters
priority: normal
severity: normal
status: open
title: Only check for print and exec parentheses cases for SyntaxError, not subclasses

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31161>
_______________________________________

From report at bugs.python.org  Wed Aug  9 10:49:18 2017
From: report at bugs.python.org (CHUA Chew Bock)
Date: Wed, 09 Aug 2017 14:49:18 +0000
Subject: [New-bugs-announce] [issue31162] urllib.request.urlopen error
Message-ID: <1502290158.78.0.927502362928.issue31162@psf.upfronthosting.co.za>


New submission from CHUA Chew Bock:

Error encountered when executing urllib.request.urlopen on IDLE3 interactive as well as run from script.

----------
components: Library (Lib)
messages: 300012
nosy: chua.chewbock at gmail.com
priority: normal
severity: normal
status: open
title: urllib.request.urlopen error
type: behavior
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31162>
_______________________________________

From report at bugs.python.org  Wed Aug  9 11:32:24 2017
From: report at bugs.python.org (Alberto Gomez-Casado)
Date: Wed, 09 Aug 2017 15:32:24 +0000
Subject: [New-bugs-announce] [issue31163] Return destination path in
 Path.rename and Path.replace
Message-ID: <1502292744.48.0.627996317333.issue31163@psf.upfronthosting.co.za>


New submission from Alberto Gomez-Casado:

Extracted from issue31154

Both calls lack any kind of return, which leads me to expect after a rename/replace the Path instance would be changed to the new path. This is not the case (reading the PEP I have seen Path instances are kind of immutable), after the call the Path instance keeps "pointing" to the previous (and quite likely now useless) path. 

Returning the new path would be a reasonable option. In any case, I think the documentation should mention this behavior explicitly.

----------
assignee: docs at python
components: Documentation, Library (Lib)
messages: 300019
nosy: Alberto Gomez-Casado, docs at python, pitrou
priority: normal
severity: normal
status: open
title: Return destination path in Path.rename and Path.replace
type: enhancement
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31163>
_______________________________________

From report at bugs.python.org  Wed Aug  9 11:53:43 2017
From: report at bugs.python.org (STINNER Victor)
Date: Wed, 09 Aug 2017 15:53:43 +0000
Subject: [New-bugs-announce] [issue31164] test_functools:
 test_recursive_pickle() stack overflow on x86 Gentoo Refleaks 3.x
Message-ID: <1502294023.01.0.985500913327.issue31164@psf.upfronthosting.co.za>


New submission from STINNER Victor:

http://buildbot.python.org/all/builders/x86%20Gentoo%20Refleaks%203.x/builds/51/steps/test/logs/stdio

0:17:12 load avg: 2.97 [ 52/406/1] test_functools crashed (Exit code -6) -- running: test_nntplib (406 sec)
beginning 6 repetitions
123456
Fatal Python error: Cannot recover from stack overflow.

Current thread 0xb74f1700 (most recent call first):
  File "/buildbot/buildarea/3.x.ware-gentoo-x86.refleak/build/Lib/test/test_functools.py", line 336 in test_recursive_pickle
  File "/buildbot/buildarea/3.x.ware-gentoo-x86.refleak/build/Lib/unittest/case.py", line 615 in run
  File "/buildbot/buildarea/3.x.ware-gentoo-x86.refleak/build/Lib/unittest/case.py", line 663 in __call__
  File "/buildbot/buildarea/3.x.ware-gentoo-x86.refleak/build/Lib/unittest/suite.py", line 122 in run
  File "/buildbot/buildarea/3.x.ware-gentoo-x86.refleak/build/Lib/unittest/suite.py", line 84 in __call__
  File "/buildbot/buildarea/3.x.ware-gentoo-x86.refleak/build/Lib/unittest/suite.py", line 122 in run
  File "/buildbot/buildarea/3.x.ware-gentoo-x86.refleak/build/Lib/unittest/suite.py", line 84 in __call__
  File "/buildbot/buildarea/3.x.ware-gentoo-x86.refleak/build/Lib/unittest/suite.py", line 122 in run
  File "/buildbot/buildarea/3.x.ware-gentoo-x86.refleak/build/Lib/unittest/suite.py", line 84 in __call__
  File "/buildbot/buildarea/3.x.ware-gentoo-x86.refleak/build/Lib/test/support/__init__.py", line 1772 in run
  File "/buildbot/buildarea/3.x.ware-gentoo-x86.refleak/build/Lib/test/support/__init__.py", line 1896 in _run_suite
  File "/buildbot/buildarea/3.x.ware-gentoo-x86.refleak/build/Lib/test/support/__init__.py", line 1940 in run_unittest
  File "/buildbot/buildarea/3.x.ware-gentoo-x86.refleak/build/Lib/test/libregrtest/runtest.py", line 171 in test_runner
  File "/buildbot/buildarea/3.x.ware-gentoo-x86.refleak/build/Lib/test/libregrtest/refleak.py", line 54 in dash_R
  File "/buildbot/buildarea/3.x.ware-gentoo-x86.refleak/build/Lib/test/libregrtest/runtest.py", line 174 in runtest_inner
  File "/buildbot/buildarea/3.x.ware-gentoo-x86.refleak/build/Lib/test/libregrtest/runtest.py", line 140 in runtest
  File "/buildbot/buildarea/3.x.ware-gentoo-x86.refleak/build/Lib/test/libregrtest/runtest_mp.py", line 71 in run_tests_slave
  File "/buildbot/buildarea/3.x.ware-gentoo-x86.refleak/build/Lib/test/libregrtest/main.py", line 519 in _main
  File "/buildbot/buildarea/3.x.ware-gentoo-x86.refleak/build/Lib/test/libregrtest/main.py", line 512 in main
  File "/buildbot/buildarea/3.x.ware-gentoo-x86.refleak/build/Lib/test/libregrtest/main.py", line 587 in main
  File "/buildbot/buildarea/3.x.ware-gentoo-x86.refleak/build/Lib/test/regrtest.py", line 46 in _main
  File "/buildbot/buildarea/3.x.ware-gentoo-x86.refleak/build/Lib/test/regrtest.py", line 50 in <module>
  File "/buildbot/buildarea/3.x.ware-gentoo-x86.refleak/build/Lib/runpy.py", line 85 in _run_code
  File "/buildbot/buildarea/3.x.ware-gentoo-x86.refleak/build/Lib/runpy.py", line 193 in _run_module_as_main

----------
components: Tests
messages: 300022
nosy: haypo
priority: normal
severity: normal
status: open
title: test_functools: test_recursive_pickle() stack overflow on x86 Gentoo Refleaks 3.x
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31164>
_______________________________________

From report at bugs.python.org  Wed Aug  9 21:43:02 2017
From: report at bugs.python.org (geeknik)
Date: Thu, 10 Aug 2017 01:43:02 +0000
Subject: [New-bugs-announce] [issue31165] null pointer deref and segfault in
 list_slice (listobject.c:455)
Message-ID: <1502329382.83.0.596499846373.issue31165@psf.upfronthosting.co.za>


New submission from geeknik:

Python 3.7 git commit 3ca9f50 compiled with afl-clang-fast on Ubuntu 16 x64. The following script triggers undefined-behavior followed by a null pointer dereference and a segfault.


import weakref
class A(object):pass
def callback(x):del lst[0]
keepali0e=[]
for i in range(1):
    lst=[str()]
    a=A()
    a.c=a
    keepali0e.append(weakref.ref(a,callback))
    del a
    while lst:keepali0e.append(lst[:])


Objects/dictobject.c:547:12: runtime error: index 16 out of bounds for type 'int8_t [8]'
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Objects/dictobject.c:547:12 in
Objects/dictobject.c:1105:18: runtime error: index 16 out of bounds for type 'int8_t [8]'
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Objects/dictobject.c:1105:18 in
Objects/dictobject.c:2739:15: runtime error: index 16 out of bounds for type 'int8_t [8]'
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Objects/dictobject.c:2739:15 in
Objects/dictobject.c:789:27: runtime error: index 16 out of bounds for type 'int8_t [8]'
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Objects/dictobject.c:789:27 in
Objects/dictobject.c:1104:18: runtime error: index 16 out of bounds for type 'int8_t [8]'
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Objects/dictobject.c:1104:18 in
Objects/dictobject.c:994:15: runtime error: index 16 out of bounds for type 'int8_t [8]'
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Objects/dictobject.c:994:15 in
Objects/dictobject.c:683:11: runtime error: index 16 out of bounds for type 'int8_t [8]'
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Objects/dictobject.c:683:11 in
Objects/dictobject.c:1024:9: runtime error: index 64 out of bounds for type 'int8_t [8]'
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Objects/dictobject.c:1024:9 in
Objects/dictobject.c:2882:31: runtime error: index 64 out of bounds for type 'int8_t [8]'
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Objects/dictobject.c:2882:31 in
Objects/dictobject.c:2346:15: runtime error: index 128 out of bounds for type 'int8_t [8]'
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Objects/dictobject.c:2346:15 in
Objects/dictobject.c:1449:11: runtime error: index 32 out of bounds for type 'int8_t [8]'
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Objects/dictobject.c:1449:11 in
Objects/dictobject.c:744:27: runtime error: index 16 out of bounds for type 'int8_t [8]'
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Objects/dictobject.c:744:27 in
Objects/dictobject.c:1631:22: runtime error: index 16 out of bounds for type 'int8_t [8]'
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Objects/dictobject.c:1631:22 in
Objects/dictobject.c:554:31: runtime error: index 16 out of bounds for type 'int8_t [8]'
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Objects/dictobject.c:554:31 in
Objects/dictobject.c:1183:15: runtime error: index 16 out of bounds for type 'int8_t [8]'
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Objects/dictobject.c:1183:15 in
Objects/dictobject.c:835:27: runtime error: index 16 out of bounds for type 'int8_t [8]'
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Objects/dictobject.c:835:27 in
Objects/dictobject.c:2036:10: runtime error: index 128 out of bounds for type 'int8_t [8]'
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Objects/dictobject.c:2036:10 in
Objects/dictobject.c:3504:38: runtime error: index 16 out of bounds for type 'int8_t [8]'
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Objects/dictobject.c:3504:38 in
Objects/dictobject.c:3422:38: runtime error: index 64 out of bounds for type 'int8_t [8]'
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Objects/dictobject.c:3422:38 in
Objects/listobject.c:455:23: runtime error: load of null pointer of type 'PyObject *' (aka 'struct _object *')
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Objects/listobject.c:455:23 in
ASAN:DEADLYSIGNAL
=================================================================
==29900==ERROR: AddressSanitizer: SEGV on unknown address 0x000000000000 (pc 0x0000007772df bp 0x7fffdd00ce30 sp 0x7fffdd00cde0 T0)
==29900==The signal is caused by a READ memory access.
==29900==Hint: address points to the zero page.
    #0 0x7772de in list_slice /root/cpython/Objects/listobject.c:455:23
    #1 0x79257b in list_subscript /root/cpython/Objects/listobject.c:2499:20
    #2 0xca195c in _PyEval_EvalFrameDefault /root/cpython/Python/ceval.c:1442:29
    #3 0xcc723c in _PyEval_EvalCodeWithName /root/cpython/Python/ceval.c:4173:14
    #4 0xc679f3 in PyEval_EvalCodeEx /root/cpython/Python/ceval.c:4200:12
    #5 0xc679f3 in PyEval_EvalCode /root/cpython/Python/ceval.c:657
    #6 0x53056e in run_mod /root/cpython/Python/pythonrun.c:982:9
    #7 0x531d77 in PyRun_FileExFlags /root/cpython/Python/pythonrun.c:935:11
    #8 0x52d219 in PyRun_SimpleFileExFlags /root/cpython/Python/pythonrun.c:398:13
    #9 0x5a958e in run_file /root/cpython/Modules/main.c:341:11
    #10 0x5a958e in Py_Main /root/cpython/Modules/main.c:901
    #11 0x500382 in main /root/cpython/./Programs/python.c:102:11
    #12 0x7f17562f83f0 in __libc_start_main /build/glibc-mXZSwJ/glibc-2.24/csu/../csu/libc-start.c:291
    #13 0x433e49 in _start (/root/cpython/python+0x433e49)

AddressSanitizer can not provide additional info.
SUMMARY: AddressSanitizer: SEGV /root/cpython/Objects/listobject.c:455:23 in list_slice
==29900==ABORTING

----------
components: Interpreter Core
messages: 300033
nosy: geeknik
priority: normal
severity: normal
status: open
title: null pointer deref and segfault in list_slice (listobject.c:455)
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31165>
_______________________________________

From report at bugs.python.org  Wed Aug  9 22:04:37 2017
From: report at bugs.python.org (geeknik)
Date: Thu, 10 Aug 2017 02:04:37 +0000
Subject: [New-bugs-announce] [issue31166] null pointer deref and segfault in
 _PyObject_Alloc (obmalloc.c:1258)
Message-ID: <1502330677.73.0.512215884622.issue31166@psf.upfronthosting.co.za>


New submission from geeknik:

Python 3.7 git commit 3ca9f50 compiled with afl-clang-fast on Ubuntu 16 x64. The following script triggers undefined-behavior followed by a null pointer dereference and a segfault.


import gc
t0ing0=object()
class A(object):
    def f():0
    x=t0ing0
r=gc.get_referrers(t0ing0)
if[0]:dct=r[0]
a=A
for i in range(1):a.f
dct["f"]=lambda:0
(a.f)


Objects/dictobject.c:547:12: runtime error: index 16 out of bounds for type 'int8_t [8]'
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Objects/dictobject.c:547:12 in
Objects/dictobject.c:1105:18: runtime error: index 16 out of bounds for type 'int8_t [8]'
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Objects/dictobject.c:1105:18 in
Objects/dictobject.c:2739:15: runtime error: index 16 out of bounds for type 'int8_t [8]'
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Objects/dictobject.c:2739:15 in
Objects/dictobject.c:789:27: runtime error: index 16 out of bounds for type 'int8_t [8]'
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Objects/dictobject.c:789:27 in
Objects/dictobject.c:1104:18: runtime error: index 16 out of bounds for type 'int8_t [8]'
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Objects/dictobject.c:1104:18 in
Objects/dictobject.c:994:15: runtime error: index 16 out of bounds for type 'int8_t [8]'
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Objects/dictobject.c:994:15 in
Objects/dictobject.c:683:11: runtime error: index 16 out of bounds for type 'int8_t [8]'
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Objects/dictobject.c:683:11 in
Objects/dictobject.c:1024:9: runtime error: index 64 out of bounds for type 'int8_t [8]'
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Objects/dictobject.c:1024:9 in
Objects/dictobject.c:2882:31: runtime error: index 64 out of bounds for type 'int8_t [8]'
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Objects/dictobject.c:2882:31 in
Objects/dictobject.c:2346:15: runtime error: index 128 out of bounds for type 'int8_t [8]'
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Objects/dictobject.c:2346:15 in
Objects/dictobject.c:1449:11: runtime error: index 32 out of bounds for type 'int8_t [8]'
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Objects/dictobject.c:1449:11 in
Objects/dictobject.c:744:27: runtime error: index 16 out of bounds for type 'int8_t [8]'
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Objects/dictobject.c:744:27 in
Objects/dictobject.c:1631:22: runtime error: index 16 out of bounds for type 'int8_t [8]'
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Objects/dictobject.c:1631:22 in
Objects/dictobject.c:554:31: runtime error: index 16 out of bounds for type 'int8_t [8]'
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Objects/dictobject.c:554:31 in
Objects/dictobject.c:1183:15: runtime error: index 16 out of bounds for type 'int8_t [8]'
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Objects/dictobject.c:1183:15 in
Objects/dictobject.c:835:27: runtime error: index 16 out of bounds for type 'int8_t [8]'
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Objects/dictobject.c:835:27 in
Objects/dictobject.c:2036:10: runtime error: index 128 out of bounds for type 'int8_t [8]'
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Objects/dictobject.c:2036:10 in
Objects/dictobject.c:3504:38: runtime error: index 16 out of bounds for type 'int8_t [8]'
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Objects/dictobject.c:3504:38 in
Objects/dictobject.c:3422:38: runtime error: index 64 out of bounds for type 'int8_t [8]'
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Objects/dictobject.c:3422:38 in
Objects/obmalloc.c:1258:36: runtime error: load of misaligned address 0xffffffffffffffff for type 'block *' (aka 'unsigned char *'), which requires 8 byte alignment
0xffffffffffffffff: note: pointer points here
<memory cannot be printed>
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Objects/obmalloc.c:1258:36 in
ASAN:DEADLYSIGNAL
=================================================================
==768==ERROR: AddressSanitizer: SEGV on unknown address 0x000000000000 (pc 0x000000506f9f bp 0x7fff0666cbc0 sp 0x7fff0666cb20 T0)
==768==The signal is caused by a READ memory access.
==768==Hint: address points to the zero page.
    #0 0x506f9e in _PyObject_Alloc /root/cpython/Objects/obmalloc.c:1258:36
    #1 0x9a669b in PyUnicode_New /root/cpython/Objects/unicodeobject.c:1296:24
    #2 0x9fea51 in _PyUnicodeWriter_PrepareInternal /root/cpython/Objects/unicodeobject.c:13561:26
    #3 0x9b9db4 in PyUnicode_DecodeUTF8Stateful /root/cpython/Objects/unicodeobject.c:4995:9
    #4 0x9c0e65 in _PyUnicode_FromId /root/cpython/Objects/unicodeobject.c:2115:22
    #5 0x89561e in _PyObject_GetAttrId /root/cpython/Objects/object.c:850:23
    #6 0x6b3f4a in _PyObject_CallMethodId /root/cpython/Objects/call.c:1086:16
    #7 0x518569 in flush_std_files /root/cpython/Python/pylifecycle.c:889:15
    #8 0x517942 in Py_FinalizeEx /root/cpython/Python/pylifecycle.c:959:9
    #9 0x5a882b in Py_Main /root/cpython/Modules/main.c:921:9
    #10 0x500382 in main /root/cpython/./Programs/python.c:102:11
    #11 0x7f94db0bb3f0 in __libc_start_main /build/glibc-mXZSwJ/glibc-2.24/csu/../csu/libc-start.c:291
    #12 0x433e49 in _start (/root/cpython/python+0x433e49)

AddressSanitizer can not provide additional info.
SUMMARY: AddressSanitizer: SEGV /root/cpython/Objects/obmalloc.c:1258:36 in _PyObject_Alloc
==768==ABORTING

----------
components: Interpreter Core
messages: 300034
nosy: geeknik
priority: normal
severity: normal
status: open
title: null pointer deref and segfault in _PyObject_Alloc (obmalloc.c:1258)
type: crash
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31166>
_______________________________________

From report at bugs.python.org  Wed Aug  9 22:04:40 2017
From: report at bugs.python.org (Joshua Li)
Date: Thu, 10 Aug 2017 02:04:40 +0000
Subject: [New-bugs-announce] [issue31167] timedelta produced by
 datetime.__sub__ does not take Daylight Savings Time into account
Message-ID: <1502330680.91.0.483669668972.issue31167@psf.upfronthosting.co.za>


New submission from Joshua Li:

See my SO answer and the corresponding question for detail: https://stackoverflow.com/a/45602760/5348393

Essentially, given two datetime.datetime instances t1 and t2, the following two syntactically different lines of code should be logically equivalent, but in fact differ by plus or minus one hour on Daylight Savings Time dates because `datetime.datetime.__sub__` does not appear to take DST into account.

`t1.timestamp()-t2.timestamp()`
`(t1-t2).total_seconds()`

I am not sure if this is by intentional design, or a behavioral bug.

----------
components: Library (Lib)
messages: 300035
nosy: JoshuaRLi
priority: normal
severity: normal
status: open
title: timedelta produced by datetime.__sub__ does not take Daylight Savings Time into account
type: behavior
versions: Python 3.3, Python 3.4, Python 3.5, Python 3.6, Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31167>
_______________________________________

From report at bugs.python.org  Wed Aug  9 22:04:45 2017
From: report at bugs.python.org (Dan Snider)
Date: Thu, 10 Aug 2017 02:04:45 +0000
Subject: [New-bugs-announce] [issue31168] IDLE hangs with absurdly long
 __repr__s
Message-ID: <1502330685.65.0.748621081688.issue31168@psf.upfronthosting.co.za>


New submission from Dan Snider:

Objects with a 500,000+ character __repr__ will cause the IDLE shell to hang and thus lose any progress on open windows. 200,000 (on a decent system) seems to be the tipping point where it can never recover.

----------
assignee: terry.reedy
components: IDLE
messages: 300036
nosy: bup, terry.reedy
priority: normal
severity: normal
status: open
title: IDLE hangs with absurdly long __repr__s
versions: Python 3.6, Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31168>
_______________________________________

From report at bugs.python.org  Wed Aug  9 22:17:46 2017
From: report at bugs.python.org (drallensmith)
Date: Thu, 10 Aug 2017 02:17:46 +0000
Subject: [New-bugs-announce] [issue31169] convert_to_error assertion failure
 in multiprocessing/managers.py
Message-ID: <1502331466.32.0.179260204496.issue31169@psf.upfronthosting.co.za>


New submission from drallensmith:

I am seeing a triggering of one of the assertions in convert_to_error, in managers.py. (This assertion is completely without any message, incidentally, which is also problematic - I've commented on this in issue5001.) This is in common across all versions of cpython that I have access to, and possibly to pypy as well. See https://travis-ci.org/drallensmith/neat-python/builds/262839326 for a set of examples. (The assertion is that "result" is a string. It is somewhat problematic when error-reporting code has an error...)

----------
components: Library (Lib)
messages: 300037
nosy: drallensmith
priority: normal
severity: normal
status: open
title: convert_to_error assertion failure in multiprocessing/managers.py
type: behavior
versions: Python 2.7, Python 3.3, Python 3.4, Python 3.5, Python 3.6, Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31169>
_______________________________________

From report at bugs.python.org  Thu Aug 10 00:48:25 2017
From: report at bugs.python.org (Lin Tian)
Date: Thu, 10 Aug 2017 04:48:25 +0000
Subject: [New-bugs-announce] [issue31170] expat: utf8_toUtf8 cannot properly
 handle exhausting buffer
Message-ID: <1502340505.45.0.797067466934.issue31170@psf.upfronthosting.co.za>


New submission from Lin Tian:

utf8_toUtf8(const ENCODING *UNUSED_P(enc),
            const char **fromP, const char *fromLim,
            char **toP, const char *toLim)
{
  char *to;
  const char *from;
  const char *fromLimInitial = fromLim;

  /* Avoid copying partial characters. */
  align_limit_to_full_utf8_characters(*fromP, &fromLim);

  for (to = *toP, from = *fromP; (from < fromLim) && (to < toLim); from++, to++)
    *to = *from;
  *fromP = from;
  *toP = to;

  if (fromLim < fromLimInitial)
    return XML_CONVERT_INPUT_INCOMPLETE;
  else if ((to == toLim) && (from < fromLim))
    // <===== Bug is here. In case (to == toLim), it's possible that
    //        from is still pointing to partial character. For example,
    //        a character with 3 bytes (A, B, C) and form is pointing to C.
    //        It means only A and B is copied to output buffer. Next
    //        scanning will start with C which could be considered as invalid
    //        byte and got dropped. After this, only "AB" is kept in memory
    //        and thus it will lead to invalid continuation byte.
    return XML_CONVERT_OUTPUT_EXHAUSTED;
  else
    return XML_CONVERT_COMPLETED;
}

----------
components: Library (Lib)
messages: 300043
nosy: Lin Tian
priority: normal
severity: normal
status: open
title: expat: utf8_toUtf8 cannot properly handle exhausting buffer
type: behavior
versions: Python 3.6, Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31170>
_______________________________________

From report at bugs.python.org  Thu Aug 10 03:21:56 2017
From: report at bugs.python.org (Hongxu Jia)
Date: Thu, 10 Aug 2017 07:21:56 +0000
Subject: [New-bugs-announce] [issue31171] multiprocessing.BoundedSemaphore
 of 32-bit python could not work while cross compiling on linux platform
Message-ID: <1502349716.95.0.155037219321.issue31171@psf.upfronthosting.co.za>


New submission from Hongxu Jia:

To build python for embedded Linux systems,
(http://www.yoctoproject.org/docs/2.3.1/yocto-project-qs/yocto-project-qs.html)
The 32-bit python's multiprocessing.BoundedSemaphore could not work.

1. Prerequisite
- Build 32bit python on 64bit host, add '-m32' to gcc

- Cross compling on linux platform, not have `-pthread'
  In configure.ac, ac_cv_pthread=no while cross compiling
  ...
  2001 AC_MSG_CHECKING(whether $CC accepts -pthread)
  2002 AC_CACHE_VAL(ac_cv_pthread,
  2003 [ac_save_cc="$CC"
  2004 CC="$CC -pthread"
  2005 AC_RUN_IFELSE([AC_LANG_SOURCE([[
  [snip]
  2018 ]])],[ac_cv_pthread=yes],[ac_cv_pthread=no],[ac_cv_pthread=no])
  ...

  ? Macro: AC_RUN_IFELSE (input, [action-if-true], [action-if-false], [action-if-cross-compiling])

2. Reproduce with simplifying the prerequisite

$ git clone https://github.com/python/cpython.git; cd ./cpython

$ ac_cv_pthread=no CC="gcc -m32" ./configure

$ make -j 10

$ ./python -c "import multiprocessing; pool_sema = multiprocessing.BoundedSemaphore(value=1); pool_sema.acquire(); pool_sema.release()"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ValueError: semaphore or lock released too many times

And the issue also caused invoking put of 'multiprocessing.Queue' hung.
$ ./python -c "import multiprocessing; queue_instance = multiprocessing.Queue(); queue_instance.put(('install'))"

----------
components: Library (Lib)
messages: 300052
nosy: hongxu
priority: normal
severity: normal
status: open
title: multiprocessing.BoundedSemaphore of 32-bit python could not work while cross compiling on linux platform
type: crash
versions: Python 2.7, Python 3.6, Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31171>
_______________________________________

From report at bugs.python.org  Thu Aug 10 03:23:51 2017
From: report at bugs.python.org (Patrick Rutkowski)
Date: Thu, 10 Aug 2017 07:23:51 +0000
Subject: [New-bugs-announce] [issue31172] Py_Main() is totally broken on
 Visual Studio 2017
Message-ID: <1502349831.97.0.859885233288.issue31172@psf.upfronthosting.co.za>


New submission from Patrick Rutkowski:

Install Visual Studio 2017

Download and unpack Python-3.6.2.tgz

Open a Visual Studio command prompt
Browse to Python-3.6.2\PCBuild
Run build.bat -p x64 -c Release
Run build.bat -p x64 -c Debug
Add the PCbuild\amd64 directory to your PATH

Create a new Visual Studio Win32 GUI project called mypythonw

Copy-paste the source code of Python-3.6.2\PCBuild\WinMain.c into your mypthonw project.

Add Python-3.6.2\Include and Python-3.6.2\PC to mypythonw's include path.

Add Python-3.6.2\PCbuild\amd64 to mypythonw's library search path

Build mypthonw

Run these two commands:

pythonw -c "import ctypes; ctypes.windll.user32.MessageBoxW(0, 'Hello World!', 'Hello', 0)

mypythonw -c "import ctypes; ctypes.windll.user32.MessageBoxW(0, 'Hello World!', 'Hello', 0)

The first will show a message box, while the second will show nothing. You won't get an error message or a debugger from mypythonw.exe. It'll just silently do nothing.

There must be something in the configuration of pythonw.vcxproj that is necessary for Python embedding to work, and which makes it work in pythonw but fail in mypythonw. The two projects are executing the exact same C code, but one works and the other doesn't. What's going on here?

----------
messages: 300055
nosy: rutski
priority: normal
severity: normal
status: open
title: Py_Main() is totally broken on Visual Studio 2017

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31172>
_______________________________________

From report at bugs.python.org  Thu Aug 10 05:20:53 2017
From: report at bugs.python.org (STINNER Victor)
Date: Thu, 10 Aug 2017 09:20:53 +0000
Subject: [New-bugs-announce] [issue31173] test_subprocess:
 test_child_terminated_in_stopped_state() leaks a zombie process
Message-ID: <1502356853.63.0.0769606130984.issue31173@psf.upfronthosting.co.za>


New submission from STINNER Victor:

The test_child_terminated_in_stopped_state() test creates a child process which calls ptrace(PTRACE_ME, 0, 0) and then crashs using SIGSEGV. The problem is that even if we read the exit status using os.waitpid() through subprocess, the process remains alive in the "t (tracing stop)" state.

I would prefer to not use ptrace() is an unit test since this API is very low-level and it's hard to use it correctly.

I suggest to either remove the functional test, or to rewrite it as an unit test using mocks to test bpo-29335 without ptrace().

haypo at selma$ ./python -m test -m test_child_terminated_in_stopped_state -F test_subprocess
Run tests sequentially
0:00:00 load avg: 0.95 [  1] test_subprocess
0:00:00 load avg: 0.95 [  2] test_subprocess
0:00:01 load avg: 0.96 [  3] test_subprocess
0:00:01 load avg: 0.96 [  4] test_subprocess
0:00:02 load avg: 0.96 [  5] test_subprocess
0:00:03 load avg: 0.96 [  6] test_subprocess
0:00:03 load avg: 0.96 [  7] test_subprocess
0:00:04 load avg: 0.96 [  8] test_subprocess
0:00:05 load avg: 0.96 [  9] test_subprocess
0:00:05 load avg: 0.96 [ 10] test_subprocess
^Z
[1]+  Stopp?                 ./python -m test -m test_child_terminated_in_stopped_state -F test_subprocess

haypo at selma$ ps
  PID TTY          TIME CMD
30359 pts/0    00:00:00 bash
31882 pts/0    00:00:00 python
31885 pts/0    00:00:00 python
31888 pts/0    00:00:00 python
31892 pts/0    00:00:00 python
31895 pts/0    00:00:00 python
31898 pts/0    00:00:00 python
31901 pts/0    00:00:00 python
31904 pts/0    00:00:00 python
31907 pts/0    00:00:00 python
31910 pts/0    00:00:00 python
31912 pts/0    00:00:00 python
31920 pts/0    00:00:00 ps

haypo at selma$ grep Stat /proc/31885/status
State:	t (tracing stop)

----------
messages: 300066
nosy: haypo
priority: normal
severity: normal
status: open
title: test_subprocess: test_child_terminated_in_stopped_state() leaks a zombie process

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31173>
_______________________________________

From report at bugs.python.org  Thu Aug 10 07:45:39 2017
From: report at bugs.python.org (STINNER Victor)
Date: Thu, 10 Aug 2017 11:45:39 +0000
Subject: [New-bugs-announce] [issue31174] test_tools leaks randomly
 references on x86 Gentoo Refleaks 3.x
Message-ID: <1502365539.11.0.343233157276.issue31174@psf.upfronthosting.co.za>


New submission from STINNER Victor:

The "x86 Gentoo Refleaks 3.x" buildbot runs tests using -u-cpu to disable the cpu resource. The problem is that DirectoryTestCase.test_files() of Lib/test/test_tools/test_unparse.py uses random:

        # Test limited subset of files unless the 'cpu' resource is specified.
        if not test.support.is_resource_enabled("cpu"):
            names = random.sample(names, 10)

So when we run the same test 7 times, each run uses different data.

I see different options:

* Reseed random using the same seed in dash_R() of regrtest
* Always test all data in test_unparse.py: all files, or select a specific set of interesting files

The random issue is more generic than just test_unparse.py, and so it would be interesting to explore this path. Maybe the random issue explains why some other tests fail randomly.

libregrtest always seeds the random RNG using a seed displayed on the standard output. We should either reuse this seed, or create a new unique seed for each test file, and display it (to be able to reproduce tests).

Reseed random before running each test file can also helps to make tests more reproductible

--

http://buildbot.python.org/all/builders/x86%20Gentoo%20Refleaks%203.x/builds/52/steps/test/logs/stdio

test_tools leaked [1, 4, 2] memory blocks, sum=7
(...)
Re-running test 'test_tools' in verbose mode
(...)
test_tools leaked [1, 2, 2] memory blocks, sum=5
(...)
1 test failed again:
    test_tools

----------
components: Tests
messages: 300075
nosy: haypo, serhiy.storchaka
priority: normal
severity: normal
status: open
title: test_tools leaks randomly references on x86 Gentoo Refleaks 3.x
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31174>
_______________________________________

From report at bugs.python.org  Thu Aug 10 09:05:08 2017
From: report at bugs.python.org (Tarmo Randel)
Date: Thu, 10 Aug 2017 13:05:08 +0000
Subject: [New-bugs-announce] [issue31175] Exception while extracting file
 from ZIP with non-matching file name in central directory
Message-ID: <1502370308.59.0.330372218363.issue31175@psf.upfronthosting.co.za>


New submission from Tarmo Randel:

The problem: miscreants are modifying ZIP file header parts so, that Python based automated analysis tools are unable to process the contents of the ZIP file but intended clients are able to open the file and extract the possibly malicious contents.

Github pull request contains patch addressing the issue so that developer can make conscious decision to allow extraction process to complete. Quite important feature for security researchers.

----------
components: Library (Lib)
files: ZIP_filename_confusion.pdf
messages: 300080
nosy: zyxtarmo
priority: normal
pull_requests: 3094
severity: normal
status: open
title: Exception while extracting file from ZIP with non-matching file name in central directory
type: behavior
versions: Python 2.7
Added file: http://bugs.python.org/file47073/ZIP_filename_confusion.pdf

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31175>
_______________________________________

From report at bugs.python.org  Thu Aug 10 09:28:01 2017
From: report at bugs.python.org (twisteroid ambassador)
Date: Thu, 10 Aug 2017 13:28:01 +0000
Subject: [New-bugs-announce] [issue31176] Is a UDP transport also a
 ReadTransport/WriteTransport?
Message-ID: <1502371681.86.0.869371272399.issue31176@psf.upfronthosting.co.za>


New submission from twisteroid ambassador:

In docs / Library Reference / asyncio / Transports and Protocols, it is mentioned that "asyncio currently implements transports for TCP, UDP, SSL, and subprocess pipes. The methods available on a transport depend on the transport?s kind." It also lists methods available on a BaseTransport, ReadTransport, WriteTransport, DatagramTransport and BaseSubprocessTransport.

However, the docs does not explain which transports have methods from which base classes, or in other words which base classes each concrete transport class inherits from. And this may not be obvious: for example, a UDP transport certainly is a DatagramTransport, but is it also a ReadTransport, or a WriteTransport?

(I feel like the answer is "no it isn't", but there are plenty of conflicting evidence. The docs show that WriteTransport has write_eof() and can_write_eof() -- methods clearly geared towards stream-like transports, and it duplicates abort() from DatagramTransport, so it would seem like WriteTransport and DatagramTransport are mutually exclusive. On the other hand, the default concrete implementation of _SelectorDatagramTransport actually inherits from Transport which inherits from both ReadTransport and WriteTransport, yet it does not inherit from DatagramTransport; As a result _SelectorDatagramTransport has all the methods from ReadTransport and WriteTransport, but many of them raise NotImplemented. This is why I'm asking this question in the first place: I found that the transport object I got from create_datagram_endpoint() has both pause_reading() and resume_reading() methods that raise NotImplemented, and thought that perhaps some event loop implementations would have these methods working, and I should try to use them. And before you say "UDP doesn't do flow control", asyncio actually does provide flow control for UDP on the writing end: see https://www.mail-archive.com/python-tulip at googlegroups.com/msg00532.html So it's not preposterous that there might be flow control on the reading end as well.)

I think it would be nice if the documentation can state the methods implemented for each type of transport, as the designers of Python intended, so there's a clear expectation of what methods will / should be available across different implementations of event loops and transports. Something along the lines of "The methods available on a transport depend on the transport?s kind: TCP transports support methods declared in BaseTransport, ReadTransport and WriteTransport below, etc."

----------
assignee: docs at python
components: Documentation, asyncio
messages: 300083
nosy: docs at python, twisteroid ambassador, yselivanov
priority: normal
severity: normal
status: open
title: Is a UDP transport also a ReadTransport/WriteTransport?
type: enhancement
versions: Python 3.5, Python 3.6, Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31176>
_______________________________________

From report at bugs.python.org  Thu Aug 10 10:22:19 2017
From: report at bugs.python.org (Hmvp)
Date: Thu, 10 Aug 2017 14:22:19 +0000
Subject: [New-bugs-announce] [issue31177] unittest mock's reset_mock throws
 an error when an attribute has been deleted
Message-ID: <1502374939.75.0.688742031254.issue31177@psf.upfronthosting.co.za>


New submission from Hmvp:

When using a mock and deleting a attribute reset_mock cannot be used anymore since it tries to call reset_mock on the _deleted sentinel value.

Reproduction path:
```
from unittest.mock import MagicMock
mock = MagicMock()
mock.a = 'test'
del mock.a
mock.reset_mock()
```

Gives:
```
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python3.5/unittest/mock.py", line 544, in reset_mock
    child.reset_mock(visited)
AttributeError: '_SentinelObject' object has no attribute 'reset_mock'
```

Expected result:
mock is reset without throwing an exception and the 'a' attribute is no longer in a deleted state

Only checked 3.5 and current master if bug is present

----------
components: Library (Lib)
messages: 300090
nosy: hmvp
priority: normal
severity: normal
status: open
title: unittest mock's reset_mock throws an error when an attribute has been deleted
type: behavior
versions: Python 3.5, Python 3.6, Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31177>
_______________________________________

From report at bugs.python.org  Thu Aug 10 11:29:44 2017
From: report at bugs.python.org (STINNER Victor)
Date: Thu, 10 Aug 2017 15:29:44 +0000
Subject: [New-bugs-announce] [issue31178] [EASY] subprocess: TypeError:
 can't concat str to bytes, in _execute_child()
Message-ID: <1502378984.02.0.493180637871.issue31178@psf.upfronthosting.co.za>


New submission from STINNER Victor:

Lib/subprocess.py contains the following code:

                try:
                    exception_name, hex_errno, err_msg = (
                            errpipe_data.split(b':', 2))
                except ValueError:
                    exception_name = b'SubprocessError'
                    hex_errno = b'0'
                    err_msg = (b'Bad exception data from child: ' +
                               repr(errpipe_data))

b'...' + repr() is wrong: it raises a "TypeError: can't concat str to bytes" when python3 is run with -bb.


Example with attached subprocess_bug.patch:

haypo at selma$ ./python -bb -m test -v test_subprocess -m test_invalid_args
(...)
======================================================================
ERROR: test_invalid_args (test.test_subprocess.ContextManagerTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/haypo/prog/python/master/Lib/subprocess.py", line 1309, in _execute_child
    errpipe_data.split(b':', 1))
ValueError: not enough values to unpack (expected 3, got 2)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/haypo/prog/python/master/Lib/test/test_subprocess.py", line 2880, in test_invalid_args
    stderr=subprocess.PIPE) as proc:
  File "/home/haypo/prog/python/master/Lib/subprocess.py", line 709, in __init__
    restore_signals, start_new_session)
  File "/home/haypo/prog/python/master/Lib/subprocess.py", line 1314, in _execute_child
    repr(errpipe_data))
TypeError: can't concat str to bytes


IMHO err_msg should be decoded using err_msg.decode(errors="surrogatepass") and then use 'Bad ...: %s' % err_msg. It would need to add an "else:" block to the try/except to do the err_msg.decode(errors="surrogatepass") when no error is raised. Well, something like that :-)

----------
components: Library (Lib)
files: subprocess_bug.patch
keywords: easy, patch
messages: 300102
nosy: haypo
priority: normal
severity: normal
status: open
title: [EASY] subprocess: TypeError: can't concat str to bytes, in _execute_child()
versions: Python 3.7
Added file: http://bugs.python.org/file47077/subprocess_bug.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31178>
_______________________________________

From report at bugs.python.org  Thu Aug 10 17:31:33 2017
From: report at bugs.python.org (Yury Selivanov)
Date: Thu, 10 Aug 2017 21:31:33 +0000
Subject: [New-bugs-announce] [issue31179] Speed-up dict.copy() up to 5.5
 times.
Message-ID: <1502400693.03.0.6389481443.issue31179@psf.upfronthosting.co.za>


New submission from Yury Selivanov:

It's possible to significantly improve performance of shallow dict copy.  Currently, PyDict_Copy creates a new empty dict object and then inserts key/values into it one by one.

My idea is to simply memcpy the whole keys/items region and do the necessary increfs after it.  This works just fine for non-key-sharing dicts.

With the following simple microbenchmark:

    import time

    N = 1000000

    for size in [0, 1, 10, 20, 50, 100, 500, 1000]:
        d = dict([(str(i), i) for i in range(size)])

        t = time.monotonic()
        for i in range(N):
            d.copy()
        e = time.monotonic() - t

        print(f'dict(size={size}).copy() x {N} times:\t {e:.4f}')


Output for 3.7 master:

    dict(size=0).copy() x 1000000 times:     0.1299
    dict(size=1).copy() x 1000000 times:     0.1499
    dict(size=10).copy() x 1000000 times:    0.3758
    dict(size=20).copy() x 1000000 times:    0.7722
    dict(size=50).copy() x 1000000 times:    1.2784
    dict(size=100).copy() x 1000000 times:   2.5128
    dict(size=500).copy() x 1000000 times:   12.8968
    dict(size=1000).copy() x 1000000 times:  25.4276


Output for patched 3.7:

    dict(size=0).copy() x 1000000 times:     0.1352
    dict(size=1).copy() x 1000000 times:     0.1285
    dict(size=10).copy() x 1000000 times:    0.1632
    dict(size=20).copy() x 1000000 times:    0.3076
    dict(size=50).copy() x 1000000 times:    0.3663
    dict(size=100).copy() x 1000000 times:   0.5140
    dict(size=500).copy() x 1000000 times:   2.3419
    dict(size=1000).copy() x 1000000 times:  4.6176

----------
components: Interpreter Core
messages: 300117
nosy: haypo, inada.naoki, yselivanov
priority: normal
severity: normal
stage: patch review
status: open
title: Speed-up dict.copy() up to 5.5 times.
type: performance
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31179>
_______________________________________

From report at bugs.python.org  Thu Aug 10 17:36:15 2017
From: report at bugs.python.org (STINNER Victor)
Date: Thu, 10 Aug 2017 21:36:15 +0000
Subject: [New-bugs-announce] [issue31180] test_multiprocessing_spawn hangs
 randomly on x86 Windows7 3.6
Message-ID: <1502400975.07.0.689018470672.issue31180@psf.upfronthosting.co.za>


New submission from STINNER Victor:

http://buildbot.python.org/all/builders/x86%20Windows7%203.6/builds/570/steps/test/logs/stdio

1:21:25 [402/405] test_frame passed -- running: test_multiprocessing_spawn (2805 sec)
1:21:32 [403/405] test_hash passed -- running: test_multiprocessing_spawn (2811 sec)
1:21:33 [404/405] test_raise passed -- running: test_multiprocessing_spawn (2812 sec)

command timed out: 1200 seconds without output running ['Tools\\buildbot\\test.bat', '-j2', '--timeout', '900'], attempting to kill
program finished with exit code 1
elapsedTime=6104.779000

----------
components: Tests
messages: 300119
nosy: haypo
priority: normal
severity: normal
status: open
title: test_multiprocessing_spawn hangs randomly on x86 Windows7 3.6
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31180>
_______________________________________

From report at bugs.python.org  Thu Aug 10 17:43:17 2017
From: report at bugs.python.org (lchin)
Date: Thu, 10 Aug 2017 21:43:17 +0000
Subject: [New-bugs-announce] [issue31181] Segfault in gcmodule.c:360
 visit_decref (PyObject_IS_GC(op))
Message-ID: <1502401397.09.0.667650470232.issue31181@psf.upfronthosting.co.za>


New submission from lchin:

When running a python script with YML data as input, it stopped with segmentation fault 

/home/appuser/pyETL/tableTrimmer.py 
 /home/appuser/pyETL/csidPyLib/etlConfig/ip4AuditSync.yml

(gdb) bt

Core was generated by `python2.7 tableTrimmer.py csidPyLib/etlConfig/ip4AuditSync.yml'.
Program terminated with signal 11, Segmentation fault.
#0  visit_decref (op=0x7f08c1afb320, data=0x0) at Modules/gcmodule.c:360
360	    if (PyObject_IS_GC(op)) {

----------
components: Interpreter Core
messages: 300121
nosy: lchin
priority: normal
severity: normal
status: open
title: Segfault in gcmodule.c:360 visit_decref (PyObject_IS_GC(op))
type: crash
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31181>
_______________________________________

From report at bugs.python.org  Fri Aug 11 04:25:56 2017
From: report at bugs.python.org (Steve Barnes)
Date: Fri, 11 Aug 2017 08:25:56 +0000
Subject: [New-bugs-announce] [issue31182] Suggested Enhancements to zipfile
 & tarfile command line interfaces
Message-ID: <1502439956.47.0.572068463872.issue31182@psf.upfronthosting.co.za>


New submission from Steve Barnes:

I would like to add to the command line of both tarfile and zipfile interface some additional options that I believe to be useful.

1. Wildcard filename matching for archiving
2. Recursive archiving (with excludes)
3. Append to archive option
4. Compression Selection
 4.1 For zipfile: zip64, bizp2 & lzma (if available)
 4.2 For tarfile: document that gz, xz & bz2 are there and determined by ext.

----------
components: Library (Lib)
messages: 300156
nosy: Steve Barnes
priority: normal
severity: normal
status: open
title: Suggested Enhancements to zipfile & tarfile command line interfaces
type: enhancement
versions: Python 2.7, Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31182>
_______________________________________

From report at bugs.python.org  Fri Aug 11 11:21:31 2017
From: report at bugs.python.org (George Collins)
Date: Fri, 11 Aug 2017 15:21:31 +0000
Subject: [New-bugs-announce] [issue31183] `Dis` module doesn't know how to
 disassemble async generator or coroutine objects
Message-ID: <1502464891.48.0.226484247582.issue31183@psf.upfronthosting.co.za>


New submission from George Collins:

Issue 21947 informed the `dis` module about the `gi_code` attribute, which stores code objects for generator objects. This allows inspection of generator objects, not just functions which return them. However, asynchronous generator objects use `ag_code` and coroutine objects use `cr_code`, so dis raises a TypeError on them. I'm making a pull request to extend the generator behavior to async generators and coroutines.

Credit to Luciano Ramalho: I tripped over this at his (great) concurrency workshop at PyBay 2017 and he identified exactly what was happening and suggested a patch.

----------
components: Extension Modules
messages: 300167
nosy: George Collins
priority: normal
severity: normal
status: open
title: `Dis` module doesn't know how to disassemble async generator or coroutine objects
type: enhancement
versions: Python 3.6, Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31183>
_______________________________________

From report at bugs.python.org  Fri Aug 11 12:22:11 2017
From: report at bugs.python.org (David Halter)
Date: Fri, 11 Aug 2017 16:22:11 +0000
Subject: [New-bugs-announce] [issue31184] Fix data descriptor detection in
 inspect.getattr_static
Message-ID: <1502468531.27.0.204472955702.issue31184@psf.upfronthosting.co.za>


New submission from David Halter:

inspect.getattr_static is currently not identifying data descriptors the right way.

Data descriptors are defined by having a __get__ attribute and at least one of the __set__ and __delete__ attributes.

Implementation detail: Both __delete__ and __get__ set the same slot called tp_descr_set in CPython.

I have attached a patch that fixes the issue IMO.

----------
files: 0001-Fix-data-descriptor-detection-in-inspect.getattr_sta.patch
keywords: patch
messages: 300170
nosy: davidhalter
priority: normal
severity: normal
status: open
title: Fix data descriptor detection in inspect.getattr_static
versions: Python 3.3, Python 3.4, Python 3.5, Python 3.6, Python 3.7
Added file: http://bugs.python.org/file47078/0001-Fix-data-descriptor-detection-in-inspect.getattr_sta.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31184>
_______________________________________

From report at bugs.python.org  Fri Aug 11 12:28:18 2017
From: report at bugs.python.org (Serhiy Storchaka)
Date: Fri, 11 Aug 2017 16:28:18 +0000
Subject: [New-bugs-announce] [issue31185] Miscellaneous errors in asyncio
 speedup module
Message-ID: <1502468898.65.0.496074575123.issue31185@psf.upfronthosting.co.za>


New submission from Serhiy Storchaka:

Proposed patch fixes miscellaneous errors in asyncio speedup module. It is hard to enumerate they all, but they are mostly NULL dereferencing, reference leaks, and handling exceptions in corner cases.

----------
components: Extension Modules, asyncio
messages: 300171
nosy: giampaolo.rodola, haypo, serhiy.storchaka, yselivanov
priority: normal
severity: normal
stage: patch review
status: open
title: Miscellaneous errors in asyncio speedup module
type: behavior
versions: Python 3.6, Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31185>
_______________________________________

From report at bugs.python.org  Sat Aug 12 01:31:52 2017
From: report at bugs.python.org (Rajath Agasthya)
Date: Sat, 12 Aug 2017 05:31:52 +0000
Subject: [New-bugs-announce] [issue31186] Support heapfix() and heapremove()
 APIs in heapq module
Message-ID: <1502515912.85.0.983303815097.issue31186@psf.upfronthosting.co.za>


New submission from Rajath Agasthya:

I'd like to suggest a couple of utility methods for the heapq module which I think are useful:

1. heapfix() - Re-establishes heap invariant after item at 'any index' in the heap changes its value.

Currently, the way to fix the heap after an arbitrary item has changed is to just call heapify() on the entire heap. This is inefficient because heapify() tries to perform _siftup() operation on half of the items of the heap to maintain the heap invariant. Doing this is unnecessary because we know exactly which element (or index) was changed and became out of place. With a heapfix() function, we can just "fix" the heap from that position.

2. heapremove() - Removes an item at 'any index' from the heap and maintains heap invariant.

Pretty much same as above. If you remove an item from an arbitrary index, you have to call heapify() to restore the heap invariant.


Supporting these methods require minimal changes to the existing _siftup() and _siftdown() methods (basically just returning position values without changing any logic at all). I have a draft implementation which I've attached as a patch file here. If there's interest in this, I can write some tests and create a Github PR.

Thanks!

----------
components: Library (Lib)
files: heapq_fix_remove.patch
keywords: patch
messages: 300190
nosy: rajathagasthya, rhettinger, stutzbach
priority: normal
severity: normal
status: open
title: Support heapfix() and heapremove() APIs in heapq module
type: enhancement
versions: Python 3.7
Added file: http://bugs.python.org/file47079/heapq_fix_remove.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31186>
_______________________________________

From report at bugs.python.org  Sat Aug 12 06:37:02 2017
From: report at bugs.python.org (Oren Milman)
Date: Sat, 12 Aug 2017 10:37:02 +0000
Subject: [New-bugs-announce] [issue31187] suboptimal code in Py_ReprEnter()
Message-ID: <1502534222.01.0.165858988283.issue31187@psf.upfronthosting.co.za>


New submission from Oren Milman:

in Objects/object.c, Py_ReprEnter() does the following:
    - try to retrieve the Py_Repr list from the thread-state dict.
    - in case the list is not in the dict, add it to the dict as an empty list.
    - check whether the received object is in the Py_Repr list, even in case the
      list was just created, and guaranteed to be empty.


I propose to put this check inside an else clause, so that it wouldn't take
place in case the list is guaranteed to be empty, i.e.:
    list = _PyDict_GetItemId(dict, &PyId_Py_Repr);
    if (list == NULL) {
        list = PyList_New(0);
        ...
    }
    else {
        i = PyList_GET_SIZE(list);
        while (--i >= 0) {
            if (PyList_GET_ITEM(list, i) == obj)
                return 1;
        }
    }

I ran the test suite, and it seems that this change doesn't break anything, so
I would be happy to open a PR for it.

----------
components: Interpreter Core
messages: 300193
nosy: Oren Milman
priority: normal
severity: normal
status: open
title: suboptimal code in Py_ReprEnter()
type: performance
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31187>
_______________________________________

From report at bugs.python.org  Sat Aug 12 13:58:52 2017
From: report at bugs.python.org (=?utf-8?b?0JTQuNC70Y/QvSDQn9Cw0LvQsNGD0LfQvtCy?=)
Date: Sat, 12 Aug 2017 17:58:52 +0000
Subject: [New-bugs-announce] [issue31188] Makefile.pre.in: commoninstall:
 reformat
Message-ID: <1502560732.74.0.724807734516.issue31188@psf.upfronthosting.co.za>


New submission from ????? ????????:

When @FRAMEWORKALTINSTALLLAST@ expands to "", the last line of recipe commoninstall in the resulting Makefile consists solely of two tabs, for which emacs complains as "suspicious line".


diff --git a/Makefile.pre.in b/Makefile.pre.in
--- a/Makefile.pre.in
+++ b/Makefile.pre.in
@@ -1069,8 +1069,7 @@ altinstall: commoninstall
 
 commoninstall:  @FRAMEWORKALTINSTALLFIRST@ \
                altbininstall libinstall inclinstall libainstall \
-               sharedinstall oldsharedinstall altmaninstall \
-               @FRAMEWORKALTINSTALLLAST@
+               sharedinstall oldsharedinstall altmaninstall @FRAMEWORKALTINSTALLLAST@
 
 # Install shared libraries enabled by Setup
 DESTDIRS=      $(exec_prefix) $(LIBDIR) $(BINLIBDEST) $(DESTSHARED)

----------
components: Build
messages: 300204
nosy: dilyan.palauzov
priority: normal
severity: normal
status: open
title: Makefile.pre.in: commoninstall: reformat
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31188>
_______________________________________

From report at bugs.python.org  Sat Aug 12 13:59:49 2017
From: report at bugs.python.org (=?utf-8?b?0JTQuNC70Y/QvSDQn9Cw0LvQsNGD0LfQvtCy?=)
Date: Sat, 12 Aug 2017 17:59:49 +0000
Subject: [New-bugs-announce] [issue31189] README.rst: installing multiple
 versions: typo
Message-ID: <1502560789.4.0.0378607685463.issue31189@psf.upfronthosting.co.za>


New submission from ????? ????????:

It speaks about pythonX.Y but means python3.${subversion}

diff --git a/README.rst b/README.rst
--- a/README.rst
+++ b/README.rst
@@ -190,7 +190,7 @@ script) you must take care that your primary python executable is not
 overwritten by the installation of a different version.  All files and
 directories installed using ``make altinstall`` contain the major and minor
 version and can thus live side-by-side.  ``make install`` also creates
-``${prefix}/bin/python3`` which refers to ``${prefix}/bin/pythonX.Y``.  If you
+``${prefix}/bin/python3`` which refers to ``${prefix}/bin/python3.X``.  If you
 intend to install multiple versions using the same prefix you must decide which
 version (if any) is your "primary" version.  Install that version using ``make
 install``.  Install all other versions using ``make altinstall``.

----------
assignee: docs at python
components: Documentation
messages: 300205
nosy: dilyan.palauzov, docs at python
priority: normal
severity: normal
status: open
title: README.rst: installing multiple versions: typo
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31189>
_______________________________________

From report at bugs.python.org  Sat Aug 12 14:04:09 2017
From: report at bugs.python.org (=?utf-8?b?0JTQuNC70Y/QvSDQn9Cw0LvQsNGD0LfQvtCy?=)
Date: Sat, 12 Aug 2017 18:04:09 +0000
Subject: [New-bugs-announce] [issue31190] ./configure: add ability to build
 and install only shared library (without static libpython3.a)
Message-ID: <1502561049.05.0.30023365478.issue31190@psf.upfronthosting.co.za>


New submission from ????? ????????:

diff --git a/Makefile.pre.in b/Makefile.pre.in
--- a/Makefile.pre.in
+++ b/Makefile.pre.in
@@ -578,7 +578,7 @@ sharedmods: $(BUILDPYTHON) pybuilddir.txt Modules/_math.o
 
 # Build static library
 # avoid long command lines, same as LIBRARY_OBJS
-$(LIBRARY): $(LIBRARY_OBJS)
+libpython$(LDVERSION).a: $(LIBRARY_OBJS)
        -rm -f $@
        $(AR) $(ARFLAGS) $@ Modules/getbuildinfo.o
        $(AR) $(ARFLAGS) $@ $(PARSER_OBJS)
diff --git a/configure.ac b/configure.ac
--- a/configure.ac
+++ b/configure.ac
@@ -987,9 +987,9 @@ esac
 
 AC_SUBST(LIBRARY)
 AC_MSG_CHECKING(LIBRARY)
-if test -z "$LIBRARY"
+if test -z "$LIBRARY" -a "x$enable_static" != "xno"
 then
-       LIBRARY='libpython$(VERSION)$(ABIFLAGS).a'
+       LIBRARY='libpython$(LDVERSION).a'
 fi
 AC_MSG_RESULT($LIBRARY)
 
@@ -1086,6 +1086,16 @@ then
 fi
 AC_MSG_RESULT($enable_shared)
 
+AC_MSG_CHECKING(for --disable-static)
+AC_ARG_ENABLE(static,
+              AS_HELP_STRING([--disable-static], [disable building static python library]))
+AC_MSG_RESULT($enable_static)
+
+if test "x$enable_static" == "xno" -a "x$enable_shared" == "xno"
+then
+       AC_MSG_ERROR(["Both static and shared library disabled"])
+fi
+
 AC_MSG_CHECKING(for --enable-profiling)
 AC_ARG_ENABLE(profiling,
               AS_HELP_STRING([--enable-profiling], [enable C-level code profiling]))

----------
components: Build
messages: 300206
nosy: dilyan.palauzov
priority: normal
severity: normal
status: open
title: ./configure: add ability to build and install only shared library (without static libpython3.a)

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31190>
_______________________________________

From report at bugs.python.org  Sat Aug 12 17:00:27 2017
From: report at bugs.python.org (Saurabh Chaturvedi)
Date: Sat, 12 Aug 2017 21:00:27 +0000
Subject: [New-bugs-announce] [issue31191] Fix grammar in threading.Barrier
 docs
Message-ID: <1502571627.6.0.886151225778.issue31191@psf.upfronthosting.co.za>


Changes by Saurabh Chaturvedi <saurabh.chaturvedi63 at gmail.com>:


----------
nosy: schedutron
priority: normal
pull_requests: 3121
severity: normal
status: open
title: Fix grammar in threading.Barrier docs
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31191>
_______________________________________

From report at bugs.python.org  Sun Aug 13 00:58:12 2017
From: report at bugs.python.org (Brian)
Date: Sun, 13 Aug 2017 04:58:12 +0000
Subject: [New-bugs-announce] [issue31192] "return await coro()" SyntaxError
 despite being "valid syntax" in PEP 492
Message-ID: <1502600292.75.0.87512622225.issue31192@psf.upfronthosting.co.za>


New submission from Brian:

PEP 492 lists the following under "valid syntax" and yet 3.5.2 raises a SyntaxError:

def foo():
    return await coro()

but this works:

def bar():
    return await (coro())

----------
components: Interpreter Core
messages: 300209
nosy: merrellb
priority: normal
severity: normal
status: open
title: "return await coro()" SyntaxError despite being "valid syntax" in PEP 492
type: behavior
versions: Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31192>
_______________________________________

From report at bugs.python.org  Sun Aug 13 09:43:47 2017
From: report at bugs.python.org (David MacIver)
Date: Sun, 13 Aug 2017 13:43:47 +0000
Subject: [New-bugs-announce] [issue31193] re.IGNORECASE strips combining
 character from lower case of LATIN CAPITAL LETTER I WITH DOT ABOVE
Message-ID: <1502631827.21.0.810598108454.issue31193@psf.upfronthosting.co.za>


New submission from David MacIver:

chr(304).lower() is a two character string - a lower case i followed by a combining chr(775) ('COMBINING DOT ABOVE').

The re module seems not to understand the combining character and a regex compiled with IGNORECASE will erroneously match a single lower case i without the required combining character. The attached file demonstrates this. I've tested this on Python 3.6.1 with my locale as ('en_GB', 'UTF-8') (I don't know whether that matters for reproducing this, but I know it can affect how lower/upper work so am including it for the sake of completeness).

The problem does not reproduce on Python 2.7.13 because on that case chr(304).lower() is 'i' without the combining character, so it fails earlier.

This is presumably related to #12728, but as that is closed as fixed while this still reproduces I don't believe it's a duplicate.

----------
components: Library (Lib)
files: casing.py
messages: 300219
nosy: David MacIver
priority: normal
severity: normal
status: open
title: re.IGNORECASE strips combining character from lower case of LATIN CAPITAL LETTER I WITH DOT ABOVE
versions: Python 3.6
Added file: http://bugs.python.org/file47080/casing.py

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31193>
_______________________________________

From report at bugs.python.org  Sun Aug 13 12:12:55 2017
From: report at bugs.python.org (Dan Snider)
Date: Sun, 13 Aug 2017 16:12:55 +0000
Subject: [New-bugs-announce] [issue31194] Inconsistent __repr__s for
 _collections objects
Message-ID: <1502640775.21.0.0795700296627.issue31194@psf.upfronthosting.co.za>


New submission from Dan Snider:

With the new C implementation of collections.OrderedDict, its repr correctly uses the subclass's name, unlike deque and defaultdict.

class Thing(_collections.OrderedDict):
    pass
>>> Thing()
Thing([])

class Thing(_collections.deque):
    pass
>>> Thing()
deque([])

----------
components: Library (Lib)
messages: 300222
nosy: bup
priority: normal
severity: normal
status: open
title: Inconsistent __repr__s for _collections objects
type: enhancement
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31194>
_______________________________________

From report at bugs.python.org  Sun Aug 13 14:41:23 2017
From: report at bugs.python.org (Brett Cannon)
Date: Sun, 13 Aug 2017 18:41:23 +0000
Subject: [New-bugs-announce] [issue31195] Make any() and all() work with
 async generators
Message-ID: <1502649683.99.0.877940152738.issue31195@psf.upfronthosting.co.za>


New submission from Brett Cannon:

It would be great if I could do something like:

  if any(x for async x in aiter()): ...

But as of right now, any() complains that "TypeError: 'async_generator' object is not iterable". I would assume that any() and all() could be updated to look for __aiter__() instead of just __iter__() and do the right thing in either case? Or am I missing something subtle here?

----------
components: Library (Lib)
messages: 300224
nosy: brett.cannon, giampaolo.rodola, haypo, yselivanov
priority: normal
severity: normal
status: open
title: Make any() and all() work with async generators
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31195>
_______________________________________

From report at bugs.python.org  Sun Aug 13 17:12:37 2017
From: report at bugs.python.org (Malcolm Smith)
Date: Sun, 13 Aug 2017 21:12:37 +0000
Subject: [New-bugs-announce] [issue31196] Blank line inconsistency between
 InteractiveConsole and standard interpreter
Message-ID: <1502658757.92.0.0359191508861.issue31196@psf.upfronthosting.co.za>


New submission from Malcolm Smith:

The standard interpreter is more eager to give an error on incomplete input, while the InteractiveConsole will keep on prompting for more:

Python 3.5.3 (default, Apr 10 2017, 07:53:16)  [GCC 6.3.0 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> try:
...
  File "<stdin>", line 2

    ^
IndentationError: expected an indented block
>>> import code
>>> code.interact()
Python 3.5.3 (default, Apr 10 2017, 07:53:16)  [GCC 6.3.0 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
(InteractiveConsole)
>>> try:
...
...
...
...

The InteractiveConsole isn't totally wrong here, because after all, a blank line in this position is syntactically correct. But the inconsistency is confusing.

----------
components: Library (Lib)
messages: 300230
nosy: Malcolm Smith
priority: normal
severity: normal
status: open
title: Blank line inconsistency between InteractiveConsole and standard interpreter
type: behavior
versions: Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31196>
_______________________________________

From report at bugs.python.org  Mon Aug 14 02:39:45 2017
From: report at bugs.python.org (Nick Coghlan)
Date: Mon, 14 Aug 2017 06:39:45 +0000
Subject: [New-bugs-announce] [issue31197] Namespace disassembly omits some
 compiled objects
Message-ID: <1502692785.51.0.58821810388.issue31197@psf.upfronthosting.co.za>


New submission from Nick Coghlan:

In reviewing the PR for issue 31183, I noticed that the criteria for deciding which values to disassemble when disassembling a namespace (objects with a __dict__ attribute) has gotten out of sync with the criteria used by the dis() itself.

The problem is that dis() checks for particular attributes that may contain code objects, while the recursive descent when processing a __dict__ attribute is based on isinstance() and a predefined list of types.

My proposed remedy for this would be:

1. Factor out a dis._get_code() helper function that returns either None or a compiled code object
2. Base the recursive descent in __dict__ processing on the value either having a __dict__ attribute, or else _get_code() returning a non-None result (in the latter case, the code object itself would be passed to the recursive call, rather than the original value from the namespace)

----------
messages: 300234
nosy: ncoghlan
priority: normal
severity: normal
status: open
title: Namespace disassembly omits some compiled objects
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31197>
_______________________________________

From report at bugs.python.org  Mon Aug 14 03:35:10 2017
From: report at bugs.python.org (Nathaniel Smith)
Date: Mon, 14 Aug 2017 07:35:10 +0000
Subject: [New-bugs-announce] [issue31198] getaddrinfo: inconsistent handling
 of port=None
Message-ID: <1502696110.72.0.954086075417.issue31198@psf.upfronthosting.co.za>


New submission from Nathaniel Smith:

socket.getaddrinfo accepts None as a port argument, and translates it into 0. This is handy, because bind() understands 0 to mean "pick a port for me", and if you want bind to pick a port for you and port=None is a slightly more obvious way to say that then port=0.

For example:

>>> socket.getaddrinfo("127.0.0.1", None)
[(<AddressFamily.AF_INET: 2>, <SocketKind.SOCK_STREAM: 1>, 6, '', ('127.0.0.1', 0)), (<AddressFamily.AF_INET: 2>, <SocketKind.SOCK_DGRAM: 2>, 17, '', ('127.0.0.1', 0)), (<AddressFamily.AF_INET: 2>, <SocketKind.SOCK_RAW: 3>, 0, '', ('127.0.0.1', 0))]

socket.getaddrinfo also accepts None as a host name; this is necessary because the underlying getaddrinfo(3) call has special handling for host=NULL, and we need some way to access it:

>>> socket.getaddrinfo(None, 0)
[(<AddressFamily.AF_INET6: 10>, <SocketKind.SOCK_STREAM: 1>, 6, '', ('::1', 0, 0, 0)), (<AddressFamily.AF_INET6: 10>, <SocketKind.SOCK_DGRAM: 2>, 17, '', ('::1', 0, 0, 0)), (<AddressFamily.AF_INET6: 10>, <SocketKind.SOCK_RAW: 3>, 0, '', ('::1', 0, 0, 0)), (<AddressFamily.AF_INET: 2>, <SocketKind.SOCK_STREAM: 1>, 6, '', ('127.0.0.1', 0)), (<AddressFamily.AF_INET: 2>, <SocketKind.SOCK_DGRAM: 2>, 17, '', ('127.0.0.1', 0)), (<AddressFamily.AF_INET: 2>, <SocketKind.SOCK_RAW: 3>, 0, '', ('127.0.0.1', 0))]

However, even though both of these features are supported separately... if you try to use them *together*, then socket.getaddrinfo errors out:

>>> socket.getaddrinfo(None, None)
socket.gaierror: [Errno -2] Name or service not known

I expected that last call to be equivalent to socket.getaddrinfo(None, 0).

----------
components: Extension Modules
messages: 300236
nosy: njs
priority: normal
severity: normal
status: open
title: getaddrinfo: inconsistent handling of port=None
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31198>
_______________________________________

From report at bugs.python.org  Mon Aug 14 04:01:57 2017
From: report at bugs.python.org (Paulo Matos)
Date: Mon, 14 Aug 2017 08:01:57 +0000
Subject: [New-bugs-announce] [issue31199] !HAVE_CLOCK_GETTIME causes
 problems with _PyTime_FromTimespec
Message-ID: <1502697717.3.0.518925337608.issue31199@psf.upfronthosting.co.za>


New submission from Paulo Matos:

On a Fedora 26 system, I run configure as:
$ ../configure --with-assertions --with-lto --with-pydebug --with-address-sanitizer --disable-ipv6

My pyconfig.h contains:
/* #undef HAVE_CLOCK */

/* Define to 1 if you have the `clock_getres' function. */
/* #undef HAVE_CLOCK_GETRES */

/* Define to 1 if you have the `clock_gettime' function. */
/* #undef HAVE_CLOCK_GETTIME */

/* Define to 1 if you have the `clock_settime' function. */
/* #undef HAVE_CLOCK_SETTIME */

/* Define if the C compiler supports computed gotos. */
/* #undef HAVE_COMPUTED_GOTOS */


When this happens, __PyTime_FromTimespec is undefined in pytime.c:747.
../Python/pytime.c:747:9: error: implicit declaration of function ?_PyTime_FromTimespec?; did 
you mean ?_PyTime_FromTimeval?? [-Werror=implicit-function-declaration]
     if (_PyTime_FromTimespec(tp, &ts, raise) < 0)
         ^~~~~~~~~~~~~~~~~~~~
         _PyTime_FromTimeval
cc1: some warnings being treated as errors
make[2]: *** [Makefile:1553: Python/pytime.o] Error 1

----------
components: Build
messages: 300238
nosy: pmatos
priority: normal
severity: normal
status: open
title: !HAVE_CLOCK_GETTIME causes problems with _PyTime_FromTimespec
type: compile error

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31199>
_______________________________________

From report at bugs.python.org  Mon Aug 14 05:21:33 2017
From: report at bugs.python.org (Paulo Matos)
Date: Mon, 14 Aug 2017 09:21:33 +0000
Subject: [New-bugs-announce] [issue31200] address sanitizer build fails
Message-ID: <1502702493.31.0.645434199194.issue31200@psf.upfronthosting.co.za>


New submission from Paulo Matos:

Build with address sanitizer fails miserably.

Configuration in Fedora 26. I attach config.log and the output of 
$ ../configure --with-assertions --with-lto --with-pydebug --with-address-sanitizer --disable-ipv6
$ make -j18 profile-opt

----------
components: Build
files: files.zip
messages: 300240
nosy: pmatos
priority: normal
severity: normal
status: open
title: address sanitizer build fails
type: compile error
Added file: http://bugs.python.org/file47081/files.zip

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31200>
_______________________________________

From report at bugs.python.org  Mon Aug 14 06:43:48 2017
From: report at bugs.python.org (Paulo Matos)
Date: Mon, 14 Aug 2017 10:43:48 +0000
Subject: [New-bugs-announce] [issue31201] module test that failed doesn't
 exist
Message-ID: <1502707428.17.0.905101363291.issue31201@psf.upfronthosting.co.za>


New submission from Paulo Matos:

Trying to rerun the tests that failed but it just says that the module doesn't exist. I am running the exact command line mentioned in the readme file:

$ make test TESTOPTS="-v test_that_failed"
running build
running build_ext

The following modules found by detect_modules() in setup.py, have been
built by the Makefile instead, as configured by the Setup files:
atexit                pwd                   time               

running build_scripts
copying and adjusting /home/pmatos/Projects/cpython/Tools/scripts/pydoc3 -> build/scripts-3.7
copying and adjusting /home/pmatos/Projects/cpython/Tools/scripts/idle3 -> build/scripts-3.7
copying and adjusting /home/pmatos/Projects/cpython/Tools/scripts/2to3 -> build/scripts-3.7
copying and adjusting /home/pmatos/Projects/cpython/Tools/scripts/pyvenv -> build/scripts-3.7
changing mode of build/scripts-3.7/pydoc3 from 664 to 775
changing mode of build/scripts-3.7/idle3 from 664 to 775
changing mode of build/scripts-3.7/2to3 from 664 to 775
changing mode of build/scripts-3.7/pyvenv from 664 to 775
renaming build/scripts-3.7/pydoc3 to build/scripts-3.7/pydoc3.7
renaming build/scripts-3.7/idle3 to build/scripts-3.7/idle3.7
renaming build/scripts-3.7/2to3 to build/scripts-3.7/2to3-3.7
renaming build/scripts-3.7/pyvenv to build/scripts-3.7/pyvenv-3.7
./python  ../Tools/scripts/run_tests.py -v test_that_failed
/home/pmatos/Projects/cpython/debug/python -u -W default -bb -E -W error::BytesWarning -m test -r -w -j 0 -u all,-largefile,-audio,-gui -v test_that_failed
== CPython 3.7.0a0 (heads/master:48d9823a0e, Aug 14 2017, 11:53:56) [GCC 7.1.1 20170622 (Red Hat 7.1.1-3)]
== Linux-4.11.11-300.fc26.x86_64-x86_64-with-fedora-26-Twenty_Six little-endian
== hash algorithm: siphash24 64bit
== cwd: /home/pmatos/Projects/cpython/debug/build/test_python_22582
== CPU count: 16
== encodings: locale=UTF-8, FS=utf-8
Testing with flags: sys.flags(debug=0, inspect=0, interactive=0, optimize=0, dont_write_bytecode=0, no_user_site=0, no_site=0, ignore_environment=1, verbose=0, bytes_warning=2, quiet=0, hash_randomization=1, isolated=0)
Using random seed 8085058
Run tests in parallel using 18 child processes
0:00:00 load avg: 15.90 [1/1/1] test_that_failed failed
test test_that_failed crashed -- Traceback (most recent call last):
  File "/home/pmatos/Projects/cpython/Lib/test/libregrtest/runtest.py", line 163, in runtest_inner
    the_module = importlib.import_module(abstest)
  File "/home/pmatos/Projects/cpython/Lib/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 994, in _gcd_import
  File "<frozen importlib._bootstrap>", line 971, in _find_and_load
  File "<frozen importlib._bootstrap>", line 953, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'test.test_that_failed'

1 test failed:
    test_that_failed
Re-running failed tests in verbose mode
Re-running test 'test_that_failed' in verbose mode
test test_that_failed crashed -- Traceback (most recent call last):
  File "/home/pmatos/Projects/cpython/Lib/test/libregrtest/runtest.py", line 163, in runtest_inner
    the_module = importlib.import_module(abstest)
  File "/home/pmatos/Projects/cpython/Lib/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 994, in _gcd_import
  File "<frozen importlib._bootstrap>", line 971, in _find_and_load
  File "<frozen importlib._bootstrap>", line 953, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'test.test_that_failed'

1 test failed again:
    test_that_failed

Total duration: 355 ms
Tests result: FAILURE
make: *** [Makefile:1019: test] Error 2

----------
components: Tests
messages: 300241
nosy: pmatos
priority: normal
severity: normal
status: open
title: module test that failed doesn't exist

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31201>
_______________________________________

From report at bugs.python.org  Mon Aug 14 09:57:32 2017
From: report at bugs.python.org (aicirbs)
Date: Mon, 14 Aug 2017 13:57:32 +0000
Subject: [New-bugs-announce] [issue31202] Windows pathlib.Path.glob(pattern)
 fixed part of the pattern changed to lowercase whereas it should be
 unchanged.
Message-ID: <1502719052.2.0.11123711046.issue31202@psf.upfronthosting.co.za>


New submission from aicirbs:

Windows pathlib.Path.glob(pattern) fixed part of the pattern to lowercase whereas it should be unchanged.
Note that this issue is different from http://bugs.python.org/issue26655 : "pathlib glob case sensitivity issue on Windows" where it was asked to get the actual case of the folder from the file system.

Assuming a directory contains a folder named 'Folder'.
On Windows, calling pathlib.Path().glob('Folder') gives 'folder', but 'Folde?' will return 'Folder'
This is an issue for instance if trying to glob files to put them in an archive to be sent to a case sensitive platform.
glob.glob() does behave properly though, Windows pathlib.Path is the only platform which has such a behavior.

I would expect Path.glob to output the same as glob.glob() for each platform.
>From comments on http://bugs.python.org/issue19718 : "Path.glob() on case-insensitive Posix filesystems" it sounds that it should even match the native shell behavior.
And it looks like it is the case for linux and macOS, I tested that with the following script, whose results on win32, darwin and linux platforms follow:

```
#!/usr/bin/env python3.6
# Let's say this path exists : ./Folder/file
from pathlib import Path
import glob
import os
import sys
import subprocess

def ls(pattern):
    if sys.platform in ('win32', 'win64'):
        process = subprocess.run(['powershell', '-Command', f'dir {pattern}'], stdout=subprocess.PIPE, stderr=subprocess.STDOUT, cwd=os.getcwd(), encoding='utf-8')
        if process.returncode:
            return []
        # expected output:
        # 
        # 
        #     Directory: C:\path_to\Folder
        # 
        # 
        # Mode                LastWriteTime         Length Name
        # ----                -------------         ------ ----
        # -a----       2017-08-14     10:16              0 file
        lines = process.stdout.splitlines()
        folder = os.path.basename(lines[2].split()[-1])
        file = lines[7].split()[-1]
        result = f"{folder}{os.path.sep}{file}"
        return [result]
    else:
        cmd = ['ls', f'{pattern}']
        process = subprocess.run(' '.join(cmd), stdout=subprocess.PIPE, stderr=subprocess.PIPE, cwd=os.getcwd(), encoding='utf-8', shell=True)
        if process.returncode:
            return []
        return [process.stdout.strip()]

def main():
    print(sys.platform)
    p = Path('.')
    tests = ['Folder/*', 'FOlder/*', 'F?lder/*', 'FOlde?/*', 'folder/*', 'f?lder/*']
    for t in tests:
        print(f'{t}:')
        print(f'    Path.glob():  {[str(f) for f in p.glob(t)]}')
        print(f'    glob.glob():  {[f for f in glob.glob(str(p/t))]}')
        print(f'    shell:        {ls(str(p/t))}')

if __name__ == '__main__':
    main()
```

```
Output:
        win32                                            darwin                                        linux
1:      Folder/*:                                        Folder/*:                                     Folder/*:
1a:         Path.glob():  ['folder\\file']                   Path.glob():  ['Folder/file']                 Path.glob():  ['Folder/file']
1b:         glob.glob():  ['Folder\\file']                   glob.glob():  ['Folder/file']                 glob.glob():  ['Folder/file']
1c:         shell:        ['Folder\\file']                   shell:        ['Folder/file']                 shell:        ['Folder/file']
2:      FOlder/*:                                        FOlder/*:                                     FOlder/*:
2a:         Path.glob():  ['folder\\file']                   Path.glob():  ['FOlder/file']                 Path.glob():  []
2b:         glob.glob():  ['FOlder\\file']                   glob.glob():  ['FOlder/file']                 glob.glob():  []
2c:         shell:        ['FOlder\\file']                   shell:        ['FOlder/file']                 shell:        []
3:      F?lder/*:                                        F?lder/*:                                     F?lder/*:
3a:         Path.glob():  ['Folder\\file']                   Path.glob():  ['Folder/file']                 Path.glob():  ['Folder/file']
3b:         glob.glob():  ['Folder\\file']                   glob.glob():  ['Folder/file']                 glob.glob():  ['Folder/file']
3c:         shell:        ['Folder\\file']                   shell:        ['Folder/file']                 shell:        ['Folder/file']
4:      FOlde?/*:                                        FOlde?/*:                                     FOlde?/*:
4a:         Path.glob():  ['Folder\\file']                   Path.glob():  []                              Path.glob():  []
4b:         glob.glob():  ['Folder\\file']                   glob.glob():  []                              glob.glob():  []
4c:         shell:        ['Folder\\file']                   shell:        []                              shell:        []
5:      folder/*:                                        folder/*:                                     folder/*:
5a:         Path.glob():  ['folder\\file']                   Path.glob():  ['folder/file']                 Path.glob():  []
5b:         glob.glob():  ['folder\\file']                   glob.glob():  ['folder/file']                 glob.glob():  []
5c:         shell:        ['folder\\file']                   shell:        ['folder/file']                 shell:        []
6:      f?lder/*:                                        f?lder/*:                                     f?lder/*:
6a:         Path.glob():  ['Folder\\file']                   Path.glob():  []                              Path.glob():  []
6b:         glob.glob():  ['Folder\\file']                   glob.glob():  []                              glob.glob():  []
6c:         shell:        ['Folder\\file']                   shell:        []                              shell:        []
```

----------
components: Library (Lib), Windows
messages: 300245
nosy: aicirbs, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
pull_requests: 3128
severity: normal
status: open
title: Windows pathlib.Path.glob(pattern) fixed part of the pattern changed to lowercase whereas it should be unchanged.
type: behavior
versions: Python 3.4, Python 3.5, Python 3.6, Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31202>
_______________________________________

From report at bugs.python.org  Mon Aug 14 10:59:44 2017
From: report at bugs.python.org (Laurent GUERBY)
Date: Mon, 14 Aug 2017 14:59:44 +0000
Subject: [New-bugs-announce] [issue31203] socket.IP_PKTINFO is missing from
 python
Message-ID: <1502722784.42.0.622750235869.issue31203@psf.upfronthosting.co.za>


New submission from Laurent GUERBY:

The constant socket.IP_PKTINFO is missing.

Definition on Linux:

http://elixir.free-electrons.com/linux/v4.12.7/source/include/uapi/linux/in.h#L96

Exemple code that would benefit from the definition:

https://chiliproject.tetaneutral.net/projects/tetaneutral/wiki/Netconsole#nagios-selectors

Note: socket.IPV6_PKTINFO is defined :).

----------
components: Library (Lib)
messages: 300248
nosy: guerby
priority: normal
severity: normal
status: open
title: socket.IP_PKTINFO is missing from python
type: enhancement
versions: Python 2.7, Python 3.3, Python 3.4, Python 3.5, Python 3.6, Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31203>
_______________________________________

From report at bugs.python.org  Mon Aug 14 20:04:36 2017
From: report at bugs.python.org (Anthony Sottile)
Date: Tue, 15 Aug 2017 00:04:36 +0000
Subject: [New-bugs-announce] [issue31204] Support __fspath__ in shlex.quote
Message-ID: <1502755476.29.0.30041692032.issue31204@psf.upfronthosting.co.za>


New submission from Anthony Sottile:

Given shlex.quote often is used closely with calls in `subprocess`, should it also support quoting __fspath__ objects?

I'll write up a quick patch for this

----------
components: Library (Lib)
messages: 300270
nosy: Anthony Sottile
priority: normal
severity: normal
status: open
title: Support __fspath__ in shlex.quote
type: enhancement
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31204>
_______________________________________

From report at bugs.python.org  Mon Aug 14 21:13:03 2017
From: report at bugs.python.org (Terry J. Reedy)
Date: Tue, 15 Aug 2017 01:13:03 +0000
Subject: [New-bugs-announce] [issue31205] IDLE,
 configdialog: Factor out KeysPage class from ConfigDialog
Message-ID: <1502759583.57.0.904846181932.issue31205@psf.upfronthosting.co.za>


New submission from Terry J. Reedy:

Do as did with #31050 and GenPage class.  This should be done *before* #31001, highlights test.

----------
assignee: terry.reedy
components: IDLE
messages: 300273
nosy: terry.reedy
priority: normal
severity: normal
stage: test needed
status: open
title: IDLE, configdialog: Factor out KeysPage class from ConfigDialog
type: enhancement
versions: Python 3.6, Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31205>
_______________________________________

From report at bugs.python.org  Mon Aug 14 21:48:57 2017
From: report at bugs.python.org (Terry J. Reedy)
Date: Tue, 15 Aug 2017 01:48:57 +0000
Subject: [New-bugs-announce] [issue31206] IDLE,
 configdialog: Factor out HighPage class from ConfigDialog
Message-ID: <1502761737.83.0.0701165546864.issue31206@psf.upfronthosting.co.za>


New submission from Terry J. Reedy:

As with #31050, #31205

----------
assignee: terry.reedy
components: IDLE
messages: 300278
nosy: csabella, terry.reedy
priority: normal
severity: normal
stage: test needed
status: open
title: IDLE, configdialog: Factor out HighPage class from ConfigDialog
type: enhancement
versions: Python 3.6, Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31206>
_______________________________________

From report at bugs.python.org  Mon Aug 14 21:52:03 2017
From: report at bugs.python.org (Terry J. Reedy)
Date: Tue, 15 Aug 2017 01:52:03 +0000
Subject: [New-bugs-announce] [issue31207] IDLE,
 configdialog: Factor out ExtPage class from ConfigDialog
Message-ID: <1502761923.78.0.599195471762.issue31207@psf.upfronthosting.co.za>


Changes by Terry J. Reedy <tjreedy at udel.edu>:


----------
assignee: terry.reedy
components: IDLE
nosy: terry.reedy
priority: normal
severity: normal
stage: test needed
status: open
title: IDLE, configdialog: Factor out ExtPage class from ConfigDialog
versions: Python 3.6, Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31207>
_______________________________________

From report at bugs.python.org  Mon Aug 14 23:49:22 2017
From: report at bugs.python.org (TaoQingyun)
Date: Tue, 15 Aug 2017 03:49:22 +0000
Subject: [New-bugs-announce] [issue31208] Simplify `low_fds_to_close` in
 subprocess.py
Message-ID: <1502768962.86.0.659487659571.issue31208@psf.upfronthosting.co.za>


Changes by TaoQingyun <845767657 at qq.com>:


----------
components: Library (Lib)
files: subprocess.patch
keywords: patch
nosy: qingyunha
priority: normal
severity: normal
status: open
title: Simplify `low_fds_to_close` in subprocess.py
type: enhancement
versions: Python 3.7
Added file: http://bugs.python.org/file47082/subprocess.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31208>
_______________________________________

From report at bugs.python.org  Tue Aug 15 00:49:11 2017
From: report at bugs.python.org (Alex Hayes)
Date: Tue, 15 Aug 2017 04:49:11 +0000
Subject: [New-bugs-announce] [issue31209] MappingProxyType can not be pickled
Message-ID: <1502772551.88.0.290451219237.issue31209@psf.upfronthosting.co.za>


New submission from Alex Hayes:

I imagine that this is by design (likely because it can't be implemented) however I wanted to check if this was the case.

Instances of types.MappingProxyType can't be pickled.

For example;

```
import pickle
from types import MappingProxyType
eggs = MappingProxyType(dict(sausage=True))
pickle.dumps(eggs)
```

Raises: TypeError: can't pickle mappingproxy objects

----

Is this the desired behaviour or is it a bug?

----------
components: Library (Lib)
messages: 300284
nosy: Alex Hayes
priority: normal
severity: normal
status: open
title: MappingProxyType can not be pickled
type: behavior
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31209>
_______________________________________

From report at bugs.python.org  Tue Aug 15 08:29:44 2017
From: report at bugs.python.org (=?utf-8?q?C=C3=A9dric_Krier?=)
Date: Tue, 15 Aug 2017 12:29:44 +0000
Subject: [New-bugs-announce] [issue31210] Can not import site from
 sys.prefix containing DELIM
Message-ID: <1502800184.6.0.906228632756.issue31210@psf.upfronthosting.co.za>


New submission from C?dric Krier:

If the path on which Python is installed contains the DELIM, the resulted sys.path is split.
I think maybe there should be a escape mechanism for the PYTHONPATH.

----------
components: Interpreter Core
messages: 300293
nosy: ced
priority: normal
severity: normal
status: open
title: Can not import site from sys.prefix containing DELIM
type: crash

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31210>
_______________________________________

From report at bugs.python.org  Tue Aug 15 12:20:30 2017
From: report at bugs.python.org (Siming Yuan)
Date: Tue, 15 Aug 2017 16:20:30 +0000
Subject: [New-bugs-announce] [issue31211] distutils/util.py get_platform()
 does not identify linux-i686 platforms
Message-ID: <1502814030.02.0.254840991446.issue31211@psf.upfronthosting.co.za>


New submission from Siming Yuan:

in CentOS installations where both 32 and 64 bit libraries can co exist, and when python is compiled to run with 32-bit libraries (i686)

>>> from distutils.util import get_platform
>>> get_platform()
'linux-x86_64'

because the api only looks at OS flavor and not the actual binary architecture.

this string is used as part of PEP425 tags in the built wheel/egg file:
my_package-3.3.0-cp34-cp34m-linux-x86_64.whl

but this creates a mismatch with PIP - pip.pep425tags returns "linux-i686" instead on the same python instance, addressed by:

# snippet pip code
def _is_running_32bit():
    return sys.maxsize == 2147483647
    
if result == "linux_x86_64" and _is_running_32bit():
    # 32 bit Python program (running on a 64 bit Linux): pip should only
    # install and run 32 bit compiled extensions in that case.
    result = "linux_i686"

so the end result is any packages built with sdist_wheel (using setuptools/distutils) is not installable on the same instance.

Of course the workaround is to overwrite that and provide --plat-name linux_i686. 

but the question is - shouldn't the tags line up?

----------
components: Distutils
messages: 300300
nosy: dstufft, merwok, siming85
priority: normal
severity: normal
status: open
title: distutils/util.py get_platform() does not identify linux-i686 platforms
versions: Python 3.4, Python 3.5, Python 3.6, Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31211>
_______________________________________

From report at bugs.python.org  Tue Aug 15 13:26:32 2017
From: report at bugs.python.org (Dave Johansen)
Date: Tue, 15 Aug 2017 17:26:32 +0000
Subject: [New-bugs-announce] [issue31212] min date can't be converted to
 timestamp
Message-ID: <1502817992.35.0.826006067807.issue31212@psf.upfronthosting.co.za>


New submission from Dave Johansen:

This worked in Python 3.6.0 and before:
```
from datetime import datetime
d = datetime(1, 1, 1, 0, 0, 0)
d.timestamp()
```

The error output is:
```
ValueError: year 0 is out of range
```

But it used to return `-62135658000.0`.

Appears to be related to https://bugs.python.org/issue29921

----------
messages: 300303
nosy: Dave Johansen
priority: normal
severity: normal
status: open
title: min date can't be converted to timestamp
type: behavior
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31212>
_______________________________________

From report at bugs.python.org  Tue Aug 15 13:43:59 2017
From: report at bugs.python.org (Christopher Stelma)
Date: Tue, 15 Aug 2017 17:43:59 +0000
Subject: [New-bugs-announce] [issue31213] __context__ reset to None in
 nested exception
Message-ID: <1502819039.28.0.810222147268.issue31213@psf.upfronthosting.co.za>


New submission from Christopher Stelma:

When I try to re-raise an exception with a __cause__ inside a nested exception, the __context__ of the outer __context__ appears to be reset to None.

e.g.

>>> try:
...     try:
...         raise Exception('foo')
...     except Exception as foo:
...         print(foo, foo.__context__, foo.__cause__)
...         try:
...             raise Exception('bar') from foo
...         except Exception as bar:
...             print(bar, bar.__context__, bar.__context__.__context__)
...             raise foo from bar
... except Exception as foo:
...     wat = foo
...
foo None None
bar foo None
>>> print(wat, wat.__context__, wat.__context__.__context__)
foo bar None
>>> print(wat, wat.__cause__, wat.__cause__.__context__)
foo bar None
>>> print(wat, wat.__cause__, wat.__cause__.__cause__)
foo bar foo

here, between "raise foo from bar" and the end, bar.__context__ goes from foo to None.  since bar is only raised once, clearly in the context of foo (twice over), I would expect bar.__context__ to stay foo.

----------
messages: 300305
nosy: Christopher Stelma
priority: normal
severity: normal
status: open
title: __context__ reset to None in nested exception
type: behavior
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31213>
_______________________________________

From report at bugs.python.org  Tue Aug 15 18:41:39 2017
From: report at bugs.python.org (Chris Lovett)
Date: Tue, 15 Aug 2017 22:41:39 +0000
Subject: [New-bugs-announce] [issue31214] os.walk has a bug on Windows
Message-ID: <1502836899.02.0.348534526331.issue31214@psf.upfronthosting.co.za>


New submission from Chris Lovett:

When I walk a directory recursively, it is tacking on an additional non-existant file from one of the subdirectories.  Here's the code:

    def copy_dir(self, src, dest):
        result = sftp.mkdir(dest)
        for dirname, dirnames, filenames in os.walk(src):
            for subdirname in dirnames:
                print("entering dir:" + subdirname)
                self.copy_dir(os.path.join(src, subdirname), os.path.join(dest, subdirname))
            for filename in filenames:
                print("copying:" + filename)

Here's the output:

entering dir:include
copying:CallbackInterface.h
copying:ClockInterface.h
entering dir:tcc
copying:CallbackInterface.tcc
copying:CMakeLists.txt
copying:darknet.i
copying:darknet.i.h
copying:darknet.obj
copying:darknet.py
copying:darknetImageNetLabels.txt
copying:darknetPYTHON_wrap.cxx
copying:darknetPYTHON_wrap.h
copying:darknet_config.json
copying:demo.py
copying:demoHelper.py
copying:OpenBLASSetup.cmake
copying:runtest.sh
copying:schoolbus.png
copying:CallbackInterface.h
copying:ClockInterface.h
copying:CallbackInterface.tcc

The last 3 files listed here doesn't exist, they are a repeat of the files found in the subdirectories.

----------
components: Windows
messages: 300313
nosy: clovett, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: os.walk has a bug on Windows
type: behavior
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31214>
_______________________________________

From report at bugs.python.org  Wed Aug 16 01:17:39 2017
From: report at bugs.python.org (Nick Coghlan)
Date: Wed, 16 Aug 2017 05:17:39 +0000
Subject: [New-bugs-announce] [issue31215] Add version changed notes for
 OpenSSL 1.1.0 compatibility
Message-ID: <1502860659.7.0.930691554308.issue31215@psf.upfronthosting.co.za>


New submission from Nick Coghlan:

https://bugs.python.org/issue26470 updated the ssl and hashlib modules in 2.7 and 3.5+ to be compatible with OpenSSL 1.1.0.

However, it's currently unclear what the *minimum* versions actually are for platforms that want to drop support for OpenSSL 1.0.x, and it's particularly unclear in 2.7, as that lacks the deprecation warning for OpenSSL 0.9.8, 1.0.0 and 1.0.1 that was introduced in Python 3.6.

This doesn't rise to the level of something to be documented as a new feature in the 2.7 What's New, but I think a version changed note in the opening section of the `ssl` module documentation would be appropriate:

2.7 branch:

    .. versionchanged: 2.7.13
       Updated to support linking with OpenSSL 1.1.0

3.6 and master branches (immediately before the 3.6 deprecation notice):

    .. versionchanged: 3.5.3
       Updated to support linking with OpenSSL 1.1.0

----------
assignee: docs at python
components: Documentation
messages: 300326
nosy: christian.heimes, docs at python, ncoghlan
priority: normal
severity: normal
stage: needs patch
status: open
title: Add version changed notes for OpenSSL 1.1.0 compatibility
type: enhancement
versions: Python 2.7, Python 3.6, Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31215>
_______________________________________

From report at bugs.python.org  Wed Aug 16 01:46:56 2017
From: report at bugs.python.org (TaoQingyun)
Date: Wed, 16 Aug 2017 05:46:56 +0000
Subject: [New-bugs-announce] [issue31216] Add ensure_ascii argument to
 json.tool
Message-ID: <1502862416.63.0.620713992739.issue31216@psf.upfronthosting.co.za>


New submission from TaoQingyun:

Sometimes I want this
```
$ echo -e '"\xe4\xbd\xa0\xe5\xa5\xbd"' | ./python -m json.tool --no-escape                         
"??"
```

not just 
```
$ echo -e '"\xe4\xbd\xa0\xe5\xa5\xbd"' | ./python -m json.tool
"\u4f60\u597d" 
```

----------
components: Library (Lib)
files: jsontool.patch
keywords: patch
messages: 300327
nosy: qingyunha
priority: normal
severity: normal
status: open
title: Add ensure_ascii argument to  json.tool
type: enhancement
versions: Python 3.7
Added file: http://bugs.python.org/file47083/jsontool.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31216>
_______________________________________

From report at bugs.python.org  Wed Aug 16 05:41:40 2017
From: report at bugs.python.org (STINNER Victor)
Date: Wed, 16 Aug 2017 09:41:40 +0000
Subject: [New-bugs-announce] [issue31217] test_code leaked [1, 1,
 1] memory blocks on x86 Gentoo Refleaks 3.6
Message-ID: <1502876500.53.0.298130981343.issue31217@psf.upfronthosting.co.za>


New submission from STINNER Victor:

http://buildbot.python.org/all/builders/x86%20Gentoo%20Refleaks%203.6/builds/58/steps/test/logs/stdio

test_code leaked [1, 1, 1] memory blocks, sum=3
(...)
Re-running test 'test_code' in verbose mode
(...)
test_code leaked [1, 1, 1] memory blocks, sum=3


I'm unable to reproduce the bug:

haypo at selma$ ./python -m test -R 3:3 test_code 
(...)
1 test OK.

----------
components: Tests
messages: 300344
nosy: haypo
priority: normal
severity: normal
status: open
title: test_code leaked [1, 1, 1] memory blocks on x86 Gentoo Refleaks 3.6
type: resource usage
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31217>
_______________________________________

From report at bugs.python.org  Wed Aug 16 06:48:18 2017
From: report at bugs.python.org (Calvin)
Date: Wed, 16 Aug 2017 10:48:18 +0000
Subject: [New-bugs-announce] [issue31218] del expects __delitem__ if
 __setitem__ is defined
Message-ID: <1502880498.02.0.663608848315.issue31218@psf.upfronthosting.co.za>


New submission from Calvin:

I noticed some odd behaviour on classes defining __setitem__.
Using del on a class defining __setitem__ but not __delitem__ results in "AttributeError: __delitem__".
On classes definig neiter __setitem__ nor __delitem__ on the other hand this results in "TypeError: 'WithoutSetItem' object doesn't support item deletion".

See the appended example script.

----------
files: delitem_example.py
messages: 300346
nosy: raumzeitkeks
priority: normal
severity: normal
status: open
title: del expects __delitem__ if __setitem__ is defined
type: behavior
versions: Python 3.6
Added file: http://bugs.python.org/file47085/delitem_example.py

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31218>
_______________________________________

From report at bugs.python.org  Wed Aug 16 09:03:51 2017
From: report at bugs.python.org (Basiruddin Syarifan Radifan)
Date: Wed, 16 Aug 2017 13:03:51 +0000
Subject: [New-bugs-announce] [issue31219] test
Message-ID: <1502888631.71.0.92474975609.issue31219@psf.upfronthosting.co.za>


Changes by Basiruddin Syarifan Radifan <newradifans at gmail.com>:


----------
components: Argument Clinic
files: indonesia.htm
nosy: Basiruddin Syarifan Radifan, larry
priority: normal
severity: normal
status: open
title: test
type: behavior
versions: Python 3.6
Added file: http://bugs.python.org/file47086/indonesia.htm

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31219>
_______________________________________

From report at bugs.python.org  Wed Aug 16 09:07:13 2017
From: report at bugs.python.org (Basiruddin Syarifan Radifan)
Date: Wed, 16 Aug 2017 13:07:13 +0000
Subject: [New-bugs-announce] [issue31220] test
Message-ID: <1502888833.1.0.837244974032.issue31220@psf.upfronthosting.co.za>


Changes by Basiruddin Syarifan Radifan <newradifans at gmail.com>:


----------
components: Argument Clinic
files: _.txt
nosy: Basiruddin Syarifan Radifan, larry
priority: normal
severity: normal
status: open
title: test
type: crash
versions: Python 3.4
Added file: http://bugs.python.org/file47087/_.txt

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31220>
_______________________________________

From report at bugs.python.org  Wed Aug 16 11:59:10 2017
From: report at bugs.python.org (STINNER Victor)
Date: Wed, 16 Aug 2017 15:59:10 +0000
Subject: [New-bugs-announce] [issue31221] Tools/scripts/patchcheck.py must
 ignore changes in Modules/expat/ and Modules/zlib/ subdirectories
Message-ID: <1502899150.0.0.5988032686.issue31221@psf.upfronthosting.co.za>


New submission from STINNER Victor:

My PR https://github.com/python/cpython/pull/3106 failed because patchcheck wants me to normalize spaces of Modules/expat/siphash.h, whereas this file is copied from libexpat: see bpo-30947.

Attached PR changes Tools/scripts/patchcheck.py to ignore changes in Modules/expat/ and Modules/zlib/ subdirectories. The change also drops support for Mercurial, since CPython migrated to Git.

----------
messages: 300372
nosy: haypo
priority: normal
severity: normal
status: open
title: Tools/scripts/patchcheck.py must ignore changes in Modules/expat/ and Modules/zlib/ subdirectories
versions: Python 2.7, Python 3.6, Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31221>
_______________________________________

From report at bugs.python.org  Wed Aug 16 13:01:45 2017
From: report at bugs.python.org (Paul G)
Date: Wed, 16 Aug 2017 17:01:45 +0000
Subject: [New-bugs-announce] [issue31222] datetime.py implementation of
 .replace inconsistent with C implementation
Message-ID: <1502902905.88.0.809330827331.issue31222@psf.upfronthosting.co.za>


New submission from Paul G:

In the .py implementation of datetime.replace (and date.replace and time.replace), the new datetime is created using the datetime type:

https://github.com/python/cpython/blob/master/Lib/datetime.py#L1578

But in the C source, it is created from type(self):

https://github.com/python/cpython/blob/master/Modules/_datetimemodule.c#L5046

I think the second should be the preferred behavior, so the datetime.py source should be updated to reflect that it's calling self.__class__(...) rather than datetime(...). I can prepare a PR if this would be desirable.

(That said, I'm not 100% clear under what circumstances the code in datetime.py is actually *used*, so I'm not sure how to write tests for it - is datetime.py essentially documentation, or is there a way to explicitly fall back to it?)

Per this issue on the pypy3 tracker:

https://bitbucket.org/pypy/pypy/issues/2635/datetimereplace-always-returns

----------
components: Interpreter Core
messages: 300377
nosy: p-ganssle
priority: normal
severity: normal
status: open
title: datetime.py implementation of .replace inconsistent with C implementation
versions: Python 3.5, Python 3.6, Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31222>
_______________________________________

From report at bugs.python.org  Wed Aug 16 14:05:41 2017
From: report at bugs.python.org (adminos)
Date: Wed, 16 Aug 2017 18:05:41 +0000
Subject: [New-bugs-announce] [issue31223] Hacked By KingSkrupellos
 Cyberizm.Org Digital Security Technological Turkish Moslem Army
Message-ID: <1502906741.75.0.248119207821.issue31223@psf.upfronthosting.co.za>


New submission from adminos:

Hacked By KingSkrupellos Cyberizm.Org Digital Security Technological Turkish Moslem Army

----------
components: Argument Clinic
messages: 300383
nosy: adminos, doko, larry
priority: normal
severity: normal
status: open
title: Hacked By KingSkrupellos Cyberizm.Org Digital Security Technological Turkish Moslem Army
type: compile error
versions: Python 3.4

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31223>
_______________________________________

From report at bugs.python.org  Thu Aug 17 06:55:01 2017
From: report at bugs.python.org (Marco Buttu)
Date: Thu, 17 Aug 2017 10:55:01 +0000
Subject: [New-bugs-announce] [issue31224] Missing definition of frozen module
Message-ID: <1502967301.1.0.268769760961.issue31224@psf.upfronthosting.co.za>


New submission from Marco Buttu:

In the doc there are several hints [*] to frozen modules, but there is no definition of them.  Do you think we should add a "frozen module" definition to the glossary?

* Doc/library/importlib.rst, Doc/library/imp.rst, Doc/reference/import.rst, ...

----------
assignee: docs at python
components: Documentation
messages: 300405
nosy: docs at python, marco.buttu
priority: normal
severity: normal
status: open
title: Missing definition of frozen module
type: enhancement
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31224>
_______________________________________

From report at bugs.python.org  Thu Aug 17 06:59:35 2017
From: report at bugs.python.org (ksqsf)
Date: Thu, 17 Aug 2017 10:59:35 +0000
Subject: [New-bugs-announce] [issue31225] allow hashlib `update' to accept
 str
Message-ID: <1502967575.03.0.97521659396.issue31225@psf.upfronthosting.co.za>


New submission from ksqsf:

It'll be nice if `update' can accept str directly, rather than str.encode first.

One of the most common use cases is "m.update(s.encode())", so it shouldn't make a big difference to just use "m.update(s)"; after all, users who want to specify encoding explicitly can still do so.

----------
components: Library (Lib)
messages: 300406
nosy: ksqsf
priority: normal
severity: normal
status: open
title: allow hashlib `update' to accept str
type: enhancement
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31225>
_______________________________________

From report at bugs.python.org  Thu Aug 17 09:58:18 2017
From: report at bugs.python.org (Vidar Fauske)
Date: Thu, 17 Aug 2017 13:58:18 +0000
Subject: [New-bugs-announce] [issue31226] shutil.rmtree fails when target
 has an internal directory junction (Windows)
Message-ID: <1502978298.87.0.60982184572.issue31226@psf.upfronthosting.co.za>


New submission from Vidar Fauske:

On Windows (Windows 10 in my case), given the following directory structure:
- rootfolder
 - a
 - b
  - junc (directory junction to ../a)

a call to `shutil.rmtree('root')` will fail with an exception `FileNotFoundError: [WinError 3]`, in a call to `os.listdir()` in `_rmtree_unsafe`. See attached minimal working example.

Note that sorting order is important: A link in 'a' pointing to 'b' does not fail. This is because `os.listdir()` raises an exception for 'b/junc' when its target ('a') has already been deleted.

Also, note that this is only for junctions, not directory links (`mklink /J` vs `mklink /D`), because:
 - Directory links flag false in the `stat.S_ISDIR(os.lstat('b/junc').st_mode)` test while junctions do not.
 - `os.islink()` returns false for both junctions, while directory links do not.

Indicated Python versions are those which I have personally tested on, and observed this behavior.

Current use case: Deleting a folder tree generated by an external tool, which creates junction links as part of its normal operation ('lerna' tool for the 'npm' javascript package manager).

----------
components: IO, Library (Lib), Windows
files: mwe.py
messages: 300424
nosy: paul.moore, steve.dower, tim.golden, vidartf, zach.ware
priority: normal
severity: normal
status: open
title: shutil.rmtree fails when target has an internal directory junction (Windows)
versions: Python 3.3, Python 3.6
Added file: http://bugs.python.org/file47090/mwe.py

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31226>
_______________________________________

From report at bugs.python.org  Thu Aug 17 11:32:04 2017
From: report at bugs.python.org (STINNER Victor)
Date: Thu, 17 Aug 2017 15:32:04 +0000
Subject: [New-bugs-announce] [issue31227] regrtest: reseed random with the
 same seed before running a test file
Message-ID: <1502983924.76.0.0545413601956.issue31227@psf.upfronthosting.co.za>


New submission from STINNER Victor:

Attached PR changes regrtest to reseed the random RNG before each test file. Use also more entropy for the seed: 2**32 (32 bits) rather than
10_000_000 (24 bits).

The change should avoid random failure of test_tools when hunting reference leaks: see bpo-31174.

Maybe it will also reduce false positive when hunting memory leaks, like bpo-31217.

----------
components: Tests
messages: 300438
nosy: haypo
priority: normal
severity: normal
status: open
title: regrtest: reseed random with the same seed before running a test file
versions: Python 2.7, Python 3.6, Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31227>
_______________________________________

From report at bugs.python.org  Thu Aug 17 13:28:25 2017
From: report at bugs.python.org (Terry J. Reedy)
Date: Thu, 17 Aug 2017 17:28:25 +0000
Subject: [New-bugs-announce] [issue31228] Subprocess.Popen crash w/ Win10,
 debug32, bad file, and PIPE
Message-ID: <1502990905.97.0.599281951083.issue31228@psf.upfronthosting.co.za>


New submission from Terry J. Reedy:

Running Debug|Win32 interpreter...
Python 3.7.0a0 (heads/master:b907abc885, Aug 17 2017, 11:41:56) [MSC v.1900 32 bit (Intel)] on win32

I get the following crash report
--------------------------------------------------------------
Microsoft Visual C++ Runtime Library
(X) Debug Assertion Failed!
Program: F:\dev\3x\PCBuild\win32\python_d.exe
File: minkernel\crts\ucrt\src\appcrt\lowio\close.cpp
Line: 48

Expression: (fh >= 0 && (unsigned)fh < (unsigned)_nhandle
...
(Press Retry to debug the application)
----------------------------------------------------------------

when running the following, extracted from test.pythoninfo

import subprocess
proc = subprocess.Popen(["gdb", "-nx", "--version"],
                        stdout=subprocess.PIPE,
                        stderr=subprocess.PIPE,
                        universal_newlines=True)

If I do any of the following, I get the expected FileNotFound error.
1. Run with installed normal 64-bit 3.6.
2. Replace 'gdb' with 'python' (but 'xyz' still crashes).
3. Remove the std... options or replace subprocess.PIPE with a real file.

----------
components: Library (Lib), Windows
messages: 300445
nosy: paul.moore, steve.dower, terry.reedy, tim.golden, zach.ware
priority: normal
severity: normal
stage: needs patch
status: open
title: Subprocess.Popen crash w/ Win10, debug32, bad file, and PIPE
type: crash
versions: Python 3.6, Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31228>
_______________________________________

From report at bugs.python.org  Thu Aug 17 14:02:16 2017
From: report at bugs.python.org (Oren Milman)
Date: Thu, 17 Aug 2017 18:02:16 +0000
Subject: [New-bugs-announce] [issue31229] wrong error messages when too many
 kwargs are received
Message-ID: <1502992936.21.0.653436354644.issue31229@psf.upfronthosting.co.za>


New submission from Oren Milman:

Some functions produce wrong error messages in case they receive too many
keyword arguments:
- in Objects/exceptions.c - ImportError_init:
    >>> ImportError(1, 2, 3, 4, a=5, b=6, c=7)
    TypeError: ImportError() takes at most 2 arguments (3 given)
- in Python/bltinmodule.c - min_max:
    >>> min(1, 2, 3, 4, a=5, b=6, c=7)
    TypeError: function takes at most 2 arguments (3 given)
    >>> max(1, 2, 3, 4, a=5, b=6, c=7)
    TypeError: function takes at most 2 arguments (3 given)
- in Modules/itertoolsmodule.c - product_new:
    >>> itertools.product(0, a=1, b=2, c=3, d=4, e=5, f=6)
    TypeError: product() takes at most 1 argument (6 given)
- in Python/bltinmodule.c - builtin_print: 
    >>> print(0, a=1, b=2, c=3, d=4, e=5, f=6)
    TypeError: print() takes at most 4 arguments (6 given)

ISTM that changing these error messages to refer to 'keyword arguments' instead
of 'arguments' is a possible solution. (e.g. the last one would become
'print() takes at most 4 keyword arguments (6 given)'

To do that, I changed two 'takes at most' PyErr_Format calls in Python/getargs.c.
I ran the test suite, and it seems that this patch doesn't break anything.
The diff file is attached.
(I didn't open a PR before hearing your opinion, as ISTM that changing code in
getargs.c is a delicate thing.)

what do you think?

----------
components: Interpreter Core
messages: 300451
nosy: Oren Milman
priority: normal
severity: normal
status: open
title: wrong error messages when too many kwargs are received
type: behavior
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31229>
_______________________________________

From report at bugs.python.org  Thu Aug 17 22:25:30 2017
From: report at bugs.python.org (Nick Coghlan)
Date: Fri, 18 Aug 2017 02:25:30 +0000
Subject: [New-bugs-announce] [issue31230] Add __code__ attributes to
 generator and coroutine objects
Message-ID: <1503023130.32.0.241759623891.issue31230@psf.upfronthosting.co.za>


New submission from Nick Coghlan:

As covered in issue 31183, correctly disassembling functions, generator-iterators, coroutines, and async generator-iterators currently requires checking for 4 different attributes:

* __code__ (functions)
* gi_code (generator iterators)
* cr_code (coroutines)
* ag_code (async generator iterators)

This could be simplified if the latter three all *also* exposed their code attributes as `__code__`.

----------
messages: 300468
nosy: ncoghlan
priority: normal
severity: normal
stage: needs patch
status: open
title: Add __code__ attributes to generator and coroutine objects
type: enhancement
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31230>
_______________________________________

From report at bugs.python.org  Fri Aug 18 02:38:47 2017
From: report at bugs.python.org (Han Lee)
Date: Fri, 18 Aug 2017 06:38:47 +0000
Subject: [New-bugs-announce] [issue31231] Travis CI mac test broken:
Message-ID: <1503038327.43.0.708294572666.issue31231@psf.upfronthosting.co.za>


New submission from Han Lee:

CPython: master branch

In mac and window create python.exe not python https://github.com/python/cpython/blob/master/Makefile.pre.in#L154

But Travis CI and docs use ./python command for test
https://docs.python.org/devguide/

https://travis-ci.org/python/cpython/jobs/265843638
/Users/travis/.travis/job_stages: line 73: ./python: is a directory

----------
messages: 300474
nosy: sn0wle0pard
priority: normal
severity: normal
status: open
title: Travis CI mac test broken:

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31231>
_______________________________________

From report at bugs.python.org  Fri Aug 18 07:07:17 2017
From: report at bugs.python.org (Nick Coghlan)
Date: Fri, 18 Aug 2017 11:07:17 +0000
Subject: [New-bugs-announce] [issue31232] Backport the new custom "print >>
 sys.stderr" error message?
Message-ID: <1503054437.52.0.692993582451.issue31232@psf.upfronthosting.co.za>


New submission from Nick Coghlan:

https://bugs.python.org/issue30721 introduces a new custom error message, such that in 3.7+ "print >> sys.stderr" will report:

```
>>> import sys
>>> print >> sys.stderr
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: unsupported operand type(s) for >>: 'builtin_function_or_method' and '_io.TextIOWrapper'. Did you mean "print(<message>, file=<output_stream>)"?
```

This is quite clearly an enhancement to the error reporting rather than a bug fix, but similar to the syntax errors for print statements without parentheses, it's an enhancement that only touches an error handling code path (specifically, the one where >> is already in the process of raising TypeError after operand coercion failed), and relates to an issue that a lot of ad hoc Python scripts are likely to encounter.

As such, before I propose it as a downstream patch for Fedora's Python 3.6 stack, I figured I'd propose it as an upstream maintenance backport first.

----------
messages: 300486
nosy: encukou, ncoghlan, ned.deily
priority: normal
severity: normal
status: open
title: Backport the new custom "print >> sys.stderr" error message?
type: enhancement
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31232>
_______________________________________

From report at bugs.python.org  Fri Aug 18 12:24:59 2017
From: report at bugs.python.org (STINNER Victor)
Date: Fri, 18 Aug 2017 16:24:59 +0000
Subject: [New-bugs-announce] [issue31233] socketserver.ThreadingMixIn leaks
 running threads after server_close()
Message-ID: <1503073499.39.0.93886151367.issue31233@psf.upfronthosting.co.za>


New submission from STINNER Victor:

test_logging has a race condition: sometimes, it leaks dangling threads on FreeBSD: bpo-30830.

I identified the root issue: socketserver.ThreadingMixIn spawns threads without waiting for their completion in server_close(). This issue is very similar to socketserver.ForkingMixIn which leaks child processes and so create zombie processes. See bpo-31151.

----------
components: Library (Lib)
messages: 300514
nosy: haypo
priority: normal
severity: normal
status: open
title: socketserver.ThreadingMixIn leaks running threads after server_close()
type: resource usage
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31233>
_______________________________________

From report at bugs.python.org  Fri Aug 18 13:12:58 2017
From: report at bugs.python.org (STINNER Victor)
Date: Fri, 18 Aug 2017 17:12:58 +0000
Subject: [New-bugs-announce] [issue31234] Make support.threading_cleanup()
 stricter
Message-ID: <1503076378.29.0.950087882836.issue31234@psf.upfronthosting.co.za>


New submission from STINNER Victor:

Currently, support.threading_cleanup() waits 1 second before emitting a warning about threads running in the background. I propose to emit immediately the warning to be able to catch bugs quicker, bugs like bpo-31233 or bpo-30830.

The problem is that currently, 12 test files still leak threads:
---
12 tests altered the execution environment:
    test_asyncio test_concurrent_futures test_fork1 test_httpservers
    test_logging test_pydoc test_ssl test_thread test_threaded_import
    test_threading test_wait3 test_xmlrpc
---

Since threading_cleanup() warnings now mark tests as failed, we should first fix these tests before being able to make support.threading_cleanup() stricter.

----------
components: Tests
messages: 300519
nosy: haypo
priority: normal
severity: normal
status: open
title: Make support.threading_cleanup() stricter
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31234>
_______________________________________

From report at bugs.python.org  Fri Aug 18 17:42:26 2017
From: report at bugs.python.org (STINNER Victor)
Date: Fri, 18 Aug 2017 21:42:26 +0000
Subject: [New-bugs-announce] [issue31235] test_logging: ResourceWarning:
 unclosed <socket.socket ...>
Message-ID: <1503092546.47.0.171395017879.issue31235@psf.upfronthosting.co.za>


New submission from STINNER Victor:

Sometimes, running test_logging emits such warning:

ResourceWarning: unclosed <socket.socket fd=6, family=AddressFamily.AF_INET, type=536870913, proto=0, laddr=('127.0.0.1', 31191), raddr=('127.0.0.1', 31192)>

I'm not sure where it does come from.

See also bpo-30830 (test_logging leaks dangling threads on FreeBSD).

----------
messages: 300534
nosy: haypo
priority: normal
severity: normal
status: open
title: test_logging: ResourceWarning: unclosed <socket.socket ...>
type: resource usage
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31235>
_______________________________________

From report at bugs.python.org  Fri Aug 18 17:51:19 2017
From: report at bugs.python.org (Oren Milman)
Date: Fri, 18 Aug 2017 21:51:19 +0000
Subject: [New-bugs-announce] [issue31236] improve some error messages of
 min() and max()
Message-ID: <1503093079.31.0.160974520345.issue31236@psf.upfronthosting.co.za>


New submission from Oren Milman:

currently, we have the following:
    >>> min(0, a=1)
    TypeError: 'a' is an invalid keyword argument for this function
    >>> max(0, a=1)
    TypeError: 'a' is an invalid keyword argument for this function
    >>> max(0, a=1, b=2, c=3)
    TypeError: function takes at most 2 arguments (3 given)
    >>> min(0, a=1, b=2, c=3)
    TypeError: function takes at most 2 arguments (3 given)

ISTM it would be preferable for min() and max() to have error messages similar
to those of int():
    >>> int(0, a=1)
    TypeError: 'a' is an invalid keyword argument for int()
    >>> int(0, a=1, b=2)
    TypeError: int() takes at most 2 arguments (3 given)

we can achieve this by making a small change in Python/bltinmodule.c in
min_max (I would open a PR soon), and by resolving #31229.

----------
components: IO
messages: 300539
nosy: Oren Milman
priority: normal
severity: normal
status: open
title: improve some error messages of min() and max()
type: behavior
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31236>
_______________________________________

From report at bugs.python.org  Fri Aug 18 19:39:31 2017
From: report at bugs.python.org (=?utf-8?q?=C5=81ukasz_Langa?=)
Date: Fri, 18 Aug 2017 23:39:31 +0000
Subject: [New-bugs-announce] [issue31237] test_gdb disables 25% of tests in
 optimized builds
Message-ID: <1503099571.25.0.918465457775.issue31237@psf.upfronthosting.co.za>


New submission from ?ukasz Langa:

We are skipping a lot of tests when optimizations are enabled (which essentially means: compiled without `--with-pydebug`). This seems overly aggressive since most Python users are using the gdb bindings with a non-debug build.

I think we should have tests for py-bt, py-up, printing globals, etc. that run on a non-debug build.

----------
components: Tests
messages: 300554
nosy: benjamin.peterson, haypo, lukasz.langa, pitrou
priority: normal
severity: normal
status: open
title: test_gdb disables 25% of tests in optimized builds
versions: Python 3.6, Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31237>
_______________________________________

From report at bugs.python.org  Fri Aug 18 19:50:45 2017
From: report at bugs.python.org (STINNER Victor)
Date: Fri, 18 Aug 2017 23:50:45 +0000
Subject: [New-bugs-announce] [issue31238] pydoc: ServerThread.stop() leaves
 a dangling thread
Message-ID: <1503100245.68.0.356327779955.issue31238@psf.upfronthosting.co.za>


New submission from STINNER Victor:

When using https://github.com/python/cpython/pull/3138 I see that test_pydoc leaves a dangling thread. The bug comes from test_server() which uses pydoc ServerThread.

ServerThread.stop() and test_pydoc don't join the thread. Moreover, ServerThread.docserver has a reference cycle through the DocServer.callback attribute.

Attached PR modifies ServerThread.stop() to join itself (the thread), to wait until the HTTP server completes, and then explicitly break the reference cycle.

With the PR, pydoc may hang if a bad HTTP client. So another option is to only modify test_pydoc to join() + break the ref cycle.

----------
components: Library (Lib)
messages: 300556
nosy: haypo
priority: normal
severity: normal
status: open
title: pydoc: ServerThread.stop() leaves a dangling thread
type: resource usage
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31238>
_______________________________________

From report at bugs.python.org  Fri Aug 18 20:34:46 2017
From: report at bugs.python.org (Vlad Shcherbina)
Date: Sat, 19 Aug 2017 00:34:46 +0000
Subject: [New-bugs-announce] [issue31239] namedtuple comparison ignores types
Message-ID: <1503102886.22.0.684881922267.issue31239@psf.upfronthosting.co.za>


New submission from Vlad Shcherbina:

Toy example:

>>> from collections import namedtuple
>>> Rectangle = namedtuple('Rectangle', 'width height')
>>> Ellipse = namedtuple('Ellipse', 'x_axis y_axis')
>>> Rectangle(width=1, height=2) == Ellipse(x_axis=1, y_axis=2)
True

I understand this happens because namedtuple inherits comparisons and hash from the regular tuple.
However, this seems like confusing and dangerous behavior.
It is especially annoying to debug when these tuples are used as dict keys (repr() shows one thing and dict item access shows another).

Why would anyone mix named tuples of different types?
Here is a use case: typing.NamedTuple together with typing.Union would be a neat way to express algebraic data types.

Haskell's
data Shape = Rectangle Int Int | Ellipse Int Int    deriving(Eq, Show)

would become
Shape = Union[Rectangle, Ellipse]

except that it does not work as expected because of the flawed comparisons.

----------
components: Library (Lib)
messages: 300562
nosy: vlad
priority: normal
severity: normal
status: open
title: namedtuple comparison ignores types
type: behavior
versions: Python 2.7, Python 3.3, Python 3.4, Python 3.5, Python 3.6, Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31239>
_______________________________________

From report at bugs.python.org  Sun Aug 20 01:14:06 2017
From: report at bugs.python.org (Jim Dennis)
Date: Sun, 20 Aug 2017 05:14:06 +0000
Subject: [New-bugs-announce] [issue31240] Add lazy evaluation support for
 dict.setdefault()
Message-ID: <1503206046.25.0.821679783552.issue31240@psf.upfronthosting.co.za>


New submission from Jim Dennis:

Code such as mydict.setdefault('eggs', []) will needlessly incur the cost of instantiating a list even when 'eggs' is already a valid key in mydict.  collections.defaultdict will not do this.  detecting and automatically calling "callable" and "type" objects (implicit laziness) would break cases where callables and types are used as first class values.

Add keyword argument: lazy?  Thus mydict.setdefault('eggs',list, lazy=True) would generate a new list only when necessary while the default would still be to append a reference to list (first class) objects).

----------
messages: 300596
nosy: jimd
priority: normal
severity: normal
status: open
title: Add lazy evaluation support for dict.setdefault()
type: enhancement

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31240>
_______________________________________

From report at bugs.python.org  Sun Aug 20 15:03:54 2017
From: report at bugs.python.org (Samuel Colvin)
Date: Sun, 20 Aug 2017 19:03:54 +0000
Subject: [New-bugs-announce] [issue31241] ast col_offset wrong for list
 comprehensions, generators and tuples
Message-ID: <1503255834.0.0.745479367375.issue31241@psf.upfronthosting.co.za>


New submission from Samuel Colvin:

With Python 3.5 and 3.6 list comprehensions, generators and tuples have the col_offset for their ast nodes off by 1:

```
import ast
ast.parse('{a for a in range(3)}').body[0].value.col_offset
>> 0  # set comprehension correct

ast.parse('{a: 1 for a in range(3)}').body[0].value.col_offset
>> 0  # dict comprehension correct

ast.parse('[a for a in range(3)]').body[0].value.col_offset
>> 1  # list comprehension wrong!

ast.parse('(a for a in range(3))').body[0].value.col_offset
>> 1  # generator comprehension wrong!

ast.parse('[1, 2, 3]').body[0].value.col_offset
>> 0  # list correct

ast.parse('{1, 2, 3}').body[0].value.col_offset
>> 0  # set correct

ast.parse('{1: 1, 2: 2, 3: 3}').body[0].value.col_offset
>> 0  # dict correct

ast.parse('(1, 2, 3)').body[0].value.col_offset
>> 1  # tuple wrong!
```

I haven't tried 3.4, the issue could be there too.

There are some other related issues #16806 and #21295 but they don't seem quite the same.

----------
components: Interpreter Core
messages: 300606
nosy: samuelcolvin
priority: normal
severity: normal
status: open
title: ast col_offset wrong for list comprehensions, generators and tuples
versions: Python 3.5, Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31241>
_______________________________________

From report at bugs.python.org  Sun Aug 20 17:49:16 2017
From: report at bugs.python.org (Ryan Finnie)
Date: Sun, 20 Aug 2017 21:49:16 +0000
Subject: [New-bugs-announce] [issue31242] Add
 SSLContext.set_verify_callback()
Message-ID: <1503265756.53.0.967490588927.issue31242@psf.upfronthosting.co.za>


New submission from Ryan Finnie:

At the moment, SSLContext.verify_mode() allows for three modes when dealing with Purpose.CLIENT_AUTH / server_side=True:

- CERT_NONE (server does not request client certificate, client does not provide it)
- CERT_OPTIONAL (server requests client certificate, raises SSLError if provided but fails verification, continues if not provided)
- CERT_REQUIRED (server requests client certificate, raises SSLError if provided but fails verification, raises SSLError if not provided)

There is currently no way to request a client certificate and manually verify it (or ignore it) if it doesn't pass OpenSSL verification.  OpenSSL provides SSL_CTX_set_cert_verify_callback for using a custom callback[0], but this is not exposed in Python.

It would be nice to have a set_verify_callback() method, similar to how set_servername_callback() does it for SSL_CTX_set_tlsext_servername_callback.

[0] https://www.openssl.org/docs/man1.0.2/ssl/SSL_CTX_set_verify.html

----------
assignee: christian.heimes
components: SSL
messages: 300607
nosy: christian.heimes, rfinnie
priority: normal
severity: normal
status: open
title: Add SSLContext.set_verify_callback()
type: enhancement

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31242>
_______________________________________

From report at bugs.python.org  Mon Aug 21 04:33:09 2017
From: report at bugs.python.org (Oren Milman)
Date: Mon, 21 Aug 2017 08:33:09 +0000
Subject: [New-bugs-announce] [issue31243] checks whether PyArg_ParseTuple
 returned a negative int
Message-ID: <1503304389.01.0.083549043961.issue31243@psf.upfronthosting.co.za>


New submission from Oren Milman:

according to the docs (https://docs.python.org/3.7/c-api/arg.html?highlight=pyarg_parsetuple#c.PyArg_ParseTuple), PyArg_ParseTuple
returns true for success or false for failure. I also looked at the
implementation in Python/getargs.c, and it seems that indeed PyArg_ParseTuple
can return only 0 or 1.

however, in some places in the codebase, there are checks whether
PyArg_ParseTuple returned a negative int.
I found a bunch in Modules/_testcapimodule.c, and one in Modules/_io/textio.c
in textiowrapper_read_chunk. (hopefully i haven't missed other places.)

this code crashes because of the check in textiowrapper_read_chunk:
import codecs
import _io
class MyDec():
    def getstate(self):
        return 420
class MyDecWrapper():
    def __call__(self, blabla):
        return MyDec()
quopri = codecs.lookup("quopri")
quopri._is_text_encoding = True
quopri.incrementaldecoder = MyDecWrapper()
t = _io.TextIOWrapper(_io.BytesIO(b'aaaaaa'),
                      newline='\n', encoding="quopri")
t.read(42)


I guess all of these checks should be changed to check whether PyArg_ParseTuple
returned 0.

also, before this specific call to PyArg_ParseTuple in textiowrapper_read_chunk,
we should check whether 'state' is a tuple.
Moreover, this call shouldn't produce a wrong error message, as explained in
#28261.

----------
components: Interpreter Core
messages: 300613
nosy: Oren Milman
priority: normal
severity: normal
status: open
title: checks whether PyArg_ParseTuple returned a negative int
type: crash
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31243>
_______________________________________

From report at bugs.python.org  Mon Aug 21 08:03:49 2017
From: report at bugs.python.org (Constantine Ketskalo)
Date: Mon, 21 Aug 2017 12:03:49 +0000
Subject: [New-bugs-announce] [issue31244] Shortcuts bug in Python IDLE
Message-ID: <1503317029.77.0.700563383458.issue31244@psf.upfronthosting.co.za>


New submission from Constantine Ketskalo:

Hi there.
Thank you for Python, I love this programming language.

I've found a bug with shortcuts. I'm from Ukraine, so I'm working with English, Ukrainian and Russian languages. But any shortcuts, where alphabet is involved (like Ctrl+Z, Ctrl+F, Ctrl+V and so on) work only when English is turned on. This is quite inconvenient. It's possible to work with it, but still confusing, because you can select something in IDLE, copy it with shortcut Ctrl+C, then try to paste it to another place and then you find out that it wasn't copied, because active language is Ukrainian. So you need to switch to English and repeat everything again.

I'm currently working with Python 3.6.1, Windows 8.1 Embedded enterprize x64.
Also I've noticed the same under Windows 7, Windows 10 in different versions of Python.
I believe there where Python 2.7, 2.11, 3.4, 3.5. Not sure about all the versions exectly, but for sure shortcuts worked the same in all the versions I've ever tried to use.

I hope this was helpful and I'm sure many developers, that work not only with English would be really happy if you would fix this.

Thanks again for your time and effort.
Best regards, Constantine.

----------
assignee: terry.reedy
components: IDLE
messages: 300623
nosy: Constantine Ketskalo, terry.reedy
priority: normal
severity: normal
status: open
title: Shortcuts bug in Python IDLE
type: behavior
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31244>
_______________________________________

From report at bugs.python.org  Mon Aug 21 10:06:59 2017
From: report at bugs.python.org (Quentin Dawans)
Date: Mon, 21 Aug 2017 14:06:59 +0000
Subject: [New-bugs-announce] [issue31245] Asyncio UNIX socket and SOCK_DGRAM
Message-ID: <1503324419.78.0.551838104662.issue31245@psf.upfronthosting.co.za>


New submission from Quentin Dawans:

Following this discussion: https://groups.google.com/forum/#!searchin/python-tulip/datagram%7Csort:relevance/python-tulip/rpScueoGf1I/WBvEWf8vCgAJ

I added the support for AF_UNIX SOCK_DGRAM is asyncio. Lots of code is coming from the other family/type of socket already implemented.

----------
components: asyncio
messages: 300625
nosy: qdawans, yselivanov
priority: normal
severity: normal
status: open
title: Asyncio UNIX socket and SOCK_DGRAM
type: enhancement
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31245>
_______________________________________

From report at bugs.python.org  Mon Aug 21 10:21:31 2017
From: report at bugs.python.org (STINNER Victor)
Date: Mon, 21 Aug 2017 14:21:31 +0000
Subject: [New-bugs-announce] [issue31246] [2.7]
 test_signal.test_setitimer_tiny() fails randomly on x86-64 El Capitan 2.7
Message-ID: <1503325291.35.0.150944992826.issue31246@psf.upfronthosting.co.za>


New submission from STINNER Victor:

http://buildbot.python.org/all/builders/x86-64%20El%20Capitan%202.7/builds/161/steps/test/logs/stdio

0:04:49 [173/403/1] test_signal failed
test test_signal failed -- Traceback (most recent call last):
  File "/Users/buildbot/buildarea/2.7.billenstein-elcapitan/build/Lib/test/test_signal.py", line 500, in test_setitimer_tiny
    self.assertEqual(self.hndl_called, True)
AssertionError: False != True

----------
components: Tests
messages: 300626
nosy: haypo
priority: normal
severity: normal
status: open
title: [2.7] test_signal.test_setitimer_tiny() fails randomly on x86-64 El Capitan 2.7
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31246>
_______________________________________

From report at bugs.python.org  Mon Aug 21 11:48:53 2017
From: report at bugs.python.org (STINNER Victor)
Date: Mon, 21 Aug 2017 15:48:53 +0000
Subject: [New-bugs-announce] [issue31247] test_xmlrpc leaks dangling threads
Message-ID: <1503330533.41.0.899332710232.issue31247@psf.upfronthosting.co.za>


New submission from STINNER Victor:

Using PR 3138 of bpo-31234, I noticed that test_xmlrpc leaks dangling threads. Example:

haypo at selma$ ./python -m test -v test_xmlrpc --fail-env-changed -m test.test_xmlrpc.MultiPathServerTestCase.test_path1
== CPython 3.7.0a0 (heads/master:0267128, Aug 21 2017, 17:23:11) [GCC 6.4.1 20170727 (Red Hat 6.4.1-1)]
(...)
Warning -- threading_cleanup() detected 0 leaked threads (count: 0, dangling: 2)
test_xmlrpc failed (env changed)
(...)

It seems like xmlrpc error handling creates reference leaks. Attached PR fixes the issue.

----------
components: Tests
messages: 300630
nosy: haypo
priority: normal
severity: normal
status: open
title: test_xmlrpc leaks dangling threads
type: resource usage
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31247>
_______________________________________

From report at bugs.python.org  Mon Aug 21 11:54:19 2017
From: report at bugs.python.org (Dan Snider)
Date: Mon, 21 Aug 2017 15:54:19 +0000
Subject: [New-bugs-announce] [issue31248] method wrapper type has invalid
 __name__/__qualname__ 'method-wrapper'
Message-ID: <1503330859.0.0.694490112113.issue31248@psf.upfronthosting.co.za>


New submission from Dan Snider:

Out of the 202 builtin/extension types easily accessible in sys.modules, that is the only one with a dash in its name.

----------
components: Library (Lib)
messages: 300632
nosy: bup
priority: normal
severity: normal
status: open
title: method wrapper type has invalid __name__/__qualname__ 'method-wrapper'
type: behavior
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31248>
_______________________________________

From report at bugs.python.org  Mon Aug 21 12:04:13 2017
From: report at bugs.python.org (STINNER Victor)
Date: Mon, 21 Aug 2017 16:04:13 +0000
Subject: [New-bugs-announce] [issue31249] test_concurrent_futures leaks
 dangling threads
Message-ID: <1503331453.05.0.480625711508.issue31249@psf.upfronthosting.co.za>


New submission from STINNER Victor:

Using PR 3138 of bpo-31234, I noticed that test_concurrent_futures leaks dangling threads:

haypo at selma$ ./python -u -m test --fail-env-changed test_concurrent_futures -v -m ThreadPoolWaitTests
== CPython 3.7.0a0 (heads/master:0267128, Aug 21 2017, 17:23:11) [GCC 6.4.1 20170727 (Red Hat 6.4.1-1)]
(...)
test_all_completed (test.test_concurrent_futures.ThreadPoolWaitTests) ... Warning -- threading_cleanup() detected 0 leaked threads (count: 0, dangling: 6)
0.11s ok
test_first_completed (test.test_concurrent_futures.ThreadPoolWaitTests) ... 1.60s ok
test_first_completed_some_already_completed (test.test_concurrent_futures.ThreadPoolWaitTests) ... 1.60s ok
test_first_exception (test.test_concurrent_futures.ThreadPoolWaitTests) ... Warning -- threading_cleanup() detected 0 leaked threads (count: 0, dangling: 6)
3.11s ok
test_first_exception_one_already_failed (test.test_concurrent_futures.ThreadPoolWaitTests) ... 2.10s ok
test_first_exception_some_already_complete (test.test_concurrent_futures.ThreadPoolWaitTests) ... Warning -- threading_cleanup() detected 0 leaked threads (count: 0, dangling: 6)
1.60s ok
test_pending_calls_race (test.test_concurrent_futures.ThreadPoolWaitTests) ... 0.12s ok
test_timeout (test.test_concurrent_futures.ThreadPoolWaitTests) ... 6.11s ok
(...)

----------
messages: 300634
nosy: haypo
priority: normal
severity: normal
status: open
title: test_concurrent_futures leaks dangling threads
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31249>
_______________________________________

From report at bugs.python.org  Mon Aug 21 12:33:56 2017
From: report at bugs.python.org (STINNER Victor)
Date: Mon, 21 Aug 2017 16:33:56 +0000
Subject: [New-bugs-announce] [issue31250] test_asyncio leaks dangling threads
Message-ID: <1503333236.01.0.923722915126.issue31250@psf.upfronthosting.co.za>


New submission from STINNER Victor:

Using PR 3138 of bpo-31234, I noticed that test_asyncio leaks dangling threads.

Tests which randomly leak threads:
---
test.test_asyncio.test_base_events.BaseEventLoopWithSelectorTests.test_create_connection_no_inet_pton
test.test_asyncio.test_base_events.BaseEventLoopWithSelectorTests.test_create_connection_service_name
test.test_asyncio.test_base_events.BaseEventLoopWithSelectorTests.test_create_connection_bluetooth
test.test_asyncio.test_base_events.BaseEventLoopWithSelectorTests.test_create_datagram_endpoint_no_addrinfo
test.test_asyncio.test_futures.DuckTests.test_wrap_future
test.test_asyncio.test_futures.CFutureTests.test_wrap_future
test.test_asyncio.test_futures.CFutureTests.test_wrap_future_use_global_loop
test.test_asyncio.test_futures.PyFutureTests.test_wrap_future
test.test_asyncio.test_futures.PyFutureTests.test_wrap_future_use_global_loop
test.test_asyncio.test_selector_events.BaseSelectorEventLoopTests.test_sock_connect_resolve_using_socket_params
test.test_asyncio.test_tasks.RunCoroutineThreadsafeTests.test_run_coroutine_threadsafe
test.test_asyncio.test_tasks.RunCoroutineThreadsafeTests.test_run_coroutine_threadsafe_with_timeout
test.test_asyncio.test_tasks.RunCoroutineThreadsafeTests.test_run_coroutine_threadsafe_task_cancelled
test.test_asyncio.test_tasks.RunCoroutineThreadsafeTests.test_run_coroutine_threadsafe_task_factory_exception
---

Write this list into "tests" and run "python3 -m test --matchfile=tests".


haypo at selma$ ./python -u -m test -v test_asyncio --matchfile=tests
== CPython 3.7.0a0 (heads/concurrent_futures:f277fa3, Aug 21 2017, 18:11:41) [GCC 6.4.1 20170727 (Red Hat 6.4.1-1)]
(...)
test_create_connection_bluetooth (test.test_asyncio.test_base_events.BaseEventLoopWithSelectorTests) ... ok
test_create_connection_no_inet_pton (test.test_asyncio.test_base_events.BaseEventLoopWithSelectorTests) ... Warning -- threading_cleanup() detected 4 leaked threads (count: 4, dangling: 5)
ok
test_create_connection_service_name (test.test_asyncio.test_base_events.BaseEventLoopWithSelectorTests) ... Warning -- threading_cleanup() detected 4 leaked threads (count: 4, dangling: 5)
ok
test_create_datagram_endpoint_no_addrinfo (test.test_asyncio.test_base_events.BaseEventLoopWithSelectorTests) ... Warning -- threading_cleanup() detected 1 leaked threads (count: 1, dangling: 2)
ok
test_wrap_future (test.test_asyncio.test_futures.CFutureTests) ... Warning -- threading_cleanup() detected 1 leaked threads (count: 1, dangling: 2)
ok
test_wrap_future_use_global_loop (test.test_asyncio.test_futures.CFutureTests) ... Warning -- threading_cleanup() detected 1 leaked threads (count: 1, dangling: 2)
ok
test_wrap_future (test.test_asyncio.test_futures.DuckTests) ... ok
test_wrap_future (test.test_asyncio.test_futures.PyFutureTests) ... Warning -- threading_cleanup() detected 1 leaked threads (count: 1, dangling: 2)
ok
test_wrap_future_use_global_loop (test.test_asyncio.test_futures.PyFutureTests) ... Warning -- threading_cleanup() detected 1 leaked threads (count: 1, dangling: 2)
ok
test_sock_connect_resolve_using_socket_params (test.test_asyncio.test_selector_events.BaseSelectorEventLoopTests) ... Warning -- threading_cleanup() detected 1 leaked threads (count: 1, dangling: 2)
ok
test_run_coroutine_threadsafe (test.test_asyncio.test_tasks.RunCoroutineThreadsafeTests)
Test coroutine submission from a thread to an event loop. ... ok
test_run_coroutine_threadsafe_task_cancelled (test.test_asyncio.test_tasks.RunCoroutineThreadsafeTests)
Test coroutine submission from a tread to an event loop ... ok
test_run_coroutine_threadsafe_task_factory_exception (test.test_asyncio.test_tasks.RunCoroutineThreadsafeTests)
Test coroutine submission from a tread to an event loop ... Warning -- threading_cleanup() detected 1 leaked threads (count: 1, dangling: 2)
ok
test_run_coroutine_threadsafe_with_timeout (test.test_asyncio.test_tasks.RunCoroutineThreadsafeTests)
Test coroutine submission from a thread to an event loop ... ok
(...)

----------
components: Tests, asyncio
messages: 300637
nosy: haypo, yselivanov
priority: normal
severity: normal
status: open
title: test_asyncio leaks dangling threads
type: resource usage
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31250>
_______________________________________

From report at bugs.python.org  Mon Aug 21 13:09:22 2017
From: report at bugs.python.org (George Lane)
Date: Mon, 21 Aug 2017 17:09:22 +0000
Subject: [New-bugs-announce] [issue31251] Diameter protocol in Python
Message-ID: <1503335362.34.0.40856899205.issue31251@psf.upfronthosting.co.za>


New submission from George Lane:

Diameter protocol in Python

----------
components: Library (Lib)
messages: 300643
nosy: George Lane
priority: normal
severity: normal
status: open
title: Diameter protocol in Python
type: resource usage
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31251>
_______________________________________

From report at bugs.python.org  Mon Aug 21 15:02:17 2017
From: report at bugs.python.org (Ashish Nitin Patil)
Date: Mon, 21 Aug 2017 19:02:17 +0000
Subject: [New-bugs-announce] [issue31252] Operator.itemgetter documentation
 should include dictionary keys example
Message-ID: <1503342137.82.0.389505495838.issue31252@psf.upfronthosting.co.za>


New submission from Ashish Nitin Patil:

Current documentation lacks the example of usage with dictionary keys.

e.g. `itemgetter('sample')({'sample': True, 'How about this': 'Nope'})`

----------
assignee: docs at python
components: Documentation
messages: 300648
nosy: ashishnitinpatil, docs at python
priority: normal
severity: normal
status: open
title: Operator.itemgetter documentation should include dictionary keys example
type: enhancement
versions: Python 2.7, Python 3.3, Python 3.4, Python 3.5, Python 3.6, Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31252>
_______________________________________

From report at bugs.python.org  Tue Aug 22 03:26:57 2017
From: report at bugs.python.org (Andrew Wall)
Date: Tue, 22 Aug 2017 07:26:57 +0000
Subject: [New-bugs-announce] [issue31253] Python fails to parse commented
 out code
Message-ID: <1503386817.16.0.0892558869333.issue31253@psf.upfronthosting.co.za>


New submission from Andrew Wall:

Python can parse and run this code:

log = list(r'..\Unknown\*.txt')


but not this:

'''
log = list(r'..\Unknown\*.txt')
'''

----------
components: Interpreter Core
messages: 300670
nosy: quamrana
priority: normal
severity: normal
status: open
title: Python fails to parse commented out code
type: compile error
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31253>
_______________________________________

From report at bugs.python.org  Tue Aug 22 03:31:12 2017
From: report at bugs.python.org (Antony Lee)
Date: Tue, 22 Aug 2017 07:31:12 +0000
Subject: [New-bugs-announce] [issue31254] WeakKeyDictionary/Mapping doesn't
 call __missing__
Message-ID: <1503387072.25.0.0553627508053.issue31254@psf.upfronthosting.co.za>


New submission from Antony Lee:

The following example, which raises a KeyError, shows that in a WeakKeyDictionary subclass that defines __missing__, that method doesn't get called.

    from weakref import WeakKeyDictionary

    class WeakKeyDictionaryWithMissing(WeakKeyDictionary):
        __missing__ = lambda: print("hello")

    class C: pass

    d = WeakKeyDictionaryWithMissing()
    d[C()]

This behavior is technically OK, as object.__missing__ is only documented in the datamodel to be called for dict subclasses, and WeakKeyDictionary is actually not a subclass of dict, but of MutableMapping.

Still, it would seem preferable if either WeakKeyDictionary did use __missing__, or perhaps, more reasonably, Mapping.__getitem__ did so.  (Or, at least, if the WeakKeyDictionary class clearly stated that it does not inherit from dict.  Note that the docs start with "Mapping class that references keys weakly. Entries in the *dictionary* etc." (emphasis mine))

----------
components: Library (Lib)
messages: 300671
nosy: Antony.Lee
priority: normal
severity: normal
status: open
title: WeakKeyDictionary/Mapping doesn't call __missing__
versions: Python 3.6, Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31254>
_______________________________________

From report at bugs.python.org  Tue Aug 22 03:40:09 2017
From: report at bugs.python.org (=?utf-8?b?5p2O5piO5a6H?=)
Date: Tue, 22 Aug 2017 07:40:09 +0000
Subject: [New-bugs-announce] [issue31255] Test getrandom before using it
Message-ID: <1503387609.26.0.161780447873.issue31255@psf.upfronthosting.co.za>


New submission from ???:

If a GNU/Linux system has glibc-2.25 and <linux-3.17, getrandom and getentropy will be declared in system headers, without a syscall implementation in the kernel.

For the build system of python, getentropy gets passed in configure, but fails at runtime, such as:

> Fatal Python error: getentropy() failed

strace gives

> syscall_318(0x7f369b1535d0, 0x18, 0, 0x13e, 0x7f369b1535d0, 0x7f369b1535e8) = -1 (errno 38)

The glibc developers regard testing the usability of a syscall to be a task of python build system.  Besides AC_TEST_FUNCS(... getentropy ...) which only test whether getentropy is defined in the system, runtime checks in configure is needed.

----------
messages: 300672
nosy: ???
priority: normal
severity: normal
status: open
title: Test getrandom before using it
type: compile error
versions: Python 3.4, Python 3.5, Python 3.6, Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31255>
_______________________________________

From report at bugs.python.org  Tue Aug 22 04:01:08 2017
From: report at bugs.python.org (bastik)
Date: Tue, 22 Aug 2017 08:01:08 +0000
Subject: [New-bugs-announce] [issue31256] xml.etree.ElementTree: add support
 for doctype in tostring method
Message-ID: <1503388868.57.0.0239596264981.issue31256@psf.upfronthosting.co.za>


New submission from bastik:

When trying to generate an XML file with 'xml.etree.ElementTree', there is no way to specify a doctype (example below).

The 'xml.etree.ElementTree.ElementTree.write' method has a 'xml_declaration' flag, which can be used as workaround, but there is no such flag for 'xml.etree.ElementTree.tostring' (and it is not always desirable to write the XML document directly to file). Please compare the corresponding flags in lxml.

Here is an example for a document I would like to generate and convert to string:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" 
    "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html><body>text</body></html>

----------
components: XML
messages: 300673
nosy: bastik
priority: normal
severity: normal
status: open
title: xml.etree.ElementTree: add support for doctype in tostring method
type: enhancement
versions: Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31256>
_______________________________________

From report at bugs.python.org  Tue Aug 22 08:26:54 2017
From: report at bugs.python.org (STINNER Victor)
Date: Tue, 22 Aug 2017 12:26:54 +0000
Subject: [New-bugs-announce] [issue31257] importlib race condition
Message-ID: <1503404814.17.0.5568940572.issue31257@psf.upfronthosting.co.za>


New submission from STINNER Victor:

While stressing the system to get system load of at least 5 (I have 4 logical CPUs), I got an import error whereas the module exists.

See recent changes to fix race conditions in importlib:

* bpo-30891: commit e72b1359f81d1dd42bd8a5c5cc2b3928b74f8023
* bpo-30891: commit 4f9a446f3fb42f800e73cd9414dd1eccb3ca4fa7
* bpo-31070: commit 9b0d1d647e3d2ec9d299e5c9f49b02fbbb810a5a
* bpo-30814: commit b4baacee1adc06edbe30ac7574d17a8cd168e2e0
* ...

I used my system_load.py script to stress my Linux:
https://github.com/haypo/misc/blob/master/bin/system_load.py


haypo at selma$ ./python -m test -r  --randseed=7323369 -R 3:3 -j2 -m test.test_robotparser.PasswordProtectedSiteTestCase.testPasswordProtectedSite test_robotparser  test_robotparser  test_robotparser  test_robotparsertest_robotparser  test_robotparser  test_robotparser  test_robotparser
Using random seed 7323369
Run tests in parallel using 2 child processes
0:00:01 load avg: 9.05 [1/7] test_robotparser passed
beginning 6 repetitions
123456
......
0:00:01 load avg: 9.05 [2/7] test_robotparser passed
beginning 6 repetitions
123456
......
0:00:02 load avg: 10.33 [3/7] test_robotparser passed
beginning 6 repetitions
123456
......
0:00:02 load avg: 10.33 [4/7] test_robotparser passed
beginning 6 repetitions
123456
......
0:00:03 load avg: 10.33 [5/7/1] test_robotparsertest_robotparser failed
test test_robotparsertest_robotparser crashed -- Traceback (most recent call last):
  File "/home/haypo/prog/python/master/Lib/test/libregrtest/runtest.py", line 163, in runtest_inner
    the_module = importlib.import_module(abstest)
  File "/home/haypo/prog/python/master/Lib/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 994, in _gcd_import
  File "<frozen importlib._bootstrap>", line 971, in _find_and_load
  File "<frozen importlib._bootstrap>", line 953, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'test.test_robotparsertest_robotparser'
0:00:03 load avg: 10.33 [6/7/1] test_robotparser passed
beginning 6 repetitions
123456
......
0:00:04 load avg: 10.33 [7/7/1] test_robotparser passed
beginning 6 repetitions
123456
......
6 tests OK.

1 test failed:
    test_robotparsertest_robotparser

Total duration: 4 sec
Tests result: FAILURE

----------
messages: 300686
nosy: brett.cannon, eric.snow, haypo, ncoghlan, serhiy.storchaka
priority: normal
severity: normal
status: open
title: importlib race condition

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31257>
_______________________________________

From report at bugs.python.org  Tue Aug 22 10:36:08 2017
From: report at bugs.python.org (STINNER Victor)
Date: Tue, 22 Aug 2017 14:36:08 +0000
Subject: [New-bugs-announce] [issue31258] [2.7] Enhance support
 reap_children() and threading_cleanup()
Message-ID: <1503412568.83.0.450478678857.issue31258@psf.upfronthosting.co.za>


New submission from STINNER Victor:

Backport reap_children() and threading_cleanup() enhancements from master to 2.7 to make tests more reliables, reduce random failures on Python CIs.

----------
components: Tests
messages: 300695
nosy: haypo
priority: normal
severity: normal
status: open
title: [2.7] Enhance support reap_children() and threading_cleanup()
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31258>
_______________________________________

From report at bugs.python.org  Tue Aug 22 10:53:06 2017
From: report at bugs.python.org (STINNER Victor)
Date: Tue, 22 Aug 2017 14:53:06 +0000
Subject: [New-bugs-announce] [issue31259] [3.6] Enhance support
 reap_children() and threading_cleanup()
Message-ID: <1503413586.26.0.662095689215.issue31259@psf.upfronthosting.co.za>


New submission from STINNER Victor:

Backport reap_children() and threading_cleanup() enhancements from master to 3.6 to make tests more reliables, reduce random failures on Python CIs.

See also bpo-31258 for Python 2.7

----------
components: Tests
messages: 300700
nosy: haypo
priority: normal
severity: normal
status: open
title: [3.6] Enhance support reap_children() and threading_cleanup()
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31259>
_______________________________________

From report at bugs.python.org  Tue Aug 22 11:18:17 2017
From: report at bugs.python.org (STINNER Victor)
Date: Tue, 22 Aug 2017 15:18:17 +0000
Subject: [New-bugs-announce] [issue31260] [2.7] Enhance PC/VS9.0/ project to
 produce python.bat, as PCbuild/
Message-ID: <1503415097.55.0.681149951058.issue31260@psf.upfronthosting.co.za>


New submission from STINNER Victor:

It would be nice to have a python.bat script produced by the PC/VS9.0/ project, as done by the PCbuild/ project.

It would fix the newly added buildbot step "pythoninfo":
https://bugs.python.org/issue30871#msg300703

Without python.bat, it's not easy to get the path to the Python executable from the buildbot configuration.

----------
components: Build, Windows
messages: 300704
nosy: haypo, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: [2.7] Enhance PC/VS9.0/ project to produce python.bat, as PCbuild/
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31260>
_______________________________________

From report at bugs.python.org  Tue Aug 22 18:26:00 2017
From: report at bugs.python.org (Micah)
Date: Tue, 22 Aug 2017 22:26:00 +0000
Subject: [New-bugs-announce] [issue31261] unittest fails to properly
 destruct objects created during setUp if a test fails
Message-ID: <1503440760.72.0.06686224084.issue31261@psf.upfronthosting.co.za>


New submission from Micah:

When running the attached example test dummy objects are created during setUp and destructed at the end of a passing test but when a test fails the dummy object that was created in setUp fails to deconstruct. This leads to cascading errors when writing integration tests where having a "ghost" object sitting in the background and listening to messages is detrimental.

This behavior became apparent when I was writing integration tests making use of a signalling package so I pulled everything that wasn't base python to see if the bug was reproduced and it was. The object that failed to deconstruct when the test failed(0x000001CA971F6208) deconstructs at the end of the entire testcase while the object declared within the test function itself(0x000001CA971F62B0) deconstructs as soon as the test fails out.

The output of the test that I ran was this:

> python -m unittest test_bug.py


test_fail (test_bug.TestBug)
Created <test_bug.DummyClass object at 0x000001CA971F6208>
Created local <test_bug.DummyClass object at 0x000001CA971F62B0>
F
deleting <test_bug.DummyClass object at 0x000001CA971F62B0>


test_pass (test_bug.TestBug)
Created <test_bug.DummyClass object at 0x000001CA971F62B0>
.
deleting <test_bug.DummyClass object at 0x000001CA971F62B0>


test_passes (test_bug.TestBug)
Created <test_bug.DummyClass object at 0x000001CA971A5400>
.
deleting <test_bug.DummyClass object at 0x000001CA971A5400>

======================================================================
FAIL: test_fail (test_bug.TestBug)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\Code\test_bug.py", line 27, in test_fail
    self.assertTrue(False)
AssertionError: False is not true

----------------------------------------------------------------------
Ran 3 tests in 0.004s

FAILED (failures=1)

deleting <test_bug.DummyClass object at 0x000001CA971F6208>

----------
components: Windows
files: test_bug.py
messages: 300725
nosy: MCBama, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: unittest fails to properly destruct objects created during setUp if a test fails
type: behavior
versions: Python 3.6
Added file: http://bugs.python.org/file47094/test_bug.py

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31261>
_______________________________________

From report at bugs.python.org  Tue Aug 22 18:52:37 2017
From: report at bugs.python.org (Dipti Sherlekar)
Date: Tue, 22 Aug 2017 22:52:37 +0000
Subject: [New-bugs-announce] [issue31262] Documentation Error
Message-ID: <1503442357.91.0.706507107768.issue31262@psf.upfronthosting.co.za>


New submission from Dipti Sherlekar:

Documentation says : In C++ terminology, normally class members (including the data members) are public  

Solution: In C++ the class memebers are private not public by default.

----------
assignee: docs at python
components: Documentation
messages: 300727
nosy: Dipti Sherlekar, docs at python
priority: normal
severity: normal
status: open
title: Documentation Error
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31262>
_______________________________________

From report at bugs.python.org  Wed Aug 23 01:02:36 2017
From: report at bugs.python.org (Isaac Elliott)
Date: Wed, 23 Aug 2017 05:02:36 +0000
Subject: [New-bugs-announce] [issue31263] Assigning to subscript/slice of
 literal is permitted
Message-ID: <1503464556.49.0.828859643321.issue31263@psf.upfronthosting.co.za>


New submission from Isaac Elliott:

In Python 3.5 and 3.6 (at least), the language reference presents a grammar that disallows assignment to literals.

For example, `(a for 1 in [1,2,3])` is a syntax error, as is `(1, a) = (2, 3)`.

However the grammar doesn't prevent assignment to subscripted or sliced literals.

For example neither `(a for [1,2,3][0] in [1,2,3])` nor `([1,2,3][0], a) = (2, 3)` are considered syntax errors.

Similar behavior is exhibited for slices.

The problem is that the `target_list` production (https://docs.python.org/3.5/reference/simple_stmts.html#grammar-token-target_list) reuses the `subscription` and `slicing` productions which both use the `primary` production, allowing literals on their left side.

----------
messages: 300740
nosy: Isaac Elliott
priority: normal
severity: normal
status: open
title: Assigning to subscript/slice of literal is permitted
type: behavior
versions: Python 3.5, Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31263>
_______________________________________

From report at bugs.python.org  Wed Aug 23 06:18:02 2017
From: report at bugs.python.org (John Torakis)
Date: Wed, 23 Aug 2017 10:18:02 +0000
Subject: [New-bugs-announce] [issue31264] Import Package/Module through
 HTTP/S repository
Message-ID: <1503483482.35.0.941565014416.issue31264@psf.upfronthosting.co.za>


New submission from John Torakis:

A module presented in "https://github.com/operatorequals/httpimport" which provides functionality for package/module loading through HTTP/S, like the golang's "import github.com/someusr/somepkg/subpkg", but for any directory served with HTTP/S.

I believe that this module has to become a core feature, after, of course, the necessary scrutiny!

----------
components: Extension Modules
hgrepos: 372
messages: 300747
nosy: operatorequals
priority: normal
severity: normal
status: open
title: Import Package/Module through HTTP/S repository
type: enhancement
versions: Python 2.7, Python 3.3, Python 3.4, Python 3.5, Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31264>
_______________________________________

From report at bugs.python.org  Wed Aug 23 06:24:19 2017
From: report at bugs.python.org (INADA Naoki)
Date: Wed, 23 Aug 2017 10:24:19 +0000
Subject: [New-bugs-announce] [issue31265] Remove doubly-linked list from C
 OrderedDict
Message-ID: <1503483859.09.0.0369181186057.issue31265@psf.upfronthosting.co.za>


New submission from INADA Naoki:

Since dict preserves insertion order, doubly linked list in OrderedDict
can be removed.

There is small performance improvement for odict creation:

$ curl https://api.github.com/orgs/python/repos > repos.json
$ ./py-patched -m perf timeit --compare-to `pwd`/py-default -s 'from collections import OrderedDict as od; import json; data=open("repos.json").read()' -- 'json.loads(data, object_pairs_hook=od)'
py-default: ..................... 1.53 ms +- 0.01 ms
py-patched: ..................... 1.30 ms +- 0.01 ms

Mean +- std dev: [py-default] 1.53 ms +- 0.01 ms -> [py-patched] 1.30 ms +- 0.01 ms: 1.18x faster (-15%)

And more memory efficient:

$ ./py-default -c 'from collections import OrderedDict; import sys; print(sys.getsizeof(OrderedDict.fromkeys(range(1000))))'
85416

$ ./py-patched -c 'from collections import OrderedDict; import sys; print(sys.getsizeof(OrderedDict.fromkeys(range(1000))))'
36992

But most important benefit is smaller code.  It make easy to maintain.

----------
components: Interpreter Core
messages: 300748
nosy: inada.naoki
priority: normal
severity: normal
status: open
title: Remove doubly-linked list from C OrderedDict
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31265>
_______________________________________

From report at bugs.python.org  Wed Aug 23 15:38:33 2017
From: report at bugs.python.org (emin mamedov)
Date: Wed, 23 Aug 2017 19:38:33 +0000
Subject: [New-bugs-announce] [issue31266] attribute error
Message-ID: <1503517113.75.0.490210479447.issue31266@psf.upfronthosting.co.za>


New submission from emin mamedov:

Hello!
I am just write some program with "string module", after starting my program python give me error about "re module".

My program is a tutorial from DrapsTV from Youtube.

Error -->>
Traceback (most recent call last):
  File "temp1.py", line 1, in <module>
    from string import Template
  File "/usr/lib64/python3.6/string.py", line 77, in <module>
    class Template(metaclass=_TemplateMetaclass):
  File "/usr/lib64/python3.6/string.py", line 82, in Template
    flags = _re.IGNORECASE
AttributeError: module 're' has no attribute 'IGNORECASE'

----------
components: Extension Modules
files: temp1.py
messages: 300763
nosy: emin mamedov
priority: normal
severity: normal
status: open
title: attribute error
type: crash
versions: Python 3.6
Added file: http://bugs.python.org/file47095/temp1.py

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31266>
_______________________________________

From report at bugs.python.org  Wed Aug 23 16:47:35 2017
From: report at bugs.python.org (Thomas Keppler)
Date: Wed, 23 Aug 2017 20:47:35 +0000
Subject: [New-bugs-announce] [issue31267] threading.Timer object is affected
 by changes to system time
Message-ID: <1503521255.96.0.0977006158294.issue31267@psf.upfronthosting.co.za>


New submission from Thomas Keppler:

Hi,

I have been playing around with threading.Timer objects as timeouts for a project and noticed that my timeouts are affected by system time changes.

To test this, I have written a small demonstration script (timer_testcase.py) which you can find in the attachments. I would expect that after 120 seconds, a "Hello!" message will appear on the screen regardless of system time changes.

If you run the script like you would normally, you will see that it will work properly and my expectations are met. Now, run it again and immediately use "date +%T -s "HH:MM:SS"" where the time is >= 2 mins in the future. You will notice that the timer will latch immediately instead of waiting those 120 seconds before latching.

I have read Lib/threading.py to a certain extent and it seems like Timer objects are using monotonic time already because they use Events which use Conditions themselves, which references a "_timer" function that is just an alias for time.monotonic as one can see at the top of the file.
Then I checked out if the monotonic time works as expected (test_monotonic.py) by just jumping back and forth "in time", everything seemed to be normal.

Am I making a mistake and if so, where?

Thanks for any of your answers.

--
Best regards
Thomas

Environment: I'm using Python 3.5.3 on Debian 9.1 "Stretch" on x86_64.

----------
components: Library (Lib)
files: timer_testcase.py
messages: 300765
nosy: winfreak
priority: normal
severity: normal
status: open
title: threading.Timer object is affected by changes to system time
type: behavior
versions: Python 3.5
Added file: http://bugs.python.org/file47096/timer_testcase.py

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31267>
_______________________________________

From report at bugs.python.org  Thu Aug 24 06:53:17 2017
From: report at bugs.python.org (desbma)
Date: Thu, 24 Aug 2017 10:53:17 +0000
Subject: [New-bugs-announce] [issue31268] Inconsistent socket timeout
 exception
Message-ID: <1503571997.66.0.245487150024.issue31268@psf.upfronthosting.co.za>


New submission from desbma:

When trying to connect a classic TCP socket to a non reachable peer, the exception reported is inconsistent if the socket has a timeout set.

See the attached program, on my system (Arch Linux with Linux 4.9 & Python 3.6.2) it outputs:

timeout timed out
timeout timed out
timeout timed out
OSError [Errno 113] No route to host
timeout timed out
timeout timed out
timeout timed out
OSError [Errno 113] No route to host
timeout timed out
timeout timed out
timeout timed out
OSError [Errno 113] No route to host
timeout timed out
timeout timed out
timeout timed out
OSError [Errno 113] No route to host
timeout timed out
timeout timed out
timeout timed out
OSError [Errno 113] No route to host

I expect one of the two exceptions to be thrown every time, not a mix of both.

Thank you

----------
components: Library (Lib)
files: socket_timeout.py
messages: 300779
nosy: desbma
priority: normal
severity: normal
status: open
title: Inconsistent socket timeout exception
type: behavior
versions: Python 3.6
Added file: http://bugs.python.org/file47098/socket_timeout.py

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31268>
_______________________________________

From report at bugs.python.org  Thu Aug 24 09:23:52 2017
From: report at bugs.python.org (Karine)
Date: Thu, 24 Aug 2017 13:23:52 +0000
Subject: [New-bugs-announce] [issue31269] bug in islink() and is_simlink()
Message-ID: <1503581032.08.0.747369734546.issue31269@psf.upfronthosting.co.za>


New submission from Karine:

Hi,

I found a problem which can be a bug and want to inform you.

it seems os.islink() and PATH(mypath).is_symlink()  functions doen't work correct when there is linked folder inside the path:

For example I have the following path:
a/b/c/test.txt

if my c folder is link these 2 functions will not show that this path is link

Only if i remove test.txt and use a/b/c path is shows that path is link.


Seems these functions checks only last part of given path not full path and doesn't find if there is link in somewere inside path and shows not correct result.

Interesting to know your opinion regarding it.

----------
messages: 300782
nosy: karaV
priority: normal
severity: normal
status: open
title: bug in islink() and is_simlink()
type: enhancement
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31269>
_______________________________________

From report at bugs.python.org  Thu Aug 24 12:03:33 2017
From: report at bugs.python.org (Raphael Michel)
Date: Thu, 24 Aug 2017 16:03:33 +0000
Subject: [New-bugs-announce] [issue31270] Simplify documentation of
 itertools.zip_longest
Message-ID: <1503590613.37.0.607996999117.issue31270@psf.upfronthosting.co.za>


New submission from Raphael Michel:

The documentation given for itertools.zip_longest contains a "roughly equivalent" pure-python implementation of the function that is intended to help the user understand what zip_longest does on a functional level.

However, the given implementation is very complicated to read for newcomers and experienced Python programmers alike, as it uses a custom-defined exception for control flow handling, a nested function, a condition that always is true if any arguments are passed ("while iterators"), as well as two other non-trivial functions from itertools (chain and repeat).

For future reference, this is the currently given implementation:

    def zip_longest(*args, **kwds):
        # zip_longest('ABCD', 'xy', fillvalue='-') --> Ax By C- D-
        fillvalue = kwds.get('fillvalue')
        iterators = [iter(it) for it in args]

        while True:
            exhausted = 0
            values = []

            for it in iterators:
                try:
                    values.append(next(it))
                except StopIteration:
                    values.append(fillvalue)
                    exhausted += 1

            if exhausted < len(args):
                yield tuple(values)
            else:
                break

This is way more complex than necessary to teach the concept of zip_longest. With this issue, I will submit a pull request with a new example implementation that seems to be the same level of "roughly equivalent" but is much easier to read, since it only uses two loops and now complicated flow 

    def zip_longest(*args, **kwds):
        # zip_longest('ABCD', 'xy', fillvalue='-') --> Ax By C- D-
        fillvalue = kwds.get('fillvalue')
        iterators = [iter(it) for it in args]

        while True:
            exhausted = 0
            values = []

            for it in iterators:
                try:
                    values.append(next(it))
                except StopIteration:
                    values.append(fillvalue)
                    exhausted += 1

            if exhausted < len(args):
                yield tuple(values)
            else:
                break


Looking at the C code of the actual implementation, I don't see that any one of the two implementations is obviously "more equivalent". I'm unsure about performance -- I haven't tried them on that but I don't think that's the point of this learning implementation.

I ran all tests from Lib/test/test_itertools.py against both the old and the new implementation. The new implementation fails at 3 tests, while the old implementation failed at four. Two of the remaining failures are related to TypeErrors not being thrown on invalid input, one of them is related to pickling the resulting object. I believe all three of them are fine to ignore in this sample, as it is not relevant to the documentation purpose.

Therefore, I believe the documentation should be changed like suggested. I'd be happy for any feedback or further ideas to improve its readability!

----------
assignee: docs at python
components: Documentation
messages: 300788
nosy: docs at python, rami
priority: normal
severity: normal
status: open
title: Simplify documentation of itertools.zip_longest
type: enhancement
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31270>
_______________________________________

From report at bugs.python.org  Thu Aug 24 14:06:29 2017
From: report at bugs.python.org (Oren Milman)
Date: Thu, 24 Aug 2017 18:06:29 +0000
Subject: [New-bugs-announce] [issue31271] an assertion failure in
 io.TextIOWrapper.write
Message-ID: <1503597989.21.0.785820469546.issue31271@psf.upfronthosting.co.za>


New submission from Oren Milman:

currently, the following causes an assertion in Modules/_io/textio.c in
_io_TextIOWrapper_write_impl() to fail:
import codecs
import io

class BadEncoder():
    def encode(self, dummy):
        return 42
def _get_bad_encoder(dummy):
    return BadEncoder()

quopri = codecs.lookup("quopri")
quopri._is_text_encoding = True
quopri.incrementalencoder = _get_bad_encoder
t = io.TextIOWrapper(io.BytesIO(b'foo'), encoding="quopri")
t.write('bar')


this is because _io_TextIOWrapper_write_impl() doesn't check whether the value
returned by encoder's encode() is a bytes object.

(I would open a PR to fix that soon.)

----------
components: IO
messages: 300795
nosy: Oren Milman
priority: normal
severity: normal
status: open
title: an assertion failure in io.TextIOWrapper.write
type: crash
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31271>
_______________________________________

From report at bugs.python.org  Thu Aug 24 14:34:24 2017
From: report at bugs.python.org (Ilia Korvigo)
Date: Thu, 24 Aug 2017 18:34:24 +0000
Subject: [New-bugs-announce] [issue31272] typing module conflicts with
 __slots__-classes
Message-ID: <1503599664.93.0.449415380288.issue31272@psf.upfronthosting.co.za>


New submission from Ilia Korvigo:

I've got conflicts between Python's typing system and `__slots__`. Here is a small reproducible example.

    from typing import TypeVar, Generic, Sequence
    
    T = TypeVar("T")

    class TestGeneric(Sequence, Generic[T]):
        __slots__ = ("test",)

        def __init__(self, test: T):
            self.test = [test]
        
        def __iter__(self):
            return iter(self.test)
        def __len__(self):
            return len(self.test)
        
        def __contains__(self, item):
            return item in self.test
        
        def __getitem__(self, _):
            return self.test[0]

Now whenever I try to specify a content type, e.g.
    
    V = TestGeneric[int]

I get 

    ValueError: 'test' in __slots__ conflicts with class variable

I use `Generics`  in classes without slots a lot, hence I think this error has to be linked to `__slots__`. Moreover, the same class works fine, if you remove the `__slots__`

----------
components: Library (Lib)
messages: 300797
nosy: grayfall
priority: normal
severity: normal
status: open
title: typing module conflicts with  __slots__-classes
type: behavior
versions: Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31272>
_______________________________________

From report at bugs.python.org  Thu Aug 24 16:37:13 2017
From: report at bugs.python.org (Nathan Buckner)
Date: Thu, 24 Aug 2017 20:37:13 +0000
Subject: [New-bugs-announce] [issue31273] Unicode support in TestCase.skip
Message-ID: <1503607033.04.0.29955978966.issue31273@psf.upfronthosting.co.za>


New submission from Nathan Buckner:

Unicode support for TestCase.skip is broken because the caught SkipTest exception is passed through a str call.
except SkipTest as e:
    self._addSkip(result, str(e))
Could be fixed with by changing to unicode(e)

----------
components: Tests, Unicode
files: issue.txt
messages: 300802
nosy: Nathan Buckner, ezio.melotti, haypo
priority: normal
severity: normal
status: open
title: Unicode support in TestCase.skip
type: crash
versions: Python 2.7
Added file: http://bugs.python.org/file47099/issue.txt

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31273>
_______________________________________

From report at bugs.python.org  Thu Aug 24 23:21:17 2017
From: report at bugs.python.org (Barry A. Warsaw)
Date: Fri, 25 Aug 2017 03:21:17 +0000
Subject: [New-bugs-announce] [issue31274] Support building against homebrew
 on macOS
Message-ID: <1503631277.04.0.477315810224.issue31274@psf.upfronthosting.co.za>


New submission from Barry A. Warsaw:

The devguide does touch on how to build Python from source using homebrew installed libraries on macOS, although I found it to be 1) a bit incomplete; 2) not so easy to discover.

It might make sense to modify configure to autodetect homebrew, or to have a switch to enable linking against homebrew libraries.

Barring that, I'd like to at least update the devguide to include details on how to also build sqlite3 and zlib modules.

----------
assignee: docs at python
components: Build, Documentation
messages: 300819
nosy: barry, docs at python
priority: normal
severity: normal
status: open
title: Support building against homebrew on macOS
versions: Python 3.6, Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31274>
_______________________________________

From report at bugs.python.org  Fri Aug 25 08:21:15 2017
From: report at bugs.python.org (Stefan Krah)
Date: Fri, 25 Aug 2017 12:21:15 +0000
Subject: [New-bugs-announce] [issue31275] Check fall-through in
 _codecs_iso2022.c
Message-ID: <1503663675.69.0.755105710638.issue31275@psf.upfronthosting.co.za>


New submission from Stefan Krah:

The last fall-through warning is in _codecs_iso2022.c. IMO the
current code is equivalent to this, but I'm no codecs expert at
all:


diff --git a/Modules/cjkcodecs/_codecs_iso2022.c b/Modules/cjkcodecs/_codecs_iso2022.c
index 1ce4218f30..abf214880f 100644
--- a/Modules/cjkcodecs/_codecs_iso2022.c
+++ b/Modules/cjkcodecs/_codecs_iso2022.c
@@ -807,15 +807,9 @@ jisx0213_encoder(const Py_UCS4 *data, Py_ssize_t *length, void *config)
     case 2: /* second character of unicode pair */
         coded = find_pairencmap((ucs2_t)data[0], (ucs2_t)data[1],
                                 jisx0213_pair_encmap, JISX0213_ENCPAIRS);
-        if (coded == DBCINV) {
-            *length = 1;
-            coded = find_pairencmap((ucs2_t)data[0], 0,
-                      jisx0213_pair_encmap, JISX0213_ENCPAIRS);
-            if (coded == DBCINV)
-                return MAP_UNMAPPABLE;
-        }
-        else
+        if (coded != DBCINV) {
             return coded;
+        /* fall through */
 
     case -1: /* flush unterminated */
         *length = 1;

----------
messages: 300834
nosy: haypo, serhiy.storchaka, skrah
priority: normal
severity: normal
status: open
title: Check fall-through in _codecs_iso2022.c

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31275>
_______________________________________

From report at bugs.python.org  Fri Aug 25 10:26:48 2017
From: report at bugs.python.org (Pauli Virtanen)
Date: Fri, 25 Aug 2017 14:26:48 +0000
Subject: [New-bugs-announce] [issue31276] PyObject_CallFinalizerFromDealloc
 is undocumented
Message-ID: <1503671208.32.0.920107204017.issue31276@psf.upfronthosting.co.za>


New submission from Pauli Virtanen:

It's unclear if PyObject_CallFinalizerFromDealloc is a public function
or not. It is not documented, but it seems there's no other way to
ensure that tp_finalize runs, at least for objects without 
Py_TPFLAGS_HAVE_GC.

In the documentation of tp_finalize (https://docs.python.org/3/c-api/typeobj.html?highlight=tp_finalize#c.PyTypeObject.tp_finalize) 
there is the sentence:

"""It is called either from the garbage collector (if the instance is
part of an isolated reference cycle) or just before the object is 
deallocated."""

However, it appears it is necessary to call it explicitly from any
user-provided tp_dealloc. Indeed, there are several calls to 
PyObject_CallFinalizerFromDealloc in cpython/Modules/* e.g. in
posixmodule.c:ScandirIterator_dealloc

----------
assignee: docs at python
components: Documentation
messages: 300842
nosy: docs at python, pv
priority: normal
severity: normal
status: open
title: PyObject_CallFinalizerFromDealloc is undocumented
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31276>
_______________________________________

From report at bugs.python.org  Fri Aug 25 13:38:57 2017
From: report at bugs.python.org (Stefan Krah)
Date: Fri, 25 Aug 2017 17:38:57 +0000
Subject: [New-bugs-announce] [issue31279] Squash new gcc warning
 (-Wstringop-overflow)
Message-ID: <1503682737.47.0.43054197609.issue31279@psf.upfronthosting.co.za>


New submission from Stefan Krah:

gcc-7.2 emits the following not-so-useful warning:


Objects/bytearrayobject.c:226:9: warning: ?memcpy?: specified size between 9223372036854775808 and 18446744073709551615 exceeds maximum object size 9223372036854775807

----------
messages: 300851
nosy: skrah
priority: normal
severity: normal
status: open
title: Squash new gcc warning (-Wstringop-overflow)

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31279>
_______________________________________

From report at bugs.python.org  Fri Aug 25 17:53:30 2017
From: report at bugs.python.org (Jim Fulton)
Date: Fri, 25 Aug 2017 21:53:30 +0000
Subject: [New-bugs-announce] [issue31280] Namespace packages in directories
 added to path aren't importable if packages from the same namespace are in
 site-packages
Message-ID: <1503698010.91.0.4365749664.issue31280@psf.upfronthosting.co.za>


New submission from Jim Fulton:

I'm having an issue importing from namespaces packages whose directories are added to sys.path and have other packages from the same namespace in site-packages.

To reproduce:

- Create a virtualenv and install zc.buildout in it (``envdir/bin/pip install zc.buildout``).

- Run the attached script with the virtual environment (``envdir/bin/python z.py``).

Note that scenarios like https://www.python.org/dev/peps/pep-0420/#id3 work fine.  This seems to related to having namespace packages in site-packages, or, presumably anywhere on the default path.

I've verified this with Python 3.4, 3.5, and 3.6.

If y'all agree that this is a bug, then I'll attempt debug it and come up with a PR.

----------
messages: 300856
nosy: j1m
priority: normal
severity: normal
status: open
title: Namespace packages in directories added to path aren't importable if packages from the same namespace are in site-packages
versions: Python 3.4, Python 3.5, Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31280>
_______________________________________

From report at bugs.python.org  Fri Aug 25 21:50:52 2017
From: report at bugs.python.org (Zhiming Wang)
Date: Sat, 26 Aug 2017 01:50:52 +0000
Subject: [New-bugs-announce] [issue31281] fileinput inplace does not work
 with pathlib.Path
Message-ID: <1503712252.67.0.949610477917.issue31281@psf.upfronthosting.co.za>


New submission from Zhiming Wang:

Consider

    import fileinput
    import pathlib
    with fileinput.input(files=(pathlib.Path('in.txt'),), inplace=True) as fp:
        for line in fp:
            print(line, end='')

which results in

    Traceback (most recent call last):
      File "./pathlib-fileinput.py", line 6, in <module>
        for line in fp:
      File "/Users/zmwang/.pyenv/versions/3.6.1/lib/python3.6/fileinput.py", line 250, in __next__
        line = self._readline()
      File "/Users/zmwang/.pyenv/versions/3.6.1/lib/python3.6/fileinput.py", line 331, in _readline
        self._filename + (self._backup or ".bak"))
    TypeError: unsupported operand type(s) for +: 'PosixPath' and 'str'

A trivial fix is converting the specified filename to str when assigning to self._filename:

    -        self._filename = self._files[0]
    +        self._filename = str(self._files[0])

----------
components: Library (Lib)
messages: 300860
nosy: zmwangx
priority: normal
severity: normal
status: open
title: fileinput inplace does not work with pathlib.Path
type: behavior
versions: Python 3.4, Python 3.5, Python 3.6, Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31281>
_______________________________________

From report at bugs.python.org  Sat Aug 26 00:51:13 2017
From: report at bugs.python.org (Xiang Zhang)
Date: Sat, 26 Aug 2017 04:51:13 +0000
Subject: [New-bugs-announce] [issue31282] C APIs called without GIL in
 PyOS_Readline
Message-ID: <1503723073.13.0.645165249007.issue31282@psf.upfronthosting.co.za>


New submission from Xiang Zhang:

When debugging our project I find something interesting. In PyOS_Readline it releases the GIL and delegate its job to PyOS_ReadlineFunctionPointer, which could be call_readline or PyOS_StdioReadline(I don't find where vms__StdioReadline is defined). But in the two functions, they use some C APIs like PyMem_Malloc/MALLOC, PyErr_SetString which need guarded by GIL. I don't understand why not doing find-grained lock control in call_readline or PyOS_StdioReadline since the code is ancient. :-(

I find this because our project makes test_cmd_line fail with `echo "Timer\n" | python -i -m timeit -n 1`.

----------
components: Interpreter Core
messages: 300862
nosy: xiang.zhang
priority: normal
severity: normal
status: open
title: C APIs called without GIL in PyOS_Readline
type: behavior
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31282>
_______________________________________

From report at bugs.python.org  Sat Aug 26 05:14:22 2017
From: report at bugs.python.org (shadowadler)
Date: Sat, 26 Aug 2017 09:14:22 +0000
Subject: [New-bugs-announce] [issue31283] Inconsistent behaviours with
 explicit and implicit inheritance from object
Message-ID: <1503738862.98.0.13501343613.issue31283@psf.upfronthosting.co.za>


New submission from shadowadler:

I discovered this while messing about with an unrelated idea, but the issue is that if you inherit explicitly from object, you get different behaviour than when you inherit implicitly. This is duplicated from my SO answer here: https://stackoverflow.com/questions/1238606/is-it-necessary-or-useful-to-inherit-from-pythons-object-in-python-3-x/45893772#45893772

If you explicitly inherit from object, what you are actually doing is inheriting from builtins.object regardless of what that points to at the time.

Therefore, I could have some (very wacky) module which overrides object for some reason. We'll call this first module "newobj.py":

import builtins

old_object = builtins.object  # otherwise cyclic dependencies

class new_object(old_object):

    def __init__(self, *args, **kwargs):
        super(new_object, self).__init__(*args, **kwargs)
        self.greeting = "Hello World!" 

builtins.object = new_object  #overrides the default object
Then in some other file ("klasses.py"):

class Greeter(object):
    pass

class NonGreeter:
    pass
Then in a third file (which we can actually run):

import newobj, klasses  # This order matters!

greeter = klasses.Greeter()
print(greeter.greeting)  # prints the greeting in the new __init__

non_greeter = NonGreeter()
print(non_greeter.greeting) # throws an attribute error
So you can see that, in the case where it is explicitly inheriting from object, we get a different behaviour than where you allow the implicit inheritance.

----------
messages: 300865
nosy: shadowadler
priority: normal
severity: normal
status: open
title: Inconsistent behaviours with explicit and implicit inheritance from object
versions: Python 3.5, Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31283>
_______________________________________

From report at bugs.python.org  Sat Aug 26 12:55:38 2017
From: report at bugs.python.org (Cheryl Sabella)
Date: Sat, 26 Aug 2017 16:55:38 +0000
Subject: [New-bugs-announce] [issue31284] IDLE: Fix WindowList invalid
 command name error when running tests
Message-ID: <1503766538.24.0.514954347408.issue31284@psf.upfronthosting.co.za>


New submission from Cheryl Sabella:

When running tests that use an EditorWindow, the following warning message occurs:

warning: callback failed in WindowList <class '_tkinter.TclError'> : invalid command name ".!menu.windows"

This warning comes from `call_callbacks` in WindowsList in windows.py when the callback is `postwindowsmenu()` from editor, which is set for the `windows` menu.  The line `end = menu.index("end")` is the line that fails.

----------
assignee: terry.reedy
components: IDLE
messages: 300885
nosy: csabella, terry.reedy
priority: normal
severity: normal
status: open
title: IDLE: Fix WindowList invalid command name error when running tests
type: enhancement
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31284>
_______________________________________

From report at bugs.python.org  Sat Aug 26 14:48:22 2017
From: report at bugs.python.org (Oren Milman)
Date: Sat, 26 Aug 2017 18:48:22 +0000
Subject: [New-bugs-announce] [issue31285] a SystemError and an assertion
 failure in warnings.warn_explicit()
Message-ID: <1503773302.07.0.816913451611.issue31285@psf.upfronthosting.co.za>


New submission from Oren Milman:

1.
the following causes an assertion failure in Python/_warnings.c in
show_warning():

import warnings

class BadLoader:
    def get_source(self, fullname):
        class BadSource:
            def splitlines(self):
                return [42]
        return BadSource()

del warnings._showwarnmsg
warnings.warn_explicit(message='foo', category=ArithmeticError, filename='bar',
                       lineno=1, module_globals={'__loader__': BadLoader(),
                                                 '__name__': 'foobar'})

in short, the assertion failure would happen in warnings.warn_explicit() in case
module_globals['__loader__'].get_source(module_globals['__name__']).splitlines()[lineno-1]
is not a str.


2.
the following raises a SystemError:

import warnings

class BadLoader:
    def get_source(self, fullname):
        class BadSource:
            def splitlines(self):
                return 42
        return BadSource()

warnings.warn_explicit(message='foo', category=UserWarning, filename='bar',
                       lineno=42, module_globals={'__loader__': BadLoader(),
                                                  '__name__': 'foobar'})

in short, warnings.warn_explicit() raises the SystemError in case
module_globals['__loader__'].get_source(module_globals['__name__']).splitlines()
is not a list.



ISTM that adding a check in warnings_warn_explicit() (in Python/_warnings.c),
to check whether
module_globals['__loader__'].get_source(module_globals['__name__'])
is a str (after existing code found out that it isn't None) would prevent both
the assertion failure and the SystemError.

What do you think? Is it OK to permit get_source() to return only None or a str?

----------
components: Interpreter Core
messages: 300892
nosy: Oren Milman
priority: normal
severity: normal
status: open
title: a SystemError and an assertion failure in warnings.warn_explicit()
type: crash
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31285>
_______________________________________

From report at bugs.python.org  Sat Aug 26 22:38:08 2017
From: report at bugs.python.org (Thomas Caswell)
Date: Sun, 27 Aug 2017 02:38:08 +0000
Subject: [New-bugs-announce] [issue31286] import in finally results in
 SystemError
Message-ID: <1503801488.97.0.284675687907.issue31286@psf.upfronthosting.co.za>


New submission from Thomas Caswell:

Code to reproduce:

def import_in_finally_fail():
    try:
        print('yo')
    finally:
        import asyncio.queues as aq


Results in:

In [68]: import_in_finally_fail()
yo
---------------------------------------------------------------------------
SystemError                               Traceback (most recent call last)
<ipython-input-68-4a6d2efcb22b> in <module>()
----> 1 import_in_finally_fail()

/tmp/py2661VBj in import_in_finally_fail()

SystemError: 'finally' pops bad exception


The exact import does matter, but it needs to be at least 2 modules deep and be aliased.  

By patching cpython to put what finally pops off the stack in the error message it looks like it is the parent module of the import.

This was found via the Matplotlib test suite (ex https://travis-ci.org/matplotlib/matplotlib/jobs/262907186 ) and reported to pytest (https://github.com/pytest-dev/pytest/issues/2674)

----------
components: Interpreter Core
messages: 300911
nosy: tcaswell
priority: normal
severity: normal
status: open
title: import in finally results in SystemError
type: crash
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31286>
_______________________________________

From report at bugs.python.org  Sun Aug 27 15:24:37 2017
From: report at bugs.python.org (Terry J. Reedy)
Date: Sun, 27 Aug 2017 19:24:37 +0000
Subject: [New-bugs-announce] [issue31287] IDLE tests: don't modify
 tkinter.messagebox.
Message-ID: <1503861877.04.0.798674063442.issue31287@psf.upfronthosting.co.za>


New submission from Terry J. Reedy:

test_configdialog twice patches tkinter.messagebox to mock askyesno without restoring it properly.  As a consequence, the outwin test in #30617 failed because the messagebox module in editor lacked askyesno.

First patch: fix configdialog and test_configdialog.  Similar to what is done elsewhere, make askyesno an instance method so that it is masked by an instance attribute mock and unmasked by deleting the instance function.  In the process, revise the messabebox import,

Follow-on patch(es): check exiting tests and apply the same idea everywhere.  In the process, end use of 'tkMessageBox'.

----------
assignee: terry.reedy
components: IDLE
messages: 300921
nosy: terry.reedy
priority: normal
severity: normal
stage: needs patch
status: open
title: IDLE tests: don't modify tkinter.messagebox.
type: behavior
versions: Python 3.6, Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31287>
_______________________________________

From report at bugs.python.org  Sun Aug 27 15:46:57 2017
From: report at bugs.python.org (Terry J. Reedy)
Date: Sun, 27 Aug 2017 19:46:57 +0000
Subject: [New-bugs-announce] [issue31288] IDLE tests: don't modify
 tkinter.messagebox.
Message-ID: <1503863217.0.0.440076905613.issue31288@psf.upfronthosting.co.za>


New submission from Terry J. Reedy:

Followup to #31287. Check existing tests for similar errors.  Apply the idea of localizing access to messageboxes everywhere.  In the process, end use of 'tkMessageBox'.

----------
assignee: terry.reedy
components: IDLE
messages: 300923
nosy: terry.reedy
priority: normal
severity: normal
stage: needs patch
status: open
title: IDLE tests: don't modify tkinter.messagebox.
type: enhancement
versions: Python 3.6, Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31288>
_______________________________________

From report at bugs.python.org  Sun Aug 27 16:17:56 2017
From: report at bugs.python.org (Paul Pinterits)
Date: Sun, 27 Aug 2017 20:17:56 +0000
Subject: [New-bugs-announce] [issue31289] File paths in exception traceback
 resolve symlinks
Message-ID: <1503865076.7.0.858933592926.issue31289@psf.upfronthosting.co.za>


New submission from Paul Pinterits:

The file paths displayed in exception tracebacks have their symlinks resolved. I would prefer if the "original" path could be displayed instead, because resolved symlinks result in unexpected paths in the traceback and can be quite confusing.

An example:

rawing at localhost ~> cat test_scripts/A.py
import B
B.throw()

rawing at localhost ~> cat test_scripts/B.py
def throw():
    raise ValueError

rawing at localhost ~> ln -s test_scripts test_symlink

rawing at localhost ~> python3 test_symlink/A.py
Traceback (most recent call last):
  File "test_symlink/A.py", line 2, in <module>
    B.throw()
  File "/home/rawing/test_scripts/B.py", line 2, in throw
    raise ValueError
ValueError

As you can see, even though both scripts reside in the same directory, the file paths displayed in the traceback look very different. At first glance, it looks like B is in a completely different place than A.
Furthermore, this behavior tends to trip up IDEs - PyCharm for example does not understand that test_scripts/B.py and test_symlink/B.py are the same file, so I end up having the same file opened in two different tabs.

Would it be possible to change this behavior and have "/home/rawing/test_symlink/B.py" show up in the traceback instead?

----------
components: Interpreter Core
messages: 300926
nosy: Paul Pinterits
priority: normal
severity: normal
status: open
title: File paths in exception traceback resolve symlinks
type: behavior
versions: Python 2.7, Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31289>
_______________________________________

From report at bugs.python.org  Mon Aug 28 03:28:43 2017
From: report at bugs.python.org (Richard)
Date: Mon, 28 Aug 2017 07:28:43 +0000
Subject: [New-bugs-announce] [issue31290] segfault on missing library symbol
Message-ID: <1503905323.73.0.492729138939.issue31290@psf.upfronthosting.co.za>


New submission from Richard:

I'm building a Python library with a C++ component composed of a number of source .cpp files.

After some changes today, compiling and loading in Python3 resulted in a segfault:


Python 3.5.3 (default, Jan 19 2017, 14:11:04) 
[GCC 6.3.0 20170118] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import mymodule
Segmentation fault (core dumped)


As you might imagine, this is not the funnest thing ever to debug.

Thankfully, when I compiled the module for Python2 and tried to load, a much more helpful and informative error message was displayed.


Python 2.7.13 (default, Jan 19 2017, 14:48:08) 
[GCC 6.3.0 20170118] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import mymodule
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "mymodule/__init__.py", line 1, in <module>
    import _mymodule
ImportError: ./_mymodule.so: undefined symbol: DBFGetRecordCount
>>> 


And indeed, it seems that in my setup.py file where I had

setuptools.Extension(
  "_mymodule",
  glob.glob('src/*.cpp') + glob.glob('lib/mylib/*.cpp') + glob.glob('lib/mylib/sublib/*.c'),

I should have had 'lib/mylib/sublib/*.cpp'.



I think the current behaviour can be improved:

1. Python3 should provide a helpful error message, like Python2 does.

2. setup.py should fail or at least provide a warning if there are missing symbols. (Perhaps this suggestion should be aimed at one of the setup utilities, but I'm not sure where to send it.)

----------
messages: 300942
nosy: immortalplants
priority: normal
severity: normal
status: open
title: segfault on missing library symbol
versions: Python 2.7, Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31290>
_______________________________________

From report at bugs.python.org  Mon Aug 28 04:44:26 2017
From: report at bugs.python.org (Oren Milman)
Date: Mon, 28 Aug 2017 08:44:26 +0000
Subject: [New-bugs-announce] [issue31291] an assertion failure in
 zipimport.zipimporter.get_data()
Message-ID: <1503909866.45.0.325412617055.issue31291@psf.upfronthosting.co.za>


New submission from Oren Milman:

on Windows, assuming the file 'foo.zip' exists, the following would cause an
assertion failure in Modules/zipimport.c in zipimport_zipimporter_get_data_impl():

import zipimport

class BadStr(str):
    def replace(self, old, new):
        return 42

zipimport.zipimporter('foo.zip').get_data(BadStr('bar'))


this is because zipimport_zipimporter_get_data_impl() assumes that
BadStr('bar').replace('/', '\\') is a string.

----------
components: Interpreter Core
messages: 300944
nosy: Oren Milman
priority: normal
severity: normal
status: open
title: an assertion failure in zipimport.zipimporter.get_data()
type: crash
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31291>
_______________________________________

From report at bugs.python.org  Mon Aug 28 09:57:45 2017
From: report at bugs.python.org (flying sheep)
Date: Mon, 28 Aug 2017 13:57:45 +0000
Subject: [New-bugs-announce] [issue31292] `python setup.py check
 --restructuredtext` fails when a include directive is present.
Message-ID: <1503928665.81.0.590770692205.issue31292@psf.upfronthosting.co.za>


New submission from flying sheep:

If I have a setup.py containing a include directive, and use the command in the title, I get


Traceback (most recent call last):
  [...]
  File "/usr/lib/python3.6/site-packages/docutils/parsers/rst/directives/misc.py", line 59, in run
    source_dir = os.path.dirname(os.path.abspath(source))
  File "/usr/lib/python3.6/posixpath.py", line 369, in abspath
    path = os.fspath(path)
TypeError: expected str, bytes or os.PathLike object, not _io.StringIO


The bug is in this method: distutils.command.check.check._check_rst_data

The first line already hints at it: `source_path = StringIO()`

StringIO instances aren?t paths, but at least the include directive (docutils.parsers.rst.directives.misc.Include) expects one.

----------
components: Distutils
files: check-test.py
messages: 300953
nosy: dstufft, flying sheep, merwok
priority: normal
severity: normal
status: open
title: `python setup.py check --restructuredtext` fails when a include directive is present.
type: crash
versions: Python 3.5, Python 3.6, Python 3.7
Added file: http://bugs.python.org/file47104/check-test.py

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31292>
_______________________________________

From report at bugs.python.org  Mon Aug 28 10:36:40 2017
From: report at bugs.python.org (Oren Milman)
Date: Mon, 28 Aug 2017 14:36:40 +0000
Subject: [New-bugs-announce] [issue31293] crashes in
 multiply_float_timedelta() and in truedivide_timedelta_float()
Message-ID: <1503931000.21.0.764494766764.issue31293@psf.upfronthosting.co.za>


New submission from Oren Milman:

both of the following true division and multiplication operations crash the
interpreter:

import datetime

class BadFloat(float):
    def as_integer_ratio(self):
        return (1 << 1000) - 1

datetime.timedelta() / BadFloat()
datetime.timedelta() * BadFloat()


this is because both multiply_float_timedelta() and truedivide_timedelta_float()
(in Modules/_datetimemodule.c) assume as_integer_ratio() returns tuple.

----------
components: Interpreter Core
messages: 300954
nosy: Oren Milman
priority: normal
severity: normal
status: open
title: crashes in multiply_float_timedelta() and in truedivide_timedelta_float()
type: crash
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31293>
_______________________________________

From report at bugs.python.org  Mon Aug 28 14:55:35 2017
From: report at bugs.python.org (Pablo)
Date: Mon, 28 Aug 2017 18:55:35 +0000
Subject: [New-bugs-announce] [issue31294] ZeroMQSocketListener and
 ZeroMQSocketHandler examples in the Logging Cookbook not working
Message-ID: <1503946535.18.0.843983247211.issue31294@psf.upfronthosting.co.za>


New submission from Pablo:

There are two documentation issues regarding the ZeroMQSocketListener and ZeroMQSocketHandler examples in the Logging Cookbook:


The first issue is that in the 'init' method for the 'ZeroMQSocketListener' the base class 'init' is never called and therefore the internal variables are not correctly initialized. This leads to an empty 'handlers' attribute and a undefined 'queue' attribute.

The second issue is that zmq.Socket.send and zmq.Socket.setsockopt only admits bytecode variables and therefore it fails in Python3 when using plain strings.

----------
assignee: docs at python
components: Documentation
messages: 300960
nosy: docs at python, pablogsal
priority: normal
pull_requests: 3273
severity: normal
status: open
title: ZeroMQSocketListener and ZeroMQSocketHandler examples in the Logging Cookbook not working

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31294>
_______________________________________

From report at bugs.python.org  Mon Aug 28 15:42:50 2017
From: report at bugs.python.org (Antony Lee)
Date: Mon, 28 Aug 2017 19:42:50 +0000
Subject: [New-bugs-announce] [issue31295] typo in __hash__ docs
Message-ID: <1503949370.09.0.113747018634.issue31295@psf.upfronthosting.co.za>


New submission from Antony Lee:

In https://docs.python.org/3.7/reference/datamodel.html#object.__hash__ I think

x.__hash__() returns an appropriate value such that x == y implies both that x is y and hash(x) == hash(y).

should be

x.__hash__() returns an appropriate value such that x == y and x is y both imply that hash(x) == hash(y).

right?

----------
assignee: docs at python
components: Documentation
messages: 300961
nosy: Antony.Lee, docs at python
priority: normal
severity: normal
status: open
title: typo in __hash__ docs
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31295>
_______________________________________

From report at bugs.python.org  Mon Aug 28 19:01:53 2017
From: report at bugs.python.org (Gregory P. Smith)
Date: Mon, 28 Aug 2017 23:01:53 +0000
Subject: [New-bugs-announce] [issue31296] support pty.fork and os.forkpty
 actions in posix subprocess module
Message-ID: <1503961313.68.0.46157569619.issue31296@psf.upfronthosting.co.za>


New submission from Gregory P. Smith:

The PyPI pexpect.spawn API is incompatible with multithreaded code thanks to its dependency on pty.fork() and running non async signal safe code between fork() and exec().
 https://github.com/pexpect/ptyprocess/issues/43
 https://github.com/pexpect/pexpect/issues/369

If subprocess supported pty.fork() and pexpect.pty_spawn.spawn like actions, pexpect could use that by default instead of the threading incompatible pty.fork().

(this is a brainstorm, see what the pexpect maintainers want in the above github issues)

----------
components: Library (Lib)
messages: 300968
nosy: gregory.p.smith
priority: normal
severity: normal
status: open
title: support pty.fork and os.forkpty actions in posix subprocess module
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31296>
_______________________________________

From report at bugs.python.org  Mon Aug 28 19:53:15 2017
From: report at bugs.python.org (Rachel Tobin)
Date: Mon, 28 Aug 2017 23:53:15 +0000
Subject: [New-bugs-announce] [issue31297] Unpickleable ModuleImportError in
 unittest patch not backported to 2.7
Message-ID: <1503964395.2.0.931089155502.issue31297@psf.upfronthosting.co.za>


New submission from Rachel Tobin:

When running the unittest runner in parallel with an module import error present in your code, this exception is raised:

_pickle.PicklingError: Can't pickle <class 'unittest.loader.ModuleImportFailure'>: attribute lookup ModuleImportFailure on unittest.loader failed

unittest should be raising standard exceptions instead.

This was fixed in Python 3.4 with this patch: https://bugs.python.org/issue22903

Given that 2.7 is still supported, this patch should be backported.

----------
components: Library (Lib)
messages: 300969
nosy: Rachel Tobin
priority: normal
severity: normal
status: open
title: Unpickleable ModuleImportError in unittest patch not backported to 2.7
type: crash
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31297>
_______________________________________

From report at bugs.python.org  Mon Aug 28 22:10:29 2017
From: report at bugs.python.org (Daniel)
Date: Tue, 29 Aug 2017 02:10:29 +0000
Subject: [New-bugs-announce] [issue31298] Error when calling numpy.astype
Message-ID: <1503972629.82.0.848961693934.issue31298@psf.upfronthosting.co.za>


New submission from Daniel:

Ubuntu 16.04.3
Python 3.5.2
Numpy version 1.11.0

Running the following two commands causes Python to crash:
import numpy as np
np.array([[1, 2],[3,4]]).astype(np.dtype([("a", np.float), ("b", np.str_)]))

The error occurs when running in an interactive session and when running those two commands as a script.

The error I get is:
*** Error in `python3': free(): invalid next size (normal): 0x0000000001a088f0 ***

A memory map printed after the error is attached.

The backtrace is
======= Backtrace: =========
/lib/x86_64-linux-gnu/libc.so.6(+0x777e5)[0x7f55ac35e7e5]
/lib/x86_64-linux-gnu/libc.so.6(+0x8037a)[0x7f55ac36737a]
/lib/x86_64-linux-gnu/libc.so.6(cfree+0x4c)[0x7f55ac36b53c]
/usr/lib/python3/dist-packages/numpy/core/multiarray.cpython-35m-x86_64-linux-gnu.so(+0x6d35e)[0x7f55ab0fa35e]
/usr/lib/python3/dist-packages/numpy/core/multiarray.cpython-35m-x86_64-linux-gnu.so(+0x45a29)[0x7f55ab0d2a29]
/usr/lib/python3/dist-packages/numpy/core/multiarray.cpython-35m-x86_64-linux-gnu.so(+0x461a2)[0x7f55ab0d31a2]
/usr/lib/python3/dist-packages/numpy/core/multiarray.cpython-35m-x86_64-linux-gnu.so(+0xcf828)[0x7f55ab15c828]
python3(PyCFunction_Call+0x77)[0x4e9bc7]
python3(PyEval_EvalFrameEx+0x614)[0x524414]
python3[0x52d2e3]
python3(PyEval_EvalCode+0x1f)[0x52dfdf]
python3[0x5fd2c2]
python3(PyRun_FileExFlags+0x9a)[0x5ff76a]
python3(PyRun_SimpleFileExFlags+0x1bc)[0x5ff95c]
python3(Py_Main+0x456)[0x63e7d6]
python3(main+0xe1)[0x4cfe41]
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf0)[0x7f55ac307830]
python3(_start+0x29)[0x5d5f29]

----------
components: Interpreter Core
files: memdump.txt
messages: 300970
nosy: droth
priority: normal
severity: normal
status: open
title: Error when calling numpy.astype
type: crash
versions: Python 3.5
Added file: http://bugs.python.org/file47105/memdump.txt

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31298>
_______________________________________

From report at bugs.python.org  Tue Aug 29 03:45:45 2017
From: report at bugs.python.org (Nick Coghlan)
Date: Tue, 29 Aug 2017 07:45:45 +0000
Subject: [New-bugs-announce] [issue31299] Add "ignore_modules" option to
 TracebackException.format()
Message-ID: <1503992745.47.0.33675937395.issue31299@psf.upfronthosting.co.za>


New submission from Nick Coghlan:

This is a proposed traceback module enhancement deriving from the "python -m" traceback verbosity discussion in issue 16217.

Specifically, I'm thinking a good way of cleaning up those tracebacks would be to use `TracebackException.format()` to get the formatted exception output, but designate particular modules to be *skipped* during the iteration (specifically `runpy`, `_frozen_importlib`, `importlib`).

I'm filing this as a separate issue so the discussion of the public API for the capability in the traceback module can be separated from the discussion of applying that API in runpy.

----------
components: Library (Lib)
messages: 300974
nosy: ncoghlan, rbcollins, vaultah
priority: normal
severity: normal
stage: needs patch
status: open
title: Add "ignore_modules" option to TracebackException.format()
type: enhancement
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31299>
_______________________________________

From report at bugs.python.org  Tue Aug 29 07:03:31 2017
From: report at bugs.python.org (Aigars Mahinovs)
Date: Tue, 29 Aug 2017 11:03:31 +0000
Subject: [New-bugs-announce] [issue31300] Traceback prints different code
 than the running module
Message-ID: <1504004611.56.0.361089712755.issue31300@psf.upfronthosting.co.za>


New submission from Aigars Mahinovs:

If you run a longer running Python program, then change the Python module that it loaded on disk and then cause a traceback to happen in the code in that module, then the traceback may be nonsensical as it will reference line numbers from the module in memory, but print code from the same line numbers from the file on disk, even if they are different.

In any case there should be a prominent warning on traceback if the module in memory does not match the module on disk, but ideally the traceback print should show the code in memory - the actual code that caused the problem.

----------
components: Library (Lib)
messages: 300980
nosy: Aigars Mahinovs
priority: normal
severity: normal
status: open
title: Traceback prints different code than the running module
type: behavior
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31300>
_______________________________________

From report at bugs.python.org  Tue Aug 29 07:20:31 2017
From: report at bugs.python.org (Cody Sixteen)
Date: Tue, 29 Aug 2017 11:20:31 +0000
Subject: [New-bugs-announce] [issue31301] Python 2.7 SIGSEGV
Message-ID: <1504005631.03.0.301240495014.issue31301@psf.upfronthosting.co.za>


New submission from Cody Sixteen:

During one fuzzing session I found that python2.7 will crash when parsing the file (attached).

Maybe you will find it useful.

----------
components: Build
files: py27-sigsegv.zip
messages: 300981
nosy: cody
priority: normal
severity: normal
status: open
title: Python 2.7 SIGSEGV
type: crash
versions: Python 2.7
Added file: http://bugs.python.org/file47106/py27-sigsegv.zip

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31301>
_______________________________________

From report at bugs.python.org  Tue Aug 29 08:56:45 2017
From: report at bugs.python.org (murphdasurf)
Date: Tue, 29 Aug 2017 12:56:45 +0000
Subject: [New-bugs-announce] [issue31302] smtplib on linux fails to log in
 correctly
Message-ID: <1504011405.04.0.710938867076.issue31302@psf.upfronthosting.co.za>


New submission from murphdasurf:

The protocol should run like this:

send: 'ehlo XXXXXXXXXXXXX\r\n'
reply: b'250-XXXXXXXXXXXXXXXXXX.com\r\n'
reply: b'250-PIPELINING\r\n'
reply: b'250-SIZE 100480000\r\n'
reply: b'250-VRFY\r\n'
reply: b'250-ETRN\r\n'
reply: b'250-STARTTLS\r\n'
reply: b'250-ENHANCEDSTATUSCODES\r\n'
reply: b'250-8BITMIME\r\n'
reply: b'250 DSN\r\n'
reply: retcode (250); Msg: b'XXXXXXXXXXXXXXXX.com\nPIPELINING\nSIZE 100480000\nVRFY\nETRN\nSTARTTLS\nENHANCEDSTATUSCODES\n8BITMIME\nDSN'
send: 'STARTTLS\r\n'
reply: b'220 2.0.0 Ready to start TLS\r\n'
reply: retcode (220); Msg: b'2.0.0 Ready to start TLS'
send: 'ehlo XXXXXXXXXXXXXXXXXXXXX\r\n'
reply: b'250-XXXXXXXXXXXXXXXXXX.com\r\n'
reply: b'250-PIPELINING\r\n'
reply: b'250-SIZE 100480000\r\n'
reply: b'250-VRFY\r\n'
reply: b'250-ETRN\r\n'
reply: b'250-AUTH LOGIN\r\n'
reply: b'250-AUTH=LOGIN\r\n'
reply: b'250-ENHANCEDSTATUSCODES\r\n'
reply: b'250-8BITMIME\r\n'
reply: b'250 DSN\r\n'
reply: retcode (250); Msg: b'XXXXXXXXXXXXXXXXXXXXXX.com\nPIPELINING\nSIZE 100480000\nVRFY\nETRN\nAUTH LOGIN\nAUTH=LOGIN\nENHANCEDSTATUSCODES\n8BITMIME\nDSN'
send: 'AUTH LOGIN XXXXXXXXXXXXXXX\r\n'
reply: b'334 RRRRRRRRRRR\r\n'
reply: retcode (334); Msg: b'RRRRRRRRR'
send: 'TTTTTTTTTTTT=\r\n'
reply: b'235 2.7.0 Authentication successful\r\n'
reply: retcode (235); Msg: b'2.7.0 Authentication successful'


And in fact it does so on Windows. However, on Linux it runs like this:

send: 'ehlo XXXXXXXXXXXXX\r\n'
reply: b'250-XXXXXXXXXXXXX.com\r\n'
reply: b'250-PIPELINING\r\n'
reply: b'250-SIZE 100480000\r\n'
reply: b'250-VRFY\r\n'
reply: b'250-ETRN\r\n'
reply: b'250-STARTTLS\r\n'
reply: b'250-ENHANCEDSTATUSCODES\r\n'
reply: b'250-8BITMIME\r\n'
reply: b'250 DSN\r\n'
reply: retcode (250); Msg: b'XXXXXXXXXXXXX.com\nPIPELINING\nSIZE 100480000\nVRFY\nETRN\nSTARTTLS\nENHANCEDSTATUSCODES\n8BITMIME\nDSN'
send: 'STARTTLS\r\n'
reply: b'220 2.0.0 Ready to start TLS\r\n'
reply: retcode (220); Msg: b'2.0.0 Ready to start TLS'
send: 'ehlo XXXXXXXXXXXXX\r\n'
reply: b'250-XXXXXXXXXXXXX.com\r\n'
reply: b'250-PIPELINING\r\n'
reply: b'250-SIZE 100480000\r\n'
reply: b'250-VRFY\r\n'
reply: b'250-ETRN\r\n'
reply: b'250-AUTH LOGIN\r\n'
reply: b'250-AUTH=LOGIN\r\n'
reply: b'250-ENHANCEDSTATUSCODES\r\n'
reply: b'250-8BITMIME\r\n'
reply: b'250 DSN\r\n'
reply: retcode (250); Msg: b'XXXXXXXXXXXXX.com\nPIPELINING\nSIZE 100480000\nVRFY\nETRN\nAUTH LOGIN\nAUTH=LOGIN\nENHANCEDSTATUSCODES\n8BITMIME\nDSN'
send: 'AUTH LOGIN XXXXXXXXXXXXXXX\r\n'
reply: b'334 RRRRRRRRRRR\r\n'
reply: retcode (334); Msg: b'RRRRRRRRRRR'
send: "b''\r\n"
reply: b'535 5.7.8 Error: authentication failed: another step is needed in authentication\r\n'
reply: retcode (535); Msg: b'5.7.8 Error: authentication failed: another step is needed in authentication'

The last sent message is an empty string and then the authentication fails. All other tokens which are replaced by XXXXXX, RRRRR or TTTTT are exactly the same.

This way the login fails and I can not use smtplib to send my mail.

----------
components: Library (Lib)
messages: 300986
nosy: murphdasurf
priority: normal
severity: normal
status: open
title: smtplib on linux fails to log in correctly
type: behavior
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31302>
_______________________________________

From report at bugs.python.org  Tue Aug 29 13:45:53 2017
From: report at bugs.python.org (Vyacheslav Rafalskiy)
Date: Tue, 29 Aug 2017 17:45:53 +0000
Subject: [New-bugs-announce] [issue31303] xml.etree.ElementTree fails to
 parse a document (regression)
Message-ID: <1504028753.39.0.976845207061.issue31303@psf.upfronthosting.co.za>


New submission from Vyacheslav Rafalskiy:

In Python 3.5.4 and 3.6.2, both on Windows and Linux, parsing a manifestly correct xml file like:

xml.etree.ElementTree.parse('bad_file.xml')

raises:
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xc3 in position 1023: invalid continuation byte

Any other Python version I tried works fine, including 2.7.13, 3.5.2 ...

----------
components: XML
files: bad_file.xml
messages: 300996
nosy: Vyacheslav.Rafalskiy
priority: normal
severity: normal
status: open
title: xml.etree.ElementTree fails to parse a document (regression)
type: crash
versions: Python 3.5, Python 3.6
Added file: http://bugs.python.org/file47108/bad_file.xml

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31303>
_______________________________________

From report at bugs.python.org  Tue Aug 29 16:13:31 2017
From: report at bugs.python.org (tamas)
Date: Tue, 29 Aug 2017 20:13:31 +0000
Subject: [New-bugs-announce] [issue31304] Update doc for starmap_async
 error_back kwarg
Message-ID: <1504037611.28.0.102125644209.issue31304@psf.upfronthosting.co.za>


New submission from tamas:

The documentation suggests that multiprocessing.pool.Pool. starmap_async has a kwarg called 'error_back', but it actually has a kwarg called 'error_callback' to handle errors, just like map_async.

Link:
https://docs.python.org/3.7/library/multiprocessing.html#multiprocessing.pool.Pool.starmap_async

----------
assignee: docs at python
components: Documentation
messages: 300999
nosy: docs at python, tamas
priority: normal
severity: normal
status: open
title: Update doc for starmap_async error_back kwarg
type: behavior
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31304>
_______________________________________

From report at bugs.python.org  Tue Aug 29 22:19:09 2017
From: report at bugs.python.org (Morris Li)
Date: Wed, 30 Aug 2017 02:19:09 +0000
Subject: [New-bugs-announce] [issue31305] 'pydoc -w import' report "no
 Python documentation found for 'import'"
Message-ID: <1504059549.17.0.252572867036.issue31305@psf.upfronthosting.co.za>


Changes by Morris Li <limuyuan91 at 126.com>:


----------
assignee: docs at python
components: Documentation
nosy: docs at python, limuyuan
priority: normal
severity: normal
status: open
title: 'pydoc -w import' report "no Python documentation found for 'import'"
type: behavior
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31305>
_______________________________________

From report at bugs.python.org  Wed Aug 30 00:00:32 2017
From: report at bugs.python.org (Terry J. Reedy)
Date: Wed, 30 Aug 2017 04:00:32 +0000
Subject: [New-bugs-announce] [issue31306] IDLE, configdialog,
 General tab: validate user entries
Message-ID: <1504065632.2.0.53368716596.issue31306@psf.upfronthosting.co.za>


New submission from Terry J. Reedy:

This was initially part of #31051.  See 2nd have of msg299262 and all of msg299263.

----------
assignee: terry.reedy
components: IDLE
messages: 301005
nosy: terry.reedy
priority: normal
severity: normal
stage: test needed
status: open
title: IDLE, configdialog, General tab: validate user entries
type: behavior
versions: Python 3.6, Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31306>
_______________________________________

From report at bugs.python.org  Wed Aug 30 10:15:47 2017
From: report at bugs.python.org (Vincent Michel)
Date: Wed, 30 Aug 2017 14:15:47 +0000
Subject: [New-bugs-announce] [issue31307] ConfigParser.read silently fails
 if filenames argument is a byte string
Message-ID: <1504102547.11.0.362577067596.issue31307@psf.upfronthosting.co.za>


New submission from Vincent Michel:

Calling `config_parser.read` with `'test'` is equivalent to:

    config_parser.read(['test'])

while calling `config_parser.read` with `b'test'` is treated as:

    config_parser.read([116, 101, 115, 116])

which means python will try to open the file descriptors 101, 115 and 116.

I don't know if byte path should be supported, but this is probably not the expected behavior.

The method code: https://github.com/python/cpython/blob/master/Lib/configparser.py#L678-L702

----------
components: Library (Lib)
messages: 301026
nosy: vxgmichel
priority: normal
severity: normal
status: open
title: ConfigParser.read silently fails if filenames argument is a byte string
type: behavior
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31307>
_______________________________________

From report at bugs.python.org  Wed Aug 30 10:27:15 2017
From: report at bugs.python.org (Antoine Pitrou)
Date: Wed, 30 Aug 2017 14:27:15 +0000
Subject: [New-bugs-announce] [issue31308] forkserver process isn't
 re-launched if it died
Message-ID: <1504103235.79.0.43802074293.issue31308@psf.upfronthosting.co.za>


New submission from Antoine Pitrou:

It may happen that the forkserver process dies (for example if SIGINT is received because the user pressed Ctrl-C) while the parent process is still alive.  In that case, if the parent tries to create a new Process instance, an exception is received.

The exception looks like this:

  File "/xxx/lib/python3.5/multiprocessing/popen_forkserver.py", line 52, in _launch
	self.sentinel, w = forkserver.connect_to_new_process(self._fds)
  File "/xxx/lib/python3.5/multiprocessing/forkserver.py", line 66, in connect_to_new_process
	client.connect(self._forkserver_address)
ConnectionRefusedError: [Errno 111] Connection refused

----------
components: Library (Lib)
messages: 301027
nosy: pitrou
priority: normal
severity: normal
stage: needs patch
status: open
title: forkserver process isn't re-launched if it died
type: behavior
versions: Python 3.6, Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31308>
_______________________________________

From report at bugs.python.org  Wed Aug 30 11:34:35 2017
From: report at bugs.python.org (Philipp)
Date: Wed, 30 Aug 2017 15:34:35 +0000
Subject: [New-bugs-announce] [issue31309] Tkinter root window does not close
 if used with matplotlib.pyplot
Message-ID: <1504107275.3.0.972371734588.issue31309@psf.upfronthosting.co.za>


New submission from Philipp:

Today I ran into a problem when using both tkinter and matplotlib.pyplot at the same time. I thought it would be best to let you know, in case it is an issue. 

The problem is/was that the root window did not close itself, when just using `askopenfilename()`. 

A complete description of the problem, including a minimal working example, screenshots and a "solution" can be found on StackOverflow: 
https://stackoverflow.com/q/45960815/7414040 

I think that the function `askopenfilename()` does not ensure that the tkinter root window, that has been opened, is destroyed after the function call ended.

----------
assignee: terry.reedy
components: IDLE, Tkinter, Windows
messages: 301028
nosy: paul.moore, psiehr, steve.dower, terry.reedy, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: Tkinter root window does not close if used with matplotlib.pyplot
type: behavior
versions: Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31309>
_______________________________________

From report at bugs.python.org  Wed Aug 30 11:52:32 2017
From: report at bugs.python.org (Antoine Pitrou)
Date: Wed, 30 Aug 2017 15:52:32 +0000
Subject: [New-bugs-announce] [issue31310] semaphore tracker isn't protected
 against crashes
Message-ID: <1504108352.86.0.17423028145.issue31310@psf.upfronthosting.co.za>


New submission from Antoine Pitrou:

Similar to bpo-31308, but pertaining to multiprocessing's semaphore tracker process.

----------
components: Library (Lib)
messages: 301029
nosy: davin, pitrou
priority: normal
severity: normal
stage: needs patch
status: open
title: semaphore tracker isn't protected against crashes
type: behavior
versions: Python 3.6, Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31310>
_______________________________________

From report at bugs.python.org  Wed Aug 30 15:29:41 2017
From: report at bugs.python.org (Oren Milman)
Date: Wed, 30 Aug 2017 19:29:41 +0000
Subject: [New-bugs-announce] [issue31311] SystemError raised by
 PyCData_setstate() in case __dict__ is not a dict
Message-ID: <1504121381.44.0.97838192699.issue31311@psf.upfronthosting.co.za>


New submission from Oren Milman:

The following code causes PyCData_setstate() (in Modules/_ctypes/_ctypes.c) to
raise a SystemError:

import ctypes
class BadStruct(ctypes.Structure):
    def __dict__(self):
        pass

BadStruct().__setstate__({}, b'foo')

this is because PyCData_setstate() assumes that the __dict__ attribute is a dict.


while we are here, I wonder whether we should change the format given to 
PyArg_ParseTuple() to "!Os#", so that the following would raise a TypeError:

import ctypes
class MyStruct(ctypes.Structure):
    pass

MyStruct().__setstate__(42, b'foo')

AttributeError: 'int' object has no attribute 'keys'


what do you think?

----------
components: Extension Modules
messages: 301034
nosy: Oren Milman
priority: normal
severity: normal
status: open
title: SystemError raised by PyCData_setstate() in case __dict__ is not a dict
type: behavior
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31311>
_______________________________________

From report at bugs.python.org  Thu Aug 31 04:15:49 2017
From: report at bugs.python.org (Wayland Zhong)
Date: Thu, 31 Aug 2017 08:15:49 +0000
Subject: [New-bugs-announce] [issue31312] Build differences caused by the
 time stamps
Message-ID: <1504167349.56.0.0314062412353.issue31312@psf.upfronthosting.co.za>


New submission from Wayland Zhong:

When I build python (version 2.7.13), I find the build results are different every time. It seems to be caused by some times tamps, and they may be generated by the code below:
    Lib/py_compile.py:106: with open(file, 'U') as f:
    Lib/py_compile.py:107:     try:
    Lib/py_compile.py:108:        timestamp = long(os.fstat(f.fileno()).st_mtime)
    Lib/py_compile.py:109:    except AttributeError:
    Lib/py_compile.py:110:        timestamp = long(os.stat(file).st_mtime)
    Lib/py_compile.py:111:   codestring = f.read()

As we know, reproducible build is a good way to counter malicious attacks that generate malicious executables, by making it easy to recreate the executable to determine if the result is correct. How can I eliminate the differences caused by the time stamps? Just remove some code? Or is there any configuration?
If we can't eliminate the difference now, can lsof support it in future versions?
Thank you.

----------
components: Library (Lib)
messages: 301042
nosy: WaylandZ
priority: normal
severity: normal
status: open
title: Build differences caused by the time stamps
type: security
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31312>
_______________________________________

From report at bugs.python.org  Thu Aug 31 05:55:01 2017
From: report at bugs.python.org (=?utf-8?b?0JzQsNGA0Log0JrQvtGA0LXQvdCx0LXRgNCz?=)
Date: Thu, 31 Aug 2017 09:55:01 +0000
Subject: [New-bugs-announce] [issue31313] Feature Add support of
 os.chflags() on Linux platform
Message-ID: <1504173301.11.0.484917875166.issue31313@psf.upfronthosting.co.za>


New submission from ???? ?????????:

Yes, chflags() is not supported by linux directly. But actually, Linux supports chflags functionality:

https://stackoverflow.com/questions/34176464/why-os-chflags-doesnt-work-under-linux

through

ioctl (fd, EXT2_IOC_SETFLAGS, <BITMASK_OF_FLAGS>)

----------
components: Library (Lib)
messages: 301043
nosy: socketpair
priority: normal
severity: normal
status: open
title: Feature Add support of os.chflags() on Linux platform
type: enhancement
versions: Python 3.6, Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31313>
_______________________________________

From report at bugs.python.org  Thu Aug 31 06:29:52 2017
From: report at bugs.python.org (Matthias Klose)
Date: Thu, 31 Aug 2017 10:29:52 +0000
Subject: [New-bugs-announce] [issue31314] email throws exception with
 oversized header input
Message-ID: <1504175392.51.0.871887341171.issue31314@psf.upfronthosting.co.za>


New submission from Matthias Klose:

[forwarded from https://bugs.debian.org/854001]

$ cat tst.py
#!/usr/bin/env python
import sys
import email

mail = email.message_from_string(
"""From: <postmaster at example.com>
To: <bounce at example.com>
Subject: demo
X-Overlong-Header-Name-causes-python-mail-to-crash-in-re-serialization-example:

Hello
""")
message = mail.as_string()
sys.stdout.write(message)

$ python tst.py 
Traceback (most recent call last):
  File "tst.py", line 13, in <module>
    message = mail.as_string()
  File "/usr/lib/python2.7/email/message.py", line 137, in as_string
    g.flatten(self, unixfrom=unixfrom)
  File "/usr/lib/python2.7/email/generator.py", line 83, in flatten
    self._write(msg)
  File "/usr/lib/python2.7/email/generator.py", line 115, in _write
    self._write_headers(msg)
  File "/usr/lib/python2.7/email/generator.py", line 164, in _write_headers
    v, maxlinelen=self._maxheaderlen, header_name=h).encode()
  File "/usr/lib/python2.7/email/header.py", line 408, in encode
    lastchunk, lastcharset = newchunks[-1]
IndexError: list index out of range

----------
components: email
messages: 301044
nosy: barry, doko, r.david.murray
priority: normal
severity: normal
status: open
title: email throws exception with oversized header input
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31314>
_______________________________________

From report at bugs.python.org  Thu Aug 31 12:25:55 2017
From: report at bugs.python.org (Oren Milman)
Date: Thu, 31 Aug 2017 16:25:55 +0000
Subject: [New-bugs-announce] [issue31315] assertion failure in
 imp.create_dynamic(), when spec.name is not a string
Message-ID: <1504196755.68.0.337811894425.issue31315@psf.upfronthosting.co.za>


New submission from Oren Milman:

The following code causes an assertion failure in get_encoded_name(), which is
called by _PyImport_LoadDynamicModuleWithSpec() (in Python/importdl.c):

import imp

class BadSpec:
    name = 42
    origin = 'foo'

imp.create_dynamic(BadSpec())


this is because _PyImport_LoadDynamicModuleWithSpec() assumes that spec.name is
a string.


should we fix this (even though imp is deprecated)?

----------
components: Extension Modules
messages: 301050
nosy: Oren Milman
priority: normal
severity: normal
status: open
title: assertion failure in imp.create_dynamic(), when spec.name is not a string
type: crash
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31315>
_______________________________________

From report at bugs.python.org  Thu Aug 31 13:01:27 2017
From: report at bugs.python.org (Andreas Jung)
Date: Thu, 31 Aug 2017 17:01:27 +0000
Subject: [New-bugs-announce] [issue31316] Frequent *** stack smashing
 detected *** with Python 3.6.2/meinheld WSGI server
Message-ID: <1504198887.45.0.0502110179306.issue31316@psf.upfronthosting.co.za>


New submission from Andreas Jung:

We are currently testing Python 3.6.2 with a BottlePY based with "meinheld" WSGI server and we see frequently these crashes....is this related to Python or meinheld in particular?

Linux unknown-device 4.7.0-1-amd64 #1 SMP Debian 4.7.2-1 (2016-08-28) x86_64 GNU/Linux


2017-08-31 17:40:55,434] WARNING JWK verification disabled
[2017-08-31 17:40:55,435] INFO pimservice version 0.2.0.dev0
[2017-08-31 17:40:55,435] INFO Importing data from /home/nils/src/festool/coa-pim-service/SCPortal_de-DE_20170418.xml.zip
[2017-08-31 17:40:55,435] INFO Locales in ZIP file: de-DE
Loading SCPortal_de-DE_20170418.xml (1/1) ? done.
Bottle v0.12.12 server starting up (using MeinheldServer())...
Listening on http://localhost:8080/
Hit Ctrl-C to quit.

"0.0.0.0 - - [31/Aug/2017:17:41:27 +0000] "GET /Search/de-DE HTTP/1.1" 200 105256 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/59.0.3071.86 Safari/537.36"
*** stack smashing detected ***: /home/nils/src/festool/coa-pim-service/bin/python terminated
======= Backtrace: =========
/lib/x86_64-linux-gnu/libc.so.6(+0x70bcb)[0x7fdcddbabbcb]
/lib/x86_64-linux-gnu/libc.so.6(__fortify_fail+0x37)[0x7fdcddc34227]
/lib/x86_64-linux-gnu/libc.so.6(__fortify_fail+0x0)[0x7fdcddc341f0]
/home/nils/src/festool/coa-pim-service/lib/python3.6/site-packages/meinheld-0.6.1-py3.6-linux-x86_64.egg/meinheld/server.cpython-36m-x86_64-linux-gnu.so(+0xebdf)[0x7fdcd49c8bdf]
/home/nils/src/festool/coa-pim-service/lib/python3.6/site-packages/meinheld-0.6.1-py3.6-linux-x86_64.egg/meinheld/server.cpython-36m-x86_64-linux-gnu.so(picoev_poll_once_internal+0x95)[0x7fdcd49ce1c5]
/home/nils/src/festool/coa-pim-service/lib/python3.6/site-packages/meinheld-0.6.1-py3.6-linux-x86_64.egg/meinheld/server.cpython-36m-x86_64-linux-gnu.so(+0xfc87)[0x7fdcd49c9c87]
/home/nils/src/festool/coa-pim-service/bin/python(_PyCFunction_FastCallKeywords+0x1a7)[0x4bdcb7]
/home/nils/src/festool/coa-pim-service/bin/python[0x54d47e]
/home/nils/src/festool/coa-pim-service/bin/python(_PyEval_EvalFrameDefault+0x2e63)[0x5512d3]
/home/nils/src/festool/coa-pim-service/bin/python[0x54c681]
/home/nils/src/festool/coa-pim-service/bin/python[0x54d5f5]
/home/nils/src/festool/coa-pim-service/bin/python(_PyEval_EvalFrameDefault+0x2e63)[0x5512d3]
/home/nils/src/festool/coa-pim-service/bin/python[0x54d0d5]
/home/nils/src/festool/coa-pim-service/bin/python(PyEval_EvalCodeEx+0x2f)[0x54dfaf]
/home/nils/src/festool/coa-pim-service/bin/python[0x488cf6]
/home/nils/src/festool/coa-pim-service/bin/python(PyObject_Call+0x3a)[0x4562fa]
/home/nils/src/festool/coa-pim-service/bin/python(_PyEval_EvalFrameDefault+0x32b6)[0x551726]
/home/nils/src/festool/coa-pim-service/bin/python[0x54d0d5]
/home/nils/src/festool/coa-pim-service/bin/python[0x54d394]
/home/nils/src/festool/coa-pim-service/bin/python(_PyEval_EvalFrameDefault+0x3178)[0x5515e8]
/home/nils/src/festool/coa-pim-service/bin/python[0x54c681]
/home/nils/src/festool/coa-pim-service/bin/python[0x54d5f5]
/home/nils/src/festool/coa-pim-service/bin/python(_PyEval_EvalFrameDefault+0x2e63)[0x5512d3]
/home/nils/src/festool/coa-pim-service/bin/python[0x54d0d5]
/home/nils/src/festool/coa-pim-service/bin/python(PyEval_EvalCode+0x23)[0x54df43]
/home/nils/src/festool/coa-pim-service/bin/python(PyRun_FileExFlags+0x167)[0x42ac17]
/home/nils/src/festool/coa-pim-service/bin/python(PyRun_SimpleFileExFlags+0xeb)[0x42ae4b]
/home/nils/src/festool/coa-pim-service/bin/python(Py_Main+0xd8f)[0x43f12f]
/home/nils/src/festool/coa-pim-service/bin/python(main+0x167)[0x421337]
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf1)[0x7fdcddb5b2b1]
/home/nils/src/festool/coa-pim-service/bin/python(_start+0x2a)[0x42140a]
======= Memory map: ========
00400000-007aa000 r-xp 00000000 fe:01 18486295                           /usr/bin/python3.6
009a9000-009aa000 r--p 003a9000 fe:01 18486295                           /usr/bin/python3.6
009aa000-00a45000 rw-p 003aa000 fe:01 18486295                           /usr/bin/python3.6
00a45000-00a77000 rw-p 00000000 00:00 0 
00fa0000-15978000 rw-p 00000000 00:00 0                                  [heap]
7fdcd4533000-7fdcd4673000 rw-p 00000000 00:00 0 
7fdcd4673000-7fdcd4679000 r-xp 00000000 fe:01 14027603                   /home/nils/src/festool/coa-pim-service/lib/python3.6/site-packages/greenlet-0.4.12-py3.6-linux-x86_64.egg/greenlet.cpython-36m-x86_64-linux-gnu.so
7fdcd4679000-7fdcd4878000 ---p 00006000 fe:01 14027603                   /home/nils/src/festool/coa-pim-service/lib/python3.6/site-packages/greenlet-0.4.12-py3.6-linux-x86_64.egg/greenlet.cpython-36m-x86_64-linux-gnu.so
7fdcd4878000-7fdcd4879000 r--p 00005000 fe:01 14027603                   /home/nils/src/festool/coa-pim-service/lib/python3.6/site-packages/greenlet-0.4.12-py3.6-linux-x86_64.egg/greenlet.cpython-36m-x86_64-linux-gnu.so
7fdcd4879000-7fdcd487a000 rw-p 00006000 fe:01 14027603                   /home/nils/src/festool/coa-pim-service/lib/python3.6/site-packages/greenlet-0.4.12-py3.6-linux-x86_64.egg/greenlet.cpython-36m-x86_64-linux-gnu.so
7fdcd487a000-7fdcd49ba000 rw-p 00000000 00:00 0 
7fdcd49ba000-7fdcd49d3000 r-xp 00000000 fe:01 29107634                   /home/nils/src/festool/coa-pim-service/lib/python3.6/site-packages/meinheld-0.6.1-py3.6-linux-x86_64.egg/meinheld/server.cpython-36m-x86_64-linux-gnu.so
7fdcd49d3000-7fdcd4bd2000 ---p 00019000 fe:01 29107634                   /home/nils/src/festool/coa-pim-service/lib/python3.6/site-packages/meinheld-0.6.1-py3.6-linux-x86_64.egg/meinheld/server.cpython-36m-x86_64-linux-gnu.so
7fdcd4bd2000-7fdcd4bd3000 r--p 00018000 fe:01 29107634                   /home/nils/src/festool/coa-pim-service/lib/python3.6/site-packages/meinheld-0.6.1-py3.6-linux-x86_64.egg/meinheld/server.cpython-36m-x86_64-linux-gnu.so
7fdcd4bd3000-7fdcd4bd5000 rw-p 00019000 fe:01 29107634                   /home/nils/src/festool/coa-pim-service/lib/python3.6/site-packages/meinheld-0.6.1-py3.6-linux-x86_64.egg/meinheld/server.cpython-36m-x86_64-linux-gnu.so
7fdcd4bd5000-7fdcd5ee1000 rw-p 00000000 00:00 0 
7fdcd5ee1000-7fdcd5eeb000 r-xp 00000000 fe:01 6422899                    /lib/x86_64-linux-gnu/libnss_files-2.24.so
7fdcd5eeb000-7fdcd60eb000 ---p 0000a000 fe:01 6422899                    /lib/x86_64-linux-gnu/libnss_files-2.24.so
7fdcd60eb000-7fdcd60ec000 r--p 0000a000 fe:01 6422899                    /lib/x86_64-linux-gnu/libnss_files-2.24.so
7fdcd60ec000-7fdcd60ed000 rw-p 0000b000 fe:01 6422899                    /lib/x86_64-linux-gnu/libnss_files-2.24.so
7fdcd60ed000-7fdcd6234000 rw-p 00000000 00:00 0 
7fdcd6234000-7fdcd626c000 r-xp 00000000 fe:01 18491635                   /usr/lib/x86_64-linux-gnu/libmpdec.so.2.4.2
7fdcd626c000-7fdcd646b000 ---p 00038000 fe:01 18491635                   /usr/lib/x86_64-linux-gnu/libmpdec.so.2.4.2
7fdcd646b000-7fdcd646c000 r--p 00037000 fe:01 18491635                   /usr/lib/x86_64-linux-gnu/libmpdec.so.2.4.2
7fdcd646c000-7fdcd646d000 rw-p 00038000 fe:01 18491635                   /usr/lib/x86_64-linux-gnu/libmpdec.so.2.4.2
7fdcd646d000-7fdcd648d000 r-xp 00000000 fe:01 18757840                   /usr/lib/python3.6/lib-dynload/_decimal.cpython-36m-x86_64-linux-gnu.so
7fdcd648d000-7fdcd668c000 ---p 00020000 fe:01 18757840                   /usr/lib/python3.6/lib-dynload/_decimal.cpython-36m-x86_64-linux-gnu.so
7fdcd668c000-7fdcd668d000 r--p 0001f000 fe:01 18757840                   /usr/lib/python3.6/lib-dynload/_decimal.cpython-36m-x86_64-linux-gnu.so
7fdcd668d000-7fdcd6696000 rw-p 00020000 fe:01 18757840                   /usr/lib/python3.6/lib-dynload/_decimal.cpython-36m-x86_64-linux-gnu.so
7fdcd6696000-7fdcd66ac000 r-xp 00000000 fe:01 6422603                    /lib/x86_64-linux-gnu/libgcc_s.so.1
7fdcd66ac000-7fdcd68ab000 ---p 00016000 fe:01 6422603                    /lib/x86_64-linux-gnu/libgcc_s.so.1
7fdcd68ab000-7fdcd68ac000 r--p 00015000 fe:01 6422603                    /lib/x86_64-linux-gnu/libgcc_s.so.1
7fdcd68ac000-7fdcd68ad000 rw-p 00016000 fe:01 6422603                    /lib/x86_64-linux-gnu/libgcc_s.so.1
7fdcd68ad000-7fdcd6a1f000 r-xp 00000000 fe:01 18481948                   /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.22
7fdcd6a1f000-7fdcd6c1f000 ---p 00172000 fe:01 18481948                   /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.22
7fdcd6c1f000-7fdcd6c29000 r--p 00172000 fe:01 18481948                   /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.22
7fdcd6c29000-7fdcd6c2b000 rw-p 0017c000 fe:01 18481948                   /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.22
7fdcd6c2b000-7fdcd6c2f000 rw-p 00000000 00:00 0 
7fdcd6c2f000-7fdcd6c42000 r-xp 00000000 fe:01 6422643                    /lib/x86_64-linux-gnu/libgpg-error.so.0.21.0
7fdcd6c42000-7fdcd6e41000 ---p 00013000 fe:01 6422643                    /lib/x86_64-linux-gnu/libgpg-error.so.0.21.0
7fdcd6e41000-7fdcd6e42000 r--p 00012000 fe:01 6422643                    /lib/x86_64-linux-gnu/libgpg-error.so.0.21.0
7fdcd6e42000-7fdcd6e43000 rw-p 00013000 fe:01 6422643                    /lib/x86_64-linux-gnu/libgpg-error.so.0.21.0
7fdcd6e43000-7fdcd86bf000 r-xp 00000000 fe:01 18486226                   /usr/lib/x86_64-linux-gnu/libicudata.so.57.1
7fdcd86bf000-7fdcd88be000 ---p 0187c000 fe:01 18486226                   /usr/lib/x86_64-linux-gnu/libicudata.so.57.1
7fdcd88be000-7fdcd88bf000 r--p 0187b000 fe:01 18486226                   /usr/lib/x86_64-linux-gnu/libicudata.so.57.1
7fdcd88bf000-7fdcd88c0000 rw-p 0187c000 fe:01 18486226                   /usr/lib/x86_64-linux-gnu/libicudata.so.57.1
7fdcd88c0000-7fdcd8a54000 r-xp 00000000 fe:01 18488734                   /usr/lib/x86_64-linux-gnu/libicuuc.so.57.1
7fdcd8a54000-7fdcd8c53000 ---p 00194000 fe:01 18488734                   /usr/lib/x86_64-linux-gnu/libicuuc.so.57.1
7fdcd8c53000-7fdcd8c65000 r--p 00193000 fe:01 18488734                   /usr/lib/x86_64-linux-gnu/libicuuc.so.57.1
7fdcd8c65000-7fdcd8c66000 rw-p 001a5000 fe:01 18488734                   /usr/lib/x86_64-linux-gnu/libicuuc.so.57.1
7fdcd8c66000-7fdcd8c68000 rw-p 00000000 00:00 0 
7fdcd8c68000-7fdcd8ed3000 r-xp 00000000 fe:01 18486229                   /usr/lib/x86_64-linux-gnu/libicui18n.so.57.1
7fdcd8ed3000-7fdcd90d2000 ---p 0026b000 fe:01 18486229                   /usr/lib/x86_64-linux-gnu/libicui18n.so.57.1
7fdcd90d2000-7fdcd90df000 r--p 0026a000 fe:01 18486229                   /usr/lib/x86_64-linux-gnu/libicui18n.so.57.1
7fdcd90df000-7fdcd90e1000 rw-p 00277000 fe:01 18486229                   /usr/lib/x86_64-linux-gnu/libicui18n.so.57.1
7fdcd90e1000-7fdcd90e2000 rw-p 00000000 00:00 0 
7fdcd90e2000-7fdcd91e9000 r-xp 00000000 fe:01 6422553                    /lib/x86_64-linux-gnu/libgcrypt.so.20.1.6
7fdcd91e9000-7fdcd93e8000 ---p 00107000 fe:01 6422553                    /lib/x86_64-linux-gnu/libgcrypt.so.20.1.6
7fdcd93e8000-7fdcd93ea000 r--p 00106000 fe:01 6422553                    /lib/x86_64-linux-gnu/libgcrypt.so.20.1.6
7fdcd93ea000-7fdcd93f1000 rw-p 00108000 fe:01 6422553                    /lib/x86_64-linux-gnu/libgcrypt.so.20.1.6
7fdcd93f1000-7fdcd93f8000 r-xp 00000000 fe:01 6422916                    /lib/x86_64-linux-gnu/librt-2.24.so
7fdcd93f8000-7fdcd95f7000 ---p 00007000 fe:01 6422916                    /lib/x86_64-linux-gnu/librt-2.24.so
7fdcd95f7000-7fdcd95f8000 r--p 00006000 fe:01 6422916                    /lib/x86_64-linux-gnu/librt-2.24.so
7fdcd95f8000-7fdcd95f9000 rw-p 00007000 fe:01 6422916                    /lib/x86_64-linux-gnu/librt-2.24.so
7fdcd95f9000-7fdcd97a9000 r-xp 00000000 fe:01 18489412                   /usr/lib/x86_64-linux-gnu/libxml2.so.2.9.4
7fdcd97a9000-7fdcd99a9000 ---p 001b0000 fe:01 18489412                   /usr/lib/x86_64-linux-gnu/libxml2.so.2.9.4
7fdcd99a9000-7fdcd99b1000 r--p 001b0000 fe:01 18489412                   /usr/lib/x86_64-linux-gnu/libxml2.so.2.9.4
7fdcd99b1000-7fdcd99b3000 rw-p 001b8000 fe:01 18489412                   /usr/lib/x86_64-linux-gnu/libxml2.so.2.9.4
7fdcd99b3000-7fdcd99b4000 rw-p 00000000 00:00 0 
7fdcd99b4000-7fdcd99c8000 r-xp 00000000 fe:01 18485011                   /usr/lib/x86_64-linux-gnu/libexslt.so.0.8.17
7fdcd99c8000-7fdcd9bc8000 ---p 00014000 fe:01 18485011                   /usr/lib/x86_64-linux-gnu/libexslt.so.0.8.17
7fdcd9bc8000-7fdcd9bc9000 r--p 00014000 fe:01 18485011                   /usr/lib/x86_64-linux-gnu/libexslt.so.0.8.17
7fdcd9bc9000-7fdcd9bca000 rw-p 00015000 fe:01 18485011                   /usr/lib/x86_64-linux-gnu/libexslt.so.0.8.17
7fdcd9bca000-7fdcd9c07000 r-xp 00000000 fe:01 18485012                   /usr/lib/x86_64-linux-gnu/libxslt.so.1.1.29
7fdcd9c07000-7fdcd9e06000 ---p 0003d000 fe:01 18485012                   /usr/lib/x86_64-linux-gnu/libxslt.so.1.1.29
7fdcd9e06000-7fdcd9e08000 r--p 0003c000 fe:01 18485012                   /usr/lib/x86_64-linux-gnu/libxslt.so.1.1.29
7fdcd9e08000-7fdcd9e09000 rw-p 0003e000 fe:01 18485012                   /usr/lib/x86_64-linux-gnu/libxslt.so.1.1.29
7fdcd9e09000-7fdcd9ff0000 r-xp 00000000 fe:01 12868319                   /home/nils/src/festool/coa-pim-service/lib/python3.6/site-packages/lxml-3.8.0-py3.6-linux-x86_64.egg/lxml/etree.cpython-36m-x86_64-linux-gnu.so
7fdcd9ff0000-7fdcda1ef000 ---p 001e7000 fe:01 12868319                   /home/nils/src/festool/coa-pim-service/lib/python3.6/site-packages/lxml-3.8.0-py3.6-linux-x86_64.egg/lxml/etree.cpython-36m-x86_64-linux-gnu.so
7fdcda1ef000-7fdcda1f0000 r--p 001e6000 fe:01 12868319                   /home/nils/src/festool/coa-pim-service/lib/python3.6/site-packages/lxml-3.8.0-py3.6-linux-x86_64.egg/lxml/etree.cpython-36m-x86_64-linux-gnu.so
7fdcda1f0000-7fdcda220000 rw-p 001e7000 fe:01 12868319                   /home/nils/src/festool/coa-pim-service/lib/python3.6/site-packages/lxml-3.8.0-py3.6-linux-x86_64.egg/lxml/etree.cpython-36m-x86_64-linux-gnu.so
7fdcda220000-7fdcda227000 rw-p 00000000 00:00 0 
7fdcda227000-7fdcda229000 r-xp 00000000 fe:01 13635416                   /home/nils/src/festool/coa-pim-service/lib/python3.6/site-packages/cryptography-2.0.3-py3.6-linux-x86_64.egg/cryptography/hazmat/bindings/_padding.abi3.so
7fdcda229000-7fdcda428000 ---p 00002000 fe:01 13635416                   /home/nils/src/festool/coa-pim-service/lib/python3.6/site-packages/cryptography-2.0.3-py3.6-linux-x86_64.egg/cryptography/hazmat/bindings/_padding.abi3.so
7fdcda428000-7fdcda429000 r--p 00001000 fe:01 13635416                   /home/nils/src/festool/coa-pim-service/lib/python3.6/site-packages/cryptography-2.0.3-py3.6-linux-x86_64.egg/cryptography/hazmat/bindings/_padding.abi3.so
7fdcda429000-7fdcda42a000 rw-p 00002000 fe:01 13635416                   /home/nils/src/festool/coa-pim-service/lib/python3.6/site-packages/cryptography-2.0.3-py3.6-linux-x86_64.egg/cryptography/hazmat/bindings/_padding.abi3.so
7fdcda42a000-7fdcda7aa000 rw-p 00000000 00:00 0 
7fdcda7aa000-7fdcda7cd000 r-xp 00000000 fe:01 13896898                   /home/nils/src/festool/coa-pim-service/lib/python3.6/site-packages/cffi-1.10.0-py3.6-linux-x86_64.egg/_cffi_backend.cpython-36m-x86_64-linux-gnu.so
7fdcda7cd000-7fdcda9cd000 ---p 00023000 fe:01 13896898                   /home/nils/src/festool/coa-pim-service/lib/python3.6/site-packages/cffi-1.10.0-py3.6-linux-x86_64.egg/_cffi_backend.cpython-36m-x86_64-linux-gnu.so
7fdcda9cd000-7fdcda9ce000 r--p 00023000 fe:01 13896898                   /home/nils/src/festool/coa-pim-service/lib/python3.6/site-packages/cffi-1.10.0-py3.6-linux-x86_64.egg/_cffi_backend.cpython-36m-x86_64-linux-gnu.so
7fdcda9ce000-7fdcda9d3000 rw-p 00024000 fe:01 13896898                   /home/nils/src/festool/coa-pim-service/lib/python3.6/site-packages/cffi-1.10.0-py3.6-linux-x86_64.egg/_cffi_backend.cpython-36m-x86_64-linux-gnu.so
7fdcda9d3000-7fdcda9d6000 rw-p 00000000 00:00 0 
7fdcda9d6000-7fdcda9d7000 r-xp 00000000 fe:01 13635417                   /home/nils/src/festool/coa-pim-service/lib/python3.6/site-packages/cryptography-2.0.3-py3.6-linux-x86_64.egg/cryptography/hazmat/bindings/_constant_time.abi3.so
7fdcda9d7000-7fdcdabd7000 ---p 00001000 fe:01 13635417                   /home/nils/src/festool/coa-pim-service/lib/python3.6/site-packages/cryptography-2.0.3-py3.6-linux-x86_64.egg/cryptography/hazmat/bindings/_constant_time.abi3.so
7fdcdabd7000-7fdcdabd8000 r--p 00001000 fe:01 13635417                   /home/nils/src/festool/coa-pim-service/lib/python3.6/site-packages/cryptography-2.0.3-py3.6-linux-x86_64.egg/cryptography/hazmat/bindings/_constant_time.abi3.so
7fdcdabd8000-7fdcdabd9000 rw-p 00002000 fe:01 13635417                   /home/nils/src/festool/coa-pim-service/lib/python3.6/site-packages/cryptography-2.0.3-py3.6-linux-x86_64.egg/cryptography/hazmat/bindings/_constant_time.abi3.so
7fdcdabd9000-7fdcdac19000 rw-p 00000000 00:00 0 
7fdcdac3a000-7fdcdb03a000 rw-p 00000000 00:00 0 
7fdcdb03a000-7fdcdb03e000 r-xp 00000000 fe:01 6422544                    /lib/x86_64-linux-gnu/libuuid.so.1.3.0
7fdcdb03e000-7fdcdb23d000 ---p 00004000 fe:01 6422544                    /lib/x86_64-linux-gnu/libuuid.so.1.3.0
7fdcdb23d000-7fdcdb23e000 r--p 00003000 fe:01 6422544                    /lib/x86_64-linux-gnu/libuuid.so.1.3.0
7fdcdb23e000-7fdcdb23f000 rw-p 00004000 fe:01 6422544                    /lib/x86_64-linux-gnu/libuuid.so.1.3.0
7fdcdb23f000-7fdcdb246000 r-xp 00000000 fe:01 18490384                   /usr/lib/x86_64-linux-gnu/libffi.so.6.0.4
7fdcdb246000-7fdcdb446000 ---p 00007000 fe:01 18490384                   /usr/lib/x86_64-linux-gnu/libffi.so.6.0.4
7fdcdb446000-7fdcdb447000 r--p 00007000 fe:01 18490384                   /usr/lib/x86_64-linux-gnu/libffi.so.6.0.4
7fdcdb447000-7fdcdb448000 rw-p 00008000 fe:01 18490384                   /usr/lib/x86_64-linux-gnu/libffi.so.6.0.4
7fdcdb448000-7fdcdb463000 r-xp 00000000 fe:01 18757835                   /usr/lib/python3.6/lib-dynload/_ctypes.cpython-36m-x86_64-linux-gnu.so
7fdcdb463000-7fdcdb662000 ---p 0001b000 fe:01 18757835                   /usr/lib/python3.6/lib-dynload/_ctypes.cpython-36m-x86_64-linux-gnu.so
7fdcdb662000-7fdcdb663000 r--p 0001a000 fe:01 18757835                   /usr/lib/python3.6/lib-dynload/_ctypes.cpython-36m-x86_64-linux-gnu.so
7fdcdb663000-7fdcdb667000 rw-p 0001b000 fe:01 18757835                   /usr/lib/python3.6/lib-dynload/_ctypes.cpython-36m-x86_64-linux-gnu.so
7fdcdb667000-7fdcdb727000 rw-p 00000000 00:00 0 
7fdcdb727000-7fdcdb73f000 r-xp 00000000 fe:01 18753773                   /usr/lib/python3.6/lib-dynload/_ssl.cpython-36m-x86_64-linux-gnu.so
7fdcdb73f000-7fdcdb93e000 ---p 00018000 fe:01 18753773                   /usr/lib/python3.6/lib-dynload/_ssl.cpython-36m-x86_64-linux-gnu.so
7fdcdb93e000-7fdcdb93f000 r--p 00017000 fe:01 18753773                   /usr/lib/python3.6/lib-dynload/_ssl.cpython-36m-x86_64-linux-gnu.so
7fdcdb93f000-7fdcdb944000 rw-p 00018000 fe:01 18753773                   /usr/lib/python3.6/lib-dynload/_ssl.cpython-36m-x86_64-linux-gnu.so
7fdcdb944000-7fdcdba04000 rw-p 00000000 00:00 0 
7fdcdba37000-7fdcdba41000 r-xp 00000000 fe:01 18757841                   /usr/lib/python3.6/lib-dynload/_json.cpython-36m-x86_64-linux-gnu.so
7fdcdba41000-7fdcdbc40000 ---p 0000a000 fe:01 18757841                   /usr/lib/python3.6/lib-dynload/_json.cpython-36m-x86_64-linux-gnu.so
7fdcdbc40000-7fdcdbc41000 r--p 00009000 fe:01 18757841                   /usr/lib/python3.6/lib-dynload/_json.cpython-36m-x86_64-linux-gnu.so
7fdcdbc41000-7fdcdbc42000 rw-p 0000a000 fe:01 18757841                   /usr/lib/python3.6/lib-dynload/_json.cpython-36m-x86_64-linux-gnu.so
7fdcdbc42000-7fdcdbc43000 r-xp 00000000 fe:01 18753771                   /usr/lib/python3.6/lib-dynload/_opcode.cpython-36m-x86_64-linux-gnu.so
7fdcdbc43000-7fdcdbe42000 ---p 00001000 fe:01 18753771                   /usr/lib/python3.6/lib-dynload/_opcode.cpython-36m-x86_64-linux-gnu.so
7fdcdbe42000-7fdcdbe43000 r--p 00000000 fe:01 18753771                   /usr/lib/python3.6/lib-dynload/_opcode.cpython-36m-x86_64-linux-gnu.so
7fdcdbe43000-7fdcdbe44000 rw-p 00001000 fe:01 18753771                   /usr/lib/python3.6/lib-dynload/_opcode.cpython-36m-x86_64-linux-gnu.so
7fdcdbe44000-7fdcdbf84000 rw-p 00000000 00:00 0 
7fdcdbf95000-7fdcdc355000 rw-p 00000000 00:00 0 
7fdcdc355000-7fdcdc367000 r-xp 00000000 fe:01 18753772                   /usr/lib/python3.6/lib-dynload/_sha3.cpython-36m-x86_64-linux-gnu.so
7fdcdc367000-7fdcdc566000 ---p 00012000 fe:01 18753772                   /usr/lib/python3.6/lib-dynload/_sha3.cpython-36m-x86_64-linux-gnu.so
7fdcdc566000-7fdcdc567000 r--p 00011000 fe:01 18753772                   /usr/lib/python3.6/lib-dynload/_sha3.cpython-36m-x86_64-linux-gnu.so
7fdcdc567000-7fdcdc569000 rw-p 00012000 fe:01 18753772                   /usr/lib/python3.6/lib-dynload/_sha3.cpython-36m-x86_64-linux-gnu.so
7fdcdc569000-7fdcdc7ce000 r-xp 00000000 fe:01 18490325                   /usr/lib/x86_64-linux-gnu/libcrypto.so.1.1
7fdcdc7ce000-7fdcdc9cd000 ---p 00265000 fe:01 18490325                   /usr/lib/x86_64-linux-gnu/libcrypto.so.1.1
7fdcdc9cd000-7fdcdc9eb000 r--p 00264000 fe:01 18490325                   /usr/lib/x86_64-linux-gnu/libcrypto.so.1.1
7fdcdc9eb000-7fdcdc9f9000 rw-p 00282000 fe:01 18490325                   /usr/lib/x86_64-linux-gnu/libcrypto.so.1.1
7fdcdc9f9000-7fdcdc9fc000 rw-p 00000000 00:00 0 
7fdcdc9fc000-7fdcdca5e000 r-xp 00000000 fe:01 18490326                   /usr/lib/x86_64-linux-gnu/libssl.so.1.1
7fdcdca5e000-7fdcdcc5e000 ---p 00062000 fe:01 18490326                   /usr/lib/x86_64-linux-gnu/libssl.so.1.1
7fdcdcc5e000-7fdcdcc62000 r--p 00062000 fe:01 18490326                   /usr/lib/x86_64-linux-gnu/libssl.so.1.1
7fdcdcc62000-7fdcdcc68000 rw-p 00066000 fe:01 18490326                   /usr/lib/x86_64-linux-gnu/libssl.so.1.1
7fdcdcc68000-7fdcdcc6d000 r-xp 00000000 fe:01 18753770                   /usr/lib/python3.6/lib-dynload/_hashlib.cpython-36m-x86_64-linux-gnu.so
7fdcdcc6d000-7fdcdce6c000 ---p 00005000 fe:01 18753770                   /usr/lib/python3.6/lib-dynload/_hashlib.cpython-36m-x86_64-linux-gnu.so
7fdcdce6c000-7fdcdce6d000 r--p 00004000 fe:01 18753770                   /usr/lib/python3.6/lib-dynload/_hashlib.cpython-36m-x86_64-linux-gnu.so
7fdcdce6d000-7fdcdce6e000 rw-p 00005000 fe:01 18753770                   /usr/lib/python3.6/lib-dynload/_hashlib.cpython-36m-x86_64-linux-gnu.so
7fdcdce6e000-7fdcdcf2e000 rw-p 00000000 00:00 0 
7fdcdcf2e000-7fdcdcf53000 r-xp 00000000 fe:01 6422594                    /lib/x86_64-linux-gnu/liblzma.so.5.2.2
7fdcdcf53000-7fdcdd152000 ---p 00025000 fe:01 6422594                    /lib/x86_64-linux-gnu/liblzma.so.5.2.2
7fdcdd152000-7fdcdd153000 r--p 00024000 fe:01 6422594                    /lib/x86_64-linux-gnu/liblzma.so.5.2.2
7fdcdd153000-7fdcdd154000 rw-p 00025000 fe:01 6422594                    /lib/x86_64-linux-gnu/liblzma.so.5.2.2
7fdcdd154000-7fdcdd15b000 r-xp 00000000 fe:01 18757843                   /usr/lib/python3.6/lib-dynload/_lzma.cpython-36m-x86_64-linux-gnu.so
7fdcdd15b000-7fdcdd35a000 ---p 00007000 fe:01 18757843                   /usr/lib/python3.6/lib-dynload/_lzma.cpython-36m-x86_64-linux-gnu.so
7fdcdd35a000-7fdcdd35b000 r--p 00006000 fe:01 18757843                   /usr/lib/python3.6/lib-dynload/_lzma.cpython-36m-x86_64-linux-gnu.so
7fdcdd35b000-7fdcdd35d000 rw-p 00007000 fe:01 18757843                   /usr/lib/python3.6/lib-dynload/_lzma.cpython-36m-x86_64-linux-gnu.so
7fdcdd35d000-7fdcdd36c000 r-xp 00000000 fe:01 6422712                    /lib/x86_64-linux-gnu/libbz2.so.1.0.4
7fdcdd36c000-7fdcdd56b000 ---p 0000f000 fe:01 6422712                    /lib/x86_64-linux-gnu/libbz2.so.1.0.4
7fdcdd56b000-7fdcdd56c000 r--p 0000e000 fe:01 6422712                    /lib/x86_64-linux-gnu/libbz2.so.1.0.4
7fdcdd56c000-7fdcdd56d000 rw-p 0000f000 fe:01 6422712                    /lib/x86_64-linux-gnu/libbz2.so.1.0.4
7fdcdd56d000-7fdcdd571000 r-xp 00000000 fe:01 18757826                   /usr/lib/python3.6/lib-dynload/_bz2.cpython-36m-x86_64-linux-gnu.so
7fdcdd571000-7fdcdd770000 ---p 00004000 fe:01 18757826                   /usr/lib/python3.6/lib-dynload/_bz2.cpython-36m-x86_64-linux-gnu.so
7fdcdd770000-7fdcdd771000 r--p 00003000 fe:01 18757826                   /usr/lib/python3.6/lib-dynload/_bz2.cpython-36m-x86_64-linux-gnu.so
7fdcdd771000-7fdcdd772000 rw-p 00004000 fe:01 18757826                   /usr/lib/python3.6/lib-dynload/_bz2.cpython-36m-x86_64-linux-gnu.so
7fdcdd772000-7fdcdd7f2000 rw-p 00000000 00:00 0 
7fdcdd7fb000-7fdcddb3b000 rw-p 00000000 00:00 0 
7fdcddb3b000-7fdcddcd0000 r-xp 00000000 fe:01 6422647                    /lib/x86_64-linux-gnu/libc-2.24.so
7fdcddcd0000-7fdcdded0000 ---p 00195000 fe:01 6422647                    /lib/x86_64-linux-gnu/libc-2.24.so
7fdcdded0000-7fdcdded4000 r--p 00195000 fe:01 6422647                    /lib/x86_64-linux-gnu/libc-2.24.so
7fdcdded4000-7fdcdded6000 rw-p 00199000 fe:01 6422647                    /lib/x86_64-linux-gnu/libc-2.24.so
7fdcdded6000-7fdcddeda000 rw-p 00000000 00:00 0 
7fdcddeda000-7fdcddfdd000 r-xp 00000000 fe:01 6422717                    /lib/x86_64-linux-gnu/libm-2.24.so
7fdcddfdd000-7fdcde1dc000 ---p 00103000 fe:01 6422717                    /lib/x86_64-linux-gnu/libm-2.24.so
7fdcde1dc000-7fdcde1dd000 r--p 00102000 fe:01 6422717                    /lib/x86_64-linux-gnu/libm-2.24.so
7fdcde1dd000-7fdcde1de000 rw-p 00103000 fe:01 6422717                    /lib/x86_64-linux-gnu/libm-2.24.so
7fdcde1de000-7fdcde1f7000 r-xp 00000000 fe:01 6425773                    /lib/x86_64-linux-gnu/libz.so.1.2.8
7fdcde1f7000-7fdcde3f6000 ---p 00019000 fe:01 6425773                    /lib/x86_64-linux-gnu/libz.so.1.2.8
7fdcde3f6000-7fdcde3f7000 r--p 00018000 fe:01 6425773                    /lib/x86_64-linux-gnu/libz.so.1.2.8
7fdcde3f7000-7fdcde3f8000 rw-p 00019000 fe:01 6425773                    /lib/x86_64-linux-gnu/libz.so.1.2.8
7fdcde3f8000-7fdcde41f000 r-xp 00000000 fe:01 6423096                    /lib/x86_64-linux-gnu/libexpat.so.1.6.2
7fdcde41f000-7fdcde61f000 ---p 00027000 fe:01 6423096                    /lib/x86_64-linux-gnu/libexpat.so.1.6.2
7fdcde61f000-7fdcde621000 r--p 00027000 fe:01 6423096                    /lib/x86_64-linux-gnu/libexpat.so.1.6.2
7fdcde621000-7fdcde622000 rw-p 00029000 fe:01 6423096                    /lib/x86_64-linux-gnu/libexpat.so.1.6.2
7fdcde622000-7fdcde624000 r-xp 00000000 fe:01 6422935                    /lib/x86_64-linux-gnu/libutil-2.24.so
7fdcde624000-7fdcde823000 ---p 00002000 fe:01 6422935                    /lib/x86_64-linux-gnu/libutil-2.24.so
7fdcde823000-7fdcde824000 r--p 00001000 fe:01 6422935                    /lib/x86_64-linux-gnu/libutil-2.24.so
7fdcde824000-7fdcde825000 rw-p 00002000 fe:01 6422935                    /lib/x86_64-linux-gnu/libutil-2.24.so
7fdcde825000-7fdcde828000 r-xp 00000000 fe:01 6422674                    /lib/x86_64-linux-gnu/libdl-2.24.so
7fdcde828000-7fdcdea27000 ---p 00003000 fe:01 6422674                    /lib/x86_64-linux-gnu/libdl-2.24.so
7fdcdea27000-7fdcdea28000 r--p 00002000 fe:01 6422674                    /lib/x86_64-linux-gnu/libdl-2.24.so
7fdcdea28000-7fdcdea29000 rw-p 00003000 fe:01 6422674                    /lib/x86_64-linux-gnu/libdl-2.24.so
7fdcdea29000-7fdcdea41000 r-xp 00000000 fe:01 6422911                    /lib/x86_64-linux-gnu/libpthread-2.24.so
7fdcdea41000-7fdcdec40000 ---p 00018000 fe:01 6422911                    /lib/x86_64-linux-gnu/libpthread-2.24.so
7fdcdec40000-7fdcdec41000 r--p 00017000 fe:01 6422911                    /lib/x86_64-linux-gnu/libpthread-2.24.so
7fdcdec41000-7fdcdec42000 rw-p 00018000 fe:01 6422911                    /lib/x86_64-linux-gnu/libpthread-2.24.so
7fdcdec42000-7fdcdec46000 rw-p 00000000 00:00 0 
7fdcdec46000-7fdcdec69000 r-xp 00000000 fe:01 6422587                    /lib/x86_64-linux-gnu/ld-2.24.so
7fdcdec9d000-7fdcdee38000 r--p 00000000 fe:01 18502511                   /usr/lib/locale/locale-archive
7fdcdee38000-7fdcdee3c000 rw-p 00000000 00:00 0 
7fdcdee5d000-7fdcdee5e000 rw-p 00000000 00:00 0 
7fdcdee5e000-7fdcdee5f000 rwxp 00000000 00:00 0 
7fdcdee5f000-7fdcdee66000 r--s 00000000 fe:01 18513329                   /usr/lib/x86_64-linux-gnu/gconv/gconv-modules.cache
7fdcdee66000-7fdcdee69000 rw-p 00000000 00:00 0 
7fdcdee69000-7fdcdee6a000 r--p 00023000 fe:01 6422587                    /lib/x86_64-linux-gnu/ld-2.24.so
7fdcdee6a000-7fdcdee6b000 rw-p 00024000 fe:01 6422587                    /lib/x86_64-linux-gnu/ld-2.24.so
7fdcdee6b000-7fdcdee6c000 rw-p 00000000 00:00 0 
7ffd8f160000-7ffd8f182000 rw-p 00000000 00:00 0                          [stack]
7ffd8f1fb000-7ffd8f1fd000 r--p 00000000 00:00 0                          [vvar]
7ffd8f1fd000-7ffd8f1ff000 r-xp 00000000 00:00 0                          [vdso]
ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0                  [vsyscall]
Abgebrochen
(coa-pim-service) nils at unknown-device:~/src/festool/coa-pim-service$

----------
messages: 301052
nosy: zopyx
priority: normal
severity: normal
status: open
title: Frequent *** stack smashing detected *** with Python 3.6.2/meinheld WSGI server
type: crash
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31316>
_______________________________________

From report at bugs.python.org  Thu Aug 31 16:18:11 2017
From: report at bugs.python.org (STINNER Victor)
Date: Thu, 31 Aug 2017 20:18:11 +0000
Subject: [New-bugs-announce] [issue31317] Memory leak in dict with shared
 keys
Message-ID: <1504210691.95.0.17825455702.issue31317@psf.upfronthosting.co.za>


New submission from STINNER Victor:

Copy attached test_leak.py to Lib/test/ and run:

haypo at selma$ ./python -m test -R 3:30 test_leak

Run tests in parallel using 1 child processes
0:00:00 load avg: 0.67 [1/1/1] test_leak failed
beginning 33 repetitions
123456789012345678901234567890123
.................................
test_leak leaked [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1] memory blocks, sum=30

1 test failed:
    test_leak

Total duration: 682 ms
Tests result: FAILURE


It seems like Thread2 constructor creates a dictionary on "self.hashvalue = (...)". It's a dictionary sharing keys, but it seems like the dict table memory is not released whne the bug occurs.


Sadly, I'm only able to reproduce the bug on a 32-bit build. Since the bug depends on exact hash values, maybe the bug exists in 64-bit builds, but is less likely?


I'm not 100% sure if the bug is related to dict or set.


The bug comes from bpo-31217.

----------
messages: 301058
nosy: haypo, inada.naoki, rhettinger, xiang.zhang
priority: normal
severity: normal
status: open
title: Memory leak in dict with shared keys
type: resource usage
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31317>
_______________________________________