From report at bugs.python.org  Sun Apr  1 01:26:13 2018
From: report at bugs.python.org (INADA Naoki)
Date: Sun, 01 Apr 2018 05:26:13 +0000
Subject: [New-bugs-announce] [issue33195] PyArg_Parse* should deprecate 'u'
 and 'z' family.
Message-ID: <1522560373.89.0.467229070634.issue33195@psf.upfronthosting.co.za>


New submission from INADA Naoki <songofacandy at gmail.com>:

https://docs.python.org/3/c-api/arg.html

Py_UNICODE usage is not deprecated in this document.
Before thinking about raising warning, deprecate it in document first.

But... from when?  Can I backport the "deprecated" doc to 3.6 or 3.5?

----------
assignee: docs at python
components: Documentation
messages: 314761
nosy: docs at python, inada.naoki
priority: normal
severity: normal
status: open
title: PyArg_Parse* should deprecate 'u' and 'z' family.
versions: Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33195>
_______________________________________

From report at bugs.python.org  Sun Apr  1 01:58:11 2018
From: report at bugs.python.org (Arcadiy Ivanov)
Date: Sun, 01 Apr 2018 05:58:11 +0000
Subject: [New-bugs-announce] [issue33196] SEGV in
 mp.synchronize.Lock.__repr__ in spawn'ed proc if ctx mismatched
Message-ID: <1522562291.13.0.467229070634.issue33196@psf.upfronthosting.co.za>


New submission from Arcadiy Ivanov <arcadiy at ivanov.biz>:

While working on GH gevent/gevent#993 I've encountered a stall trying to read from an mp.Queue passed to mp.Process's target as an argument. Trying to print out the lock state in child process I encountered as SEGV in Lock's __repr__. I originally thought it was due to gevent/greenlet stack magic, but it wasn't. 

This happens when `fork` context Queue (default) is used with `spawn` context Process (obvious stupidity on my part, alas shouldn't crash).

Python 3.6.4 from PyEnv
Fedora 27

```
$ python test_lock_sigsegv.py 
Parent r_q: <Lock(owner=None)>, <Lock(owner=None)>, <BoundedSemaphore(value=2147483647, maxvalue=2147483647)>
-11
```

```
Program terminated with signal SIGSEGV, Segmentation fault.
#0  __new_sem_getvalue (sem=0x7fc877f54000, sval=sval at entry=0x7fffb130db9c) at sem_getvalue.c:38
38        *sval = atomic_load_relaxed (&isem->data) & SEM_VALUE_MASK;
...
#0  __new_sem_getvalue (sem=0x7fc877f54000, sval=sval at entry=0x7fffb130db9c) at sem_getvalue.c:38
#1  0x00007f1116aeb202 in semlock_getvalue (self=<optimized out>) at /tmp/python-build.20171219170845.6548/Python-3.6.4/Modules/_multiprocessing/semaphore.c:531
```

At a minimum I think there should be a check trying to reduce arguments via incompatible context's process to prevent a SEGV.

Test attached.

----------
components: Library (Lib)
files: test_lock_sigsegv.py
messages: 314762
nosy: arcivanov
priority: normal
severity: normal
status: open
title: SEGV in mp.synchronize.Lock.__repr__ in spawn'ed proc if ctx mismatched
type: crash
versions: Python 3.6
Added file: https://bugs.python.org/file47510/test_lock_sigsegv.py

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33196>
_______________________________________

From report at bugs.python.org  Sun Apr  1 03:14:32 2018
From: report at bugs.python.org (Antony Lee)
Date: Sun, 01 Apr 2018 07:14:32 +0000
Subject: [New-bugs-announce] [issue33197] Confusing error message when
 constructing invalid inspect.Parameters
Message-ID: <1522566872.11.0.467229070634.issue33197@psf.upfronthosting.co.za>


New submission from Antony Lee <anntzer.lee at gmail.com>:

Python 3.6.4

    In [15]: inspect.Parameter("foo", kind=inspect.Parameter.VAR_KEYWORD, default=42)
    ---------------------------------------------------------------------------
    ValueError                                Traceback (most recent call last)
    <ipython-input-15-7ed8c4fd15f3> in <module>()
    ----> 1 inspect.Parameter("foo", kind=inspect.Parameter.VAR_KEYWORD, default=42)

    /usr/lib/python3.6/inspect.py in __init__(self, name, kind, default, annotation)
    2442             if kind in (_VAR_POSITIONAL, _VAR_KEYWORD):
    2443                 msg = '{} parameters cannot have default values'.format(kind)
    -> 2444                 raise ValueError(msg)
    2445         self._default = default
    2446         self._annotation = annotation

    ValueError: 4 parameters cannot have default values

Note the "interesting" error message that starts with "4 parameters ..." (yes, I guess that inspect.Parameter == 4 internally...).

Probably just a matter of making the error f'{kind.name} parameters ...'.

----------
components: Library (Lib)
messages: 314764
nosy: Antony.Lee
priority: normal
severity: normal
status: open
title: Confusing error message when constructing invalid inspect.Parameters
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33197>
_______________________________________

From report at bugs.python.org  Sun Apr  1 04:38:58 2018
From: report at bugs.python.org (Florian Schulze)
Date: Sun, 01 Apr 2018 08:38:58 +0000
Subject: [New-bugs-announce] [issue33198] Build on Linux with
 --enable-optimizations fails
Message-ID: <1522571938.21.0.467229070634.issue33198@psf.upfronthosting.co.za>


New submission from Florian Schulze <florian.schulze at gmx.net>:

While the tests run there is a segfault:
https://travis-ci.org/collective/buildout.python/jobs/360190312#L2529
Started with b2 and still happens in b3, b1 built fine.

----------
components: Build, Tests
messages: 314767
nosy: fschulze
priority: normal
severity: normal
status: open
title: Build on Linux with --enable-optimizations fails
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33198>
_______________________________________

From report at bugs.python.org  Sun Apr  1 04:56:59 2018
From: report at bugs.python.org (pdox)
Date: Sun, 01 Apr 2018 08:56:59 +0000
Subject: [New-bugs-announce] [issue33199] PyDict_Copy() can leave
 'ma_version_tag' uninitialized
Message-ID: <1522573019.92.0.467229070634.issue33199@psf.upfronthosting.co.za>


New submission from pdox <pdox at alum.mit.edu>:

PyDict_Copy leaves 'ma_version_tag' uninitialized when the dictionary being copied has a split table.

----------
components: Interpreter Core
messages: 314768
nosy: pdox
priority: normal
severity: normal
status: open
title: PyDict_Copy() can leave 'ma_version_tag' uninitialized
type: behavior
versions: Python 3.4, Python 3.5, Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33199>
_______________________________________

From report at bugs.python.org  Sun Apr  1 06:29:26 2018
From: report at bugs.python.org (Serhiy Storchaka)
Date: Sun, 01 Apr 2018 10:29:26 +0000
Subject: [New-bugs-announce] [issue33200] Optimize the empty set "literal"
Message-ID: <1522578566.75.0.467229070634.issue33200@psf.upfronthosting.co.za>


New submission from Serhiy Storchaka <storchaka+cpython at gmail.com>:

The following PR optimizes bytecode for the empty set "literal": {*()}.

Currently it is compiled to

    LOAD_CONST               0 (())
    BUILD_SET_UNPACK         1

It will optimized to

    BUILD_SET                0

$ ./python -m perf timeit --duplicate 1000 '{*()}'

Unpatched:  Mean +- std dev: 68.6 ns +- 1.1 ns
Patched:    Mean +- std dev: 31.8 ns +- 2.8 ns

----------
components: Interpreter Core
messages: 314769
nosy: benjamin.peterson, brett.cannon, ncoghlan, serhiy.storchaka, yselivanov
priority: normal
severity: normal
status: open
title: Optimize the empty set "literal"
type: performance
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33200>
_______________________________________

From report at bugs.python.org  Sun Apr  1 15:03:03 2018
From: report at bugs.python.org (Antoine Pitrou)
Date: Sun, 01 Apr 2018 19:03:03 +0000
Subject: [New-bugs-announce] [issue33201] Modernize "Extension types"
 documentation
Message-ID: <1522609383.56.0.467229070634.issue33201@psf.upfronthosting.co.za>


New submission from Antoine Pitrou <pitrou at free.fr>:

The "Defining New Types" in the C extension docs uses outdated idioms (e.g. pre-C99 struct initialization) and could do with a good refresh.

----------
assignee: docs at python
components: Documentation
messages: 314782
nosy: docs at python, pitrou
priority: normal
severity: normal
stage: needs patch
status: open
title: Modernize "Extension types" documentation
type: behavior
versions: Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33201>
_______________________________________

From report at bugs.python.org  Sun Apr  1 15:21:50 2018
From: report at bugs.python.org (=?utf-8?q?Andr=C3=A9s_Delfino?=)
Date: Sun, 01 Apr 2018 19:21:50 +0000
Subject: [New-bugs-announce] [issue33202] os.walk mentions os.listdir
 instead of os.scandir
Message-ID: <1522610510.25.0.467229070634.issue33202@psf.upfronthosting.co.za>


New submission from Andr?s Delfino <adelfino at gmail.com>:

Documentation states that, for walk, "errors from the listdir() call are ignored". That's no longer the case since 3.5. Change mention to listdir() to scandir().

----------
assignee: docs at python
components: Documentation
messages: 314786
nosy: adelfino, docs at python
priority: normal
pull_requests: 6049
severity: normal
status: open
title: os.walk mentions os.listdir instead of os.scandir
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33202>
_______________________________________

From report at bugs.python.org  Sun Apr  1 16:04:10 2018
From: report at bugs.python.org (Wolfgang Maier)
Date: Sun, 01 Apr 2018 20:04:10 +0000
Subject: [New-bugs-announce] [issue33203] random.choice: raise IndexError on
 empty sequence even when not using getrandbits internally
Message-ID: <1522613050.8.0.467229070634.issue33203@psf.upfronthosting.co.za>


New submission from Wolfgang Maier <wolfgang.maier at biologie.uni-freiburg.de>:

from https://docs.python.org/3/library/random.html#random.choice:

Return a random element from the non-empty sequence seq. If seq is empty, raises IndexError.

Indeed:
>>> import random
>>> random.choice([])
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/wolma/cpython/random.py", line 259, in choice
    raise IndexError('Cannot choose from an empty sequence') from None
IndexError: Cannot choose from an empty sequence

but when not using getrandbits internally:
>>> class MyRandom(random.Random):
...     def random(self):
...         return super().random()
... 
>>> my_random=MyRandom()
>>> my_random.choice([])
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/wolma/cpython/random.py", line 257, in choice
    i = self._randbelow(len(seq))
  File "/home/wolma/cpython/random.py", line 245, in _randbelow
    rem = maxsize % n
ZeroDivisionError: integer division or modulo by zero

This is because the ValueError that random.choice tries to catch gets raised only in the getrandbits-dependent branch of Random._randbelow, but not in the branch using only Random.random (even though Random._randbelow suggests uniform behaviour.

----------
components: Library (Lib)
messages: 314787
nosy: rhettinger, serhiy.storchaka, wolma
priority: normal
severity: normal
status: open
title: random.choice: raise IndexError on empty sequence even when not using getrandbits internally
type: behavior
versions: Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33203>
_______________________________________

From report at bugs.python.org  Sun Apr  1 23:37:29 2018
From: report at bugs.python.org (Terry J. Reedy)
Date: Mon, 02 Apr 2018 03:37:29 +0000
Subject: [New-bugs-announce] [issue33204] IDLE: remove \b from colorizer
 string prefix
Message-ID: <1522640249.89.0.467229070634.issue33204@psf.upfronthosting.co.za>


New submission from Terry J. Reedy <tjreedy at udel.edu>:

Lib\idlelib\colorizer.py, line 25, is:
  stringprefix = r"(?i:\br|u|f|fr|rf|b|br|rb)?"

The r prefix, but only the r prefix, must be preceded by a non-word character.

On pydev thread "IDLE colorizer", MRAB noted: "The \b will apply only to the first choice. Shouldn't it be more like:
  stringprefix = r"(?:\b(?i:r|u|f|fr|rf|b|br|rb))?"

I think the \b should be removed instead.  Currently uf'a' gets f'a' colored, while ur'a' only gets  'a' colored.  (Tim Peters pointed out the difference with a different example.) I prefer coloring the maximal legal string rather than the minimum.    I think the contrast between two chars legal by themselves, but differently colored when put together, makes the bug more obvious.

----------
assignee: terry.reedy
components: IDLE
messages: 314793
nosy: terry.reedy
priority: normal
severity: normal
stage: test needed
status: open
title: IDLE: remove \b from colorizer string prefix
type: behavior
versions: Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33204>
_______________________________________

From report at bugs.python.org  Mon Apr  2 07:55:51 2018
From: report at bugs.python.org (INADA Naoki)
Date: Mon, 02 Apr 2018 11:55:51 +0000
Subject: [New-bugs-announce] [issue33205] GROWTH_RATE prevents dict shrinking
Message-ID: <1522670151.04.0.467229070634.issue33205@psf.upfronthosting.co.za>


New submission from INADA Naoki <songofacandy at gmail.com>:

GROWTH_RATE is changed from (used*2) to (used*2 + dk_size/2) in #17563, at Python 3.4.
It was for avoid resizing dict for massive del/insert use case, by increasing possibility of overwriting DUMMY entry.

>From Python 3.6, there are no DUMMY entry.  While there are dummy keys, we resize (repack) when entries are full.
So there are no benefit from "possibility of overwriting dummy entry".

(Of course, there are still benefit from slow repacking rate for new dict.
So I don't propose to change it back to (used*2), but (used*3).)


This GROWTH_RATE prevents dict is shrinked in insertion_resize().

For example, consider this dict:

>>> d = dict.fromkeys(range(10900))
>>> len(d)
10900
>>> sys.getsizeof(d)
295008
>>> for i in range(10900):
...     del d[i]
...
>>> len(d)
0
>>> sys.getsizeof(d)
295008

`del d[i]` doesn't shrink the dict.  This is another issue (#32623).

Current dk_size is 16384 and entries length is dk_size * 2 // 3 = 10922.
So dictresize will called when next 923 entries are added.

New dk_size is round_up_to_power_of_two(922 + 16384/2) = 16384.
So dict is not shrinked!

>>> for i in range(923):
...     d[i] = i
...
>>> sys.getsizeof(d)
295008

round_up_to_power_of_two(used + dk_size/2) means dict is shrinked only when used == 0.

I propose changing GROWTH_RATE again to `used*3` from `used*2 + dk_size/2`

In case of dict growing without deletion, dk_size is doubled for each resize as current behavior.
When there are deletion, dk_size is growing aggressively than Python 3.3 (used*2 -> used*3).  And it allows dict shrinking after massive deletions.

>>> import sys
>>> d = dict.fromkeys(range(10900))
>>> sys.getsizeof(d)
295008
>>> for i in range(10900):
...     del d[i]
...
>>> len(d)
0
>>> sys.getsizeof(d)
295008
>>> for i in range(923):
...     d[i] = i
...
>>> sys.getsizeof(d)
36968


I want to backport this change to Python 3.7.  While it's beta3, "dict can't be shrinked unless empty" behavior looks resource usage bug to me.

----------
components: Interpreter Core
messages: 314806
nosy: Mark.Shannon, Yury.Selivanov, eric.snow, inada.naoki, rhettinger, vstinner
priority: normal
severity: normal
status: open
title: GROWTH_RATE prevents dict shrinking
type: resource usage
versions: Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33205>
_______________________________________

From report at bugs.python.org  Mon Apr  2 13:18:03 2018
From: report at bugs.python.org (Vitaly Kruglikov)
Date: Mon, 02 Apr 2018 17:18:03 +0000
Subject: [New-bugs-announce] [issue33206] Windows inet_pton(socket.AF_INET6,
 'localhost') raises ValueError instead of socket.error
Message-ID: <1522689483.28.0.467229070634.issue33206@psf.upfronthosting.co.za>


New submission from Vitaly Kruglikov <vitaly.krug at gmail.com>:

I am seeing this with `socket.AF_INET6` on Windows running python 2.7.14
```
[00:02:33]   File "C:\projects\pika\pika\adapters\host_tcp_connector.py", line 153, in _check_already_resolved
[00:02:33]     socket.inet_pton(socket.AF_INET6, 'localhost')
[00:02:33]   File "C:\Python27\lib\site-packages\twisted\python\compat.py", line 68, in inet_pton
[00:02:33]     raise ValueError("Illegal characters: %r" % (''.join(x),))
[00:02:33] ValueError: Illegal characters: 't'
```

With `socket.AF_INET`, `socket.inet_pton(socket.AF_INET, 'localhost')` raises `socket.error` as expected.

For comparison, with Python 2.7.10 running on OS X, both AF_INET and AF_INET6 raise `socket.error`.

----------
components: Library (Lib), Windows
messages: 314814
nosy: paul.moore, steve.dower, tim.golden, vitaly.krug, zach.ware
priority: normal
severity: normal
status: open
title: Windows inet_pton(socket.AF_INET6, 'localhost') raises ValueError instead of socket.error
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33206>
_______________________________________

From report at bugs.python.org  Mon Apr  2 18:01:55 2018
From: report at bugs.python.org (Will T)
Date: Mon, 02 Apr 2018 22:01:55 +0000
Subject: [New-bugs-announce] [issue33207] typing.Generic does not correctly
 call super().__init_subclass__
Message-ID: <1522706515.12.0.467229070634.issue33207@psf.upfronthosting.co.za>


New submission from Will T <timwilloney at gmail.com>:

Per the docs ( https://docs.python.org/3/reference/datamodel.html#object.__init_subclass__ ) this should be chain-calling super but currently doesn't, and thus breaks base classes listed after it which depend on this functionality. Attached a test for a real-world usecase of mine. Originally noted in https://bugs.python.org/issue32162? but on a new ticket as requested. Thanks :)

----------
components: Library (Lib)
files: generic_init_subclass.py
messages: 314828
nosy: wrmsr
priority: normal
severity: normal
status: open
title: typing.Generic does not correctly call super().__init_subclass__
type: behavior
versions: Python 3.7
Added file: https://bugs.python.org/file47513/generic_init_subclass.py

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33207>
_______________________________________

From report at bugs.python.org  Mon Apr  2 18:20:41 2018
From: report at bugs.python.org (Lukasz)
Date: Mon, 02 Apr 2018 22:20:41 +0000
Subject: [New-bugs-announce] [issue33208] Lib2to3  grammar.txt error
Message-ID: <1522707641.37.0.467229070634.issue33208@psf.upfronthosting.co.za>


New submission from Lukasz <Lswierzewski1 at gmail.com>:

Hello I have problem with script
Im am totaly beginner in python
I'm using Qpython ( python 2.7 on android)

So  this  is  my error:
/data/user/0/org.qpython.qpy/files/bin/qpython-android5.sh "/storage/emulated/0/qpython/scripts/InstaPy-master/quickstart.py" && exit
ipts/InstaPy-master/quickstart.py" && exit     <
Traceback (most recent call last):
  File "/storage/emulated/0/qpython/scripts/InstaPy-master/quickstart.py", line 7, in <module>
    from instapy import InstaPy
  File "/storage/emulated/0/qpython/scripts/InstaPy-master/instapy/__init__.py", line 2, in <module>
    from .instapy import InstaPy
  File "/storage/emulated/0/qpython/scripts/InstaPy-master/instapy/instapy.py", line 22, in <module>
    from .clarifai_util import check_image
  File "/storage/emulated/0/qpython/scripts/InstaPy-master/instapy/clarifai_util.py", line 3, in <module>
    from clarifai.rest import ClarifaiApp, Image as ClImage
  File "/storage/emulated/0/qpython/lib/python2.7/site-packages/clarifai/rest/__init__.py", line 3, in <module>
    from .client import ApiClient, ApiError, UserError, TokenError
  File "/storage/emulated/0/qpython/lib/python2.7/site-packages/clarifai/rest/client.py", line 21, in <module>
    from past.builtins import basestring
  File "/data/user/0/org.qpython.qpy/files/lib/python2.7/site-packages/future-0.16.0-py2.7.egg/past/__init__.py", line 88, in <module>
    from past.translation import install_hooks as autotranslate
  File "/data/user/0/org.qpython.qpy/files/lib/python2.7/site-packages/future-0.16.0-py2.7.egg/past/translation/__init__.py", line 42, in <module>
    from lib2to3.refactor import RefactoringTool
  File "/data/user/0/org.qpython.qpy/files/lib/python27.zip/lib2to3/refactor.py", line 27, in <module>
  File "/data/user/0/org.qpython.qpy/files/lib/python27.zip/lib2to3/fixer_util.py", line 9, in <module>
  File "/data/user/0/org.qpython.qpy/files/lib/python27.zip/lib2to3/pygram.py", line 32, in <module>
  File "/data/user/0/org.qpython.qpy/files/lib/python27.zip/lib2to3/pgen2/driver.py", line 121, in load_grammar
  File "/data/user/0/org.qpython.qpy/files/lib/python27.zip/lib2to3/pgen2/pgen.py", line 385, in generate_grammar
  File "/data/user/0/org.qpython.qpy/files/lib/python27.zip/lib2to3/pgen2/pgen.py", line 15, in __init__
IOError: [Errno 20] Not a directory: '/data/user/0/org.qpython.qpy/files/lib/python27.zip/lib2to3/Grammar.txt'
1|dream2lte:/ $

----------
messages: 314830
nosy: Lukasz00500
priority: normal
severity: normal
status: open
title: Lib2to3  grammar.txt error

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33208>
_______________________________________

From report at bugs.python.org  Mon Apr  2 21:08:23 2018
From: report at bugs.python.org (Serhiy Storchaka)
Date: Tue, 03 Apr 2018 01:08:23 +0000
Subject: [New-bugs-announce] [issue33209] Repeated Pickler.dump() doesn't
 reset the state in the C implementation of pickle
Message-ID: <1522717703.98.0.467229070634.issue33209@psf.upfronthosting.co.za>


New submission from Serhiy Storchaka <storchaka+cpython at gmail.com>:

test_clear_pickler_memo tested only the default protocol and fails when bump the default protocol to 4. For some reasons C implementation writes FRAME before PROTO when call Pickler.dump() repeatedly.

Python implementation:
b'\x80\x04\x95\x13\x00\x00\x00\x00\x00\x00\x00]\x94(\x8c\x07abcdefg\x94h\x01K,e.'
    0: \x80 PROTO      4
    2: \x95 FRAME      19
   11: ]    EMPTY_LIST
   12: \x94 MEMOIZE    (as 0)
   13: (    MARK
   14: \x8c     SHORT_BINUNICODE 'abcdefg'
   23: \x94     MEMOIZE    (as 1)
   24: h        BINGET     1
   26: K        BININT1    44
   28: e        APPENDS    (MARK at 13)
   29: .    STOP
highest protocol among opcodes = 4

C implementation:
b'\x95\x15\x00\x00\x00\x00\x00\x00\x00\x80\x04]\x94(\x8c\x07abcdefg\x94h\x01K,e.'
    0: \x95 FRAME      21
    9: \x80 PROTO      4
   11: ]    EMPTY_LIST
   12: \x94 MEMOIZE    (as 0)
   13: (    MARK
   14: \x8c     SHORT_BINUNICODE 'abcdefg'
   23: \x94     MEMOIZE    (as 1)
   24: h        BINGET     1
   26: K        BININT1    44
   28: e        APPENDS    (MARK at 13)
   29: .    STOP
highest protocol among opcodes = 4

----------
components: Extension Modules
messages: 314847
nosy: alexandre.vassalotti, pitrou, serhiy.storchaka
priority: normal
severity: normal
status: open
title: Repeated Pickler.dump() doesn't reset the state in the C implementation of pickle
type: behavior
versions: Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33209>
_______________________________________

From report at bugs.python.org  Mon Apr  2 22:25:38 2018
From: report at bugs.python.org (Cyker Way)
Date: Tue, 03 Apr 2018 02:25:38 +0000
Subject: [New-bugs-announce] [issue33210] pkgutil.walk_packages gives
 incomplete results
Message-ID: <1522722338.23.0.467229070634.issue33210@psf.upfronthosting.co.za>


New submission from Cyker Way <cykerway at gmail.com>:

The current implementation of `pkgutil.walk_packages()` is confusing. Users may be given incomplete results while no exception is raised if they don't explicitly provide the `prefix` parameter. The doc says:

>   prefix is a string to output on the front of every module name on output.

But the fact is, `prefix` is not merely an output formatter at all. This function cannot handle an empty prefix (which is the default) and will often encounter import errors which are then simply ignored without an `onerror` function (which is default again).

See test program for details.

Doc:

https://docs.python.org/3.6/library/pkgutil.html#pkgutil.walk_packages

Source:

https://github.com/python/cpython/blob/3.6/Lib/pkgutil.py

----------
components: Library (Lib)
files: test.py
messages: 314850
nosy: cykerway
priority: normal
severity: normal
status: open
title: pkgutil.walk_packages gives incomplete results
type: behavior
versions: Python 3.6
Added file: https://bugs.python.org/file47516/test.py

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33210>
_______________________________________

From report at bugs.python.org  Tue Apr  3 01:13:29 2018
From: report at bugs.python.org (Gil Forcada)
Date: Tue, 03 Apr 2018 05:13:29 +0000
Subject: [New-bugs-announce] [issue33211] lineno and col_offset are wrong on
 function definitions with decorators
Message-ID: <1522732409.34.0.467229070634.issue33211@psf.upfronthosting.co.za>


New submission from Gil Forcada <gil.gnome at gmail.com>:

Given the following code:

class MyClass(object):

    @property
    def my_function(self): pass

Parsing it with ast module, the lineno and col_offset of the ast.FunctionDef is reported to be where the decorator starts (i.e. line 3 column 4) rather than where the actual def statement is (i.e. line 4 column 4).

This is due to the decorator that is part of the ast.FunctionDef, but as  there can be multiple decorators (which they don't provide their own lineno and col_offset arguments) and they can span across multiple lines, there is no reliable way to actually know where the actual def statement starts physically.

See this bug report on flake8-builtins plugin (I'm the author) reported by @dhood user on github:

https://github.com/gforcada/flake8-builtins/issues/22#issuecomment-377961642

----------
components: Library (Lib)
messages: 314861
nosy: gforcada
priority: normal
severity: normal
status: open
title: lineno and col_offset are wrong on function definitions with decorators
type: behavior
versions: Python 2.7, Python 3.4, Python 3.5, Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33211>
_______________________________________

From report at bugs.python.org  Tue Apr  3 02:20:11 2018
From: report at bugs.python.org (umedoblock)
Date: Tue, 03 Apr 2018 06:20:11 +0000
Subject: [New-bugs-announce] [issue33212] add several options to msgfmt.py
Message-ID: <1522736411.29.0.467229070634.issue33212@psf.upfronthosting.co.za>


New submission from umedoblock <umedoblock at gmail.com>:

= Why do I add several options ?
I'd like to effectively know a progress of translate about some po files
when KiCad dev team announce to public KiCad 5.0 major.
Because KiCad dev team separates PCB design tools to exe files.
And I used the poedit to translate work.
Then I'd like to know progress per a po file.

Please take my idea in msgfmt.py.

== help
{{{
Usage: msgfmt.py [OPTIONS] filename.po

Options:
...
    -d
    --dump
        Show all msg_id_NO, msgid and msgstr in filename.po.

    -p
    --progress
        Show translate progress.

    -r
    --remaining
        Show no translated msgid with msg_id_NO.
...
}}}

== progress option
{{{
$ python3 ~/KiCad/kicad-doc/src/translate-set/msgfmt.py --progress ~/KiCad/kicad-doc/src/eeschema/po/ja.po 
translated: 1151 of 1427 (80%), Remaining: 276
}}}

== dump option
{{{
$ python3 ~/KiCad/kicad-doc/src/translate-set/msgfmt.py --dump ~/KiCad/kicad-doc/src/eeschema/po/ja.po | head -20
msg_id_NO=1
Eeschema
Eeschema

msg_id_NO=2
_Reference manual_
_????????????_ 

msg_id_NO=3
*Copyright*

*???*
...
}}}

== remaining option
{{{
$ python3 ~/KiCad/kicad-doc/src/translate-set/msgfmt.py --remaining ~/KiCad/kicad-doc/src/eeschema/po/ja.po | head -20
msg_id_NO=4
This document is Copyright (C) 2010-2018 by its contributors as listed below. Yo
u may distribute it and/or modify it under the terms of either the GNU General Public License (http://www.gnu.org/licenses/gpl.html), version 3 or later, or the Creative Commons Attribution License (http://creativecommons.org/licenses/by/3.0/), version 3.0 or later.

msg_id_NO=12
About KiCad translation: https://github.com/KiCad/kicad-i18n/issues

msg_id_NO=14
Published on May 30, 2015.

msg_id_NO=15
Symbol Annotation Tool
...
}}}

----------
components: Demos and Tools
files: msgfmt.py
messages: 314862
nosy: umedoblock
priority: normal
severity: normal
status: open
title: add several options to msgfmt.py
type: enhancement
versions: Python 3.6
Added file: https://bugs.python.org/file47517/msgfmt.py

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33212>
_______________________________________

From report at bugs.python.org  Tue Apr  3 04:26:58 2018
From: report at bugs.python.org (Ron Reiter)
Date: Tue, 03 Apr 2018 08:26:58 +0000
Subject: [New-bugs-announce] [issue33213] crypt function not hashing properly
Message-ID: <1522744018.72.0.467229070634.issue33213@psf.upfronthosting.co.za>


New submission from Ron Reiter <ron.reiter at gmail.com>:

import crypt

Expected result:
>>> crypt.crypt("test") == crypt.crypt("test")
False
>>> crypt.crypt("test", crypt.mksalt()) == crypt.crypt("test", crypt.mksalt())
False

Unexpected results:
>>> crypt.crypt("test", crypt.METHOD_SHA512) == crypt.crypt("test", crypt.METHOD_SHA512)
True
>>> crypt.crypt("test", crypt.mksalt(crypt.METHOD_SHA512)) == crypt.crypt("test", crypt.mksalt(crypt.METHOD_SHA512))
False

----------
components: Extension Modules
messages: 314866
nosy: Ron Reiter
priority: normal
severity: normal
status: open
title: crypt function not hashing properly
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33213>
_______________________________________

From report at bugs.python.org  Tue Apr  3 10:33:53 2018
From: report at bugs.python.org (Javier Dehesa)
Date: Tue, 03 Apr 2018 14:33:53 +0000
Subject: [New-bugs-announce] [issue33214] join method for list and tuple
Message-ID: <1522766033.22.0.467229070634.issue33214@psf.upfronthosting.co.za>


New submission from Javier Dehesa <javidcf at gmail.com>:

It is pretty trivial to concatenate a sequence of strings:

    ''.join([str1, str2, ...])

Concatenating a sequence of lists is for some reason significantly more convoluted. Some current options include:

    sum([lst1, lst2, ...], [])
    [x for y [lst1, lst2, ...] for x in y]
    list(itertools.chain(lst1, lst2, ...))

The first one being the less recomendable but more intuitive and the third one being the faster but most cumbersome (see https://stackoverflow.com/questions/49631326/why-is-itertools-chain-faster-than-a-flattening-list-comprehension ). None of these looks like "the one obvious way to do it" to me. Furthermore, I feel a dedicated concatenation method could be more efficient than any of these approaches.

If we accept that ''.join(...) is an intuitive idiom, why not provide the syntax:

    [].join([lst1, lst2, ...])

And while we are at it:

    ().join([tpl1, tpl2, ...])

Like with str, these methods should only accept sequences of objects of their own class (e.g. we could do [].join(list(s) for s in seqs) if seqs contains lists, tuples and generators). The use case for non-empty joiners would probably be less frequent than for strings, but it also solves a problem that has no clean solution with the current tools. Here is what I would probably do to join a sequence of lists with [None, 'STOP', None]:

lsts = [lst1, lst2, ...]
joiner = [None, 'STOP', None]
lsts_joined = list(itertools.chain.from_iterable(lst + joiner for lst in lsts))[:-len(joiner)]

Which is awful and inefficient (I am not saying this is the best or only possible way to solve it, it is just what I, self-considered experienced Python developer, might write).

----------
components: Library (Lib)
messages: 314881
nosy: Javier Dehesa
priority: normal
severity: normal
status: open
title: join method for list and tuple
type: enhancement

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33214>
_______________________________________

From report at bugs.python.org  Tue Apr  3 12:11:15 2018
From: report at bugs.python.org (Marius Gedminas)
Date: Tue, 03 Apr 2018 16:11:15 +0000
Subject: [New-bugs-announce] [issue33215] PyPI API wiki pages could link to
 Warehous docs please
Message-ID: <1522771875.25.0.467229070634.issue33215@psf.upfronthosting.co.za>


New submission from Marius Gedminas <marius at gedmin.as>:

Now that Warehouse is preparing to replace the old PyPI codebase I think it would be a good idea if the old PyPI API documentation pages on the Python wiki would link to the new Warehouse API documentation.

Specifically,

- https://wiki.python.org/moin/PyPISimple ought to link to https://warehouse.readthedocs.io/api-reference/legacy/

- https://wiki.python.org/moin/PyPIJSON ought to link to https://warehouse.readthedocs.io/api-reference/json/

- https://wiki.python.org/moin/PyPIXmlRpc ought to link to https://warehouse.readthedocs.io/api-reference/xml-rpc/

The wiki pages are immutable so I cannot do this myself.

(I'm not sure if this is the right bug tracker.  Apologies if I got it wrong!)

----------
assignee: docs at python
components: Documentation
messages: 314886
nosy: docs at python, mgedmin
priority: normal
severity: normal
status: open
title: PyPI API wiki pages could link to Warehous docs please
type: enhancement

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33215>
_______________________________________

From report at bugs.python.org  Tue Apr  3 15:38:13 2018
From: report at bugs.python.org (Manuel Vazquez Acosta)
Date: Tue, 03 Apr 2018 19:38:13 +0000
Subject: [New-bugs-announce] [issue33216] CALL_FUNCTION_VAR
Message-ID: <1522784293.5.0.467229070634.issue33216@psf.upfronthosting.co.za>


Change by Manuel Vazquez Acosta <manuel at merchise.org>:


----------
nosy: mvaled
priority: normal
severity: normal
status: open
title: CALL_FUNCTION_VAR

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33216>
_______________________________________

From report at bugs.python.org  Tue Apr  3 16:16:12 2018
From: report at bugs.python.org (Dutcho)
Date: Tue, 03 Apr 2018 20:16:12 +0000
Subject: [New-bugs-announce] [issue33217] x in enum.Flag() is True when x is
 no Flag
Message-ID: <1522786572.96.0.467229070634.issue33217@psf.upfronthosting.co.za>


New submission from Dutcho <dutcho at ziggo.nl>:

While `Flag() in Flag()` and `Flag() | Flag()` result in the expected outcomes, e.g. `str() in Flag()` unexpectedly returns `True`, whereas `Flag() | str()` as expected raises a TypeError.
    >>> import enum
    >>> ABC = enum.Flag('ABC', 'a, b, c')
    >>> ac = ABC.a | ABC.c
    >>> def test():
    ...     for x in (*ABC, 'test'):
    ...         print(x, end=' ')
    ...         try:
    ...             print(x in ac, end=' ')
    ...         except TypeError as e:
    ...             print(e, end=' ')
    ...         try:
    ...             print(x | ac)
    ...         except TypeError as e:
    ...             print(e)
    >>> test()
    ABC.a True ABC.c|a
    ABC.b False ABC.c|b|a
    ABC.c True ABC.c|a
    test True unsupported operand type(s) for |: 'str' and 'ABC'

Likely cause is modelling of Flag.__contains__ after Flag.__and__ etc., which is incorrect as __contains__ doesn't have a reflected version like __and__ etc. have. The returned `NotImplemented` is therefore not handled by the interpreter, although it is converted to bool to satisfy __contains__ return type.

This can be fixed by redefinition of Flag.__contains__ to raise TypeError:
    >>> def __contains__(self, other):
    ...     if not isinstance(other, self.__class__):
    ...         raise TypeError(f"unsupported operand type(s) for 'in': "
    ...                         f"{type(other).__qualname__!r} and {type(self).__qualname__!r}")
    ...     return other & self == other
    >>> enum.Flag.__contains__ = __contains__
    >>> test()
    ABC.a True ABC.c|a
    ABC.b False ABC.c|b|a
    ABC.c True ABC.c|a
    test unsupported operand type(s) for 'in': 'str' and 'ABC' unsupported operand type(s) for |: 'str' and 'ABC'

----------
components: Library (Lib)
messages: 314890
nosy: Dutcho
priority: normal
severity: normal
status: open
title: x in enum.Flag() is True when x is no Flag
type: behavior
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33217>
_______________________________________

From report at bugs.python.org  Tue Apr  3 16:26:04 2018
From: report at bugs.python.org (arjun v)
Date: Tue, 03 Apr 2018 20:26:04 +0000
Subject: [New-bugs-announce] [issue33218] Fix instances in documentation
 where dictionaries are referenced as unordered
Message-ID: <1522787164.53.0.467229070634.issue33218@psf.upfronthosting.co.za>


New submission from arjun v <arjunvkp at gmail.com>:

In python 3.6 (and above hopefully), dictionary keys are going to be ordered, while their documentation still reference them as being unordered.

https://docs.python.org/3/tutorial/datastructures.html#dictionaries 

1: `It is best to think of a dictionary as an unordered set of key: value pairs,...`

2: `Performing list(d.keys()) on a dictionary returns a list of all the keys used in the dictionary, in arbitrary order..`

The examples should also be changed to reflect the same:
```
>>> tel = {'jack': 4098, 'sape': 4139}
>>> tel['guido'] = 4127
>>> tel
{'sape': 4139, 'guido': 4127, 'jack': 4098}
>>> tel['jack']
4098
>>> del tel['sape']
>>> tel['irv'] = 4127
>>> tel
{'guido': 4127, 'irv': 4127, 'jack': 4098}
>>> list(tel.keys())
['irv', 'guido', 'jack']
>>> sorted(tel.keys())
['guido', 'irv', 'jack']
>>> 'guido' in tel
True
>>> 'jack' not in tel
False
```

```
>>> dict([('sape', 4139), ('guido', 4127), ('jack', 4098)])
{'sape': 4139, 'guido': 4127, 'jack': 4098}
```

```
>>> dict(sape=4139, guido=4127, jack=4098)
{'sape': 4139, 'guido': 4127, 'jack': 4098}
```

----------
assignee: docs at python
components: Documentation
messages: 314892
nosy: arjunv, docs at python
priority: normal
severity: normal
status: open
title: Fix instances in documentation where dictionaries are referenced as unordered
versions: Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33218>
_______________________________________

From report at bugs.python.org  Tue Apr  3 16:38:20 2018
From: report at bugs.python.org (Dutcho)
Date: Tue, 03 Apr 2018 20:38:20 +0000
Subject: [New-bugs-announce] [issue33219] x in IntFlag() should test int x's
 inclusion in IntFlag
Message-ID: <1522787900.11.0.467229070634.issue33219@psf.upfronthosting.co.za>


New submission from Dutcho <dutcho at ziggo.nl>:

While `enum.IntFlag.__and__` accepts an int arg `other` and converts it to `IntFlag` before masking, `enum.IntFlag.__contains__` handles an int arg `other` no different from a different type arg `other` (i.e. returns `True` in Python 3.6 due to issue 33217, but would raise `TypeError` after that's fixed):
    >>> import enum
    >>> ABC = enum.Flag('ABC', 'a, b, c')
    >>> ac = ABC.a | ABC.c
    >>> ABC.b in ac # works
    False
    >>> 2 in ac # should be the same; no exception due to issue 33217
    True
    >>> ac & 3 # works, equivalent to ac & ABC(3)
    <ABC.a: 1>

This is caused by a lack of specialized `IntFlag.__contains__`, so `Flag.__contains__` does the work. Can be fixed by adding a specialization, which (like in `IntFlag.__and__`) tests for `isinstance(other, (IntFlag, int))`.

    >>> def __contains__(self, other):
    ...     if not isinstance(other, (self.__class__, int)):
    ...         return TypeError
    ...     return other & self == other # conversion of int to IntFlag implicitly handled by IntFlag.__and__
    >>> IntFlag.__contains__ = __contains__

----------
components: Library (Lib)
messages: 314893
nosy: Dutcho
priority: normal
severity: normal
status: open
title: x in IntFlag() should test int x's inclusion in IntFlag
type: enhancement
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33219>
_______________________________________

From report at bugs.python.org  Tue Apr  3 18:24:02 2018
From: report at bugs.python.org (Brett Rasmussen)
Date: Tue, 03 Apr 2018 22:24:02 +0000
Subject: [New-bugs-announce] [issue33220] Antivirus hits on
 python-2.7.14.amd64.msi file
Message-ID: <1522794242.81.0.467229070634.issue33220@psf.upfronthosting.co.za>


New submission from Brett Rasmussen <brett.rasmussen at inl.gov>:

The antivirus program 'AVG Business Edition' reported that the file 
python-2.7.14.amd64.msi contained "Trojan horse SCGeneric_c6.HJQ".

The virustotal.com web site reported a match on MD5
  370014d73c3059f610c27365def62058  for file python-2.7.14.amd64.msi

  (i.e. Baidu  Win32.Trojan.WisdomEyes.16070401...)

Hopefully these are just 'false positives' ?

Thanks,
BR

----------
messages: 314904
nosy: brett.rasmussen at inl.gov
priority: normal
severity: normal
status: open
title: Antivirus hits on python-2.7.14.amd64.msi file
type: behavior
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33220>
_______________________________________

From report at bugs.python.org  Tue Apr  3 18:33:22 2018
From: report at bugs.python.org (Andrew Svetlov)
Date: Tue, 03 Apr 2018 22:33:22 +0000
Subject: [New-bugs-announce] [issue33221] Add stats for asyncio task usage.
Message-ID: <1522794802.06.0.467229070634.issue33221@psf.upfronthosting.co.za>


New submission from Andrew Svetlov <andrew.svetlov at gmail.com>:

I suggest adding a `Task.stats()` method.
The method should return a dict with the task usage statistics.
Dict keys:
- total_time: a time between task creation and a moment of the call (or task finishing timestamp if the task has finished). The value includes a time for `await func()` waiting.
- real_time: a cumulative time for `Task._step()` executions
- switch_count: a count of `Task._step()` calls.

It can help with tasks activity analyzing.

Yuri, what do you think about?

----------
components: asyncio
messages: 314905
nosy: asvetlov, yselivanov
priority: normal
severity: normal
status: open
title: Add stats for asyncio task usage.
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33221>
_______________________________________

From report at bugs.python.org  Wed Apr  4 10:00:04 2018
From: report at bugs.python.org (Jeroen Demeyer)
Date: Wed, 04 Apr 2018 14:00:04 +0000
Subject: [New-bugs-announce] [issue33222] Various test failures if
 PYTHONUSERBASE is not canonicalized
Message-ID: <1522850404.57.0.682650639539.issue33222@psf.upfronthosting.co.za>


New submission from Jeroen Demeyer <J.Demeyer at UGent.be>:

Setting PYTHONUSERBASE=/tmp/x/.. causes the Python test suite to fail:

======================================================================
FAIL: test_user_similar (test.test_sysconfig.TestSysConfig)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/local/src/sage-config/local/src/cpython/Lib/test/test_sysconfig.py", line 276, in test_user_similar
    self.assertEqual(user_path, global_path.replace(base, user, 1))
AssertionError: '/tmp/lib/python3.8' != '/tmp/x/../lib/python3.8'
- /tmp/lib/python3.8
+ /tmp/x/../lib/python3.8
?     +++++

======================================================================
FAIL: test_get_path (test.test_site.HelperFunctionsTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/local/src/sage-config/local/src/cpython/Lib/test/test_site.py", line 192, in test_get_path
    sysconfig.get_path('purelib', scheme))
AssertionError: '/tmp/x/../lib/python3.8/site-packages' != '/tmp/lib/python3.8/site-packages'
- /tmp/x/../lib/python3.8/site-packages
?     -----
+ /tmp/lib/python3.8/site-packages


======================================================================
FAIL: test_s_option (test.test_site.HelperFunctionsTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/local/src/sage-config/local/src/cpython/Lib/test/test_site.py", line 199, in test_s_option
    self.assertIn(usersite, sys.path)
AssertionError: '/tmp/x/../lib/python3.8/site-packages' not found in ['/usr/local/src/sage-config/local/src/cpython', '/usr/local/src/sage-config/local/lib/python38.zip', '/usr/local/src/sage-config/local/src/cpython/Lib', '/usr/local/src/sage-config/local/src/cpython/build/lib.linux-x86_64-3.8', '/usr/local/src/sage-config/local/lib/python3.8/site-packages']

----------
components: Tests
messages: 314931
nosy: jdemeyer
priority: normal
severity: normal
status: open
title: Various test failures if PYTHONUSERBASE is not canonicalized
type: behavior
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33222>
_______________________________________

From report at bugs.python.org  Wed Apr  4 10:13:39 2018
From: report at bugs.python.org (Ioannis Valasakis)
Date: Wed, 04 Apr 2018 14:13:39 +0000
Subject: [New-bugs-announce] [issue33223] test_posix fails ERRNO 0
Message-ID: <1522851219.93.0.682650639539.issue33223@psf.upfronthosting.co.za>


New submission from Ioannis Valasakis <code at wizofe.uk>:

I am on mac OSX high Sierra 10.13.4. I am using the Apple LLVM version 9.1.0 (clang-902.0.37.1) from the XCode toolchain. 

My tests fail on posix and if run the interpreter I get

--- pylog ---
./python.exe
Python 3.8.0a0 (heads/master:da58533ac6, Apr  3 2018, 16:54:53)
[Clang 9.1.0 (clang-902.0.37.1)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> os.getgrouplist('wizofe', 20)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
OSError: [Errno 0] Error
--- pylog ---

My system is defined as case-sensitive in the setup.

----------
components: Tests, ctypes, macOS
messages: 314934
nosy: ned.deily, ronaldoussoren, wizofe
priority: normal
severity: normal
status: open
title: test_posix fails ERRNO 0
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33223>
_______________________________________

From report at bugs.python.org  Wed Apr  4 11:18:24 2018
From: report at bugs.python.org (Jeff Kaufman)
Date: Wed, 04 Apr 2018 15:18:24 +0000
Subject: [New-bugs-announce] [issue33224] "RuntimeError: generator raised
 StopIteration" in difflib.mdiff
Message-ID: <1522855104.15.0.682650639539.issue33224@psf.upfronthosting.co.za>


New submission from Jeff Kaufman <jeff.t.kaufman at gmail.com>:

With python built at HEAD (c51d8c9b) and at 3.7b3 (fcd4e03e08) the code:

    import difflib
    for fromdata, todata, flag in difflib._mdiff(
        ["2"], ["3"], 1):
      pass

produces:

    Traceback (most recent call last):
      File "/home/jefftk/cpython/Lib/difflib.py", line 1638, in _mdiff
        from_line, to_line, found_diff = next(line_pair_iterator)
    StopIteration

    The above exception was the direct cause of the following exception:

    Traceback (most recent call last):
      File "/home/jefftk/icdiff/repro.py", line 3, in <module>
        ["2"], ["3"], 1):
    RuntimeError: generator raised StopIteration

In python 3.5 and 3.6 I don't get an error.

This is probably due to https://bugs.python.org/issue32670 which implements PEP 479, but I this this isn't supposed to happen in library code?

----------
components: Library (Lib)
messages: 314936
nosy: Jeff.Kaufman
priority: normal
severity: normal
status: open
title: "RuntimeError: generator raised StopIteration" in difflib.mdiff
type: behavior
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33224>
_______________________________________

From report at bugs.python.org  Wed Apr  4 13:40:42 2018
From: report at bugs.python.org (Yang Yu)
Date: Wed, 04 Apr 2018 17:40:42 +0000
Subject: [New-bugs-announce] [issue33225] imaplib module IMAP4.append()
 unexpected response BAD Command Error
Message-ID: <1522863642.75.0.682650639539.issue33225@psf.upfronthosting.co.za>


New submission from Yang Yu <yangyudev at gmail.com>:

part of bytestring in message in .append() includes 'text-align: center;\r\n}</style><!--[if gte mso 9]><xml>'

append always fails on such messages with 
`abort: command: APPEND => unexpected response: b'}</style><!--[if BAD Command Error. 10'`

Some spam emails includes this in body. I pipe incoming emails to spamassassin and pipe back stdout to be appended to spam mailbox. This type of failures always causes IMAPS connection to be closed.

I can reproduce this every time. IMAP server is Microsoft Exchange I believe

----------
components: Extension Modules
messages: 314945
nosy: yuy
priority: normal
severity: normal
status: open
title: imaplib module IMAP4.append() unexpected response BAD Command Error
type: behavior
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33225>
_______________________________________

From report at bugs.python.org  Wed Apr  4 19:31:07 2018
From: report at bugs.python.org (Seyeong Kim)
Date: Wed, 04 Apr 2018 23:31:07 +0000
Subject: [New-bugs-announce] [issue33226] In some envrionment using unicode,
 formatwarning shows ascii error
Message-ID: <1522884667.65.0.682650639539.issue33226@psf.upfronthosting.co.za>


New submission from Seyeong Kim <seyeong.kim at canonical.com>:

In some circumstances using unicode, formatwarning show me ascii error

so I should prefix on below line to remove this crash
s =  "%s: %s: %s\n" % (lineno, category.__name__, message)
to
s =  u"%s: %s: %s\n" % (lineno, category.__name__, message)

below are logs

Apr  4 22:22:44 juju-7a19d0-0-lxd-0 cinder-api[5302]: 2018-04-04 22:22:44.441 5316 ERROR cinder.api.middleware.fault   File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 144, in __call__
Apr  4 22:22:44 juju-7a19d0-0-lxd-0 cinder-api[5302]: 2018-04-04 22:22:44.441 5316 ERROR cinder.api.middleware.fault     return resp(environ, start_response)
Apr  4 22:22:44 juju-7a19d0-0-lxd-0 cinder-api[5302]: 2018-04-04 22:22:44.441 5316 ERROR cinder.api.middleware.fault   File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 130, in __call__
Apr  4 22:22:44 juju-7a19d0-0-lxd-0 cinder-api[5302]: 2018-04-04 22:22:44.441 5316 ERROR cinder.api.middleware.fault     resp = self.call_func(req, *args, **self.kwargs)
Apr  4 22:22:44 juju-7a19d0-0-lxd-0 cinder-api[5302]: 2018-04-04 22:22:44.441 5316 ERROR cinder.api.middleware.fault   File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 195, in call_func
Apr  4 22:22:44 juju-7a19d0-0-lxd-0 cinder-api[5302]: 2018-04-04 22:22:44.441 5316 ERROR cinder.api.middleware.fault     return self.func(req, *args, **kwargs)
Apr  4 22:22:44 juju-7a19d0-0-lxd-0 cinder-api[5302]: 2018-04-04 22:22:44.441 5316 ERROR cinder.api.middleware.fault   File "/usr/lib/python2.7/dist-packages/cinder/api/openstack/wsgi.py", line 1061, in __call__
Apr  4 22:22:44 juju-7a19d0-0-lxd-0 cinder-api[5302]: 2018-04-04 22:22:44.441 5316 ERROR cinder.api.middleware.fault     content_type, body, accept)
Apr  4 22:22:44 juju-7a19d0-0-lxd-0 cinder-api[5302]: 2018-04-04 22:22:44.441 5316 ERROR cinder.api.middleware.fault   File "/usr/lib/python2.7/dist-packages/cinder/api/openstack/wsgi.py", line 1123, in _process_stack
Apr  4 22:22:44 juju-7a19d0-0-lxd-0 cinder-api[5302]: 2018-04-04 22:22:44.441 5316 ERROR cinder.api.middleware.fault     action_result = self.dispatch(meth, request, action_args)
Apr  4 22:22:44 juju-7a19d0-0-lxd-0 cinder-api[5302]: 2018-04-04 22:22:44.441 5316 ERROR cinder.api.middleware.fault   File "/usr/lib/python2.7/dist-packages/cinder/api/openstack/wsgi.py", line 1221, in dispatch
Apr  4 22:22:44 juju-7a19d0-0-lxd-0 cinder-api[5302]: 2018-04-04 22:22:44.441 5316 ERROR cinder.api.middleware.fault     return method(req=request, **action_args)
Apr  4 22:22:44 juju-7a19d0-0-lxd-0 cinder-api[5302]: 2018-04-04 22:22:44.441 5316 ERROR cinder.api.middleware.fault   File "/usr/lib/python2.7/dist-packages/cinder/api/v2/volumes.py", line 405, in create
Apr  4 22:22:44 juju-7a19d0-0-lxd-0 cinder-api[5302]: 2018-04-04 22:22:44.441 5316 ERROR cinder.api.middleware.fault     **kwargs)
Apr  4 22:22:44 juju-7a19d0-0-lxd-0 cinder-api[5302]: 2018-04-04 22:22:44.441 5316 ERROR cinder.api.middleware.fault   File "/usr/lib/python2.7/dist-packages/cinder/volume/api.py", line 327, in create
Apr  4 22:22:44 juju-7a19d0-0-lxd-0 cinder-api[5302]: 2018-04-04 22:22:44.441 5316 ERROR cinder.api.middleware.fault     flow_engine.run()
Apr  4 22:22:44 juju-7a19d0-0-lxd-0 cinder-api[5302]: 2018-04-04 22:22:44.441 5316 ERROR cinder.api.middleware.fault   File "/usr/lib/python2.7/dist-packages/taskflow/engines/action_engine/engine.py", line 230, in run
Apr  4 22:22:44 juju-7a19d0-0-lxd-0 cinder-api[5302]: 2018-04-04 22:22:44.441 5316 ERROR cinder.api.middleware.fault     for _state in self.run_iter(timeout=timeout):
Apr  4 22:22:44 juju-7a19d0-0-lxd-0 cinder-api[5302]: 2018-04-04 22:22:44.441 5316 ERROR cinder.api.middleware.fault   File "/usr/lib/python2.7/dist-packages/taskflow/engines/action_engine/engine.py", line 308, in run_iter
Apr  4 22:22:44 juju-7a19d0-0-lxd-0 cinder-api[5302]: 2018-04-04 22:22:44.441 5316 ERROR cinder.api.middleware.fault     failure.Failure.reraise_if_any(fails)
Apr  4 22:22:44 juju-7a19d0-0-lxd-0 cinder-api[5302]: 2018-04-04 22:22:44.441 5316 ERROR cinder.api.middleware.fault   File "/usr/lib/python2.7/dist-packages/taskflow/types/failure.py", line 336, in reraise_if_any
Apr  4 22:22:44 juju-7a19d0-0-lxd-0 cinder-api[5302]: 2018-04-04 22:22:44.441 5316 ERROR cinder.api.middleware.fault     failures[0].reraise()
Apr  4 22:22:44 juju-7a19d0-0-lxd-0 cinder-api[5302]: 2018-04-04 22:22:44.441 5316 ERROR cinder.api.middleware.fault   File "/usr/lib/python2.7/dist-packages/taskflow/types/failure.py", line 343, in reraise
Apr  4 22:22:44 juju-7a19d0-0-lxd-0 cinder-api[5302]: 2018-04-04 22:22:44.441 5316 ERROR cinder.api.middleware.fault     six.reraise(*self._exc_info)
Apr  4 22:22:44 juju-7a19d0-0-lxd-0 cinder-api[5302]: 2018-04-04 22:22:44.441 5316 ERROR cinder.api.middleware.fault   File "/usr/lib/python2.7/dist-packages/taskflow/engines/action_engine/executor.py", line 82, in _execute_task
Apr  4 22:22:44 juju-7a19d0-0-lxd-0 cinder-api[5302]: 2018-04-04 22:22:44.441 5316 ERROR cinder.api.middleware.fault     result = task.execute(**arguments)
Apr  4 22:22:44 juju-7a19d0-0-lxd-0 cinder-api[5302]: 2018-04-04 22:22:44.441 5316 ERROR cinder.api.middleware.fault   File "/usr/lib/python2.7/dist-packages/cinder/volume/flows/api/create_volume.py", line 527, in execute
Apr  4 22:22:44 juju-7a19d0-0-lxd-0 cinder-api[5302]: 2018-04-04 22:22:44.441 5316 ERROR cinder.api.middleware.fault     volume = objects.Volume(context=context, **volume_properties)
Apr  4 22:22:44 juju-7a19d0-0-lxd-0 cinder-api[5302]: 2018-04-04 22:22:44.441 5316 ERROR cinder.api.middleware.fault   File "/usr/lib/python2.7/dist-packages/cinder/objects/volume.py", line 180, in __init__
Apr  4 22:22:44 juju-7a19d0-0-lxd-0 cinder-api[5302]: 2018-04-04 22:22:44.441 5316 ERROR cinder.api.middleware.fault     super(Volume, self).__init__(*args, **kwargs)
Apr  4 22:22:44 juju-7a19d0-0-lxd-0 cinder-api[5302]: 2018-04-04 22:22:44.441 5316 ERROR cinder.api.middleware.fault   File "/usr/lib/python2.7/dist-packages/oslo_versionedobjects/base.py", line 304, in __init__
Apr  4 22:22:44 juju-7a19d0-0-lxd-0 cinder-api[5302]: 2018-04-04 22:22:44.441 5316 ERROR cinder.api.middleware.fault     setattr(self, key, kwargs[key])
Apr  4 22:22:44 juju-7a19d0-0-lxd-0 cinder-api[5302]: 2018-04-04 22:22:44.441 5316 ERROR cinder.api.middleware.fault   File "/usr/lib/python2.7/dist-packages/oslo_versionedobjects/base.py", line 72, in setter
Apr  4 22:22:44 juju-7a19d0-0-lxd-0 cinder-api[5302]: 2018-04-04 22:22:44.441 5316 ERROR cinder.api.middleware.fault     field_value = field.coerce(self, name, value)
Apr  4 22:22:44 juju-7a19d0-0-lxd-0 cinder-api[5302]: 2018-04-04 22:22:44.441 5316 ERROR cinder.api.middleware.fault   File "/usr/lib/python2.7/dist-packages/oslo_versionedobjects/fields.py", line 192, in coerce
Apr  4 22:22:44 juju-7a19d0-0-lxd-0 cinder-api[5302]: 2018-04-04 22:22:44.441 5316 ERROR cinder.api.middleware.fault     return self._type.coerce(obj, attr, value)
Apr  4 22:22:44 juju-7a19d0-0-lxd-0 cinder-api[5302]: 2018-04-04 22:22:44.441 5316 ERROR cinder.api.middleware.fault   File "/usr/lib/python2.7/dist-packages/oslo_versionedobjects/fields.py", line 325, in coerce
Apr  4 22:22:44 juju-7a19d0-0-lxd-0 cinder-api[5302]: 2018-04-04 22:22:44.441 5316 ERROR cinder.api.middleware.fault     u"for further details" % value, FutureWarning)
Apr  4 22:22:44 juju-7a19d0-0-lxd-0 cinder-api[5302]: 2018-04-04 22:22:44.441 5316 ERROR cinder.api.middleware.fault   File "/usr/lib/python2.7/logging/__init__.py", line 1724, in _showwarning
Apr  4 22:22:44 juju-7a19d0-0-lxd-0 cinder-api[5302]: 2018-04-04 22:22:44.441 5316 ERROR cinder.api.middleware.fault     s = warnings.formatwarning(message, category, filename, lineno, line)
Apr  4 22:22:44 juju-7a19d0-0-lxd-0 cinder-api[5302]: 2018-04-04 22:22:44.441 5316 ERROR cinder.api.middleware.fault   File "/usr/lib/python2.7/warnings.py", line 50, in formatwarning
Apr  4 22:22:44 juju-7a19d0-0-lxd-0 cinder-api[5302]: 2018-04-04 22:22:44.441 5316 ERROR cinder.api.middleware.fault     s =  "%s: %s: %s\n" % (lineno, category.__name__, message)
Apr  4 22:22:44 juju-7a19d0-0-lxd-0 cinder-api[5302]: 2018-04-04 22:22:44.441 5316 ERROR cinder.api.middleware.fault UnicodeEncodeError: 'ascii' codec can't encode character u'\u0101' in position 1: ordinal not in range(128)
Apr  4 22:22:44 juju-7a19d0-0-lxd-0 cinder-api[5302]: 2018-04-04 22:22:44.441 5316 ERROR cinder.api.middleware.fault

----------
components: Library (Lib)
messages: 314961
nosy: xtrusia
priority: normal
severity: normal
status: open
title: In some envrionment using unicode, formatwarning shows ascii error
type: crash
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33226>
_______________________________________

From report at bugs.python.org  Thu Apr  5 02:26:53 2018
From: report at bugs.python.org (Oz Tiram)
Date: Thu, 05 Apr 2018 06:26:53 +0000
Subject: [New-bugs-announce] [issue33227] Cmd do_something only accepts one
 argument
Message-ID: <1522909613.34.0.682650639539.issue33227@psf.upfronthosting.co.za>


New submission from Oz Tiram <nahumoz at gmail.com>:

Considering that I want to build an CLI interactive program using cmd.Cmd,
which has multiple functions that accept two or more arguments,
I always have to do parsing of arg inside my do_methods for example:

class Calc(cmd.Cmd):

    do_add(self, arg):
       a, b = parse_ints(arg)
       return a+b

    do_mult(self, arg):
       a, b = parse_ints(arg)
       return a*b

This is easy enough for a simple calculator. But for more logic I need to create more parse_type and explicitly call them at the top of each do_stuff.
Considering this I have created a patch for Cmd.cmd, that keeps the old functionality, but adds the ability to pass multiple arguments without having to parse them every time. 
  
The functionality is as follows:


class MyProg(cmd.Cmd):

    do_simple(self, arg):
       print(arg)

    do_greet(self, name="World")
       print("Hello %s" %s)

    do_add(self, a, b):
       print(a+b)
    

Now in the cmd prompt:

(Cmd) simple foo
foo  
(Cmd) # keeps the old functionallity 
(Cmd) add 2 3
5
# add two properly add the values
(Cmd) add 2
# expects two arguments, so it will print the help for add
(Cmd) greet
Hello World
# does not complain because it has a default value
(Cmd) greet Guido
Hello Guido
(Cmd) greet name=Raymond
Hello Raymond
# works too

None of these example guess the type. It is simply handles as a string. but I am playing around with automatically detecting types of variables using function annotations in python 3.5 and later.

Also, although, not demonstrated here, one can also define:

    def do_some_method_with_kwargs(self, arg1, arg2, **kwargs):
        ... do fancy stuff ...

(Cmd) some_method_with_kwargs foo bar kwargs='{"name": "John", "age"=35}'

kwargs is parsed to a dict (using json.loads) when escaped properly.
This is a bit esoteric, hence not shown here (but the patch patch includes it).
 
I was wondering if this kind of enhancement would be considered for this module? 
I will happily create a PR with tests and documentation explaining this enhanced functionality.

----------
files: cmd.patch
keywords: patch
messages: 314968
nosy: Oz.Tiram
priority: normal
severity: normal
status: open
title: Cmd do_something only accepts one argument
Added file: https://bugs.python.org/file47520/cmd.patch

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33227>
_______________________________________

From report at bugs.python.org  Thu Apr  5 04:50:52 2018
From: report at bugs.python.org (Wolfgang Maier)
Date: Thu, 05 Apr 2018 08:50:52 +0000
Subject: [New-bugs-announce] [issue33228] Use Random.choices in tempfile
Message-ID: <1522918252.74.0.682650639539.issue33228@psf.upfronthosting.co.za>


New submission from Wolfgang Maier <wolfgang.maier at biologie.uni-freiburg.de>:

A rather trivial change: tempfile._RandomNameSequence could make use of
random.Random.choices introduced in 3.6.
IMO, the suggested change would give clearer and also faster code.
It also updates the docstring of the class, which currently talks about
a six-character string, when an eight-character string gets returned.

----------
messages: 314969
nosy: wolma
priority: normal
severity: normal
status: open
title: Use Random.choices in tempfile
type: performance
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33228>
_______________________________________

From report at bugs.python.org  Thu Apr  5 05:08:37 2018
From: report at bugs.python.org (Mikhail Zakharov)
Date: Thu, 05 Apr 2018 09:08:37 +0000
Subject: [New-bugs-announce] =?utf-8?q?=5Bissue33229=5D_Documentation_-__?=
 =?utf-8?q?io_=E2=80=94_Core_tools_for_working_with_streams_-_seek=28=29?=
Message-ID: <1522919317.94.0.682650639539.issue33229@psf.upfronthosting.co.za>


New submission from Mikhail Zakharov <zmey20000 at yahoo.com>:

Documentation for io.IOBase class and it's seek() method at https://docs.python.org/3/library/io.html mentions SEEK_* constants like:

"    SEEK_SET or 0 ? start of the stream (the default); offset should be zero or positive
    SEEK_CUR or 1 ? current stream position; offset may be negative
    SEEK_END or 2 ? end of the stream; offset is usually negative
"

It seems, they actually should be used as os.SEEK_SET, os.SEEK_CUR and os.SEEK_END.

----------
assignee: docs at python
components: Documentation
messages: 314970
nosy: Mikhail Zakharov, docs at python
priority: normal
severity: normal
status: open
title: Documentation -  io ? Core tools for working with streams - seek()
type: enhancement
versions: Python 2.7, Python 3.4, Python 3.5, Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33229>
_______________________________________

From report at bugs.python.org  Thu Apr  5 06:19:14 2018
From: report at bugs.python.org (Hubert Holin)
Date: Thu, 05 Apr 2018 10:19:14 +0000
Subject: [New-bugs-announce] [issue33230] _decimal build failure
 (unsupported platform for that module) - evolution possible?
Message-ID: <1522923554.41.0.682650639539.issue33230@psf.upfronthosting.co.za>


New submission from Hubert Holin <Hubert.Holin at free.fr>:

_decimal fails to build on my platform due to lack of support:

In file included from /Developer/Python/3.6/Python/Python-3.6.5/Modules/_decimal/_decimal.c:34:
/usr/local/include/mpdecimal.h:201:4: error: "unsupported platform: need mpd_size_t == mpd_uint_t"
  #error "unsupported platform: need mpd_size_t == mpd_uint_t"

My host system type: x86_64-apple-darwin17.5.0
My target system type: x86_64-apple-darwin17.5.0
Compiler used: clang+llvm-6.0.0-x86_64-apple-darwin
Hardware: 3,7 GHz Quad-Core Intel Xeon E5

Relevant configurations:

--enable-universalsdk... /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.13.sdk

--with-universal-archs... intel

MACHDEP... darwin

Could this module be upgraded?

----------
components: Extension Modules
messages: 314976
nosy: Hubert Holin
priority: normal
severity: normal
status: open
title: _decimal build failure (unsupported platform for that module) - evolution possible?
type: compile error
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33230>
_______________________________________

From report at bugs.python.org  Thu Apr  5 08:34:19 2018
From: report at bugs.python.org (INADA Naoki)
Date: Thu, 05 Apr 2018 12:34:19 +0000
Subject: [New-bugs-announce] [issue33231] Potential memory leak in
 normalizestring()
Message-ID: <1522931659.19.0.682650639539.issue33231@psf.upfronthosting.co.za>


Change by INADA Naoki <songofacandy at gmail.com>:


----------
components: Interpreter Core
nosy: inada.naoki
priority: normal
severity: normal
status: open
title: Potential memory leak in normalizestring()
type: resource usage
versions: Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33231>
_______________________________________

From report at bugs.python.org  Thu Apr  5 10:49:04 2018
From: report at bugs.python.org (Alexander Sturm)
Date: Thu, 05 Apr 2018 14:49:04 +0000
Subject: [New-bugs-announce] [issue33232] Segmentation fault in
 operator.attrgetter
Message-ID: <1522939744.33.0.682650639539.issue33232@psf.upfronthosting.co.za>


New submission from Alexander Sturm <asturm at hbk.com>:

This issue can be reproduced using Python 3.5.2 - 3.5.5 compiled with VS2015 (tested both the official Python builds as well as local builds using VS2015 Version 14.0.25431.01 Update 3) on 64-bit Windows.

To reproduce, run the attached file (or python -c "import operator; operator.attrgetter('x'*1000000)").

The segfault itself occurs in attrgetter_new [1], while scanning for dots inside a unicode string. From my reading of the code, the code itself is actually correct - instead, it triggers a compiler bug in VS2015, which causes it to miscompile the code by applying an incorrect optimization.

The optimization VS2015 applies is twofold: It uses SIMD instructions to avoid looking at every byte of the input string individually, and it tries to eliminate the condition in PyUnicode_READ [2] by executing all three branches (UCS1, UCS2 and UCS4) at once and then in a separate step discarding the results for the two branches that would not be taken.

When combining these optimizations, the generated code incorrectly only checks the precondition of the UCS1 branch to decide whether it can use the optimized SIMD version. Since the UCS1 branch looks at 8 bytes of the buffer in each iteration, it checks whether the buffer contains at least 8 bytes. However, each iteration of the UCS2/UCS4 branches look at 16/32 bytes, respectively.

As a result, when passing a UCS1 string to attrgetter(), we end up reading 24 bytes past the end of the buffer for every 8 bytes the string contains. In the reproduction example, we thus read about 24 MB past the buffer, almost guaranteeing a crash.

The issue does not occur when compiling Python using VS2017 (i.e. PlatformToolset v141, MSVC version 14.13.26128). Presumably this means the compiler bug has been fixed upstream.

[1]: https://github.com/python/cpython/blob/v3.5.5/Modules/_operator.c#L587
[2]: https://github.com/python/cpython/blob/v3.5.5/Include/unicodeobject.h#L521

----------
components: Interpreter Core
files: repro.py
messages: 314987
nosy: asturm
priority: normal
severity: normal
status: open
title: Segmentation fault in operator.attrgetter
versions: Python 3.5
Added file: https://bugs.python.org/file47521/repro.py

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33232>
_______________________________________

From report at bugs.python.org  Thu Apr  5 14:51:29 2018
From: report at bugs.python.org (Ned Deily)
Date: Thu, 05 Apr 2018 18:51:29 +0000
Subject: [New-bugs-announce] [issue33233] Suggest third-party cmd2 module as
 alternative to cmd
Message-ID: <1522954289.49.0.682650639539.issue33233@psf.upfronthosting.co.za>


New submission from Ned Deily <nad at python.org>:

The cmd module in the standard library has languished for many years.  In the mean time, third-party replacements for it have arisen.  Perhaps the most popular is cmd2 which seems to be actively maintained, provides upward compatibility from cmd along with many desirable new features, and is permissively licensed.

I suggest we consider at a minimum adding a "See also:" note referencing cmd2 to the cmd documentation in the Standard Library document, similar to what we do for the third-party "requests" module in the "urllib.request" documentation.

We could be even bolder and officially deprecate "cmd" and consider closing open enhancement issues for it on b.p.o.




https://pypi.org/project/cmd2/
https://cmd2.readthedocs.io/

----------
assignee: docs at python
components: Documentation
messages: 315002
nosy: Catherine.Devlin, docs at python, lukasz.langa, ned.deily
priority: normal
severity: normal
status: open
title: Suggest third-party cmd2 module as alternative to cmd
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33233>
_______________________________________

From report at bugs.python.org  Thu Apr  5 17:46:53 2018
From: report at bugs.python.org (Raymond Hettinger)
Date: Thu, 05 Apr 2018 21:46:53 +0000
Subject: [New-bugs-announce] [issue33234] Improve list() pre-sizing for
 inputs with known lengths
Message-ID: <1522964813.15.0.682650639539.issue33234@psf.upfronthosting.co.za>


New submission from Raymond Hettinger <raymond.hettinger at gmail.com>:

The list() constructor isn't taking full advantage of known input lengths or length hints.  Ideally, it should pre-size and not over-allocate when the input size is known or can be reasonably estimated.

Python 3.8.0a0 (heads/master:091e95e900, Apr  5 2018, 09:48:33)
[Clang 9.1.0 (clang-902.0.39.1)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from sys import getsizeof
>>> getsizeof([0] * 10)
144
>>> getsizeof(list([0] * 10))
200
>>> getsizeof(list(range(10)))
200

----------
components: Interpreter Core
messages: 315006
nosy: rhettinger
priority: normal
severity: normal
status: open
title: Improve list() pre-sizing for inputs with known lengths
type: performance
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33234>
_______________________________________

From report at bugs.python.org  Fri Apr  6 06:13:47 2018
From: report at bugs.python.org (Paddy McCarthy)
Date: Fri, 06 Apr 2018 10:13:47 +0000
Subject: [New-bugs-announce] [issue33235] Better help text for
 dict.setdefault
Message-ID: <1523009627.66.0.682650639539.issue33235@psf.upfronthosting.co.za>


New submission from Paddy McCarthy <paddy3118 at gmail.com>:

Hi, I was answering some question and used dict.setdefault as part of the solution and posted the help() on it as part of my answer.

The help was this:

In [15]: help(mapper.setdefault)
Help on built-in function setdefault:

setdefault(...) method of builtins.dict instance
    D.setdefault(k[,d]) -> D.get(k,d), also set D[k]=d if k not in D

This seems the wrong way around. Is it not better expressed as

    D.setdefault(k[,d]) -> set D[k]=d if k not in D and then D.get(k,d)

----------
assignee: docs at python
components: Documentation
messages: 315015
nosy: Paddy McCarthy, docs at python
priority: normal
severity: normal
status: open
title: Better help text for dict.setdefault
type: enhancement
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33235>
_______________________________________

From report at bugs.python.org  Fri Apr  6 07:50:07 2018
From: report at bugs.python.org (Robert)
Date: Fri, 06 Apr 2018 11:50:07 +0000
Subject: [New-bugs-announce] [issue33236] MagicMock().__iter__.return_value
 is different from MagicMock().__iter__()
Message-ID: <1523015407.46.0.682650639539.issue33236@psf.upfronthosting.co.za>


New submission from Robert <lma at posteo.de>:

According to the documentation .return_value should be identical to the object returned when calling the mock ("assert m() is m.return_value")

This is the case except on objects returned by __iter__ on MagicMocks. The following script demonstrates the problem:
----
from unittest.mock import MagicMock
m = MagicMock()
assert x.__iter__() is x.__iter__.return_value    # <- fails
----

In fact __iter__() returns the object "iter([])" (which matches the documentation) while __iter__.return_value return a MagicMock object (which does not match the documentation).

When replacing "__iter__" with any other special function MagicMock works as expected.

----------
components: Library (Lib)
messages: 315016
nosy: mrh1997
priority: normal
severity: normal
status: open
title: MagicMock().__iter__.return_value is different from MagicMock().__iter__()
versions: Python 2.7, Python 3.4, Python 3.5, Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33236>
_______________________________________

From report at bugs.python.org  Fri Apr  6 11:43:32 2018
From: report at bugs.python.org (Serhiy Storchaka)
Date: Fri, 06 Apr 2018 15:43:32 +0000
Subject: [New-bugs-announce] [issue33237] Improve AttributeError message for
 partially initialized module
Message-ID: <1523029412.67.0.682650639539.issue33237@psf.upfronthosting.co.za>


New submission from Serhiy Storchaka <storchaka+cpython at gmail.com>:

Cyclic import usually leads to an AttributeError "module 'spam' has no attribute 'ham'" which usually is confusing because in normal case 'spam.ham' exists, and the user can have no ideas why it is disappeared.

The proposed PR allows to specialize the AttributeError message for partially initialized module. Any suggestions about the error message?

----------
components: Interpreter Core
messages: 315019
nosy: brett.cannon, eric.snow, ncoghlan, serhiy.storchaka
priority: normal
severity: normal
status: open
title: Improve AttributeError message for partially initialized module
type: enhancement
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33237>
_______________________________________

From report at bugs.python.org  Fri Apr  6 14:02:45 2018
From: report at bugs.python.org (Jason Haydaman)
Date: Fri, 06 Apr 2018 18:02:45 +0000
Subject: [New-bugs-announce] [issue33238] AssertionError on await of Future
 returned by asyncio.wrap_future
Message-ID: <1523037765.71.0.682650639539.issue33238@psf.upfronthosting.co.za>


New submission from Jason Haydaman <jhaydaman at sightlineinnovation.com>:

When the concurrent Future wrapped by asyncio.wrap_future has set_result or set_exception called multiple times, I get the following traceback:

Traceback (most recent call last):
  File "/usr/local/lib/python3.6/asyncio/events.py", line 145, in _run
    self._callback(*self._args)
  File "/usr/local/lib/python3.6/asyncio/futures.py", line 399, in _set_state
    _copy_future_state(other, future)
  File "/usr/local/lib/python3.6/asyncio/futures.py", line 369, in _copy_future_state
    assert not dest.done()
AssertionError


Minimal reproducible example:

import asyncio
import concurrent.futures

f = concurrent.futures.Future()
async_f = asyncio.wrap_future(f)
f.set_result(1)
f.set_result(1)

loop = asyncio.get_event_loop()
loop.run_until_complete(async_f)


The documentation says that set_result and set_exception are only meant for Executors, so, arguably using them outside of that context would fall into undocumented behavior rather than a bug. But it still seems like it should be the second set_result that raises something like InvalidStateError. Alternatively, this can be avoided if _copy_future_state checks for done in addition to cancellation. What should happen here?

----------
components: Library (Lib), asyncio
messages: 315030
nosy: Jason Haydaman, asvetlov, yselivanov
priority: normal
severity: normal
status: open
title: AssertionError on await of Future returned by asyncio.wrap_future
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33238>
_______________________________________

From report at bugs.python.org  Sat Apr  7 15:22:42 2018
From: report at bugs.python.org (Martin Falatic)
Date: Sat, 07 Apr 2018 19:22:42 +0000
Subject: [New-bugs-announce] [issue33239] tempfile module: functions with
 the 'buffering' option are incorrectly documented
Message-ID: <1523128962.9.0.682650639539.issue33239@psf.upfronthosting.co.za>


New submission from Martin Falatic <martin at falatic.com>:

The documentation for the tempfile module in Python 3.x for the `buffering` option is incorrect:

https://docs.python.org/3/library/tempfile.html

TemporaryFile, NamedTemporaryFile, and SpooledTemporaryFile all take the `buffering` option, which in turn appears to correlate to the Python 2.7 option `bufsize`, which was and continues to be an integer (per the source).

In the 3.x documentation the default signature for TemporaryFile, NamedTemporaryFile, and SpooledTemporaryFile includes `buffering=None`. Actually specifying None as a default for this will cause an exception (`TypeError: an integer is required (got type NoneType)`).

There is a cross-reference in the 3.x tempfile docs to `open` (https://docs.python.org/3/library/functions.html#open) which in turn shows the correct signature to use for `buffering`. Additionally, the source code is clearly documented (https://github.com/python/cpython/blob/master/Lib/tempfile.py)

A good correction would be to ensure `buffering=-1` is documented as the default for the three functions in tempfile, with an additional note explicitly stating that -1 == no buffering, and the existing `open` cross-reference retained.

----------
assignee: docs at python
components: Documentation
messages: 315072
nosy: MartyMacGyver, docs at python
priority: normal
severity: normal
status: open
title: tempfile module: functions with the 'buffering' option are incorrectly documented
versions: Python 3.4, Python 3.5, Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33239>
_______________________________________

From report at bugs.python.org  Sat Apr  7 22:51:56 2018
From: report at bugs.python.org (Yu Liu)
Date: Sun, 08 Apr 2018 02:51:56 +0000
Subject: [New-bugs-announce] [issue33240] shutil.rmtree fails when the inner
 floder is opened in Explorer on Windows
Message-ID: <1523155916.86.0.682650639539.issue33240@psf.upfronthosting.co.za>


New submission from Yu Liu <neophyliam at gmail.com>:

Given the following directory structure on a Windows machine:
- foo
 - bar

a call to `shutil.rmtree("foo")` will fail when the inner folder `bar` is opened in an Explorer. The error message indicates the `foo` directory is not empty, while after the execution, although it failed, the `foo` directory is empty. So the inner folder `bar` was removed successfully, but `foo` was not. And the error message is misleading.

It will not fail when `foo` is opened in an Explorer, neither on Linux system.

----------
components: Library (Lib), Windows
messages: 315077
nosy: giampaolo.rodola, paul.moore, philius, steve.dower, tarek, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: shutil.rmtree fails when the inner floder is opened in Explorer on Windows
type: behavior
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33240>
_______________________________________

From report at bugs.python.org  Sun Apr  8 14:19:34 2018
From: report at bugs.python.org (Louis Martin)
Date: Sun, 08 Apr 2018 18:19:34 +0000
Subject: [New-bugs-announce] [issue33241] Tooltip not display with macOS
 64-bit installer 3.6.5 but work with macOS 64-bit/32-bit installer
Message-ID: <1523211574.88.0.682650639539.issue33241@psf.upfronthosting.co.za>


Change by Louis Martin <louis.martin.uqam at gmail.com>:


----------
components: Tkinter
nosy: louis-martin
priority: normal
severity: normal
status: open
title: Tooltip not display with macOS 64-bit installer 3.6.5 but work with macOS 64-bit/32-bit installer
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33241>
_______________________________________

From report at bugs.python.org  Sun Apr  8 17:19:25 2018
From: report at bugs.python.org (Matthias Urlichs)
Date: Sun, 08 Apr 2018 21:19:25 +0000
Subject: [New-bugs-announce] [issue33242] Support binary symbol names
Message-ID: <1523222365.46.0.682650639539.issue33242@psf.upfronthosting.co.za>


New submission from Matthias Urlichs <smurf at smurf.noris.de>:

ctypes should support binary symbols.

Rationale: There's no requirement that the symbol name in question is encoded as ASCII or UTF-8.

>>> import ctypes
>>> t = type('iface', (ctypes.Structure,), {'_fields_': [(b'c_string_symbol', ctypes.CFUNCTYPE(ctypes.c_uint32))]})
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: '_fields_' must be a sequence of (name, C type) pairs

----------
components: ctypes
messages: 315096
nosy: smurfix
priority: normal
severity: normal
status: open
title: Support binary symbol names
type: enhancement

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33242>
_______________________________________

From report at bugs.python.org  Sun Apr  8 23:07:30 2018
From: report at bugs.python.org (Ali Abbas)
Date: Mon, 09 Apr 2018 03:07:30 +0000
Subject: [New-bugs-announce] [issue33243] nltk is not working properly
Message-ID: <1523243250.91.0.682650639539.issue33243@psf.upfronthosting.co.za>


New submission from Ali Abbas <aliabbas367 at gmail.com>:

nltk is not working properly, showing this error

Traceback (most recent call last):
  File "token.py", line 1, in <module>
    from nltk.tokenize import word_tokenize, sent_tokenize
  File "D:\PYTHON36\lib\site-packages\nltk\__init__.py", line 89, in <module>
    from nltk.internals import config_java
  File "D:\PYTHON36\lib\site-packages\nltk\internals.py", line 11, in <module>
    import subprocess
  File "D:\PYTHON36\lib\subprocess.py", line 126, in <module>
    import threading
  File "D:\PYTHON36\lib\threading.py", line 7, in <module>
    from traceback import format_exc as _format_exc
  File "D:\PYTHON36\lib\traceback.py", line 5, in <module>
    import linecache
  File "D:\PYTHON36\lib\linecache.py", line 11, in <module>
    import tokenize
  File "D:\PYTHON36\lib\tokenize.py", line 35, in <module>
    from token import *
  File "C:\Users\Ali Abbas\Desktop\token\token.py", line 1, in <module>
    from nltk.tokenize import word_tokenize, sent_tokenize
  File "D:\PYTHON36\lib\site-packages\nltk\tokenize\__init__.py", line 67, in <m
odule>
    from nltk.tokenize.mwe      import MWETokenizer
  File "D:\PYTHON36\lib\site-packages\nltk\tokenize\mwe.py", line 31, in <module
>
    from nltk.util import Trie
  File "D:\PYTHON36\lib\site-packages\nltk\util.py", line 15, in <module>
    import pydoc
  File "D:\PYTHON36\lib\pydoc.py", line 72, in <module>
    from traceback import format_exception_only
ImportError: cannot import name 'format_exception_only'

----------
components: Library (Lib)
files: error.txt
messages: 315106
nosy: Ali Abbas
priority: normal
severity: normal
status: open
title: nltk is not working properly
type: compile error
versions: Python 3.6
Added file: https://bugs.python.org/file47524/error.txt

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33243>
_______________________________________

From report at bugs.python.org  Sun Apr  8 23:08:17 2018
From: report at bugs.python.org (Vignesh)
Date: Mon, 09 Apr 2018 03:08:17 +0000
Subject: [New-bugs-announce] [issue33244] Overflow error
Message-ID: <1523243297.05.0.682650639539.issue33244@psf.upfronthosting.co.za>


New submission from Vignesh <Vigneshprasanth01 at gmail.com>:

Help me to recover this.......

----------
assignee: terry.reedy
components: IDLE
files: New Doc 2018-04-09.pdf
messages: 315107
nosy: Vignesh, terry.reedy
priority: normal
severity: normal
status: open
title: Overflow error
type: behavior
versions: Python 2.7, Python 3.4
Added file: https://bugs.python.org/file47525/New Doc 2018-04-09.pdf

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33244>
_______________________________________

From report at bugs.python.org  Sun Apr  8 23:47:30 2018
From: report at bugs.python.org (Ofek Lev)
Date: Mon, 09 Apr 2018 03:47:30 +0000
Subject: [New-bugs-announce] [issue33245] Unable to send CTRL_BREAK_EVENT
Message-ID: <1523245650.34.0.682650639539.issue33245@psf.upfronthosting.co.za>


New submission from Ofek Lev <ofekmeister at gmail.com>:

Vault (https://github.com/hashicorp/vault) requires the use of signals to trigger certain output https://www.vaultproject.io/docs/internals/telemetry.html.

The required signal isn't sent on py2.7:

>>> import os
>>> import signal
>>> import psutil
>>> p = psutil.Process([p.info for p in psutil.process_iter(attrs=['pid', 'name']) if 'vault' in p.info['name']][0]['pid'])
>>> p.exe()
'C:\\Users\\Ofek\\Desktop\\vault.exe'
>>> p.pid
15536
>>> os.kill(p.pid, signal.CTRL_BREAK_EVENT)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
WindowsError: [Error 87] The parameter is incorrect

Interestingly, on py3.6 that code works but instead produces this:

OSError: [WinError 87] The parameter is incorrect

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
SystemError: <built-in function kill> returned a result with an error set

----------
components: Windows
messages: 315108
nosy: Ofekmeister, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: Unable to send CTRL_BREAK_EVENT
type: behavior
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33245>
_______________________________________

From report at bugs.python.org  Mon Apr  9 03:14:29 2018
From: report at bugs.python.org (=?utf-8?q?Markus_J=C3=A4rvisalo?=)
Date: Mon, 09 Apr 2018 07:14:29 +0000
Subject: [New-bugs-announce] =?utf-8?q?=5Bissue33246=5D_Note_in_18=2E2=2E?=
 =?utf-8?q?_json_=E2=80=94_JSON_encoder_and_decoder_is_incorrect?=
Message-ID: <1523258069.7.0.682650639539.issue33246@psf.upfronthosting.co.za>


New submission from Markus J?rvisalo <markus.jarvisalo at gmail.com>:

The note in https://docs.python.org/2/library/json.html section "18.2. json ? JSON encoder and decoder".

The note says incorrectly that JSON is a subset of YAML 1.2 but if you follow the text written in YAML 1.2 specification it states that:

"The primary objective of this revision is to bring YAML into compliance with JSON as an official subset."

So it should be that YAML is a subset of JSON and not the other way around.

----------
assignee: docs at python
components: Documentation
messages: 315111
nosy: Markus J?rvisalo, docs at python
priority: normal
severity: normal
status: open
title: Note in 18.2. json ? JSON encoder and decoder is incorrect
type: behavior
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33246>
_______________________________________

From report at bugs.python.org  Mon Apr  9 03:48:52 2018
From: report at bugs.python.org (=?utf-8?q?S=C5=82awomir_Nizio?=)
Date: Mon, 09 Apr 2018 07:48:52 +0000
Subject: [New-bugs-announce] [issue33247] File "<string>" is opened and read
 unconditionally
Message-ID: <1523260132.01.0.682650639539.issue33247@psf.upfronthosting.co.za>


New submission from S?awomir Nizio <poczta-sn at gazeta.pl>:

This may lead to e.g. DoS or data leak.

When an exception occurs, the Python interpreter displays the stack trace:

$ python3 a.py
Traceback (most recent call last):
  File "a.py", line 2, in <module>
    print(1/0)
ZeroDivisionError: division by zero

and does so also when there is no source file:

$ python3 -c 'print(1/0)'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ZeroDivisionError: division by zero

It opens the file in question to read the needed bits, and an the latter
case it tries to open file named literally <string>.

There is a possibility of a denial of service (a), data leak (b) and
other types of problems (c).

a) If "<string>" is a symbolic link for example to /dev/stdin or an
"empty" fifo, the script hangs.

$ ln -s /dev/stdin '<string>'
$ python3 -c 'print(1/0)'
Traceback (most recent call last):
  File "<string>", line 1, in <module>

# hangs here

b) If <string> is a symbolic link, its content will be read as well.
This can be a file with some kind of secret data, readable by the user
executing the script, but which should not be normally displayed to
screen (or for example sent back with HTTP to display the error message).

User who can run a certain script (for example using sudo) that calls
python -c as root or other user may be able to see parts of otherwise
inaccessible files.

c) Other attacks (or misbehaviour) is possible, for example file
<string> could be a symbolic link to a device file which upon reading
triggers some system wide action.

An interesting possiblity (if not necessarity a security issue by
itself) is a way to trick a user about the cause of the exception, like
in the example.

$ echo 'print(0/1)' > '<string>'
$ python3 -c 'print(1/0)'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
    print(0/1)
ZeroDivisionError: division by zero




As it turns out, the file doesn't need to be in the current directory
but anywhere in sys.path.

I was able to confirm the behaviour on all versions I tried: 2.7.14,
3.5.4 and 3.6.5.


This is in a way similar to the problem with Python modules loaded from
current working directory:
	https://bugs.python.org/issue16202 (sys.path[0] security issues)
but the essential differences are that:
- upon validation system (e.g. a server) may reject Python files but
leave out "<string>,"
- system can sanitize sys.path which helps with the sys.path[0] bug but
not the one being described.

----------
components: Interpreter Core
messages: 315113
nosy: snizio
priority: normal
severity: normal
status: open
title: File "<string>" is opened and read unconditionally
type: security

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33247>
_______________________________________

From report at bugs.python.org  Mon Apr  9 06:10:20 2018
From: report at bugs.python.org (Jonas Obrist)
Date: Mon, 09 Apr 2018 10:10:20 +0000
Subject: [New-bugs-announce] [issue33248] Different behavior on macos and
 linux (docker) with __await__
Message-ID: <1523268620.33.0.682650639539.issue33248@psf.upfronthosting.co.za>


New submission from Jonas Obrist <ojiidotch at gmail.com>:

The attached code runs fine on MacOS using 3.6.5 from homebrew. However on Windows (I tested on 3.6.4 with the 32bit installer from the website) and Linux (using the python:3.6.5 docker image) it errors with "TypeError: cannot 'yield from' a coroutine object in a non-coroutine generator".

On MacOS I tried both _UnixSelectorEventLoop with Kqueue and Select selectors, on Linux _UnixSelectorEventLoop with Epoll and Select.

Which behavior is the expected one?

As an aside, what I'm trying to achieve is to have an awaitable object that which if unawaited evaluates to False if used in if statements.

----------
components: asyncio
files: nta.py
messages: 315119
nosy: Jonas Obrist, asvetlov, yselivanov
priority: normal
severity: normal
status: open
title: Different behavior on macos and linux (docker) with __await__
type: behavior
versions: Python 3.6
Added file: https://bugs.python.org/file47526/nta.py

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33248>
_______________________________________

From report at bugs.python.org  Mon Apr  9 08:37:25 2018
From: report at bugs.python.org (Alexander Neumann)
Date: Mon, 09 Apr 2018 12:37:25 +0000
Subject: [New-bugs-announce] [issue33249] Pickling objects with recursive
 references and partials fail
Message-ID: <1523277445.09.0.682650639539.issue33249@psf.upfronthosting.co.za>


New submission from Alexander Neumann <aleneum at gmail.com>:

The code below causes a 

   AttributeError: 'Event' object has no attribute 'name'

It seems like the entries of `Machine.events` are not available during `__setstate__`. However, the behaviour depends on the Python version.

For Python 3.6.4 it's always the 'first' (as in it's always 'to_A' which has been added first) element in `Machine.events` that is incomplete. For Python 3.4.8 and 3.5.5 it looks like a racing condition since sometimes there is no error, sometimes its the first or the second element which is incomplete.
Letting `Machine` serve as its own model and pickling that does work for all three versions.

Tested on MacOS High Sierra.

```
import pickle
from functools import partial


class Event:

    def __init__(self, name, machine):
        self.name = name
        self.machine = machine

    def trigger(self):
        pass


class Machine:

    def __init__(self, model, states):
        self.events = {}
        self.model = model if model != 'self' else self

        for state in states:
            trigger = 'to_%s' % state
            self.events[trigger] = Event(trigger, self)
            trig_func = partial(self.events[trigger].trigger)
            setattr(self.model, trigger, trig_func)

    def __getstate__(self):
        return self.__dict__

    def __setstate__(self, state):
        self.__dict__.update(state)
        print(self.events['to_B'].name)
        print(self.events['to_A'].name)


class Model:

    def __init__(self):
        self.machine = Machine(self, states=['A', 'B'])

model = Model()
dump = pickle.dumps(model)
model2 = pickle.loads(dump)
```

----------
components: Interpreter Core
messages: 315127
nosy: aleneum
priority: normal
severity: normal
status: open
title: Pickling objects with recursive references and partials fail
type: behavior
versions: Python 3.4, Python 3.5, Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33249>
_______________________________________

From report at bugs.python.org  Mon Apr  9 09:29:07 2018
From: report at bugs.python.org (handle)
Date: Mon, 09 Apr 2018 13:29:07 +0000
Subject: [New-bugs-announce] [issue33250] Move VERSION attribute output up
 in pydoc (via help() builtin)
Message-ID: <1523280547.57.0.682650639539.issue33250@psf.upfronthosting.co.za>


New submission from handle <ke13bd at gmail.com>:

I'd like to suggest moving the output of a modules version, e.g. 

VERSION 
   __version__ 

up as it can "buried" by PACKAGE CONTENTS and DATA items.

https://github.com/python/cpython/blob/3.6/Lib/pydoc.py#L1181

The other attributes (date, author, credits) could be moved up, too, e.g. to https://github.com/python/cpython/blob/3.6/Lib/pydoc.py#L1117

Of course this depends on the use case (what are we looking for), but assuming these attributes are rather limited and constant in size, they would not be blocking access to the further items.

----------
components: Library (Lib)
messages: 315128
nosy: handle
priority: normal
severity: normal
status: open
title: Move VERSION attribute output up in pydoc (via help() builtin)
type: enhancement
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33250>
_______________________________________

From report at bugs.python.org  Mon Apr  9 16:04:26 2018
From: report at bugs.python.org (Tim Shaffer)
Date: Mon, 09 Apr 2018 20:04:26 +0000
Subject: [New-bugs-announce] [issue33251] ConfigParser.items returns items
 present in vars
Message-ID: <1523304266.46.0.682650639539.issue33251@psf.upfronthosting.co.za>


New submission from Tim Shaffer <timshaffer at me.com>:

The documentation for ConfigParser.items(section, raw=False, vars=None) says the following:

> Changed in version 3.2: Items present in vars no longer appear in the result. The previous behaviour mixed actual parser options with variables provided for interpolation.

https://docs.python.org/3/library/configparser.html#configparser.ConfigParser.items

However, this does not seem to be the case. The keys from vars are present in the output. Tested on 3.6.5.

This example shows the issue:

    import configparser

    config = configparser.ConfigParser()

    config.add_section('example')
    config.set('example', 'name', 'timster %(suffix)s')

    data = config.items('example', vars={'suffix': 'user'})

    print(data)


Expected output:

    [('name', 'timster user')]

Actual output:

    [('name', 'timster user'), ('suffix', 'user')]

----------
components: Library (Lib)
messages: 315148
nosy: timster
priority: normal
severity: normal
status: open
title: ConfigParser.items returns items present in vars
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33251>
_______________________________________

From report at bugs.python.org  Mon Apr  9 18:13:44 2018
From: report at bugs.python.org (Ed Morley)
Date: Mon, 09 Apr 2018 22:13:44 +0000
Subject: [New-bugs-announce] [issue33252] Clarify ResourceWarning
 documentation
Message-ID: <1523312024.12.0.682650639539.issue33252@psf.upfronthosting.co.za>


New submission from Ed Morley <emorley at mozilla.com>:

The `ResourceWarning` warning has always been ignored by default, since it was added in Python 3.2 in:
https://github.com/python/cpython/commit/08be72d0aa0112118b79d271479598c218adfd23#diff-db4e2b9efea108a38c53e06fa99cdd77R391

However there are several places in the docs where the "ignored by default" aspect is not mentioned even though it is for `DeprecationWarning` and similar - which caused me quite a bit of confusion today.

The docs were partly improved in Python 3.7 onwards by:
https://github.com/python/cpython/commit/9b99747386b690007027c3be2a5d7cfe3d3634f5

...however:
(a) there is still at least one place that still needs updating on master (https://docs.python.org/3.8/library/warnings.html#warning-categories)
(b) it would be good to backport the docs fixes to 3.4/3.5/3.6 (I was looking at the 3.6 docs today)

I'm happy to open PRs to fix this; but filing this issue so I have something to reference.

----------
assignee: docs at python
components: Documentation
messages: 315153
nosy: docs at python, edmorley
priority: normal
severity: normal
status: open
title: Clarify ResourceWarning documentation
type: enhancement
versions: Python 3.4, Python 3.5, Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33252>
_______________________________________

From report at bugs.python.org  Mon Apr  9 18:46:08 2018
From: report at bugs.python.org (Gorialis)
Date: Mon, 09 Apr 2018 22:46:08 +0000
Subject: [New-bugs-announce] [issue33253] xxsubtype.bench does not function
 correctly on CPython 3+
Message-ID: <1523313968.82.0.682650639539.issue33253@psf.upfronthosting.co.za>


New submission from Gorialis <sansgorialis+bpo at gmail.com>:

Due to the `OS|i` signature of `xxsubtype.bench`, it accepts `bytes` as its second argument in CPython 3+, however, it does a call `PyObject_GetAttr` which only accepts `PyString`.

This means that if you give the function `bytes`, it fails on the getattr call, but if you give it anything else, it rejects it as it doesn't match the signature.

While the module itself does not contribute to CPython's functionality (and is infact optional), since it serves as an example, it should actually function, as it does in CPython 2.

----------
components: Extension Modules
messages: 315154
nosy: Gorialis
priority: normal
severity: normal
status: open
title: xxsubtype.bench does not function correctly on CPython 3+
type: behavior
versions: Python 3.4, Python 3.5, Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33253>
_______________________________________

From report at bugs.python.org  Mon Apr  9 20:27:02 2018
From: report at bugs.python.org (Brett Cannon)
Date: Tue, 10 Apr 2018 00:27:02 +0000
Subject: [New-bugs-announce] [issue33254] importlib.resources.contents()
 incorrectly yields an empty list
Message-ID: <1523320022.42.0.682650639539.issue33254@psf.upfronthosting.co.za>


New submission from Brett Cannon <brett at python.org>:

If you look at https://github.com/python/cpython/blob/0f914b5b5f6ba186afd7112fc851c97247076f70/Lib/importlib/resources.py#L247 you will notice that the generator for importlib.resources.contents() return an empty list. While the intent was to have no values be returned by contents(), by virtue of being a generator that empty list is actually being returned as a value.

----------
assignee: brett.cannon
components: Library (Lib)
messages: 315155
nosy: barry, brett.cannon
priority: normal
severity: normal
stage: test needed
status: open
title: importlib.resources.contents() incorrectly yields an empty list
type: behavior
versions: Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33254>
_______________________________________

From report at bugs.python.org  Tue Apr 10 05:21:14 2018
From: report at bugs.python.org (=?utf-8?q?Nicol=C3=A1s_Hatcher?=)
Date: Tue, 10 Apr 2018 09:21:14 +0000
Subject: [New-bugs-announce] [issue33255] json.dumps has different behaviour
 if encoding='utf-8' or encoding='utf8'
Message-ID: <1523352074.24.0.682650639539.issue33255@psf.upfronthosting.co.za>


New submission from Nicol?s Hatcher <nicoliere at gmail.com>:

Hey I'm new here, so please let me know what incorrect things I am doing!

I _think_ `json.dumps(o, ensure_ascii=False)` is doing the wrong thing when `o` has both unicode and str keys/values. For instance:

```
import json
o = {u"greeting": "hi", "currency": "?"}
json.dumps(o, ensure_ascii=False, encoding="utf8")
json.dumps(o, ensure_ascii=False)
```

The first `dumps` will work while the second will fail. the reason is:

https://github.com/python/cpython/blob/2.7/Lib/json/encoder.py#L198

This will decode any str if the encoding is not 'utf-8'. In the mixed case (unicode and str) this will blow. I workaround is to use any of the aliases for 'utf-8' like 'utf8' or 'u8'.

I would be crazy happy to provide a PR if this is really an issue.
Let me know if extra clarification is needed.
Nicol?s

----------
components: Unicode
messages: 315164
nosy: ezio.melotti, nhatcher, vstinner
priority: normal
severity: normal
status: open
title: json.dumps has different behaviour if encoding='utf-8' or encoding='utf8'
type: behavior
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33255>
_______________________________________

From report at bugs.python.org  Tue Apr 10 07:59:39 2018
From: report at bugs.python.org (=?utf-8?q?St=C3=A9phane_Blondon?=)
Date: Tue, 10 Apr 2018 11:59:39 +0000
Subject: [New-bugs-announce] [issue33256] module is not displayed by
 cgitb.html
Message-ID: <1523361579.44.0.682650639539.issue33256@psf.upfronthosting.co.za>


New submission from St?phane Blondon <stephane.blondon at gmail.com>:

The html produced by cgitb.html() does not display <module> because the square brackets are interpreted as a html tag (see the picture in attachement).
This bug occurs if the code is called directly in the module, not inside a function or a class.

----------
components: Library (Lib)
files: traceback_cgi.gif
messages: 315167
nosy: sblondon
priority: normal
severity: normal
status: open
title: module is not displayed by cgitb.html
Added file: https://bugs.python.org/file47528/traceback_cgi.gif

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33256>
_______________________________________

From report at bugs.python.org  Tue Apr 10 12:41:09 2018
From: report at bugs.python.org (Ivan Pozdeev)
Date: Tue, 10 Apr 2018 16:41:09 +0000
Subject: [New-bugs-announce] [issue33257] Race conditions in Tkinter with
 non-threaded Tcl
Message-ID: <1523378469.47.0.682650639539.issue33257@psf.upfronthosting.co.za>


New submission from Ivan Pozdeev <ivan_pozdeev at mail.ru>:

(Marked only 2.7 as affected but this would affect any branch if built with nonthreaded Tcl.)

When running the attached TkinterCrash2-2.py repeatedly with 2.7.14 and 2.7 head, win7 x64, two kinds of errors pop up randomly:

1. Crashes and freezes.
2. Exceptions on the console like:

Exception in thread Thread-14:
Traceback (most recent call last):
  File "C:\Users\Sasha\Documents\cpython\lib\threading.py", line 801, in __boots
trap_inner
    self.run()
  File "../tkt/TkinterCrash2-2.py", line 50, in run
    self.deliverToqueue((self.target, z, y))
  File "../tkt/TkinterCrash2-2.py", line 133, in arrival_122
    new_yz[1])
  File "C:\Users\Sasha\Documents\cpython\lib\lib-tk\Tkinter.py", line 2328, in create_line
    return self._create('line', args, kw)
  File "C:\Users\Sasha\Documents\cpython\lib\lib-tk\Tkinter.py", line 2310, in _create
    *(args + self._options(cnf, kw))))
ValueError: invalid literal for int() with base 10: 'None'


Running the same code with the minimal required changes (attached as TkinterCrash3-2-2.py) under 3.6 (same platform) goes without any errors.


Diagnostics showed:

1. Under debug Python, a crash became an MSVC double-free assertion error. The stacktrace is: Tkapp_Call->Tkapp_CallDeallocArgs->Tcl_DecrRefCount->TclFreeObj->free

2. The exceptions are caused by a `<canvas> create line` Tk call randomly returning "None" (a string) instead of an integer -- which it should never do according to its doc. Since it happens inconsistently, this also suggests a race condition.

3. In Tkapp_Call and SetVar, Tcl lock isn't held when creating/destroying Tcl objects for the call. The underlying fns modify the global state (free objects list, reference counters), so it should be held.
These two are the only such places in the code.


I've fixed this, will file the PR shortly. Holding the lock when calling Tkapp_CallDeallocArgs in Tkapp_Call eliminated the crashes and freezes, locking objects' creation in the same fn decreased the number of exceptions greatly, and locking creation in SetVar, too, eliminated them completely.


I did not check if objects created with AsObj in SetVar need to be disposed of like in Tkapp_Call.

Also have no idea how to autotest the fix. If someone does, I'm all ears.

----------
components: Tkinter
files: TkinterCrash2-2.py
messages: 315172
nosy: Ivan.Pozdeev
priority: normal
severity: normal
status: open
title: Race conditions in Tkinter with non-threaded Tcl
type: crash
versions: Python 2.7
Added file: https://bugs.python.org/file47529/TkinterCrash2-2.py

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33257>
_______________________________________

From report at bugs.python.org  Tue Apr 10 13:05:03 2018
From: report at bugs.python.org (Carl)
Date: Tue, 10 Apr 2018 17:05:03 +0000
Subject: [New-bugs-announce] [issue33258] Unable to install 3.6.5 on Windows
 Server 2008
Message-ID: <1523379903.12.0.682650639539.issue33258@psf.upfronthosting.co.za>


New submission from Carl <carl.henning.ctr at dla.mil>:

Hello,

I am trying to install python version 3.6.5 on a windows 2008 2008 rc2 SP1 server.

I have tired both the installer for python-3.6.5.exe and python-3.6.5-amd64.exe installers. 

Both will not run on the server either from the gui or the command line prompt with admin privileges. It appears to run but nothing happens, no response or no error is provided. Any there any other methods to install this version?

----------
components: Windows
messages: 315173
nosy: hpo0016, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: Unable to install 3.6.5 on Windows Server 2008
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33258>
_______________________________________

From report at bugs.python.org  Tue Apr 10 17:34:20 2018
From: report at bugs.python.org (=?utf-8?q?G=C3=A9ry_Ogam?=)
Date: Tue, 10 Apr 2018 21:34:20 +0000
Subject: [New-bugs-announce] [issue33259] Encoding issue in the name of the
 local DST timezone
Message-ID: <1523396060.7.0.682650639539.issue33259@psf.upfronthosting.co.za>


New submission from G?ry Ogam <gery.ogam at gmail.com>:

There seems to be an encoding bug in Python 3.6.5 on Windows with the [timezone constant](https://docs.python.org/3/library/time.html#timezone-constants) `time.tzname`:

    >>> import time
    >>> time.tzname
    ('Paris, Madrid', 'Paris, Madrid (heure d\x92?t?)')

In the second string (the name of the local *DST* timezone), the escape sequence `\x92` is (since it is in a *character* string, not in a byte string) the Unicode code point [U+0092 PRIVATE USE 2 (PU2)](https://en.wikipedia.org/wiki/List_of_Unicode_characters), instead of the Unicode code point [U+2019 RIGHT SINGLE QUOTATION MARK](https://en.wikipedia.org/wiki/List_of_Unicode_characters) as expected, which would have been displayed as `?` or `\u2019`, so `'Paris, Madrid (heure d??t?)'`.

This `\x92` obviously comes from the 0x92 byte of the [CP-1252 encoding](https://en.wikipedia.org/wiki/Windows-1252) for the `?` character, but the byte has been badly handled in `time.tzname` somehow.

Indeed, quoting the [?Lexical analysis?](https://docs.python.org/3/reference/lexical_analysis.html#string-and-bytes-literals) chapter from the *Language Reference*:

> In a bytes literal, hexadecimal and octal escapes denote the byte with
> the given value. In a string literal, these escapes denote a Unicode
> character with the given value.

----------
components: Library (Lib)
messages: 315181
nosy: maggyero
priority: normal
severity: normal
status: open
title: Encoding issue in the name of the local DST timezone
type: behavior
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33259>
_______________________________________

From report at bugs.python.org  Wed Apr 11 03:17:24 2018
From: report at bugs.python.org (Serhiy Storchaka)
Date: Wed, 11 Apr 2018 07:17:24 +0000
Subject: [New-bugs-announce] [issue33260] Update token.py etc for ASYNC and
 AWAIT
Message-ID: <1523431044.29.0.682650639539.issue33260@psf.upfronthosting.co.za>


New submission from Serhiy Storchaka <storchaka+cpython at gmail.com>:

After making 'async' and 'await' proper keywords (issue30406), token.py was not regenerated. The proposed patch updates token.py, tokenize.py and the documentation.

----------
components: Library (Lib)
messages: 315184
nosy: serhiy.storchaka
priority: normal
severity: normal
status: open
title: Update token.py etc for ASYNC and AWAIT
type: behavior
versions: Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33260>
_______________________________________

From report at bugs.python.org  Wed Apr 11 04:59:53 2018
From: report at bugs.python.org (Jeroen Demeyer)
Date: Wed, 11 Apr 2018 08:59:53 +0000
Subject: [New-bugs-announce] [issue33261] inspect.isgeneratorfunction fails
 on hand-created methods
Message-ID: <1523437193.82.0.682650639539.issue33261@psf.upfronthosting.co.za>


New submission from Jeroen Demeyer <J.Demeyer at UGent.be>:

The inspect functions isgeneratorfunction, iscoroutinefunction, isasyncgenfunction can fail on methods that do not have a __code__ attribute:

>>> from types import MethodType
>>> class Callable:
...     def __call__(self, *args):
...         return args
>>> m = MethodType(Callable(), 42)
>>> m()
(42,)
>>> import inspect
>>> inspect.iscoroutinefunction(m)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib64/python3.6/inspect.py", line 186, in iscoroutinefunction
    object.__code__.co_flags & CO_COROUTINE)
AttributeError: 'Callable' object has no attribute '__code__'

This was discovered while working on PEP 575, but it is really an independent issue that should be fixed anyway.

----------
components: Library (Lib)
messages: 315187
nosy: jdemeyer
priority: normal
severity: normal
status: open
title: inspect.isgeneratorfunction fails on hand-created methods
type: behavior
versions: Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33261>
_______________________________________

From report at bugs.python.org  Wed Apr 11 06:28:49 2018
From: report at bugs.python.org (Christian Heimes)
Date: Wed, 11 Apr 2018 10:28:49 +0000
Subject: [New-bugs-announce] [issue33262] Deprecate shlex.split(None) to
 read from stdin.
Message-ID: <1523442529.49.0.682650639539.issue33262@psf.upfronthosting.co.za>


New submission from Christian Heimes <lists at cheimes.de>:

The shlex module implements simple tokenize for a shell-like mini language. The shlex.split() function splits a string into subcomponents just like a typical Unix shell. However function has a surprising feature. When None is passed into shlex.split().

Note: Since the split() function instantiates a shlex instance, passing None for s will read the string to split from standard input. 

https://docs.python.org/3/library/shlex.html#shlex.split


This is both surprising, unnecessary and potentially dangerous. Reading from sys.stdin is a blocking operation. In case an application doesn't account for None, shlex.split(value) could lead to a blocked server application. I suggest to deprecate and eventually remove this mis-feature.

Credits: David R. MacIver reported the bug on Twitter: https://twitter.com/DRMacIver/status/984001867985367040

----------
messages: 315189
nosy: christian.heimes
priority: normal
severity: normal
stage: needs patch
status: open
title: Deprecate shlex.split(None) to read from stdin.
type: behavior
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33262>
_______________________________________

From report at bugs.python.org  Wed Apr 11 13:13:48 2018
From: report at bugs.python.org (Vlad Starostin)
Date: Wed, 11 Apr 2018 17:13:48 +0000
Subject: [New-bugs-announce] [issue33263] Asyncio server enters an invalid
 state after a request with SO_LINGER
Message-ID: <1523466828.6.0.682650639539.issue33263@psf.upfronthosting.co.za>


New submission from Vlad Starostin <drtyrsa at yandex.ru>:

Long story short, if you have a TCP asyncio-server (implementing asyncio.Protocol) that sends something to socket in connection_made callback, it will leak fds and won't be able to handle some consequent requests, if you will send to it RST right after TCP-handshake.

In my case, these requests are sent by keepalived demon (it makes TCP-checks that way), but they can be easily hand-crafted.


Example server:

import asyncio

class EchoServerClientProtocol(asyncio.Protocol):
    def connection_made(self, transport):
        self.transport = transport
        self.transport.write(b'Say something')

    def data_received(self, data):
        self.transport.write(data)
        self.transport.close()

loop = asyncio.get_event_loop()
coro = loop.create_server(EchoServerClientProtocol, '127.0.0.1', 8888)
server = loop.run_until_complete(coro)
loop.run_forever()


Example client:

import socket
import struct
import time

# first request
sock = socket.socket()
l_onoff = 1
l_linger = 0
sock.setsockopt(socket.SOL_SOCKET, socket.SO_LINGER, struct.pack('ii', l_onoff, l_linger))
sock.connect(('127.0.0.1', 8022))
sock.close()

time.sleep(1)

# second request
sock = socket.socket()
sock.connect(('127.0.0.1', 8888))
print('Got', sock.recv(1024))
sock.sendall(b'Hi there')
print('Got', sock.recv(1024))  # <-- Will hang here
sock.close()


Why is this happening?
1. First request
  1.1. _SelectorSocketTransport.__init__ schedules connection_made and loop.add_reader.
  1.2. connection_made tries to send data to the socket, it fails, _SelectorTransport._fatal_error is called, then _SelectorTransport._force_close is called.
  1.3. _SelectorTransport._force_close removes the reader from the loop (there is none, it's ok), and schedules closing the socket in _SelectorTransport._call_connection_lost.
  1.4. loop.add_reader is called (scheduled on step 1), it calls epoll_ctl, no error is returned because the socket isn't closed yet.
  1.5. the socket is closed by _SelectorTransport._call_connection_lost (scheduled on step 3). The corresponding entry in _SelectorMapping remains and isn't cleaned up.

2. Second request
  2.1. FD from the first request is reused by the OS
  2.2. BaseSelectorEventLoop._add_reader sees that the FD is already in BaseSelectorEventLoop._selector, so it calls _selector.modify
  2.3 _BaseSelectorImpl.modify sees that the events mask is the same, so it uses a shortcut, changing only the callback, but without calling epoll_ctl. The socket won't be polled.

I think the source of the error is in step 1.4, we shouldn't add the reader if we are already closing the transport. I've made a simple PR fixing this, but maybe there are other options here.

----------
components: asyncio
messages: 315196
nosy: asvetlov, drtyrsa, giampaolo.rodola, yselivanov
priority: normal
severity: normal
status: open
title: Asyncio server enters an invalid state after a request with SO_LINGER
type: behavior
versions: Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33263>
_______________________________________

From report at bugs.python.org  Wed Apr 11 18:57:04 2018
From: report at bugs.python.org (=?utf-8?q?Andr=C3=A9s_Delfino?=)
Date: Wed, 11 Apr 2018 22:57:04 +0000
Subject: [New-bugs-announce] [issue33264] Remove to-be-deprecated
 urllib.request.urlretrieve function reference from HOWTO
Message-ID: <1523487424.62.0.682650639539.issue33264@psf.upfronthosting.co.za>


New submission from Andr?s Delfino <adelfino at gmail.com>:

Documentation says urllib.request.urlretrieve "might become deprecated at some point in the future".

IMHO, it's better to provide examples that do not make users rely on API that has this label.

I propose the example in the pull request to accomplish exactly the same task while not relying on urllib.request.urlretrieve.

----------
assignee: docs at python
components: Documentation
messages: 315204
nosy: adelfino, docs at python
priority: normal
severity: normal
status: open
title: Remove to-be-deprecated urllib.request.urlretrieve function reference from HOWTO
type: enhancement
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33264>
_______________________________________

From report at bugs.python.org  Thu Apr 12 01:24:51 2018
From: report at bugs.python.org (Jeroen Demeyer)
Date: Thu, 12 Apr 2018 05:24:51 +0000
Subject: [New-bugs-announce] [issue33265] contextlib.ExitStack abuses
 __self__
Message-ID: <1523510691.14.0.682650639539.issue33265@psf.upfronthosting.co.za>


New submission from Jeroen Demeyer <J.Demeyer at UGent.be>:

In contextlib, there is code which roughly looks like

        def _exit_wrapper(exc_type, exc, tb):
            return cm_exit(cm, exc_type, exc, tb)
        _exit_wrapper.__self__ = cm

This creates a new function _exit_wrapper from a given function cm_exit by prepending the __self__ attribute to *args. Now this is exactly what a method does too.

It would be better to use an actual method for this: it's cleaner, faster and it doesn't abuse a double-underscore attribute. The latter will actually break with PEP 575, as __self__ will become a special name  instead of an arbitrary attribute.

----------
components: Library (Lib)
messages: 315212
nosy: jdemeyer
priority: normal
severity: normal
status: open
title: contextlib.ExitStack abuses __self__

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33265>
_______________________________________

From report at bugs.python.org  Thu Apr 12 04:24:59 2018
From: report at bugs.python.org (Zsolt Dollenstein)
Date: Thu, 12 Apr 2018 08:24:59 +0000
Subject: [New-bugs-announce] [issue33266] 2to3 doesn't parse all valid
 string literals
Message-ID: <1523521499.6.0.682650639539.issue33266@psf.upfronthosting.co.za>


New submission from Zsolt Dollenstein <zsol.zsol at gmail.com>:

For example `rf"blah"` is not tokenized properly.

----------
components: 2to3 (2.x to 3.x conversion tool)
messages: 315215
nosy: Zsolt Dollenstein
priority: normal
severity: normal
status: open
title: 2to3 doesn't parse all valid string literals
type: behavior

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33266>
_______________________________________

From report at bugs.python.org  Thu Apr 12 04:45:09 2018
From: report at bugs.python.org (Eric Wieser)
Date: Thu, 12 Apr 2018 08:45:09 +0000
Subject: [New-bugs-announce] [issue33267] ctypes array types create
 reference cycles
Message-ID: <1523522709.53.0.682650639539.issue33267@psf.upfronthosting.co.za>


New submission from Eric Wieser <wieser.eric+pybug at gmail.com>:

Discovered in https://github.com/numpy/numpy/pull/10882/files#r180813166

A reproduction:
```
In [1]: import ctypes

In [2]: def make_array_ctype(shape):
   ...:     import ctypes
   ...:     ct = ctypes.c_uint8
   ...:     for i in shape:
   ...:         ct = i * ct
   ...:     return ct
   ...:

# all on one line to keep ipython out of this
In [3]: gc.collect(); x = make_array_ctype((1,)); del x; gc.collect()
```

Using the proposed function in https://github.com/numpy/numpy/pull/10891, we get a few more details:
```
In [4]: from numpy.testing import assert_no_gc_cycles

In [5]: assert_no_gc_cycles(make_array_ctype, (1,))
AssertionError: Reference cycles were found when calling make_array_ctype: 7 objects were collected, of which 6 are shown below:
  tuple object with id=2822255556536:
    (<class '_ctypes.Array'>,)
  PyCArrayType object with id=2822226500408:
    <class '__main__.c_ubyte_Array_1'>
  getset_descriptor object with id=2822252062256:
    <attribute '__dict__' of 'c_ubyte_Array_1' objects>
  getset_descriptor object with id=2822252062184:
    <attribute '__weakref__' of 'c_ubyte_Array_1' objects>
  tuple object with id=2822243712440:
    (<class '__main__.c_ubyte_Array_1'>,
     <class '_ctypes.Array'>,
     <class '_ctypes._CData'>,
     <class 'object'>)
  StgDict object with id=2822226211928:
    {'__dict__': <attribute '__dict__' of 'c_ubyte_Array_1' objects>,
     '__doc__': None,
     '__module__': '__main__',
     '__weakref__': <attribute '__weakref__' of 'c_ubyte_Array_1' objects>,
     '_length_': 1,
     '_type_': <class 'ctypes.c_ubyte'>}
```

I suppose this isn't really a bug, but it's not clear to me why a cycle needs to be created here.

----------
components: ctypes
messages: 315216
nosy: Eric.Wieser
priority: normal
severity: normal
status: open
title: ctypes array types create reference cycles
versions: Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33267>
_______________________________________

From report at bugs.python.org  Thu Apr 12 05:55:09 2018
From: report at bugs.python.org (tkhyn)
Date: Thu, 12 Apr 2018 09:55:09 +0000
Subject: [New-bugs-announce] [issue33268] iteration over attrs in metaclass'
 __new__ affects class' __name__
Message-ID: <1523526909.26.0.682650639539.issue33268@psf.upfronthosting.co.za>


New submission from tkhyn <thomas.khyn_python at m4x.org>:

The following script, run with python 3.6.5, highlights an issue with the class' __name__ attribute being set incorrectly just because of a loop in the metaclass' __new__ method:

class MC(type):
    def __new__(mcs, name, bases, attrs):
        for name, attr in attrs.items():
            pass
        return super(MC, mcs).__new__(mcs, name, bases, attrs)

class C(metaclass=MC):
    a = None

print(C.__name__)


Expected output: "C"
Actual output: "a"

Comment the for loop and you get the expected output!

On Python 2.7.13, the amended code exposes the same bug:

class MC(type):
    def __new__(mcs, name, bases, attrs):
        for name, attr in attrs.items():
            pass
        return super(MC, mcs).__new__(mcs, name, bases, attrs)

class C(object):
    __metaclass__ = MC
    a = None

print C.__name__

output is "__metaclass__" and should be "C"

----------
components: Interpreter Core
messages: 315218
nosy: tkhyn
priority: normal
severity: normal
status: open
title: iteration over attrs in metaclass' __new__ affects class' __name__
versions: Python 2.7, Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33268>
_______________________________________

From report at bugs.python.org  Thu Apr 12 10:55:59 2018
From: report at bugs.python.org (Kadir Haldenbilen)
Date: Thu, 12 Apr 2018 14:55:59 +0000
Subject: [New-bugs-announce] [issue33269] InteractiveConsole behaves
 differently when used on terminal and used within script
Message-ID: <1523544959.58.0.682650639539.issue33269@psf.upfronthosting.co.za>


New submission from Kadir Haldenbilen <khaldenbilen at gmail.com>:

on terminal push and runcode accepts indentation where required (like for loop etc), within script gives error message

on terminal import works properly and objects can be found  as normal, within script you may need to add module name upfront

simple example
import code
ii = code.InteractiveConsole()
ii.push("for ii in range(3):")
ii.push("    print('i', i)")

you will get normal expected output on terminal, but indentation error within script

ii.push("from time import sleep")
ii.push("sleep(1)")
will sleep 1 sec on terminal, but will give name error

----------
messages: 315224
nosy: Kadir Haldenbilen
priority: normal
severity: normal
status: open
title: InteractiveConsole behaves differently when used on terminal and used within script
type: behavior
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33269>
_______________________________________

From report at bugs.python.org  Thu Apr 12 16:32:14 2018
From: report at bugs.python.org (Daniel Moisset)
Date: Thu, 12 Apr 2018 20:32:14 +0000
Subject: [New-bugs-announce] [issue33270] tags for anonymous code objects
 should be interned
Message-ID: <1523565134.3.0.682650639539.issue33270@psf.upfronthosting.co.za>


New submission from Daniel Moisset <dmoisset at machinalis.com>:

In compile.c, some strings are created to be used as name of anonymous code objects: "<module>", "<lambda>", "<genexpr>", "<listcomp>", "<setcomp>" and "<dictcomp>".

Only the first two of those are interned (created with PyUnicode_InternFromString ), the rest are created with "PyUnicode_FromString". Even if they use a static variable to avoid creating multiple instances, not interning results in multiple copies when code objects are marshaled into pyc files and reloaded (one copy per module).

Always using PyUnicode_InternFromString should be more consistent, and slightly more efficient

----------
components: Interpreter Core
messages: 315232
nosy: Daniel Moisset
priority: normal
severity: normal
status: open
title: tags for anonymous code objects should be interned

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33270>
_______________________________________

From report at bugs.python.org  Fri Apr 13 03:39:34 2018
From: report at bugs.python.org (Michael McCoy)
Date: Fri, 13 Apr 2018 07:39:34 +0000
Subject: [New-bugs-announce] [issue33271] Exception handling matches
 subtypes, not subclasses
Message-ID: <1523605174.57.0.682650639539.issue33271@psf.upfronthosting.co.za>


New submission from Michael McCoy <michael.b.mccoy at gmail.com>:

Exception handling matches subtypes, not subclasses


# Example


    from abc import ABC
    
    class MyException(Exception, ABC):
        pass
    
    class OtherException(Exception):
        pass
    
    MyException.register(OtherException)
    
    try:
        raise OtherException
    except MyException:
        print("Correct: Caught MyException")
    except Exception:
        print("Wrong: Caught something else")
        
    # "Wrong: Caught something else"


# Background and evidence of bug-ness

Issue 2534 [1] (10 years ago!) introduced the behavior, but only in the Python 3 patch [2]. During code review, the correct function call was used [3], but the function's name got switched in the final python3 patch without any comment.

The current Python 2 code uses `PyObject_IsSubclass`, and produces the correct behavior in the example above (using `__metaclass__ = ABCMeta`, of course). This leads me to strongly suspect that this is a bug, not a feature. The note below regarding unittest for further evidence that this code has eight legs.

Given the ancient nature of this bug, it affects all versions of python3.

[1] https://bugs.python.org/issue2534
[2] https://bugs.python.org/file11257/isinstance3k-2.patch
[3] https://codereview.appspot.com/483/diff/1/21#newcode114
[4] https://github.com/python/cpython/blob/2.7/Python/errors.c#L119


# Solution

Coming very soon in a PR on Github, but in short, we do the following:

  1. Switch `PyType_IsSubtype` to  `PyObject_IsSubclass`.
  2. Revert the changes made to remove ?dead code? in https://bugs.python.org/issue31091. The code was dead because the wrong function was used?the PR left only the bug.
  3. Add tests. Note that `unittest`?s `self.assertRaises` function uses `issubclass` and does not alert to this bug. (Different symptom, same cause.)


# Note

This bug has nothing to do with the `abc` package, beyond being a simple way to generate the error.


-Mike

Gitub: mbmccoy

----------
components: Interpreter Core
messages: 315241
nosy: Michael McCoy
priority: normal
severity: normal
status: open
title: Exception handling matches subtypes, not subclasses
versions: Python 3.4, Python 3.5, Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33271>
_______________________________________

From report at bugs.python.org  Fri Apr 13 07:12:03 2018
From: report at bugs.python.org (Mikhail)
Date: Fri, 13 Apr 2018 11:12:03 +0000
Subject: [New-bugs-announce] [issue33272] Which are reasonable reason for
 recursion limit in function _vformat of class Formatter lib string?
Message-ID: <1523617923.76.0.682650639539.issue33272@psf.upfronthosting.co.za>


New submission from Mikhail <mikhail.v.gavrilov at gmail.com>:

The presence of the restriction of recursion prevent making nested conditions for the superformatter: https://github.com/ebrehault/superformatter

for example:

import string


class SuperFormatter(string.Formatter):
    """World's simplest Template engine."""

    def format_field(self, value, spec):
        if spec.startswith('repeat'):
            template = spec.partition(':')[-1]
            if type(value) is dict:
                value = value.items()
            return ''.join([template.format(item=item) for item in value])
        elif spec == 'call':
            return value()
        elif spec.startswith('if'):
            return (value and spec.partition(':')[-1]) or ''
        else:
            return super(SuperFormatter, self).format_field(value, spec)


data = {
	'a1':1,
	'a2':2,
	'a3':3
}

tmpl = '''
{a1:if:
	{a2:if:
		{a2}
	}
	{a1}
}
'''

sf = SuperFormatter()
out = sf.format(tmpl, **data)
print(out)


Causes error:

$ python3.6 my_progs/test.py 
Traceback (most recent call last):
  File "my_progs/test.py", line 37, in <module>
    out = sf.format(tmpl, **data)
  File "/usr/lib64/python3.6/string.py", line 190, in format
    return self.vformat(format_string, args, kwargs)
  File "/usr/lib64/python3.6/string.py", line 194, in vformat
    result, _ = self._vformat(format_string, args, kwargs, used_args, 2)
  File "/usr/lib64/python3.6/string.py", line 244, in _vformat
    auto_arg_index=auto_arg_index)
  File "/usr/lib64/python3.6/string.py", line 244, in _vformat
    auto_arg_index=auto_arg_index)
  File "/usr/lib64/python3.6/string.py", line 244, in _vformat
    auto_arg_index=auto_arg_index)
  File "/usr/lib64/python3.6/string.py", line 201, in _vformat
    raise ValueError('Max string recursion exceeded')
ValueError: Max string recursion exceeded


If there was not this restriction, then the example code is worked.

----------
components: Library (Lib)
messages: 315242
nosy: mv.gavrilov
priority: normal
severity: normal
status: open
title: Which are reasonable reason for recursion limit in function _vformat of class Formatter lib string?
type: behavior
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33272>
_______________________________________

From report at bugs.python.org  Fri Apr 13 10:33:23 2018
From: report at bugs.python.org (Julian DeMille)
Date: Fri, 13 Apr 2018 14:33:23 +0000
Subject: [New-bugs-announce] [issue33273] Allow multiple imports from one
 module while preserving its namespace
Message-ID: <1523630003.29.0.682650639539.issue33273@psf.upfronthosting.co.za>


New submission from Julian DeMille <julian.demille at demilletech.net>:

An example of this would be to have something like `import <insert lib here>.{ <mod1>, <mod2>, ... }`

----------
components: Interpreter Core
messages: 315247
nosy: jdemilledt
priority: normal
severity: normal
status: open
title: Allow multiple imports from one module while preserving its namespace
type: enhancement

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33273>
_______________________________________

From report at bugs.python.org  Fri Apr 13 14:36:43 2018
From: report at bugs.python.org (Ari Krupnik)
Date: Fri, 13 Apr 2018 18:36:43 +0000
Subject: [New-bugs-announce] [issue33274] minidom removeAttributeNode
 returns None
Message-ID: <1523644603.75.0.682650639539.issue33274@psf.upfronthosting.co.za>


New submission from Ari Krupnik <ari at lib.aero>:

W3C DOM Level 1[1] requires removeAttributeNode() to return the removed node:

removeAttributeNode: Removes the specified attribute.
Return Value: The Attr node that was removed.

Minidom implementation returns None.

[1]https://www.w3.org/TR/1998/REC-DOM-Level-1-19981001/level-one-core.html#method-removeAttributeNode

----------
components: Library (Lib), XML
messages: 315253
nosy: iter
priority: normal
pull_requests: 6157
severity: normal
status: open
title: minidom removeAttributeNode returns None
versions: Python 2.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33274>
_______________________________________

From report at bugs.python.org  Fri Apr 13 14:38:46 2018
From: report at bugs.python.org (Ben FrantzDale)
Date: Fri, 13 Apr 2018 18:38:46 +0000
Subject: [New-bugs-announce] [issue33275] glob.glob should explicitly note
 that results aren't sorted
Message-ID: <1523644726.46.0.682650639539.issue33275@psf.upfronthosting.co.za>


New submission from Ben FrantzDale <ben at formlabs.com>:

The sortedness of glob.glob's output is platform-dependent. While the docs do not mention sorting, and so are strictly correct, if you are on a platform where its output is sorted, it's easy to believe that the output is always sorted.

I propose we a Note maybe next to "Note: Using the ?**? pattern in large directory trees may consume an inordinate amount of time." that says "Note: While the output of glob.glob may be sorted on some architectures, ordering is not guaranteed. Use `sort(glob.glob(...))` if ordering is important."

This wrong assumption burned us when scripts inexplicably stopped working on OSX High Sierra.

----------
components: Library (Lib)
messages: 315254
nosy: Ben FrantzDale
priority: normal
severity: normal
status: open
title: glob.glob should explicitly note that results aren't sorted
type: enhancement
versions: Python 2.7, Python 3.4, Python 3.5, Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33275>
_______________________________________

From report at bugs.python.org  Fri Apr 13 15:08:16 2018
From: report at bugs.python.org (Brett Cannon)
Date: Fri, 13 Apr 2018 19:08:16 +0000
Subject: [New-bugs-announce] [issue33276] Clarify that __path__ can't be set
 to just anything
Message-ID: <1523646496.5.0.682650639539.issue33276@psf.upfronthosting.co.za>


New submission from Brett Cannon <brett at python.org>:

https://docs.python.org/3/reference/import.html#module-path says that "regardless of its value", __path__'s mere existence is key. But then later the docs say "__path__ must be an iterable of strings, but it may be empty". Technically the latter is more accurate, so the docs should get touched up to match itself better.

----------
assignee: brett.cannon
components: Documentation
keywords: easy
messages: 315256
nosy: brett.cannon
priority: normal
severity: normal
stage: needs patch
status: open
title: Clarify that __path__ can't be set to just anything
versions: Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33276>
_______________________________________

From report at bugs.python.org  Fri Apr 13 17:48:22 2018
From: report at bugs.python.org (Brett Cannon)
Date: Fri, 13 Apr 2018 21:48:22 +0000
Subject: [New-bugs-announce] [issue33277] Deprecate __loader__, __package__,
 __file__, and __cached__ on modules
Message-ID: <1523656102.14.0.682650639539.issue33277@psf.upfronthosting.co.za>


New submission from Brett Cannon <brett at python.org>:

If you look at https://docs.python.org/3/reference/import.html#import-related-module-attributes you will notice there are a lot of attributes on modules. But since the introduction of module specs (https://docs.python.org/3/library/importlib.html#importlib.machinery.ModuleSpec), all of those attributes became redundant.

Now some attributes can never be taken away based on common idioms in the Python community. __name__ can't go due to `if __name__ == '__main__'`. __path__ can't go due to people manipulating it with e.g. pkg_resources or pkgutil. The rest, though, don't have such strong idioms tied to them (and in the case of __package__, has actually been made officially redundant by the import system since Python 3.6). That means setting them is unnecessary and any time someone wants to read or potentially manipulate them they have to choose between those attributes or the spec (or update both). Either way it can lead to bugs (I know I have nearly forgotten to set both the spec and the attribute before).

I don't know if we can easily deprecate these attributes using __getattr__ on modules as added in Python 3.7, or if this is more of a documentation thing and need a long period of clearly messaging that these redundant attributes will no longer be set at some point.

----------
components: Library (Lib)
messages: 315269
nosy: brett.cannon, eric.snow
priority: normal
severity: normal
stage: test needed
status: open
title: Deprecate __loader__, __package__, __file__, and __cached__ on modules
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33277>
_______________________________________

From report at bugs.python.org  Sat Apr 14 10:29:15 2018
From: report at bugs.python.org (Marc-Andre Lemburg)
Date: Sat, 14 Apr 2018 14:29:15 +0000
Subject: [New-bugs-announce] [issue33278] libexpat uses
 HAVE_SYSCALL_GETRANDOM instead of HAVE_GETRANDOM_SYSCALL
Message-ID: <1523716155.58.0.682650639539.issue33278@psf.upfronthosting.co.za>


New submission from Marc-Andre Lemburg <mal at egenix.com>:

See https://github.com/python/cpython/blob/3.6/Modules/expat/xmlparse.c#L87

The Python configure script tests and sets the variable HAVE_GETRANDOM_SYSCALL.

The solution would be to have Python's config script define HAVE_SYSCALL_GETRANDOM as well, in case it detects the function.

----------
components: XML
messages: 315289
nosy: lemburg
priority: normal
severity: normal
status: open
title: libexpat uses HAVE_SYSCALL_GETRANDOM instead of HAVE_GETRANDOM_SYSCALL
versions: Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33278>
_______________________________________

From report at bugs.python.org  Sat Apr 14 14:32:01 2018
From: report at bugs.python.org (Dan Snider)
Date: Sat, 14 Apr 2018 18:32:01 +0000
Subject: [New-bugs-announce] [issue33279] Py_Build value is causing crashes
 with the "L" and "K" format characters when upcasting sub 64-bit integer
 types
Message-ID: <1523730721.0.0.682650639539.issue33279@psf.upfronthosting.co.za>


New submission from Dan Snider <mr.assume.away at gmail.com>:

New to both C and the CPython api so I'm not sure what compiler details matter but this is with VS14 and default compile parameters on windows 10 with 32bit Python.

The following function:

static PyObject *spam(PyObject *module) {
    digit w = 9;
    int x = 9;
    long y = 9;
    int32_t z = 9;
    return Py_BuildValue("(HIK)(lll)(LLL)", w,w,w,x,y,z,x,y,z);  
}

returns:

>>> spam()
((9, 9, 38654705673), (9, 9, 9), (38654705673, 438244858264171835, 6133859984))

Above was the most I could fit in a single Py_BuildValue call without causing a crash (it instantly crashes if "O" or "N" fields are present). I believe this happens because the ints are being upcasted as pointers, which is undefined behavior.

----------
components: Interpreter Core, Windows
messages: 315300
nosy: bup, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: Py_Build value is causing crashes with the "L" and "K" format characters when upcasting sub 64-bit integer types
type: crash
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33279>
_______________________________________

From report at bugs.python.org  Sat Apr 14 16:12:42 2018
From: report at bugs.python.org (=?utf-8?q?Andr=C3=A9s_Delfino?=)
Date: Sat, 14 Apr 2018 20:12:42 +0000
Subject: [New-bugs-announce] [issue33280] Update link to Tcl/Tk 8.6 man
 pages in tkinter.rst
Message-ID: <1523736762.33.0.682650639539.issue33280@psf.upfronthosting.co.za>


New submission from Andr?s Delfino <adelfino at gmail.com>:

There are two links to the Tcl/Tk documentation on tkinter.rst. One of them points to 8.5 man pages.

----------
assignee: docs at python
components: Documentation
messages: 315303
nosy: adelfino, docs at python
priority: normal
severity: normal
status: open
title: Update link to Tcl/Tk 8.6 man pages in tkinter.rst
type: enhancement
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33280>
_______________________________________

From report at bugs.python.org  Sun Apr 15 01:24:20 2018
From: report at bugs.python.org (Ian Burgwin)
Date: Sun, 15 Apr 2018 05:24:20 +0000
Subject: [New-bugs-announce] [issue33281] ctypes.util.find_library not
 working on macOS
Message-ID: <1523769860.6.0.682650639539.issue33281@psf.upfronthosting.co.za>


New submission from Ian Burgwin <ihaveamac at icloud.com>:

On Python 3.7.0a4 and later (including 3.7.0b4), find_library currently always returns None on macOS. It works on 3.7.0a3 and earlier. Tested on macOS 10.11 and 10.13.

Expected result: Tested on 3.6.5, 3.7.0a1 and 3.7.0a3:

>>> from ctypes.util import find_library
>>> find_library('iconv')
'/usr/lib/libiconv.dylib'
>>> find_library('c')
'/usr/lib/libc.dylib'
>>>

Current output on 3.7.0a4 to 3.7.0b3:

>>> from ctypes.util import find_library
>>> find_library('iconv')
>>> find_library('c')
>>>

----------
components: ctypes
messages: 315309
nosy: Ian Burgwin (ihaveahax)
priority: normal
severity: normal
status: open
title: ctypes.util.find_library not working on macOS
type: behavior
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33281>
_______________________________________

From report at bugs.python.org  Sun Apr 15 07:25:34 2018
From: report at bugs.python.org (Jatin Goel)
Date: Sun, 15 Apr 2018 11:25:34 +0000
Subject: [New-bugs-announce] [issue33282] Subprocess Popen communicate hung
 if stdin not given to Popen, even though script not expecting any input
Message-ID: <1523791534.02.0.682650639539.issue33282@psf.upfronthosting.co.za>


New submission from Jatin Goel <jatingoel1037 at gmail.com>:

Hi
needed a small help related to the subprocess module
I'm executing a powershell process using the subprocess module

process = subprocess.Popen(
    [
        'powershell.exe',
        script
    ],
    stdout=subprocess.PIPE,
    stderr=subprocess.PIPE,
    shell=True
)

output, error = process.communicate()

but the call to the communicate method is getting hung on some machines, and not all

i tried to debug the subprocess module but couldn't find anything helpful

the script is not expecting any input from the user

on the machines where it failed, if I just add stdin=subprocess.PIPE argument, it works

process = subprocess.Popen(
    [
        'powershell.exe',
        script
    ],
    stdin=subprocess.PIPE,
    stdout=subprocess.PIPE,
    stderr=subprocess.PIPE,
    shell=True
)

above works

----------
components: Windows
messages: 315317
nosy: JatinGoel, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: Subprocess Popen communicate hung if stdin not given to Popen, even though script not expecting any input
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33282>
_______________________________________

From report at bugs.python.org  Sun Apr 15 10:23:12 2018
From: report at bugs.python.org (=?utf-8?q?Andr=C3=A9s_Delfino?=)
Date: Sun, 15 Apr 2018 14:23:12 +0000
Subject: [New-bugs-announce] [issue33283] Mention PNG as a supported image
 format by Tcl/Tk
Message-ID: <1523802192.37.0.682650639539.issue33283@psf.upfronthosting.co.za>


New submission from Andr?s Delfino <adelfino at gmail.com>:

PNG is supported since Tcl/Tk 8.6. Update the doc/source accordingly.

----------
assignee: docs at python
components: Documentation
messages: 315324
nosy: adelfino, docs at python
priority: normal
severity: normal
status: open
title: Mention PNG as a supported image format by Tcl/Tk
type: enhancement
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33283>
_______________________________________

From report at bugs.python.org  Sun Apr 15 14:14:55 2018
From: report at bugs.python.org (Barry Devlin)
Date: Sun, 15 Apr 2018 18:14:55 +0000
Subject: [New-bugs-announce] [issue33284] Increase test coverage for
 numbers.py
Message-ID: <1523816095.6.0.682650639539.issue33284@psf.upfronthosting.co.za>


New submission from Barry Devlin <barrypatrickdevlin at gmail.com>:

The __bool__ method in the complex class in numbers is not tested.

----------
components: Tests
messages: 315337
nosy: Barry Devlin
priority: normal
severity: normal
status: open
title: Increase test coverage for numbers.py
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33284>
_______________________________________

From report at bugs.python.org  Sun Apr 15 17:47:49 2018
From: report at bugs.python.org (Bram Borger-Johnson)
Date: Sun, 15 Apr 2018 21:47:49 +0000
Subject: [New-bugs-announce] [issue33285] pip upgrade runtime crash
Message-ID: <1523828869.12.0.682650639539.issue33285@psf.upfronthosting.co.za>


New submission from Bram Borger-Johnson <Bramnet at live.com>:

I was using pip to install a package when I got a warning message that a new version of pip was available.
I ran the prompted command and got the following back.
E:\GitHub\discord_feedbot>pip install --upgrade pip
Collecting pip
  Downloading pip-10.0.0-py2.py3-none-any.whl (1.3MB)
    100% |????????????????????????????????| 1.3MB 654kB/s
Installing collected packages: pip
  Found existing installation: pip 9.0.1
    Uninstalling pip-9.0.1:
Exception:
Traceback (most recent call last):
  File "e:\programs\python36-32\lib\shutil.py", line 544, in move
    os.rename(src, real_dst)
OSError: [WinError 17] The system cannot move the file to a different disk drive: 'e:\\programs\\python36-32\\scripts\\pip.exe' -> 'C:\\Users\\Bram\\AppData\\Local\\Temp\\pip-nahcxphl-uninstall\\programs\\python36-32\\scripts\\pip.exe'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "e:\programs\python36-32\lib\site-packages\pip\basecommand.py", line 215, in main
  File "e:\programs\python36-32\lib\site-packages\pip\commands\install.py", line 342, in run
  File "e:\programs\python36-32\lib\site-packages\pip\req\req_set.py", line 778, in install
  File "e:\programs\python36-32\lib\site-packages\pip\req\req_install.py", line 754, in uninstall
  File "e:\programs\python36-32\lib\site-packages\pip\req\req_uninstall.py", line 115, in remove
  File "e:\programs\python36-32\lib\site-packages\pip\utils\__init__.py", line 267, in renames
  File "e:\programs\python36-32\lib\shutil.py", line 559, in move
    os.unlink(src)
PermissionError: [WinError 5] Access is denied: 'e:\\programs\\python36-32\\scripts\\pip.exe'

Now, whenever I run pip at all, it results in 
Traceback (most recent call last):
  File "e:\programs\python36-32\lib\runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "e:\programs\python36-32\lib\runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "E:\Programs\Python36-32\Scripts\pip.exe\__main__.py", line 5, in <module>
ModuleNotFoundError: No module named 'pip'

I sent an email help at python.org who directed me to create a bug report, stating that it looks like the people behind pip "didn't expect your temp directory to be on a different drive from your library directory."

----------
components: Windows
messages: 315344
nosy: Bram Borger-Johnson, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: pip upgrade runtime crash
type: crash
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33285>
_______________________________________

From report at bugs.python.org  Mon Apr 16 06:21:46 2018
From: report at bugs.python.org (schwemro)
Date: Mon, 16 Apr 2018 10:21:46 +0000
Subject: [New-bugs-announce] [issue33286] Conflict between tqdm and
 multiprocessing on windows
Message-ID: <1523874106.93.0.682650639539.issue33286@psf.upfronthosting.co.za>


New submission from schwemro <rschwemmle at yahoo.de>:

Apparently, there occurs a conflict between tqdm and multiprocessing. There is an AttributeError displayed. I provide here an minimal working example. It works on UNIX but unfortunately not on windows.

----------
components: Windows
files: tqdm_multi.py
messages: 315353
nosy: paul.moore, schwemro, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: Conflict between tqdm and multiprocessing on windows
type: compile error
versions: Python 3.6
Added file: https://bugs.python.org/file47537/tqdm_multi.py

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33286>
_______________________________________

From report at bugs.python.org  Mon Apr 16 08:37:45 2018
From: report at bugs.python.org (antonio95100)
Date: Mon, 16 Apr 2018 12:37:45 +0000
Subject: [New-bugs-announce] [issue33287] "pip iinstall packageName.py" fails
Message-ID: <1523882265.62.0.682650639539.issue33287@psf.upfronthosting.co.za>


New submission from antonio95100 <antonio95100 at gmail.com>:

Hi all. I'm struggling with the installation of a pyton package under Ubuntu 16.4 and Python 7. I follow the statements of the pakage authors, but I obtaint the following errors (note that I have the same error for different packages installation attempt). Any suggestion is more than welcome.

$ pip install path.py
Collecting path.py
Exception:
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/pip/basecommand.py", line 209, in main
    status = self.run(options, args)
  File "/usr/lib/python2.7/dist-packages/pip/commands/install.py", line 328, in run
    wb.build(autobuilding=True)
  File "/usr/lib/python2.7/dist-packages/pip/wheel.py", line 748, in build
    self.requirement_set.prepare_files(self.finder)
  File "/usr/lib/python2.7/dist-packages/pip/req/req_set.py", line 360, in prepare_files
    ignore_dependencies=self.ignore_dependencies))
  File "/usr/lib/python2.7/dist-packages/pip/req/req_set.py", line 512, in _prepare_file
    finder, self.upgrade, require_hashes)
  File "/usr/lib/python2.7/dist-packages/pip/req/req_install.py", line 273, in populate_link
    self.link = finder.find_requirement(self, upgrade)
  File "/usr/lib/python2.7/dist-packages/pip/index.py", line 442, in find_requirement
    all_candidates = self.find_all_candidates(req.name)
  File "/usr/lib/python2.7/dist-packages/pip/index.py", line 400, in find_all_candidates
    for page in self._get_pages(url_locations, project_name):
  File "/usr/lib/python2.7/dist-packages/pip/index.py", line 545, in _get_pages
    page = self._get_page(location)
  File "/usr/lib/python2.7/dist-packages/pip/index.py", line 648, in _get_page
    return HTMLPage.get_page(link, session=self.session)
  File "/usr/lib/python2.7/dist-packages/pip/index.py", line 757, in get_page
    "Cache-Control": "max-age=600",
  File "/usr/share/python-wheels/requests-2.9.1-py2.py3-none-any.whl/requests/sessions.py", line 480, in get
    return self.request('GET', url, **kwargs)
  File "/usr/lib/python2.7/dist-packages/pip/download.py", line 378, in request
    return super(PipSession, self).request(method, url, *args, **kwargs)
  File "/usr/share/python-wheels/requests-2.9.1-py2.py3-none-any.whl/requests/sessions.py", line 468, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/share/python-wheels/requests-2.9.1-py2.py3-none-any.whl/requests/sessions.py", line 576, in send
    r = adapter.send(request, **kwargs)
  File "/usr/share/python-wheels/CacheControl-0.11.5-py2.py3-none-any.whl/cachecontrol/adapter.py", line 46, in send
    resp = super(CacheControlAdapter, self).send(request, **kw)
  File "/usr/share/python-wheels/requests-2.9.1-py2.py3-none-any.whl/requests/adapters.py", line 376, in send
    timeout=timeout
  File "/usr/share/python-wheels/urllib3-1.13.1-py2.py3-none-any.whl/urllib3/connectionpool.py", line 610, in urlopen
    _stacktrace=sys.exc_info()[2])
  File "/usr/share/python-wheels/urllib3-1.13.1-py2.py3-none-any.whl/urllib3/util/retry.py", line 228, in increment
    total -= 1
TypeError: unsupported operand type(s) for -=: 'Retry' and 'int'

----------
components: Installation
messages: 315354
nosy: antonio95100
priority: normal
severity: normal
status: open
title: "pip iinstall packageName.py" fails
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33287>
_______________________________________

From report at bugs.python.org  Mon Apr 16 14:37:10 2018
From: report at bugs.python.org (Steven Steven)
Date: Mon, 16 Apr 2018 18:37:10 +0000
Subject: [New-bugs-announce] [issue33288] Diethealth
Message-ID: <1523903830.84.0.682650639539.issue33288@psf.upfronthosting.co.za>


New submission from Steven Steven <stevenalbertsteven at gmail.com>:

sue

----------
messages: 315363
nosy: stevensalbert
priority: normal
severity: normal
status: open
title: Diethealth

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33288>
_______________________________________

From report at bugs.python.org  Mon Apr 16 17:49:41 2018
From: report at bugs.python.org (Bryan Oakley)
Date: Mon, 16 Apr 2018 21:49:41 +0000
Subject: [New-bugs-announce] [issue33289] askcolor is returning floats for r, g,
 b values instead of ints
Message-ID: <1523915381.82.0.682650639539.issue33289@psf.upfronthosting.co.za>


New submission from Bryan Oakley <bryan.oakley at gmail.com>:

Even though the underlying tcl/tk interpreter is returning ints, askcolor is converting the values to floats. My guess is this is an oversight related to the change in functionality of the / operator in python3.

this:

    return (r/256, g/256, b/256), str(result)

should probably be this:

    return (r//256, g//256, b//256), str(result)

----------
components: Tkinter
messages: 315367
nosy: Bryan.Oakley
priority: normal
severity: normal
status: open
title: askcolor is returning floats for r,g,b values instead of ints
versions: Python 3.4, Python 3.5, Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33289>
_______________________________________

From report at bugs.python.org  Mon Apr 16 19:09:40 2018
From: report at bugs.python.org (Gilbert Wilson)
Date: Mon, 16 Apr 2018 23:09:40 +0000
Subject: [New-bugs-announce] [issue33290] Python.org macOS pkg installs pip3
 as pip
Message-ID: <1523920180.91.0.682650639539.issue33290@psf.upfronthosting.co.za>


New submission from Gilbert Wilson <gilw at dropbox.com>:

The python-3.6.5-macosx10.6.pkg installs pip3 as pip. This means if you have both python2.7.x and 3.6.x you get unexpected and undesirable behavior. According to the release notes in 3.6.5:

Python 3 and Python 2 Co-existence

Python.org Python 3.6 and 2.7.x versions can both be installed on your system and will not conflict. Command names for Python 3 contain a 3 in them, python3 (or python3.6), idle3 (or idle3.6), pip3 (or pip3.6), etc.  Python 2.7 command names contain a 2 or no digit: python2 (or python2.7 or python), idle2 (or idle2.7 or idle), etc.

The release notes for Python2.7.14 have a similarly worded note on Python 3 and Python 2 co-existence.

For both Pythons to properly coexist you must install Python2.7.x after installing Python3.6.x or manually fix the changes that the Python installers make to your ~/.profile PATH environmental variable.

$ which pip3
/Library/Frameworks/Python.framework/Versions/3.6/bin/pip3

$ ls -l $(dirname $(which pip3))
[SNIP]
-rwxr-xr-x  1 gilw  admin    263 Apr 16 13:05 pip
-rwxr-xr-x  1 gilw  admin    263 Apr 16 13:05 pip3
-rwxr-xr-x  1 gilw  admin    263 Apr 16 13:05 pip3.6
[SNIP]

----------
components: macOS
messages: 315369
nosy: dbxgil, ned.deily, ronaldoussoren
priority: normal
severity: normal
status: open
title: Python.org macOS pkg installs pip3 as pip
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33290>
_______________________________________

From report at bugs.python.org  Mon Apr 16 21:21:19 2018
From: report at bugs.python.org (Adam Klinger)
Date: Tue, 17 Apr 2018 01:21:19 +0000
Subject: [New-bugs-announce] [issue33291] Cannot Install Stegano Package -
 Python 3.6.5 Base
Message-ID: <1523928079.77.0.682650639539.issue33291@psf.upfronthosting.co.za>


New submission from Adam Klinger <adamjklinger at gmail.com>:

There seems to be an issue in the default setup.py which comes with the Python 3.6.5 installer. Upon trying to install an additional package through pip the below is observed:

C:\Users\adamj\Desktop>pip install stegano
Collecting stegano
  Downloading https://files.pythonhosted.org/packages/77/76/07a61c042ac1a1cb3445689b4f140800b16d0e883a46049e221d7a92de66/Stegano-0.8.4.tar.gz (243kB)
    100% |????????????????????????????????| 245kB 3.3MB/s
    Complete output from command python setup.py egg_info:
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "C:\Users\adamj\AppData\Local\Temp\pip-install-sh3rd6wj\stegano\setup.py", line 26, in <module>
        readme = f.read()
      File "c:\python36\lib\encodings\cp1252.py", line 23, in decode
        return codecs.charmap_decode(input,self.errors,decoding_table)[0]
    UnicodeDecodeError: 'charmap' codec can't decode byte 0x90 in position 1569: character maps to <undefined>

----------
components: Unicode
messages: 315373
nosy: Adam Klinger, ezio.melotti, vstinner
priority: normal
severity: normal
status: open
title: Cannot Install Stegano Package - Python 3.6.5 Base
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33291>
_______________________________________

From report at bugs.python.org  Mon Apr 16 23:09:36 2018
From: report at bugs.python.org (Vex Woo)
Date: Tue, 17 Apr 2018 03:09:36 +0000
Subject: [New-bugs-announce] [issue33292] Fix secrets.randbelow docstring
Message-ID: <1523934576.27.0.682650639539.issue33292@psf.upfronthosting.co.za>


New submission from Vex Woo <hap.ddup at gmail.com>:

- https://github.com/python/cpython/blob/master/Lib/secrets.py#L28

Please fix

    """Return a random int in the range [0, n)."""

to 

    """Return a random int in the range(0, n)."""

----------
components: Library (Lib)
messages: 315376
nosy: Nixawk
priority: normal
severity: normal
status: open
title: Fix secrets.randbelow docstring
type: resource usage
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33292>
_______________________________________

From report at bugs.python.org  Tue Apr 17 01:20:37 2018
From: report at bugs.python.org (Han Shaowen)
Date: Tue, 17 Apr 2018 05:20:37 +0000
Subject: [New-bugs-announce] [issue33293] Using
 datetime.datetime.utcnow().timestamp() in Python3.6.0 can't get correct UTC
 timestamp.
Message-ID: <1523942437.5.0.682650639539.issue33293@psf.upfronthosting.co.za>


New submission from Han Shaowen <bolide2005 at gmail.com>:

What I am talking is like:

Python 3.6.0 (default, Feb 28 2018, 15:41:04)
[GCC 4.2.1 Compatible Apple LLVM 9.0.0 (clang-900.0.39.2)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import time
>>> from datetime import datetime
>>> time.time()
1523942154.3787892
>>> datetime.now().timestamp()
1523942165.202865
>>> datetime.utcnow().timestamp()
1523913372.362377

Apparently, datetime.now().timestamp() give me right unix timestamp while utcnow().timestamp() doesn't.

Fellas what do you think about this?

----------
components: Extension Modules
messages: 315377
nosy: Han Shaowen
priority: normal
severity: normal
status: open
title: Using datetime.datetime.utcnow().timestamp() in Python3.6.0 can't get correct UTC timestamp.
type: behavior
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33293>
_______________________________________

From report at bugs.python.org  Tue Apr 17 03:02:12 2018
From: report at bugs.python.org (=?utf-8?q?Martin_Li=C5=A1ka?=)
Date: Tue, 17 Apr 2018 07:02:12 +0000
Subject: [New-bugs-announce] [issue33294] Support complex expressions for
 py-print command.
Message-ID: <1523948532.18.0.682650639539.issue33294@psf.upfronthosting.co.za>


New submission from Martin Li?ka <marxin.liska at gmail.com>:

The patch is based on a blog post: http://kouk.surukle.me/2014/09/25/debugging-python-objects-and-fields-with-gdb/.
Adding him: @kouk

Purpose of the pull request is to support more complex expressions for py-print command.
Small example:

(gdb) py-print self
local 'self' = <OptimizationLevel(level='', options=[<MarchFlag(name='-mtune=', options={'x86_64': ('native', 'i386', 'i486', 'i586', 'pentium', 'lakemont', 'pentium-mmx', 'pentiumpro', 'i686', 'pentium2', 'pentium3', 'pentium3m', 'pentium-m', 'pentium4', 'pentium4m', 'prescott', 'nocona', 'core2', 'nehalem', 'westmere', 'sandybridge', 'ivybridge', 'haswell', 'broadwell', 'skylake', 'bonnell', 'silvermont', 'knl', 'skylake-avx512', 'k6', 'k6-2', 'k6-3', 'athlon', 'athlon-tbird', 'athlon-4', 'athlon-xp', 'athlon-mp', 'k8', 'opteron', 'athlon64', 'athlon-fx', 'k8-sse3', 'opteron-sse3', 'athlon64-sse3', 'amdfam10', 'barcelona', 'bdver1', 'bdver2', 'bdver3', 'bdver4', 'znver1', 'btver1', 'btver2', 'winchip-c6', 'winchip2', 'c3', 'c3-2', 'geode'), 'ppc64': ['401', '403', '405', '405fp', '440', '440fp', '464', '464fp', '476', '476fp', '505', '601', '602', '603', '603e', '604', '604e', '620', '630', '740', '7400', '7450', '750', '801', '821', '823', '860', '970', '8540', 'a2', 'e300c2', 'e300c3', 'e500mc', 'e500mc64', 'e5500', 'e6...(truncated)

(gdb) py-print self.options.0.options.x86_64.3
local 'self.options.0.options.x86_64.3' = 'i586'
I consider it very handy. For now I support objects, dictionary, tuple and list types of objects.
I guess a test-case and a documentation entry will be needed. But I would first like to get a feedback about the intention of the change.

----------
components: Demos and Tools
messages: 315381
nosy: Martin Li?ka
priority: normal
severity: normal
status: open
title: Support complex expressions for py-print command.
type: enhancement

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33294>
_______________________________________

From report at bugs.python.org  Tue Apr 17 05:15:32 2018
From: report at bugs.python.org (INADA Naoki)
Date: Tue, 17 Apr 2018 09:15:32 +0000
Subject: [New-bugs-announce] [issue33295] ERROR:
 test_sites_no_connection_close (test.test_urllib2net.OtherNetworkTests)
Message-ID: <1523956532.44.0.682650639539.issue33295@psf.upfronthosting.co.za>


New submission from INADA Naoki <songofacandy at gmail.com>:

https://github.com/python/cpython/blob/5fbc511f56688654a05b9eba23d140318bb9b2d5/Lib/test/test_urllib2net.py#L180-L198

This test uses http://www.imdb.com/, but it is gone.
So This test start failing.

Is there any good URL for this test case?
May I skip this test for now?

----------
components: Tests
messages: 315384
nosy: inada.naoki
priority: critical
severity: normal
status: open
title: ERROR: test_sites_no_connection_close (test.test_urllib2net.OtherNetworkTests)
versions: Python 2.7, Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33295>
_______________________________________

From report at bugs.python.org  Tue Apr 17 05:16:46 2018
From: report at bugs.python.org (Shlomo Anglister)
Date: Tue, 17 Apr 2018 09:16:46 +0000
Subject: [New-bugs-announce] [issue33296] datetime.datetime.utcfromtimestamp
 call decimal causes precision loss
Message-ID: <1523956606.05.0.682650639539.issue33296@psf.upfronthosting.co.za>


New submission from Shlomo Anglister <shlomo.anglister at gmail.com>:

Output for python2.7.11

2018-04-12-05:23:08.436252
2018-04-12-05:23:08.436252

Output for python3.4 and python3.6

2018-04-12-05:23:08.436252
2018-04-12-05:23:08.000000

----------
components: Library (Lib)
files: timestamp_decimal_test.py
messages: 315385
nosy: anglister
priority: normal
severity: normal
status: open
title: datetime.datetime.utcfromtimestamp call decimal causes precision loss
type: behavior
versions: Python 3.4, Python 3.5, Python 3.6
Added file: https://bugs.python.org/file47538/timestamp_decimal_test.py

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33296>
_______________________________________

From report at bugs.python.org  Tue Apr 17 07:12:06 2018
From: report at bugs.python.org (=?utf-8?q?Andr=C3=A9s_Delfino?=)
Date: Tue, 17 Apr 2018 11:12:06 +0000
Subject: [New-bugs-announce] [issue33297] Mention Pillow package on
 tkinter.rst to work with more image formats
Message-ID: <1523963526.08.0.682650639539.issue33297@psf.upfronthosting.co.za>


New submission from Andr?s Delfino <adelfino at gmail.com>:

Given that the default list of supported formats is quite limited, IMHO we should point to Pillow, or any other package that provides support for more formats.

----------
assignee: docs at python
components: Documentation
messages: 315393
nosy: adelfino, docs at python
priority: normal
severity: normal
status: open
title: Mention Pillow package on tkinter.rst to work with more image formats
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33297>
_______________________________________

From report at bugs.python.org  Tue Apr 17 12:21:41 2018
From: report at bugs.python.org (Serhiy Storchaka)
Date: Tue, 17 Apr 2018 16:21:41 +0000
Subject: [New-bugs-announce] [issue33298] Wrap only constants with
 _PyCode_ConstantKey() in the compiler.
Message-ID: <1523982101.74.0.682650639539.issue33298@psf.upfronthosting.co.za>


New submission from Serhiy Storchaka <storchaka+cpython at gmail.com>:

Currently every constant is wrapped with _PyCode_ConstantKey() in the compiler. This is necessary for distinguishing numbers 1, 1.0 and 1.0+0j, 0.0 and -0.0, and avoiding warnings from comparing string and bytes constants.

But the change that introduced using _PyCode_ConstantKey() for constants (in issue25843) also caused to wrapping names with _PyCode_ConstantKey(). This is superfluous because all names are strings. This just consumes memory for 2-tuples for every name and spends CPU time for creating these tuples for every name.

There are two ways of getting rid of this overhead.

1. Make _PyCode_ConstantKey() returning just a string itself for string argument. Strings are never equal to tuples. This requires minimal changes.

2. Use different helpers for adding constants and names, and use _PyCode_ConstantKey() only for names. This will clean up the code for names, but needs adding few special functions and macros.

I'm going to apply both solution. The first one will add a benefit for string constants (and other common types), the second one will help moving the folding tuples of constants from peephole.c to compile.c and generalizing it for lists, sets and dicts of constants, constant default values, constant arguments, etc.

The following PR implements the second option.

----------
assignee: serhiy.storchaka
components: Interpreter Core
messages: 315405
nosy: benjamin.peterson, brett.cannon, ncoghlan, serhiy.storchaka, yselivanov
priority: normal
severity: normal
status: open
title: Wrap only constants with _PyCode_ConstantKey() in the compiler.
type: performance
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33298>
_______________________________________

From report at bugs.python.org  Tue Apr 17 12:54:12 2018
From: report at bugs.python.org (Serhiy Storchaka)
Date: Tue, 17 Apr 2018 16:54:12 +0000
Subject: [New-bugs-announce] [issue33299] Return an object itself for some
 types in _PyCode_ConstantKey()
Message-ID: <1523984052.37.0.682650639539.issue33299@psf.upfronthosting.co.za>


New submission from Serhiy Storchaka <storchaka+cpython at gmail.com>:

_PyCode_ConstantKey() wraps a constant into a tuple that contains other items besides the wrapped object. This makes booleans different from 0 and 1, makes numbers of different type different (1, 1.0 and 1.0+0j), makes floating point zeros with different signs different (0.0 and -0.0), and avoids a ByteWarning from comparing strings and bytes.

But for some types objects are always different from objects of other types and from tuples. _PyCode_ConstantKey() can be made faster and using less memory if return the object of these types unwrapped.

----------
components: Interpreter Core
messages: 315407
nosy: serhiy.storchaka
priority: normal
severity: normal
status: open
title: Return an object itself for some types in _PyCode_ConstantKey()
type: performance
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33299>
_______________________________________

From report at bugs.python.org  Tue Apr 17 16:02:15 2018
From: report at bugs.python.org (Glen Neff)
Date: Tue, 17 Apr 2018 20:02:15 +0000
Subject: [New-bugs-announce] [issue33300] Bad usage example in id() DocString
Message-ID: <1523995335.26.0.682650639539.issue33300@psf.upfronthosting.co.za>


New submission from Glen Neff <gneff at trioptimum.com>:

The DocString for id() in 3.5.x & 3.6.x seems to have an incorrect usage example, specifically ``id(obj, /)``.

This is present in 3.5.x & 3.6.x.  It does not appear to be present in 2.7.x or 3.4.x.:

Python 3.6.4 (v3.6.4:d48eceb, Dec 19 2017, 06:54:40) [MSC v.1900 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> help(id)
Help on built-in function id in module builtins:

id(obj, /)
    Return the identity of an object.

    This is guaranteed to be unique among simultaneously existing objects.
    (CPython uses the object's memory address.)

>>>

----------
assignee: docs at python
components: Documentation
messages: 315410
nosy: docs at python, gneff
priority: normal
severity: normal
status: open
title: Bad usage example in id() DocString
versions: Python 3.5, Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33300>
_______________________________________

From report at bugs.python.org  Tue Apr 17 17:23:21 2018
From: report at bugs.python.org (Alok Singh)
Date: Tue, 17 Apr 2018 21:23:21 +0000
Subject: [New-bugs-announce] [issue33301] Add __contains__ to pathlib
Message-ID: <1524000201.84.0.682650639539.issue33301@psf.upfronthosting.co.za>


New submission from Alok Singh <alokbeniwal at gmail.com>:

I was thinking today that it would be natural for paths to support __contains__ since then you could write statements like `if file in dir` or `if subdir in dir` cleanly.

The library plumbum appears to already have this, but I think it could be useful in the standard library:

https://plumbum.readthedocs.io/en/latest/_modules/plumbum/path/base.html#Path.__contains__

----------
components: Library (Lib)
messages: 315422
nosy: Alok Singh
priority: normal
severity: normal
status: open
title: Add __contains__ to pathlib
type: enhancement
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33301>
_______________________________________

From report at bugs.python.org  Tue Apr 17 17:48:27 2018
From: report at bugs.python.org (Matthew Woodcraft)
Date: Tue, 17 Apr 2018 21:48:27 +0000
Subject: [New-bugs-announce] [issue33302] The search for pyvenv.cfg doesn't
 match PEP 405
Message-ID: <1524001707.98.0.682650639539.issue33302@psf.upfronthosting.co.za>


New submission from Matthew Woodcraft <matthew at woodcraft.me.uk>:

PEP 405 says that the pyvenv.cfg file is found as follows:

? a pyvenv.cfg file is found either adjacent to the Python executable or one directory above it (if the executable is a symlink, it is not dereferenced), ?

But in cpython if the executable is a symlink, it _is_ dereferenced before searching for pyvenv.cfg .

I've checked this behaviour with Python 3.5, 3.6, and today's 3.7 tip.

Looking in 3.7's getpath.c, calculate_path_impl() calls, in order:

   calculate_program_full_path()
   calculate_argv0_path()
   calculate_read_pyenv()

It looks like the symlink resolution happens near the end of calculate_argv0_path().


I think this means that the 'home=' line in pyvenv.cfg files generated by the 'venv' module is irrelevant (and seems likely to confuse anyone who might try to construct a Python environment by imitating a built venv).

----------
components: Interpreter Core
messages: 315424
nosy: mattheww
priority: normal
severity: normal
status: open
title: The search for pyvenv.cfg doesn't match PEP 405
type: behavior
versions: Python 3.5, Python 3.6, Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33302>
_______________________________________

From report at bugs.python.org  Tue Apr 17 20:33:12 2018
From: report at bugs.python.org (John Burnett)
Date: Wed, 18 Apr 2018 00:33:12 +0000
Subject: [New-bugs-announce] [issue33303] ElementTree Comment text isn't
 escaped
Message-ID: <1524011592.66.0.682650639539.issue33303@psf.upfronthosting.co.za>


New submission from John Burnett <python at johnburnett.com>:

The _serialize_xml function in ElementTree.py doesn't escape Comment.text values when writing output.  This means the following code:

    import sys
    import xml.etree.ElementTree
    elem = xml.etree.ElementTree.Comment()
    elem.text = 'hi --> bye'
    tree = xml.etree.ElementTree.ElementTree(elem)
    tree.write(sys.stdout)

...will output the following invalid xml:

    <!--hi --> bye-->

In Python 3.7, changing the _serialize_xml function on line 903/904 from this:

    if tag is Comment:
        write("<!--%s-->" % text)

...to this:

    if tag is Comment:
        write("<!--%s-->" % _escape_cdata(text))

...writes something more expected:

    <!--hi --&gt; bye-->

----------
components: XML
messages: 315428
nosy: eli.bendersky, johnburnett, scoder
priority: normal
severity: normal
status: open
title: ElementTree Comment text isn't escaped
type: behavior
versions: Python 2.7, Python 3.4, Python 3.5, Python 3.6, Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33303>
_______________________________________

From report at bugs.python.org  Tue Apr 17 21:11:07 2018
From: report at bugs.python.org (jackb)
Date: Wed, 18 Apr 2018 01:11:07 +0000
Subject: [New-bugs-announce] [issue33304] Syntax Error on leading 0 in
 integer tokens
Message-ID: <1524013867.24.0.682650639539.issue33304@psf.upfronthosting.co.za>


New submission from jackb <bhjwla at gmail.com>:

Entering 007 gives a syntax error. Should return 7.
00 correctly returns 0.
007. correctly returns 7.0.

----------
components: Interpreter Core
messages: 315429
nosy: jackb
priority: normal
severity: normal
status: open
title: Syntax Error on leading 0 in integer tokens
type: behavior
versions: Python 3.4

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33304>
_______________________________________

From report at bugs.python.org  Tue Apr 17 21:47:11 2018
From: report at bugs.python.org (Steven D'Aprano)
Date: Wed, 18 Apr 2018 01:47:11 +0000
Subject: [New-bugs-announce] [issue33305] Improve syntax error for numbers
 with leading zero
Message-ID: <1524016031.89.0.682650639539.issue33305@psf.upfronthosting.co.za>


New submission from Steven D'Aprano <steve+python at pearwood.info>:

The Python 2.x syntax for octal integers is a syntax error in 3.x, but the error message is very uninformative:

SyntaxError: invalid token


Can this be improved? Perhaps to something like:

invalid token, use 0o prefix for octal integers


(see also #33304)

----------
messages: 315431
nosy: steven.daprano
priority: normal
severity: normal
status: open
title: Improve syntax error for numbers with leading zero
type: behavior
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33305>
_______________________________________

From report at bugs.python.org  Wed Apr 18 05:22:20 2018
From: report at bugs.python.org (Serhiy Storchaka)
Date: Wed, 18 Apr 2018 09:22:20 +0000
Subject: [New-bugs-announce] [issue33306] Improving SyntaxError for
 unmatched parentheses
Message-ID: <1524043340.4.0.682650639539.issue33306@psf.upfronthosting.co.za>


New submission from Serhiy Storchaka <storchaka+cpython at gmail.com>:

SyntaxError messages were improved recently in PyPy. [1] [2]  The part of this improvement in CPython (showing an expected token) is discussed in issue1634034. The following PR implements other part, related to unmatched parentheses.

Showing the number of the line containing an unmatched opening parenthesis looks as enhancement. But I'm not sure that this change is always an enhancement (in comparison with issue1634034). Syntax error for unmatched parentheses will be raised even if the code contains other syntax errors before unmatched parenthesis.

[1] https://morepypy.blogspot.de/2018/04/improving-syntaxerror-in-pypy.html
[2] https://bitbucket.org/pypy/pypy/commits/e66f24650dafbcd2ac1c443af7417eddf8e8f093

----------
components: Interpreter Core
messages: 315441
nosy: serhiy.storchaka
priority: normal
severity: normal
status: open
title: Improving SyntaxError for unmatched parentheses
type: enhancement
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33306>
_______________________________________

From report at bugs.python.org  Wed Apr 18 06:52:55 2018
From: report at bugs.python.org (Kaulkwappe)
Date: Wed, 18 Apr 2018 10:52:55 +0000
Subject: [New-bugs-announce] [issue33307] socket.send() fails to send large
 amount of bytes
Message-ID: <1524048775.4.0.682650639539.issue33307@psf.upfronthosting.co.za>


New submission from Kaulkwappe <bugs.python.org at prvy.eu>:

socket.setblocking(0)
socket.send(b'a' * 32 * 1024 * 1024)

In the example above socket.send() fails with this error:

Error in connection handler
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/websockets/server.py", line 81, in handler
    yield from self.ws_handler(self, path)
  File "/var/www/vhosts/.../app/sockets/core/Daemon.py", line 286, in websocketClientHandler
    self.api.connection.send(data)
  File "/usr/lib/python3.5/ssl.py", line 869, in send
    return self._sslobj.write(data)
  File "/usr/lib/python3.5/ssl.py", line 594, in write
    return self._sslobj.write(data)
ssl.SSLWantWriteError: The operation did not complete (write) (_ssl.c:1949)

----------
assignee: christian.heimes
components: SSL
messages: 315444
nosy: Kaulkwappe, christian.heimes
priority: normal
severity: normal
status: open
title: socket.send() fails to send large amount of bytes
type: crash
versions: Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33307>
_______________________________________

From report at bugs.python.org  Wed Apr 18 11:32:26 2018
From: report at bugs.python.org (Brett Cannon)
Date: Wed, 18 Apr 2018 15:32:26 +0000
Subject: [New-bugs-announce] [issue33308] parser.st2list(...,
 col_info=True) triggers a SystemError
Message-ID: <1524065546.01.0.682650639539.issue33308@psf.upfronthosting.co.za>


New submission from Brett Cannon <brett at python.org>:

`parser.st2list(parser.suite(""), col_info=True)` causes:

Fatal Python error: a function returned a result with an error set
IndexError: list assignment index out of range

```
The above exception was the direct cause of the following exception:

SystemError: <built-in function st2list> returned a result with an error set

Current thread 0x00007fff9ca99380 (most recent call first):
  File "/private/tmp/tester/lazy/sub.py", line 3 in <module>
fish: '/Users/brettcannon/Repositories?' terminated by signal SIGABRT (Abort)
```

----------
components: Library (Lib)
messages: 315455
nosy: brett.cannon
priority: normal
severity: normal
stage: test needed
status: open
title: parser.st2list(..., col_info=True) triggers a SystemError
type: behavior
versions: Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33308>
_______________________________________

From report at bugs.python.org  Wed Apr 18 11:53:04 2018
From: report at bugs.python.org (Nathaniel Duarte)
Date: Wed, 18 Apr 2018 15:53:04 +0000
Subject: [New-bugs-announce] [issue33309] Unittest Mock objects do not
 freeze arguments they are called with
Message-ID: <1524066784.49.0.682650639539.issue33309@psf.upfronthosting.co.za>


New submission from Nathaniel Duarte <slacknate at gmail.com>:

It is possible to make a call with a Mock object, for example, where a dictionary is passed to the mock and later modified. If a call assertion is then made against the mock using the initial value/contents of the dictionary the assertion will fail.

----------
components: Library (Lib)
files: mock_issue.py
messages: 315456
nosy: slacknate
priority: normal
severity: normal
status: open
title: Unittest Mock objects do not freeze arguments they are called with
type: behavior
versions: Python 3.7
Added file: https://bugs.python.org/file47539/mock_issue.py

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33309>
_______________________________________

From report at bugs.python.org  Wed Apr 18 14:38:33 2018
From: report at bugs.python.org (Serhiy Storchaka)
Date: Wed, 18 Apr 2018 18:38:33 +0000
Subject: [New-bugs-announce] [issue33310] Update references to PIL-style
 arrays
Message-ID: <1524076713.71.0.682650639539.issue33310@psf.upfronthosting.co.za>


New submission from Serhiy Storchaka <storchaka+cpython at gmail.com>:

The documentation for buffer protocol and memoryview objects contains references to "PIL-style arrays". Since PIL [1] seems dead (it was not ported to Python 3) and superseded by Pillow [2], I think it is worth to update these references. Either rename "PIL-style arrays" to "Pillow-style arrays" or change the wording to more general.

[1] http://www.pythonware.com/products/pil/
[2] http://python-pillow.org/

----------
assignee: docs at python
components: Documentation
messages: 315458
nosy: docs at python, pitrou, serhiy.storchaka, skrah
priority: normal
severity: normal
status: open
title: Update references to PIL-style arrays
type: enhancement
versions: Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33310>
_______________________________________

From report at bugs.python.org  Wed Apr 18 14:39:52 2018
From: report at bugs.python.org (=?utf-8?q?St=C3=A9phane_Blondon?=)
Date: Wed, 18 Apr 2018 18:39:52 +0000
Subject: [New-bugs-announce] [issue33311] cgitb: remove parentheses when the
 error is in module
Message-ID: <1524076792.36.0.682650639539.issue33311@psf.upfronthosting.co.za>


New submission from St?phane Blondon <stephane.blondon at gmail.com>:

The cgitb module displays a traceback in text or html.

When a module is called, there are no parameters (displayed as '()'). I
think they are unnecessary and the parentheses should be removed.

### example for the text version ###
$ python3 demo.py
[...]
 /home/stephane/src/cgitest/demo.py in <module>()
    7 def func1(a, b):                         ^-- to be removed?
[...]
### end of example ###

It occurs in both text and html versions.

----------
components: Library (Lib)
messages: 315459
nosy: sblondon
priority: normal
severity: normal
status: open
title: cgitb: remove parentheses when the error is in module
type: enhancement

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33311>
_______________________________________

From report at bugs.python.org  Wed Apr 18 17:59:39 2018
From: report at bugs.python.org (Gregory P. Smith)
Date: Wed, 18 Apr 2018 21:59:39 +0000
Subject: [New-bugs-announce] [issue33312] ubsan undefined behavior sanitizer
 flags struct _dictkeysobject (PyDictKeysObj)
Message-ID: <1524088779.29.0.682650639539.issue33312@psf.upfronthosting.co.za>


New submission from Gregory P. Smith <greg at krypto.org>:

Build CPython (master in this case - though I originally noticed the problem when building a 3.6 tree) as follows with clang installed:

build$ LD=clang-5.0 LDFLAGS=-fsanitize=undefined CC=clang-5.0 CXX=clang-5.0 CFLAGS=-fsanitize=undefined CXXFLAGS=-fsanitize=undefined ../gpshead/configure
build$ make -j12

...

notice many of the warnings scroll by during the build itself as it executes the interpreter

then execute it yourself at the end and you'll get a bunch of these:

../gpshead/Objects/dictobject.c:547:12: runtime error: index 16 out of bounds for type 'int8_t [8]'
../gpshead/Objects/dictobject.c:1145:18: runtime error: index 16 out of bounds for type 'int8_t [8]'
../gpshead/Objects/dictobject.c:2817:15: runtime error: index 16 out of bounds for type 'int8_t [8]'
../gpshead/Objects/dictobject.c:831:27: runtime error: index 16 out of bounds for type 'int8_t [8]'
../gpshead/Objects/dictobject.c:1144:18: runtime error: index 16 out of bounds for type 'int8_t [8]'
../gpshead/Objects/dictobject.c:1034:15: runtime error: index 16 out of bounds for type 'int8_t [8]'
../gpshead/Objects/dictobject.c:728:11: runtime error: index 16 out of bounds for type 'int8_t [8]'
../gpshead/Objects/dictobject.c:1064:9: runtime error: index 64 out of bounds for type 'int8_t [8]'
../gpshead/Objects/dictobject.c:2960:31: runtime error: index 64 out of bounds for type 'int8_t [8]'
../gpshead/Objects/dictobject.c:1489:11: runtime error: index 32 out of bounds for type 'int8_t [8]'
../gpshead/Objects/dictobject.c:637:27: runtime error: index 128 out of bounds for type 'int8_t [8]'
../gpshead/Objects/dictobject.c:788:27: runtime error: index 16 out of bounds for type 'int8_t [8]'
../gpshead/Objects/dictobject.c:1671:22: runtime error: index 16 out of bounds for type 'int8_t [8]'
../gpshead/Objects/dictobject.c:554:31: runtime error: index 16 out of bounds for type 'int8_t [8]'
../gpshead/Objects/dictobject.c:1223:15: runtime error: index 16 out of bounds for type 'int8_t [8]'
../gpshead/Objects/dictobject.c:876:27: runtime error: index 16 out of bounds for type 'int8_t [8]'
../gpshead/Objects/dictobject.c:2396:15: runtime error: index 32 out of bounds for type 'int8_t [8]'
../gpshead/Objects/dictobject.c:2078:10: runtime error: index 128 out of bounds for type 'int8_t [8]'
../gpshead/Objects/dictobject.c:3584:38: runtime error: index 16 out of bounds for type 'int8_t [8]'
../gpshead/Objects/dictobject.c:3502:38: runtime error: index 64 out of bounds for type 'int8_t [8]'

At issue is the hash table here: https://github.com/python/cpython/blob/3.7/Objects/dict-common.h

which is intentionally meant to be indexed "out of bounds" off the end of the struct.

I'm not a strict C language definition so I don't know if that is _supposed_ to be defined behavior as we all tend to assume it is in C or not.  If it is supposed to be okay, we should be able to annotate it as such to avoid the warning under ubsan builds.

If it is not, we need to change the way this is written.

----------
messages: 315464
nosy: gregory.p.smith
priority: normal
severity: normal
stage: needs patch
status: open
title: ubsan undefined behavior sanitizer flags struct _dictkeysobject (PyDictKeysObj)
type: compile error
versions: Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33312>
_______________________________________

From report at bugs.python.org  Wed Apr 18 18:17:43 2018
From: report at bugs.python.org (Pablo Galindo Salgado)
Date: Wed, 18 Apr 2018 22:17:43 +0000
Subject: [New-bugs-announce] [issue33313] pwritev,
 preadv and posix_spawn are missing from "What's new in 3.7"
Message-ID: <1524089863.52.0.682650639539.issue33313@psf.upfronthosting.co.za>


New submission from Pablo Galindo Salgado <pablogsal at gmail.com>:

The "What's new in 3.7" is missing the functions exposed in issue31368 and issue20104: pwritev, preadv and posix_spawn.

----------
messages: 315465
nosy: gregory.p.smith, pablogsal, serhiy.storchaka
priority: normal
severity: normal
status: open
title: pwritev, preadv and posix_spawn are missing from "What's new in 3.7"

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33313>
_______________________________________

From report at bugs.python.org  Wed Apr 18 18:30:31 2018
From: report at bugs.python.org (Pablo Galindo Salgado)
Date: Wed, 18 Apr 2018 22:30:31 +0000
Subject: [New-bugs-announce] [issue33314] Bad rendering in the documentation
 for the os module
Message-ID: <1524090631.89.0.682650639539.issue33314@psf.upfronthosting.co.za>


New submission from Pablo Galindo Salgado <pablogsal at gmail.com>:

Currently, there are rendering issues in the os module documentation for the constants os.RWF_HIPRI and os.RWF_NOWAIT:

https://docs.python.org/3.7/library/os.html#os.RWF_HIPRI
https://docs.python.org/3.7/library/os.html#os.RWF_NOWAIT

----------
assignee: pablogsal
components: Documentation
messages: 315467
nosy: pablogsal
priority: high
severity: normal
stage: needs patch
status: open
title: Bad rendering in the documentation for the os module
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33314>
_______________________________________

From report at bugs.python.org  Thu Apr 19 08:02:56 2018
From: report at bugs.python.org (Semyon Proshev)
Date: Thu, 19 Apr 2018 12:02:56 +0000
Subject: [New-bugs-announce] [issue33315] Allow queue.Queue to be used in
 type annotations
Message-ID: <1524139376.68.0.682650639539.issue33315@psf.upfronthosting.co.za>


New submission from Semyon Proshev <s-proshev at ya.ru>:

from queue import Queue

a: Queue[int]

This code throws a TypeError in runtime.
But its pyi stub allows to use it in such a way (https://github.com/python/typeshed/blob/master/stdlib/3/queue.pyi)

I'd suggest to update classes in queue module to allow them to be used in type annotations.

----------
components: Library (Lib)
messages: 315488
nosy: sproshev
priority: normal
severity: normal
status: open
title: Allow queue.Queue to be used in type annotations
versions: Python 3.5, Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33315>
_______________________________________

From report at bugs.python.org  Thu Apr 19 20:18:56 2018
From: report at bugs.python.org (Ivan Pozdeev)
Date: Fri, 20 Apr 2018 00:18:56 +0000
Subject: [New-bugs-announce] [issue33316] PyThread_release_lock always fails
Message-ID: <1524183536.18.0.682650639539.issue33316@psf.upfronthosting.co.za>


New submission from Ivan Pozdeev <ivan_pozdeev at mail.ru>:

In win7 x64 debug mode with thread_debug=1, every PyThread_release_lock() is accompanied with a message on stderr:

Could not PyThread_release_lock(<id>) error: 0

----------
components: Interpreter Core, Windows
messages: 315497
nosy: Ivan.Pozdeev, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: PyThread_release_lock always fails
type: behavior
versions: Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33316>
_______________________________________

From report at bugs.python.org  Fri Apr 20 03:24:42 2018
From: report at bugs.python.org (=?utf-8?q?Pekka_Kl=C3=A4rck?=)
Date: Fri, 20 Apr 2018 07:24:42 +0000
Subject: [New-bugs-announce] [issue33317] `repr()` of string in NFC and NFD
 forms does not differ
Message-ID: <1524209082.19.0.682650639539.issue33317@psf.upfronthosting.co.za>


New submission from Pekka Kl?rck <pekka.klarck at gmail.com>:

If I have two strings that look the same but have different Unicode form, it's very hard to see where the problem actually is:

>>> a = 'hyv\xe4'
>>> b = 'hyva\u0308'
>>> print(a)
hyv?
>>> print(b)
hyva?
>>> a == b
False
>>> print(repr(a))
'hyv?'
>>> print(repr(b))
'hyva?'

This affects, for example, test automation frameworks using `repr()` in error reporting. For example, both unittest and pytest report `self.assertEqual('hyv\xe4', 'hyva\u0308')` like this:

AssertionError: 'hyv?' != 'hyva?'
- hyv?
+ hyva?

Because the NFC form is used by strings by default, I would propose that `repr()` would show the decomposed form if the string is in NFD. In practice I'd like `repr('hyva\0308')` to yield `'hyva\0308'`.

----------
messages: 315504
nosy: pekka.klarck
priority: normal
severity: normal
status: open
title: `repr()` of string in NFC and NFD forms does not differ

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33317>
_______________________________________

From report at bugs.python.org  Fri Apr 20 08:57:21 2018
From: report at bugs.python.org (Serhiy Storchaka)
Date: Fri, 20 Apr 2018 12:57:21 +0000
Subject: [New-bugs-announce] [issue33318] Move folding tuples of constants
 into compiler.c from peephole.c
Message-ID: <1524229041.8.0.682650639539.issue33318@psf.upfronthosting.co.za>


New submission from Serhiy Storchaka <storchaka+cpython at gmail.com>:

Explicit tuples of constants are folded in the AST optimizer since 3.7. But peephole.c still contains the code for folding tuples of constants, because there are tuples created by the compiler. For example "[1, 2, *a]", "f(1, 2, *a)", "def f(a=1, b=2)" all create a tuple (1, 2).

The following PR moves this code from peephole.c into compiler.c. This makes the code a tiny bit clearer, because it works on higher level than  a bytecode. An obvious benefit -- the optimization is performed before calculating the depth of the stack, thus it will be more exact.

----------
components: Interpreter Core
messages: 315508
nosy: benjamin.peterson, brett.cannon, inada.naoki, ncoghlan, pitrou, rhettinger, serhiy.storchaka, yselivanov
priority: normal
severity: normal
status: open
title: Move folding tuples of constants into compiler.c from peephole.c
type: enhancement
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33318>
_______________________________________

From report at bugs.python.org  Fri Apr 20 09:05:14 2018
From: report at bugs.python.org (=?utf-8?q?Pekka_Kl=C3=A4rck?=)
Date: Fri, 20 Apr 2018 13:05:14 +0000
Subject: [New-bugs-announce] [issue33319] `subprocess.run` documentation
 doesn't tell is using `stdout=PIPE` safe
Message-ID: <1524229514.84.0.682650639539.issue33319@psf.upfronthosting.co.za>


New submission from Pekka Kl?rck <pekka.klarck at gmail.com>:

I'm porting old scripts from Python 2.7 to 3.6 and plan to change `subprocess.call()` to `subprocess.run()` at the same time. When using `call()` I've used `tempfile.TemporaryFile` as stdout because it's documentation has this warning:

    Note: Do not use stdout=PIPE or stderr=PIPE with this function. The child process will block if it generates enough output to a pipe to fill up the OS pipe buffer as the pipes are not being read from.

Interestingly there is no such note in the docs of `run()`, and based on my (possibly inadequate) testing I couldn't get it to hang either. I'm still somewhat worried about using `stdout=PIPE` with it because the docs don't explicitly say it would be safe. I'm especially worried because the docs of `call()` nowadays say that it's equivalent to `run(...).returncode`. If that's the case, then I would expect the warning in `call()` to apply also to `run()`. Or is the warning nowadays outdated altogether?

----------
messages: 315510
nosy: pekka.klarck
priority: normal
severity: normal
status: open
title: `subprocess.run` documentation doesn't tell is using `stdout=PIPE` safe

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33319>
_______________________________________

From report at bugs.python.org  Fri Apr 20 10:15:01 2018
From: report at bugs.python.org (=?utf-8?q?Andr=C3=A9s_Delfino?=)
Date: Fri, 20 Apr 2018 14:15:01 +0000
Subject: [New-bugs-announce] [issue33320] Clarify who can add the "skip
 issue" label on GitHub
Message-ID: <1524233701.12.0.682650639539.issue33320@psf.upfronthosting.co.za>


New submission from Andr?s Delfino <adelfino at gmail.com>:

IMHO, the wording of https://devguide.python.org/pullrequest/ is somewhat confusing regarding the "skip issue" label.

I thought that I (with no commit access) should add the label, but, as far I understand, that is a responsibility of developers with commit access.

The PR offers a wording to clarify this.

----------
assignee: docs at python
components: Documentation
messages: 315512
nosy: adelfino, docs at python
priority: normal
severity: normal
status: open
title: Clarify who can add the "skip issue" label on GitHub
type: enhancement

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33320>
_______________________________________

From report at bugs.python.org  Fri Apr 20 19:48:21 2018
From: report at bugs.python.org (Gregory P. Smith)
Date: Fri, 20 Apr 2018 23:48:21 +0000
Subject: [New-bugs-announce] [issue33321] Add a Linux clang ubsan undefined
 behavior sanitizer buildbot
Message-ID: <1524268101.98.0.682650639539.issue33321@psf.upfronthosting.co.za>


New submission from Gregory P. Smith <greg at krypto.org>:

CPython needs to be regularly built and vetted for C undefined behavior.

CFLAGS=-fsanitize=undefined LDFLAGS=-fsanitize=undefined CC=clang LD=clang ./configure

is all it takes to start with.

if you do this today, test_ctypes will knock your socks off with errors.  We should detect this type of build and intentionally skip its tests on ubsan, tracking that with another yet to be filed issue.

I'll look into having Google run one.

----------
assignee: gregory.p.smith
components: Build
keywords: buildbot
messages: 315542
nosy: gregory.p.smith
priority: normal
severity: normal
status: open
title: Add a Linux clang ubsan undefined behavior sanitizer buildbot
type: compile error
versions: Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33321>
_______________________________________

From report at bugs.python.org  Sat Apr 21 02:47:32 2018
From: report at bugs.python.org (Jacob Thalman)
Date: Sat, 21 Apr 2018 06:47:32 +0000
Subject: [New-bugs-announce] [issue33322] Overridden __getitem__ not called
 on use of slice syntax when inheriting from tuple
Message-ID: <1524293252.15.0.682650639539.issue33322@psf.upfronthosting.co.za>


New submission from Jacob Thalman <jpthalman at gmail.com>:

class MyTuple(tuple):
    def __getitem__(self, item):
        print "Getting {}".format(item)

t = MyTuple((1, 2))
t[0] -> "Getting 0"
t[1] -> "Getting 1"
t[slice(None)] -> "Getting slice(None, None, None)"
t[:] -> (1, 2)
t[slice(None, 1)] -> "Getting slice(None, 1, None)"
t[:1] -> (1,)

Overriding __getattribute__ confirms that the overridden __getitem__ is never called when syntactic slice syntax is used.

----------
messages: 315551
nosy: jpthalman
priority: normal
severity: normal
status: open
title: Overridden __getitem__ not called on use of slice syntax when inheriting from tuple
type: behavior
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33322>
_______________________________________

From report at bugs.python.org  Sat Apr 21 06:53:20 2018
From: report at bugs.python.org (Naris R)
Date: Sat, 21 Apr 2018 10:53:20 +0000
Subject: [New-bugs-announce] [issue33323] inconsistent stack trace for
 exceptions thrown in generators passed to min/max
Message-ID: <1524308000.84.0.682650639539.issue33323@psf.upfronthosting.co.za>


New submission from Naris R <nariscatboy at gmail.com>:

if a generator passed to min/max throws an exception, the stack trace is normally shown on the line that caused the exception, but if the exception is a StopIteration, the trace only shows the line where the max/min function was called.

I was writing a minimax and alphabeta search with generator expression and list comprehension and accidentally passed an empty iterator to the next function while computing states for minimax and it took a very long time to find out where the error was actually caused.

----------
messages: 315561
nosy: Naris R
priority: normal
severity: normal
status: open
title: inconsistent stack trace for exceptions thrown in generators passed to min/max
type: behavior
versions: Python 2.7, Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33323>
_______________________________________

From report at bugs.python.org  Sat Apr 21 07:50:02 2018
From: report at bugs.python.org (Alfonso Chavez)
Date: Sat, 21 Apr 2018 11:50:02 +0000
Subject: [New-bugs-announce] [issue33324] Bug in documentation 3.6:
 string-methods - str.center
Message-ID: <1524311402.18.0.682650639539.issue33324@psf.upfronthosting.co.za>


New submission from Alfonso Chavez <alfonsochc at gmail.com>:

Hi. In https://docs.python.org/3/library/stdtypes.html#string-methods says:

str.center(width[, fillchar])

But, I tried in Windows 10 and Ubuntu Server:

>>> var = 'Variable de prueba'
>>> str.center(50, var)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: descriptor 'center' requires a 'str' object but received a 'int'

Then, I tried this:

>>> str.center(var, 50)
'                Variable de prueba                '

I think the documentation should read:
str.center(fillchar[, width])

Version in Windows: Python 3.6.5
Version in Ubuntu Server: Python 3.5.2

----------
assignee: docs at python
components: Documentation
messages: 315562
nosy: Alfonso Chavez, docs at python
priority: normal
severity: normal
status: open
title: Bug in documentation 3.6: string-methods - str.center
type: enhancement
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33324>
_______________________________________

From report at bugs.python.org  Sat Apr 21 08:09:20 2018
From: report at bugs.python.org (Serhiy Storchaka)
Date: Sat, 21 Apr 2018 12:09:20 +0000
Subject: [New-bugs-announce] [issue33325] Optimize sequences of constants in
 the compiler
Message-ID: <1524312560.56.0.682650639539.issue33325@psf.upfronthosting.co.za>


New submission from Serhiy Storchaka <storchaka+cpython at gmail.com>:

The following PR makes three optimizations in the compiler.

1. A sequence of several LOAD_CONSTs is replaced with a single LOAD_CONST followed by UNPACK_SEQUENCE.

For example, "{'a': 1, 'b': 2, 'c': 3}" is currently compiled to

  1           0 LOAD_CONST               0 (1)
              2 LOAD_CONST               1 (2)
              4 LOAD_CONST               2 (3)
              6 LOAD_CONST               3 (('a', 'b', 'c'))
              8 BUILD_CONST_KEY_MAP      3
             10 POP_TOP
             12 LOAD_CONST               4 (None)
             14 RETURN_VALUE

With this optimization it will be compiled to:

  1           0 LOAD_CONST               5 ((('a', 'b', 'c'), 3, 2, 1))
              2 UNPACK_SEQUENCE          4
              4 BUILD_CONST_KEY_MAP      3
              6 POP_TOP
              8 LOAD_CONST               4 (None)
             10 RETURN_VALUE

2. Optimized building lists and sets of constants. [1, 2, 3, 4, 5] will be compiled to [*(1, 2, 3, 4, 5)], and {1, 2, 3, 4, 5} will be compiled to {*frozenset(1, 2, 3, 4, 5)}, where (1, 2, 3, 4, 5) and frozenset(1, 2, 3, 4, 5) are just constants.

x = [1, 2, 3, 4, 5]
y = {1, 2, 3, 4, 5}

currently is compiled to

  1           0 LOAD_CONST               0 (1)
              2 LOAD_CONST               1 (2)
              4 LOAD_CONST               2 (3)
              6 LOAD_CONST               3 (4)
              8 LOAD_CONST               4 (5)
             10 BUILD_LIST               5
             12 STORE_NAME               0 (x)

  2          14 LOAD_CONST               0 (1)
             16 LOAD_CONST               1 (2)
             18 LOAD_CONST               2 (3)
             20 LOAD_CONST               3 (4)
             22 LOAD_CONST               4 (5)
             24 BUILD_SET                5
             26 STORE_NAME               1 (y)
             28 LOAD_CONST               5 (None)
             30 RETURN_VALUE

With optimization 1 it will be compiled to

  1           0 LOAD_CONST               6 ((5, 4, 3, 2, 1))
              2 UNPACK_SEQUENCE          5
              4 BUILD_LIST               5
              6 STORE_NAME               0 (x)

  2           8 LOAD_CONST               6 ((5, 4, 3, 2, 1))
             10 UNPACK_SEQUENCE          5
             12 BUILD_SET                5
             14 STORE_NAME               1 (y)
             16 LOAD_CONST               5 (None)
             18 RETURN_VALUE

And with optimization 2 it will be compiled to

  1           0 LOAD_CONST               0 ((1, 2, 3, 4, 5))
              2 BUILD_LIST_UNPACK        1
              4 STORE_NAME               0 (x)

  2           6 LOAD_CONST               1 (frozenset({1, 2, 3, 4, 5}))
              8 BUILD_SET_UNPACK         1
             10 STORE_NAME               1 (y)
             12 LOAD_CONST               2 (None)
             14 RETURN_VALUE

3. Remove unused constants.

After folding tuples of constants created at code generation level, eliminating unreachable code, and after the above two optimizations, unused constants are left in the co_consts tuple. The third optimization removes them and reenumerate constants in the order of occurrence. The above example will be compiled to:

  1           0 LOAD_CONST               0 ((1, 2, 3, 4, 5))
              2 BUILD_LIST_UNPACK        1
              4 STORE_NAME               0 (x)

  2           6 LOAD_CONST               1 (frozenset({1, 2, 3, 4, 5}))
              8 BUILD_SET_UNPACK         1
             10 STORE_NAME               1 (y)
             12 LOAD_CONST               2 (None)
             14 RETURN_VALUE

See issue28813 for the implementation of this optimization on the level of raw bytecode (in peephole.c). 

These optimizations are useful not only for initializing collections of constants. Calling function with constant arguments "f(x, a=1, b=2)":

Current:
  1           0 LOAD_NAME                0 (f)
              2 LOAD_NAME                1 (x)
              4 LOAD_CONST               0 (None)
              6 LOAD_CONST               1 (1)
              8 LOAD_CONST               2 (2)
             10 LOAD_CONST               3 (('a', 'b'))
             12 CALL_FUNCTION_KW         4
             14 POP_TOP
             16 LOAD_CONST               0 (None)
             18 RETURN_VALUE

Optimized:
  1           0 LOAD_NAME                0 (f)
              2 LOAD_NAME                1 (x)
              4 LOAD_CONST               0 ((('a', 'b'), 2, 1, None))
              6 UNPACK_SEQUENCE          4
              8 CALL_FUNCTION_KW         4
             10 POP_TOP
             12 LOAD_CONST               1 (None)
             14 RETURN_VALUE

This issue depends on issue33318.

----------
components: Interpreter Core
messages: 315563
nosy: benjamin.peterson, brett.cannon, inada.naoki, ncoghlan, pitrou, rhettinger, serhiy.storchaka, yselivanov
priority: normal
severity: normal
status: open
title: Optimize sequences of constants in the compiler
type: enhancement
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33325>
_______________________________________

From report at bugs.python.org  Sat Apr 21 14:52:42 2018
From: report at bugs.python.org (Kirill Balunov)
Date: Sat, 21 Apr 2018 18:52:42 +0000
Subject: [New-bugs-announce] [issue33326] Convert collections (cmp_op,
 hasconst, hasname and others) in opcode module to more optimal type
Message-ID: <1524336762.03.0.682650639539.issue33326@psf.upfronthosting.co.za>


New submission from Kirill Balunov <kirill.balunov at gmail.com>:

The opcode module contains several collections:

`cmp_op`
`hasconst`
`hasname`
`hasjrel`
...

which are only used for `in` checks. At the same time, they are stored as `list`s and `cmp_op` as tuple. Both these types are not optimal for `__contains__` checks. Maybe it is worth at least to convert them to `frozenset` type after they are filled?

----------
components: Library (Lib)
messages: 315576
nosy: godaygo
priority: normal
severity: normal
status: open
title: Convert collections (cmp_op, hasconst, hasname and others) in opcode module to more optimal type
type: performance
versions: Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33326>
_______________________________________

From report at bugs.python.org  Sat Apr 21 15:10:03 2018
From: report at bugs.python.org (Matej Cepl)
Date: Sat, 21 Apr 2018 19:10:03 +0000
Subject: [New-bugs-announce] [issue33327] Add a method to move messages to
 IMAPlib
Message-ID: <1524337803.72.0.682650639539.issue33327@psf.upfronthosting.co.za>


New submission from Matej Cepl <mcepl at cepl.eu>:

I am in the process of writing script working with IMAP first time using Python 3 for it (unfortunately, most of servers where I run other code is so ancient that even Python 2.7 is a stretch), and it is really nice experience so far. Many problems which are dealt with on the StackExchange with arcane workarounds are now resolved in email or imaplib libraries. Thank you, everybody!

However, it seems to me that few higher level commands could greatly improve useability of imaplib library. For example, moving messages (not fault of imaplib) is still horrible mess. In the end I had to write this monstrosity just to make moving messages working:


    Capas = collections.namedtuple('Capas', ['MOVE', 'UIDPLUS'])

    def __login(self, host='localhost', username=None, password=None, ssl=None):
        self.box = imaplib.IMAP4_SSL(host=host)
        ok, data = self.box.login(username, password)
        if ok != 'OK':
            raise IOError('Cannot login with credentials %s' %
                          str((host, username, password,)))

        ok, data = box.capability()
        capas = data[0].decode()
        box.features_present = Capas._make(['MOVE' in capas, 'UIDPLUS' in capas])

    def move_messages(self, target, messages):
        if self.box.features_present.MOVE:
            ok, data = self.box.uid('MOVE', '%s %s' % (messages, target))
            if ok != 'OK':
                raise IOError('Cannot move messages to folder %s' % target)
        elif self.box.features_present.UIDPLUS:
            ok, data = self.box.uid('COPY', '%s %s' % (messages, target))
            if ok != 'OK':
                raise IOError('Cannot copy messages to folder %s' % target)
            ok, data = self.box.uid('STORE',
                                    r'+FLAGS.SILENT (\DELETED) %s' % messages)
            if ok != 'OK':
                raise IOError('Cannot delete messages.')
            ok, data = self.box.uid('EXPUNGE', messages)
            if ok != 'OK':
                raise IOError('Cannot expunge messages.')
        else:
            ok, data = self.box.uid('COPY', '%s %s' % (messages, target))
            if ok != 'OK':
                raise IOError('Cannot copy messages to folder %s' % target)
            ok, data = self.box.uid('STORE',
                                    r'+FLAGS.SILENT (\DELETED) %s' % messages)
            if ok != 'OK':
                raise IOError('Cannot delete messages.')

It would be nice if some capabilities detection (see issue 18921) was embedded into login method, and if some version of move_messages was included in imaplib itself.

----------
components: Library (Lib)
messages: 315578
nosy: mcepl
priority: normal
severity: normal
status: open
title: Add a method to move messages to IMAPlib
type: enhancement
versions: Python 2.7, Python 3.4, Python 3.5, Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33327>
_______________________________________

From report at bugs.python.org  Sat Apr 21 23:03:53 2018
From: report at bugs.python.org (Rick Teachey)
Date: Sun, 22 Apr 2018 03:03:53 +0000
Subject: [New-bugs-announce] [issue33328] pdb error when stepping through
 generator
Message-ID: <1524366233.1.0.682650639539.issue33328@psf.upfronthosting.co.za>


New submission from Rick Teachey <ricky at teachey.org>:

Hello: 

The attached python file runs a pdb interactive session for a generator. It produces an error in Python 3.7 Beta 3, but no error in 3.6.

I searched for this issue and there do seem to be things that are related to it already, but I haven't investigated the cause of this issue and am not acquainted with the details of the pdb module, so I was unable to determine whether previous reports discuss this same problem.

----------
files: demo.py
messages: 315586
nosy: Ricyteach
priority: normal
severity: normal
status: open
title: pdb error when stepping through generator
versions: Python 3.7
Added file: https://bugs.python.org/file47545/demo.py

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33328>
_______________________________________

From report at bugs.python.org  Sun Apr 22 04:07:51 2018
From: report at bugs.python.org (=?utf-8?q?Miro_Hron=C4=8Dok?=)
Date: Sun, 22 Apr 2018 08:07:51 +0000
Subject: [New-bugs-announce] [issue33329] test_multiprocessing_fork,
 _forkserver, _spawn hangs: ValueError: signal number 32 out of range
Message-ID: <1524384471.45.0.682650639539.issue33329@psf.upfronthosting.co.za>


New submission from Miro Hron?ok <miro at hroncok.cz>:

Originally reported in https://bugs.python.org/issue31463

This started to bother us in Fedora rawhide for various Python versions, so chances are something changed on the system level.

# python3.7 -m test.regrtest test_multiprocessing_fork 
Run tests sequentially
0:00:00 load avg: 1.24 [1/1] test_multiprocessing_fork
/usr/lib64/python3.7/multiprocessing/semaphore_tracker.py:55: UserWarning: semaphore_tracker: process died unexpectedly, relaunching.  Some semaphores might leak.
  warnings.warn('semaphore_tracker: process died unexpectedly, '
Exception in thread Thread-26:
Traceback (most recent call last):
  File "/usr/lib64/python3.7/threading.py", line 917, in _bootstrap_inner
    self.run()
  File "/usr/lib64/python3.7/threading.py", line 865, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/lib64/python3.7/multiprocessing/resource_sharer.py", line 139, in _serve
    signal.pthread_sigmask(signal.SIG_BLOCK, range(1, signal.NSIG))
  File "/usr/lib64/python3.7/signal.py", line 60, in pthread_sigmask
    sigs_set = _signal.pthread_sigmask(how, mask)
ValueError: signal number 32 out of range


(hangs...)

^CProcess Process-184:
Traceback (most recent call last):
  File "/usr/lib64/python3.7/multiprocessing/process.py", line 297, in _bootstrap
    self.run()
  File "/usr/lib64/python3.7/multiprocessing/process.py", line 99, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/lib64/python3.7/test/_test_multiprocessing.py", line 3328, in child_access
    w = conn.recv()
  File "/usr/lib64/python3.7/multiprocessing/connection.py", line 251, in recv
    return _ForkingPickler.loads(buf.getbuffer())
  File "/usr/lib64/python3.7/multiprocessing/connection.py", line 960, in rebuild_connection
    fd = df.detach()
  File "/usr/lib64/python3.7/multiprocessing/resource_sharer.py", line 57, in detach
    with _resource_sharer.get_connection(self._id) as conn:
  File "/usr/lib64/python3.7/multiprocessing/resource_sharer.py", line 87, in get_connection
    c = Client(address, authkey=process.current_process().authkey)
  File "/usr/lib64/python3.7/multiprocessing/connection.py", line 498, in Client
    answer_challenge(c, authkey)
  File "/usr/lib64/python3.7/multiprocessing/connection.py", line 741, in answer_challenge
    message = connection.recv_bytes(256)         # reject large message
  File "/usr/lib64/python3.7/multiprocessing/connection.py", line 216, in recv_bytes
    buf = self._recv_bytes(maxlength)
  File "/usr/lib64/python3.7/multiprocessing/connection.py", line 407, in _recv_bytes
    buf = self._recv(4)
  File "/usr/lib64/python3.7/multiprocessing/connection.py", line 379, in _recv
    chunk = read(handle, remaining)
KeyboardInterrupt
Warning -- multiprocessing.process._dangling was modified by test_multiprocessing_fork
  Before: <_weakrefset.WeakSet object at 0x7f2abd5f6d30>
  After:  <_weakrefset.WeakSet object at 0x7f2abd5f6128> 
Warning -- threading._dangling was modified by test_multiprocessing_fork
  Before: <_weakrefset.WeakSet object at 0x7f2abd5f66d8>
  After:  <_weakrefset.WeakSet object at 0x7f2abc835048> 

Test suite interrupted by signal SIGINT.
1 test omitted:
    test_multiprocessing_fork

Total duration: 3 min 43 sec
Tests result: INTERRUPTED

--------------------------------

# python3.7 -m test.regrtest test_multiprocessing_forkserver
Run tests sequentially
0:00:00 load avg: 1.00 [1/1] test_multiprocessing_forkserver
Exception in thread Thread-26:
Traceback (most recent call last):
  File "/usr/lib64/python3.7/threading.py", line 917, in _bootstrap_inner
    self.run()
  File "/usr/lib64/python3.7/threading.py", line 865, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/lib64/python3.7/multiprocessing/resource_sharer.py", line 139, in _serve
    signal.pthread_sigmask(signal.SIG_BLOCK, range(1, signal.NSIG))
  File "/usr/lib64/python3.7/signal.py", line 60, in pthread_sigmask
    sigs_set = _signal.pthread_sigmask(how, mask)
ValueError: signal number 32 out of range

(hangs...)

^CProcess Process-184:
Traceback (most recent call last):
  ...
KeyboardInterrupt
Warning -- multiprocessing.process._dangling was modified by test_multiprocessing_forkserver
  Before: <_weakrefset.WeakSet object at 0x7f0323ecde10>
  After:  <_weakrefset.WeakSet object at 0x7f0323ecd208> 
Warning -- threading._dangling was modified by test_multiprocessing_forkserver
  Before: <_weakrefset.WeakSet object at 0x7f0323ecd7b8>
  After:  <_weakrefset.WeakSet object at 0x7f0323ecddd8> 

Test suite interrupted by signal SIGINT.
1 test omitted:
    test_multiprocessing_forkserver

Total duration: 55 sec
Tests result: INTERRUPTED

--------------------------------

# python3.7 -m test.regrtest test_multiprocessing_spawn
Run tests sequentially
0:00:00 load avg: 1.49 [1/1] test_multiprocessing_spawn
Exception in thread Thread-26:
Traceback (most recent call last):
  File "/usr/lib64/python3.7/threading.py", line 917, in _bootstrap_inner
    self.run()
  File "/usr/lib64/python3.7/threading.py", line 865, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/lib64/python3.7/multiprocessing/resource_sharer.py", line 139, in _serve
    signal.pthread_sigmask(signal.SIG_BLOCK, range(1, signal.NSIG))
  File "/usr/lib64/python3.7/signal.py", line 60, in pthread_sigmask
    sigs_set = _signal.pthread_sigmask(how, mask)
ValueError: signal number 32 out of range

(hangs...)

^CProcess Process-184:
Traceback (most recent call last):
  ...
KeyboardInterrupt
Warning -- multiprocessing.process._dangling was modified by test_multiprocessing_spawn
  Before: <_weakrefset.WeakSet object at 0x7fb2134a9dd8>
  After:  <_weakrefset.WeakSet object at 0x7fb2134a91d0> 
Warning -- threading._dangling was modified by test_multiprocessing_spawn
  Before: <_weakrefset.WeakSet object at 0x7fb2134a9780>
  After:  <_weakrefset.WeakSet object at 0x7fb2134a9da0> 

Test suite interrupted by signal SIGINT.
1 test omitted:
    test_multiprocessing_spawn

Total duration: 54 sec
Tests result: INTERRUPTED

--------------------------------

Happens with all 3 tests on 3.4 to 3.7.

To test in docker, one can do:

$ docker run -ti fedora:rawhide /bin/bash
# dnf update
# dnf install python37  # or python3-test for 3.6, or python35, python34

Note that without `dnf update`, the tests work for me (for now) so attaching a full package diff that "starts" this.

----------
components: Tests
files: upgraded_packages.txt
messages: 315590
nosy: hroncok
priority: normal
severity: normal
status: open
title: test_multiprocessing_fork, _forkserver, _spawn hangs: ValueError: signal number 32 out of range
versions: Python 3.4, Python 3.5, Python 3.6, Python 3.7
Added file: https://bugs.python.org/file47546/upgraded_packages.txt

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33329>
_______________________________________

From report at bugs.python.org  Sun Apr 22 04:56:30 2018
From: report at bugs.python.org (Serhiy Storchaka)
Date: Sun, 22 Apr 2018 08:56:30 +0000
Subject: [New-bugs-announce] [issue33330] Better error handling in
 PyImport_Cleanup()
Message-ID: <1524387390.7.0.682650639539.issue33330@psf.upfronthosting.co.za>


New submission from Serhiy Storchaka <storchaka+cpython at gmail.com>:

The proposed PR adds missed checks for errors and PyErr_Clear() calls in PyImport_Cleanup().

It may be worth to use PyErr_WriteUnraisable() in place of PyErr_Clear() (newly added and existing) in most cases. But this can add an unexpected noise, and I don't wont to do this at this stage of 3.7. It could be done in 3.8 though, and backported to 3.7 later.

I'm not sure that PyImport_Cleanup() works properly if sys.modules is not a dict. At this stage a custom mapping will likely be broken when most of modules are destroyed.

----------
components: Interpreter Core
messages: 315592
nosy: brett.cannon, eric.snow, ncoghlan, pitrou, serhiy.storchaka, vstinner
priority: normal
severity: normal
status: open
title: Better error handling in PyImport_Cleanup()
type: behavior
versions: Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33330>
_______________________________________

From report at bugs.python.org  Sun Apr 22 06:38:18 2018
From: report at bugs.python.org (Serhiy Storchaka)
Date: Sun, 22 Apr 2018 10:38:18 +0000
Subject: [New-bugs-announce] [issue33331] Clean modules in the reversed order
Message-ID: <1524393498.13.0.682650639539.issue33331@psf.upfronthosting.co.za>


New submission from Serhiy Storchaka <storchaka+cpython at gmail.com>:

Since dict is ordered, modules in sys.modules are ordered for the time of importing. Currently they are cleaned in PyImport_Cleanup() in the direct order -- from imported first to imported later. I wonder if cleaning them in the reversed order can solve some problems with the interpreter shutdown.

For example reverting the order fixes issue33328 and may help in other cases.

If revert the order, should only iterating weaklist be reverted (with setting all module globals to None), or iterating sys.modules (with setting sys.module values to None) too?

----------
components: Interpreter Core
messages: 315603
nosy: brett.cannon, eric.snow, ncoghlan, pitrou, serhiy.storchaka
priority: normal
severity: normal
status: open
title: Clean modules in the reversed order
versions: Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33331>
_______________________________________

From report at bugs.python.org  Sun Apr 22 07:03:27 2018
From: report at bugs.python.org (Antoine Pitrou)
Date: Sun, 22 Apr 2018 11:03:27 +0000
Subject: [New-bugs-announce] [issue33332] Expose valid signal set
 (sigfillset())
Message-ID: <1524395007.64.0.682650639539.issue33332@psf.upfronthosting.co.za>


New submission from Antoine Pitrou <pitrou at free.fr>:

See issue33329: it would be nice to expose a signal module function to get the set of user-actionable signals.  Hopefully sigfillset() does that.

----------
messages: 315606
nosy: pitrou
priority: normal
severity: normal
stage: needs patch
status: open
title: Expose valid signal set (sigfillset())
type: enhancement
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33332>
_______________________________________

From report at bugs.python.org  Sun Apr 22 07:20:36 2018
From: report at bugs.python.org (Chris Bradbury)
Date: Sun, 22 Apr 2018 11:20:36 +0000
Subject: [New-bugs-announce] [issue33333] ConfigParser.items returns items
 present in `DEFAULTSECT` when section argument given.
Message-ID: <1524396036.58.0.682650639539.issue33333@psf.upfronthosting.co.za>


New submission from Chris Bradbury <viff3000 at gmail.com>:

According to `ConfigParser.items` docs:

> When section is not given, return a list of section_name, section_proxy pairs, including DEFAULTSECT.

> Otherwise, return a list of name, value pairs for the options in the given section. Optional arguments have the same meaning as for the get() method.

https://docs.python.org/3/library/configparser.html#configparser.ConfigParser.items

However due to `d = self._defaults.copy()` on line 843 of configparser.py the default section is always included, even when a section argument is specified.

----------
messages: 315608
nosy: chrBrd
priority: normal
severity: normal
status: open
title: ConfigParser.items returns items present in `DEFAULTSECT` when section argument given.
type: behavior
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33333>
_______________________________________

From report at bugs.python.org  Sun Apr 22 13:11:17 2018
From: report at bugs.python.org (Serhiy Storchaka)
Date: Sun, 22 Apr 2018 17:11:17 +0000
Subject: [New-bugs-announce] [issue33334] Add support of NOP and
 EXTENDED_ARG in stack_effect()
Message-ID: <1524417077.19.0.682650639539.issue33334@psf.upfronthosting.co.za>


New submission from Serhiy Storchaka <storchaka+cpython at gmail.com>:

Currently dis.stack_effect() doesn't support opcodes NOP and EXTENDED_ARG. NOP is never emitted by the standard compiler (it is temporary added in the peephole optimizer, but later it is removed). EXTENDED_ARG is a special case, it is considered as a part of long instructions.

dis.stack_effect() itself is not used in the dis module. It can be used in the third-party code, and the third-party code can produce bytecode with non-standard use of NOP and EXTENDED_ARG. Supporting them in dis.stack_effect() can avoid the need of special casing them in the third-party code.

I don't know whether this is a bug fix or a new feature.

----------
components: Library (Lib)
messages: 315622
nosy: larry, serhiy.storchaka
priority: normal
severity: normal
status: open
title: Add support of NOP and EXTENDED_ARG in stack_effect()
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33334>
_______________________________________

From report at bugs.python.org  Sun Apr 22 14:34:36 2018
From: report at bugs.python.org (J E)
Date: Sun, 22 Apr 2018 18:34:36 +0000
Subject: [New-bugs-announce] [issue33335] turtle.onkey doesn't pass key
 information when key is None
Message-ID: <1524422076.99.0.682650639539.issue33335@psf.upfronthosting.co.za>


New submission from J E <jepst79 at gmail.com>:

I use the turtle module to teach Python to university students. The turtle.onkey function allows a program to register a callback function to handle a keyboard when the turtle window has a focus. For example, this call will result in myfun() getting called when the "s" key is pressed:

def myfun():
  print("You pressed s")

turtle.onkey(myfun, "s")
turtle.listen()

But what if I want to handle all keys? I could create a bunch of functions, and register a separate callback for each possible; that's not a great solution. A better solution is to let turtle call a single function when any key is pressed. Fortunately, this is possible by passing None in for the key:

def myfun():
  print("You pressed some key")

turtle.onkey(myfun, None)
turtle.listen()

However: there is no way for myfun to determine which key was pressed. It is called with no parameters, and has no way to recover information about which key was pressed, so this approach is useless. The only way to handle all keyboard events it to call onkey, with a separate handler, for every key you potentially want to handle.

>From the source [1] this looks like it was intended that when key is None, turtle would pass a Tkinter event to the handler, so that it could recover information about the event, but this doesn't happen.

[1] https://hg.python.org/cpython/file/3.6/Lib/turtle.py#l700

I would argue that correct behavior is not to expose Tkinter event to users of the higher-level turtle library, but rather to pass the name of the key as a string parameter, so you could write this:

def myfun(key):
  print("You pressed "+key)

turtle.onkey(myfun, None)
turtle.listen()

This is an easy fix and I'd be happy to submit a patch.

----------
components: Tkinter
messages: 315627
nosy: je1234
priority: normal
severity: normal
status: open
title: turtle.onkey doesn't pass key information when key is None
type: behavior
versions: Python 2.7, Python 3.4, Python 3.5, Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33335>
_______________________________________

From report at bugs.python.org  Sun Apr 22 18:29:42 2018
From: report at bugs.python.org (Matej Cepl)
Date: Sun, 22 Apr 2018 22:29:42 +0000
Subject: [New-bugs-announce] [issue33336] [imaplib] MOVE is a legal command
Message-ID: <1524436182.1.0.682650639539.issue33336@psf.upfronthosting.co.za>


New submission from Matej Cepl <mcepl at cepl.eu>:

When running

    box = IMAP4_SSL(host)
    box.login(user, pass)
    msgs = somehowget_uids_of_messsages_to_move()
    box.uid('MOVE', msgs, 'target')

I get an error "Unknown IMAP4 UID command: MOVE". The problem is that imaplib contains a list Commands of the permitted IMAP commands, and MOVE is not included, even though it is perfectly legal (and quite widely supported) command according to RFC-6851.

----------
components: Library (Lib)
messages: 315632
nosy: mcepl
priority: normal
severity: normal
status: open
title: [imaplib] MOVE is a legal command
versions: Python 3.4, Python 3.5, Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33336>
_______________________________________

From report at bugs.python.org  Sun Apr 22 21:04:07 2018
From: report at bugs.python.org (=?utf-8?q?=C5=81ukasz_Langa?=)
Date: Mon, 23 Apr 2018 01:04:07 +0000
Subject: [New-bugs-announce] [issue33337] Provide a supported Concrete
 Syntax Tree implementation in the standard library
Message-ID: <1524445447.99.0.682650639539.issue33337@psf.upfronthosting.co.za>


New submission from ?ukasz Langa <lukasz at langa.pl>:

Python includes a set of batteries that enable parsing of Python code.  This
includes its own AST (provided in the standard library under the `ast` module),
as well as a pure Python tokenizer (provided in the standard library under
`tokenize` and `token`).  It also provides an undocumented CST under lib2to3,
which contains its own outdated and patched copies of `tokenize` and `token`.

This situation causes the following issues for users of Python:
- the built-in AST does not preserve comments or whitespace;
- the built-in AST increasingly modifies the tree before presenting it to user
  code (constant folding moved to the AST in Python 3.7);
- the built-in tokenize.py can only be used to parse Python 3.7+ code;
- the version in lib2to3 is partially customized and partially outdated,
  leaving bits of new grammar not supported; new bits of grammar very often get
  overlooked in lib2to3.
- lib2to3 is not documented.

So if users want to write tools that manipulate Python code, the standard
library doesn't provide them with great options.

I suggest the following plan:

1. Bring Lib/lib2to3/pgen2/tokenize.py to the same state as Lib/tokenize.py
   (leaving the bits that allow for parsing of Python 3.6 and older files).

2. Merge the two tokenizers in Python 3.8 so that Lib/tokenize.py now
   officially supports tokenizing Python 2.7 - 3.7 code.

3. Update Lib/lib2to3/pgen2 and move it under Lib/pgen.  Document it as the
   built-in CST provided by Python for use in applications which require code
   modification.  Make it still officially support parsing of Python 2.7 - 3.7
   code.

All three changes are made in a backwards-compatible fashion, existing code
should NOT break.  That being said, the parser under Lib/pgen might grow some
new behavior compared to the compatibility mode for lib2to3, I specifically
seek to improve handling of comments and error recovery.

----------
components: Library (Lib)
messages: 315638
nosy: benjamin.peterson, gregory.p.smith, gvanrossum, lukasz.langa, serhiy.storchaka
priority: normal
severity: normal
status: open
title: Provide a supported Concrete Syntax Tree implementation in the standard library
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33337>
_______________________________________

From report at bugs.python.org  Sun Apr 22 21:04:56 2018
From: report at bugs.python.org (=?utf-8?q?=C5=81ukasz_Langa?=)
Date: Mon, 23 Apr 2018 01:04:56 +0000
Subject: [New-bugs-announce] [issue33338] [lib2to3] Synchronize token.py and
 tokenize.py with the standard library
Message-ID: <1524445496.4.0.682650639539.issue33338@psf.upfronthosting.co.za>


New submission from ?ukasz Langa <lukasz at langa.pl>:

lib2to3's token.py and tokenize.py were initially copies of the respective
files from the standard library.  They were copied to allow Python 3 to read
Python 2's grammar.

Since 2006, lib2to3 grew to be widely used as a Concrete Syntax Tree, also for
parsing Python 3 code.  Additions to support Python 3 grammar were added but
sadly, the main token.py and tokenize.py diverged.

This change brings them back together, minimizing the differences to the bare
minimum that is in fact required by lib2to3.  Before this change, almost every
line in lib2to3/pgen2/tokenize.py was different from tokenize.py.  After this
change, the diff between the two files is only 175 lines long and is entirely
filled with relevant Python 2 compatibility bits.

Merging the implementations, there's numerous fixes to the lib2to3 tokenizer:

+ docstrings made as similar as possible
+ ported `TokenInfo`
+ ported `tokenize.tokenize()` and `tokenize.open()`
+ removed Python 2-only implementation cruft
+ fixes Unicode identifier handling
+ fixes string prefix handling
+ fixes Ellipsis handling
+ Untokenizer backported bugfixes:
	- 5e6db313686c200da425a54d2e0c95fa40107b1d
	- 9dc3a36c849c15c227a8af218cfb215abe7b3c48
	- 5b8d2c3af76e704926cf5915ad0e6af59a232e61
	- e411b6629fb5f7bc01bec89df75737875ce6d8f5
        - BPO-2495
+ tokenizer doesn't crash on missing newline at the end of the
stream (added \Z (end of string) to PseudoExtras) - BPO-16152
+ `find_cookie` includes file name in error messages, if available
+ `find_cookie` raises SyntaxError on invalid encodings: BPO-14990

Improvements to lib2to3/pgen2/token.py:

+ taken from the current Lib/token.py
+ tokens renumbered to match Lib/token.py
+ `__all__` properly defined
+ ASYNC, AWAIT and BACKQUOTE exist under different numbers (100 + old number)
+ ELLIPSIS added
+ ENCODING added

----------
components: 2to3 (2.x to 3.x conversion tool), Library (Lib)
messages: 315639
nosy: lukasz.langa
priority: normal
severity: normal
status: open
title: [lib2to3] Synchronize token.py and tokenize.py with the standard library
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33338>
_______________________________________

From report at bugs.python.org  Mon Apr 23 04:38:32 2018
From: report at bugs.python.org (=?utf-8?q?Pekka_Kl=C3=A4rck?=)
Date: Mon, 23 Apr 2018 08:38:32 +0000
Subject: [New-bugs-announce] [issue33339] Using default encoding with
 `subprocess.run()` is not obvious
Message-ID: <1524472712.16.0.682650639539.issue33339@psf.upfronthosting.co.za>


New submission from Pekka Kl?rck <pekka.klarck at gmail.com>:

It is possible to use `subprocess.run()` with the system's default encoding by using `universal_newlines=True`. This is very handy, but it's not at all obvious (at least for me) that setting such option has effect on encoding. This is especially true when the docs don't explain this explicitly:

"""If encoding or errors are specified, or universal_newlines is true, file objects for stdin, stdout and stderr are opened in text mode using the specified encoding and errors or the io.TextIOWrapper default."""

This is such a useful feature that it ought to be documented better, preferably with an example.

>From code reading point of view, I would also consider `encoding=True` to be more explicit that `universal_newlines=True`. I can submit a separate issue about that if others feel the same.

----------
messages: 315652
nosy: pekka.klarck
priority: normal
severity: normal
status: open
title: Using default encoding with `subprocess.run()` is not obvious

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33339>
_______________________________________

From report at bugs.python.org  Mon Apr 23 05:51:36 2018
From: report at bugs.python.org (sam_b)
Date: Mon, 23 Apr 2018 09:51:36 +0000
Subject: [New-bugs-announce] [issue33340] Inaccurate docs on `import`
 behaviour
Message-ID: <1524477096.11.0.682650639539.issue33340@psf.upfronthosting.co.za>


New submission from sam_b <sam at sambrown.eu>:

The docs https://docs.python.org/3/tutorial/modules.html#the-module-search-path describe:

> When a module named spam is imported, the interpreter first searches for a built-in module with that name. If not found, it then searches for a file named spam.py in a list of directories given by the variable sys.path. sys.path is initialized from these locations:

> -   The directory containing the input script (or the current directory when no file is specified).
> -   PYTHONPATH (a list of directory names, with the same syntax as the shell variable PATH).
> -   The installation-dependent default.


However, it seems like "the directory containing the input script" is checked *before* the standard library:

?  tmp more logging.py
def foo():
    print('bar')
?  tmp python
Python 2.7.15rc1 (default, Apr 15 2018, 21:51:34)
[GCC 7.3.0] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import logging
>>> logging.foo()
bar
>>> logging.WARNING
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'module' object has no attribute 'WARNING'
>>> 


Am I misunderstanding the docs?

----------
assignee: docs at python
components: Documentation
messages: 315653
nosy: docs at python, sam_b
priority: normal
severity: normal
status: open
title: Inaccurate docs on `import` behaviour
type: behavior
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33340>
_______________________________________

From report at bugs.python.org  Mon Apr 23 06:12:22 2018
From: report at bugs.python.org (Gianfranco)
Date: Mon, 23 Apr 2018 10:12:22 +0000
Subject: [New-bugs-announce] [issue33341] python3 fails to build if
 directory or sysroot contains "*icc*" string
Message-ID: <1524478342.39.0.682650639539.issue33341@psf.upfronthosting.co.za>


New submission from Gianfranco <costamagnagianfranco at yahoo.it>:

Hello, as said, in yocto we pass in $CC environment, also the --sysroot keyword.

If --sysroot contains the "*icc*" or "*gcc*" in path, the build fails because of wrong autoconf checks.

checking in a case switch for *icc* seems bad to me, I propose to change it with something like
"*icc" so we might exclude when icc is not the compiler.

I know this isn't a strong patch, but maybe we can apply it while writing some better detect code.

The build fails as descripted in the yocto bug
https://bugzilla.yoctoproject.org/show_bug.cgi?id=12703

----------
components: Build
files: patch
messages: 315656
nosy: locutusofborg
priority: normal
severity: normal
status: open
title: python3 fails to build if directory or sysroot contains "*icc*" string
type: compile error
versions: Python 3.8
Added file: https://bugs.python.org/file47547/patch

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33341>
_______________________________________

From report at bugs.python.org  Mon Apr 23 09:44:30 2018
From: report at bugs.python.org (benaryorg)
Date: Mon, 23 Apr 2018 13:44:30 +0000
Subject: [New-bugs-announce] [issue33342] urllib IPv6 parsing fails with
 special characters in passwords
Message-ID: <1524491070.85.0.682650639539.issue33342@psf.upfronthosting.co.za>


New submission from benaryorg <binary at benary.org>:

The documentation specifies to follow RFC 2396 (https://tools.ietf.org/html/rfc2396.html) but fails to parse a user:password at host url in urllib.parse.urlsplit (https://docs.python.org/3/library/urllib.parse.html#urllib.parse.urlsplit) when the password contains an '[' character.
This is because the urlsplit code does not strip the authority part (everything from index 0 up to and including the last '@') before checking whether the hostname contains '[' for detecting whether it's an IPv6 address (https://github.com/python/cpython/blob/8a6f4b4bba950fb8eead1b176c58202d773f2f70/Lib/urllib/parse.py#L416-L418).

----------
components: Library (Lib)
messages: 315668
nosy: benaryorg
priority: normal
severity: normal
status: open
title: urllib IPv6 parsing fails with special characters in passwords
versions: Python 2.7, Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33342>
_______________________________________

From report at bugs.python.org  Mon Apr 23 13:46:18 2018
From: report at bugs.python.org (Victor Porton)
Date: Mon, 23 Apr 2018 17:46:18 +0000
Subject: [New-bugs-announce] [issue33343] Subcommand abbreviations
Message-ID: <1524505578.79.0.682650639539.issue33343@psf.upfronthosting.co.za>


New submission from Victor Porton <porton at narod.ru>:

https://docs.python.org/3/library/argparse.html does not allow abbreviations for subparsers ("subcommands").

It seems inconsistent that long options can be abbreviated but subcommands cannot.

Either implement subcommand abbreviations or explain in the docs why there is none.

----------
components: Library (Lib)
messages: 315671
nosy: porton
priority: normal
severity: normal
status: open
title: Subcommand abbreviations
type: enhancement
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33343>
_______________________________________

From report at bugs.python.org  Tue Apr 24 03:31:12 2018
From: report at bugs.python.org (Kasra Vand)
Date: Tue, 24 Apr 2018 07:31:12 +0000
Subject: [New-bugs-announce] [issue33344] Use logical negation of integers
 directly in arithmatic propostions and equations
Message-ID: <1524555072.69.0.682650639539.issue33344@psf.upfronthosting.co.za>


New submission from Kasra Vand <kasraavand at gmail.com>:

Logical Negation of integers in Python always returns a Boolean result which can be achieve using `not`. Sometimes it's necessary to use this result directly in a proposition within a list comprehension (mostly). But if we use `not` directly in such arithmatic proposition it will raise a `SyntaxError.

```
In [46]: 3 + not(4)
  File "<ipython-input-46-e87dc6b7d800>", line 1
    3 + not(4)
          ^
SyntaxError: invalid syntax

```

Isn't that possible to make this work for integers in future releases? I was more curious to know what would be the drawbacks of such feature if there are any?

----------
components: Interpreter Core
messages: 315688
nosy: Kasra Vand
priority: normal
severity: normal
status: open
title: Use logical negation of integers directly in arithmatic propostions and equations
type: behavior
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33344>
_______________________________________

From report at bugs.python.org  Tue Apr 24 03:44:04 2018
From: report at bugs.python.org (Steve Piercy)
Date: Tue, 24 Apr 2018 07:44:04 +0000
Subject: [New-bugs-announce] [issue33345] Documentation for PowerShell
 instructions
Message-ID: <1524555844.08.0.682650639539.issue33345@psf.upfronthosting.co.za>


New submission from Steve Piercy <web at stevepiercy.com>:

Effective 2017-11-20, PowerShell is now the default command shell on Windows 10, and possibly other versions.  Command Prompt (cmd.exe) is still present, just no longer the default.

https://support.microsoft.com/en-us/help/4027690/windows-powershell-is-replacing-command-prompt

Current instructions for installing and configuring Python on Windows reference only Command Prompt.  There may be other places.

https://docs.python.org/3/using/windows.html#configuring-python

FYI, we're in the process of updating our documentation for Pyramid where we have both PowerShell and Command Prompt.  At some point in the future, we may drop the latter.

https://github.com/Pylons/pyramid/issues/3260

Let me know what y'all think is the best way to proceed, even if that is no action.

----------
assignee: docs at python
components: Documentation
messages: 315690
nosy: docs at python, stevepiercy
priority: normal
severity: normal
status: open
title: Documentation for PowerShell instructions
type: enhancement
versions: Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33345>
_______________________________________

From report at bugs.python.org  Tue Apr 24 07:05:41 2018
From: report at bugs.python.org (Pablo Galindo Salgado)
Date: Tue, 24 Apr 2018 11:05:41 +0000
Subject: [New-bugs-announce] [issue33346] Syntax error with async generator
 inside dictionary comprehension
Message-ID: <1524567941.57.0.682650639539.issue33346@psf.upfronthosting.co.za>


New submission from Pablo Galindo Salgado <pablogsal at gmail.com>:

Given this async function:

async def elements(n):
    yield n
    yield n*2
    yield n*3
    yield n*4

This definition is considered invalid:

async def test():
    return { n: [x async for x in elements(n)] for n in range(3)}

SyntaxError: asynchronous comprehension outside of an asynchronous function

The reason (I suspect) is because the dict comprehension is not an async context (it would be if range was an async iterable and we use `async for n in range(3)`). 

Is this expected behaviour or something it needs to change?

----------
components: asyncio
messages: 315692
nosy: asvetlov, pablogsal, yselivanov
priority: normal
severity: normal
status: open
title: Syntax error with async generator inside dictionary comprehension
type: behavior
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33346>
_______________________________________

From report at bugs.python.org  Tue Apr 24 07:06:00 2018
From: report at bugs.python.org (Lucian Cristian)
Date: Tue, 24 Apr 2018 11:06:00 +0000
Subject: [New-bugs-announce] [issue33347] zlibmodule undefined reference
Message-ID: <1524567960.98.0.682650639539.issue33347@psf.upfronthosting.co.za>


New submission from Lucian Cristian <lucian.cristian at gmail.com>:

trying to compile a utility in Openwrt and when pip-install is selected and zlib is linked to python I get this:


staging_dir/hostpkg/lib/libpython2.7.a(zlibmodule.o): In function `PyZlib_crc32':
build_dir/hostpkg/Python-2.7.14/./Modules/zlibmodule.c:1045: undefined reference to `crc32'
build_dir/hostpkg/Python-2.7.14/./Modules/zlibmodule.c:1053: undefined reference to `crc32'
staging_dir/hostpkg/lib/libpython2.7.a(zlibmodule.o): In function `PyZlib_adler32':
build_dir/hostpkg/Python-2.7.14/./Modules/zlibmodule.c:1013: undefined reference to `adler32'
build_dir/hostpkg/Python-2.7.14/./Modules/zlibmodule.c:1021: undefined reference to `adler32'
staging_dir/hostpkg/lib/libpython2.7.a(zlibmodule.o): In function `Decomp_dealloc':
build_dir/hostpkg/Python-2.7.14/./Modules/zlibmodule.c:469: undefined reference to `inflateEnd'
staging_dir/hostpkg/lib/libpython2.7.a(zlibmodule.o): In function `Comp_dealloc':
build_dir/hostpkg/Python-2.7.14/./Modules/zlibmodule.c:459: undefined reference to `deflateEnd'
staging_dir/hostpkg/lib/libpython2.7.a(zlibmodule.o): In function `PyZlib_objcompress':
build_dir/hostpkg/Python-2.7.14/./Modules/zlibmodule.c:504: undefined reference to `deflate'
staging_dir/hostpkg/lib/libpython2.7.a(zlibmodule.o): In function `PyZlib_copy':
build_dir/hostpkg/Python-2.7.14/./Modules/zlibmodule.c:772: undefined reference to `deflateCopy'
staging_dir/hostpkg/lib/libpython2.7.a(zlibmodule.o): In function `PyZlib_uncopy':
build_dir/hostpkg/Python-2.7.14/./Modules/zlibmodule.c:821: undefined reference to `inflateCopy'
staging_dir/hostpkg/lib/libpython2.7.a(zlibmodule.o): In function `PyZlib_decompressobj':
build_dir/hostpkg/Python-2.7.14/./Modules/zlibmodule.c:434: undefined reference to `inflateInit2_'
staging_dir/hostpkg/lib/libpython2.7.a(zlibmodule.o): In function `PyZlib_compressobj':
build_dir/hostpkg/Python-2.7.14/./Modules/zlibmodule.c:398: undefined reference to `deflateInit2_'
staging_dir/hostpkg/lib/libpython2.7.a(zlibmodule.o): In function `PyZlib_objdecompress':
build_dir/hostpkg/Python-2.7.14/./Modules/zlibmodule.c:632: undefined reference to `inflate'
staging_dir/hostpkg/lib/libpython2.7.a(zlibmodule.o): In function `PyZlib_decompress':
build_dir/hostpkg/Python-2.7.14/./Modules/zlibmodule.c:308: undefined reference to `inflateInit2_'
build_dir/hostpkg/Python-2.7.14/./Modules/zlibmodule.c:335: undefined reference to `inflate'
build_dir/hostpkg/Python-2.7.14/./Modules/zlibmodule.c:360: undefined reference to `inflateEnd'
build_dir/hostpkg/Python-2.7.14/./Modules/zlibmodule.c:318: undefined reference to `inflateEnd'
build_dir/hostpkg/Python-2.7.14/./Modules/zlibmodule.c:344: undefined reference to `inflateEnd'
build_dir/hostpkg/Python-2.7.14/./Modules/zlibmodule.c:349: undefined reference to `inflateEnd'
build_dir/hostpkg/Python-2.7.14/./Modules/zlibmodule.c:330: undefined reference to `inflateEnd'
staging_dir/hostpkg/lib/libpython2.7.a(zlibmodule.o):build_dir/hostpkg/Python-2.7.14/./Modules/zlibmodule.c:365: more undefined references to `inflateEnd' follow
staging_dir/hostpkg/lib/libpython2.7.a(zlibmodule.o): In function `PyZlib_unflush':
build_dir/hostpkg/Python-2.7.14/./Modules/zlibmodule.c:892: undefined reference to `inflate'
build_dir/hostpkg/Python-2.7.14/./Modules/zlibmodule.c:918: undefined reference to `inflateEnd'
staging_dir/hostpkg/lib/libpython2.7.a(zlibmodule.o): In function `PyZlib_flush':
build_dir/hostpkg/Python-2.7.14/./Modules/zlibmodule.c:713: undefined reference to `deflate'
build_dir/hostpkg/Python-2.7.14/./Modules/zlibmodule.c:728: undefined reference to `deflateEnd'
staging_dir/hostpkg/lib/libpython2.7.a(zlibmodule.o): In function `PyZlib_compress':
build_dir/hostpkg/Python-2.7.14/./Modules/zlibmodule.c:220: undefined reference to `deflateInit_'
build_dir/hostpkg/Python-2.7.14/./Modules/zlibmodule.c:233: undefined reference to `deflateEnd'
build_dir/hostpkg/Python-2.7.14/./Modules/zlibmodule.c:250: undefined reference to `deflate'
build_dir/hostpkg/Python-2.7.14/./Modules/zlibmodule.c:245: undefined reference to `deflateEnd'
build_dir/hostpkg/Python-2.7.14/./Modules/zlibmodule.c:254: undefined reference to `deflateEnd'
build_dir/hostpkg/Python-2.7.14/./Modules/zlibmodule.c:265: undefined reference to `deflateEnd'
collect2: error: ld returned 1 exit status

python has access to zlib headers and libraries

----------
components: Build
messages: 315693
nosy: Lucian Cristian
priority: normal
severity: normal
status: open
title: zlibmodule undefined reference
type: compile error
versions: Python 2.7, Python 3.4, Python 3.5, Python 3.6, Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33347>
_______________________________________

From report at bugs.python.org  Tue Apr 24 08:11:17 2018
From: report at bugs.python.org (Zsolt Dollenstein)
Date: Tue, 24 Apr 2018 12:11:17 +0000
Subject: [New-bugs-announce] [issue33348] lib2to3 doesn't parse f(*[] or [])
Message-ID: <1524571877.77.0.682650639539.issue33348@psf.upfronthosting.co.za>


Change by Zsolt Dollenstein <zsol.zsol at gmail.com>:


----------
components: 2to3 (2.x to 3.x conversion tool)
nosy: zsol
priority: normal
severity: normal
status: open
title: lib2to3 doesn't parse f(*[] or [])
type: behavior

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33348>
_______________________________________

From report at bugs.python.org  Tue Apr 24 09:02:45 2018
From: report at bugs.python.org (Zsolt Dollenstein)
Date: Tue, 24 Apr 2018 13:02:45 +0000
Subject: [New-bugs-announce] [issue33349] 2to3 fails to parse async
 generators in non-async functions
Message-ID: <1524574965.95.0.682650639539.issue33349@psf.upfronthosting.co.za>


New submission from Zsolt Dollenstein <zsol.zsol at gmail.com>:

This is valid since python3.7:

```
def f():
    return (i * 2 async for i in arange(42))
```

but lib2to3 doesn't parse this properly, because `async` is tokenized as a `NAME` instead of `ASYNC`

----------
components: 2to3 (2.x to 3.x conversion tool)
messages: 315700
nosy: zsol
priority: normal
severity: normal
status: open
title: 2to3 fails to parse async generators in non-async functions
type: behavior
versions: Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33349>
_______________________________________

From report at bugs.python.org  Tue Apr 24 12:51:16 2018
From: report at bugs.python.org (Alisue Lambda)
Date: Tue, 24 Apr 2018 16:51:16 +0000
Subject: [New-bugs-announce] [issue33350] WinError 10038 is raised when
 loop.sock_connect is wrapped with asyncio.wait_for
Message-ID: <1524588676.0.0.682650639539.issue33350@psf.upfronthosting.co.za>


New submission from Alisue Lambda <lambdalisue at hashnote.net>:

This is my first time to create an issue on the python bug tracker so let me know if I don't follow the rule which I need to follow.

# Summary

Using 'loop.sock_connect' with 'asyncio.wait_for' raises 'OSError [WinError 10038]' in Windows 10 Pro when timed-out.

# Detail

I use 'loop.sock_connect' to establish a TCP connection for checking if a particular port on a target host is available.
However, when I wrap the coroutine with 'asyncio.wait_for', the following exception is raised when the wrapped coroutine has timed-out.

    Traceback (most recent call last):
      File "C:\Users\alisue/test.py", line 41, in <module>
        loop.run_until_complete(test(loop))
      File "C:\Python36\lib\asyncio\base_events.py", line 454, in run_until_complete
        self.run_forever()
      File "C:\Python36\lib\asyncio\base_events.py", line 421, in run_forever
        self._run_once()
      File "C:\Python36\lib\asyncio\base_events.py", line 1395, in _run_once
        event_list = self._selector.select(timeout)
      File "C:\Python36\lib\selectors.py", line 323, in select
        r, w, _ = self._select(self._readers, self._writers, [], timeout)
      File "C:\Python36\lib\selectors.py", line 314, in _select
        r, w, x = select.select(r, w, w, timeout)
    OSError: [WinError 10038] ...........

While it is raised from 'lib\selectors.py', I cannot catch this exception so the event loop has halted.

The attached 'test.py' is a minimum script to reproduce the error.

Thanks.

----------
components: asyncio
files: test.py
messages: 315707
nosy: Alisue Lambda, asvetlov, yselivanov
priority: normal
severity: normal
status: open
title: WinError 10038 is raised when loop.sock_connect is wrapped with asyncio.wait_for
type: behavior
versions: Python 3.6
Added file: https://bugs.python.org/file47548/test.py

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33350>
_______________________________________

From report at bugs.python.org  Wed Apr 25 00:36:44 2018
From: report at bugs.python.org (Ethan Smith)
Date: Wed, 25 Apr 2018 04:36:44 +0000
Subject: [New-bugs-announce] [issue33351] Support compiling with clang-cl on
 Windows
Message-ID: <1524631004.17.0.682650639539.issue33351@psf.upfronthosting.co.za>


New submission from Ethan Smith <ethan at ethanhs.me>:

The clang folks have been hard at work making an ABI compatible backend to clang for Windows. Additionally they have created a cl compatible driver for clang, which can be used in lieu of cl itself. Clang-cl has been adopted to build Chrome on Windows http://blog.llvm.org/2018/03/clang-is-now-used-to-build-chrome-for.html, so I think it is stable enough to be considered for use.

Clang-cl has several advantages, such as computed goto support and many other optimizations which would make Python faster on Windows.

I would be happy to start contributing patches to further this goal, I already have a couple of small patches.

----------
components: Build
messages: 315721
nosy: Ethan Smith
priority: normal
severity: normal
status: open
title: Support compiling with clang-cl on Windows
type: enhancement
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33351>
_______________________________________

From report at bugs.python.org  Wed Apr 25 09:46:37 2018
From: report at bugs.python.org (STINNER Victor)
Date: Wed, 25 Apr 2018 13:46:37 +0000
Subject: [New-bugs-announce] [issue33352] Windows: test_regrtest fails on
 installed Python
Message-ID: <1524663997.77.0.682650639539.issue33352@psf.upfronthosting.co.za>


New submission from STINNER Victor <victor.stinner at gmail.com>:

Failure on Windows 10 with Python 3.7 beta 3:

======================================================================
ERROR: test_pcbuild_rt (test.test_regrtest.ProgramsTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\Users\vstinner\AppData\Local\Programs\Python\Python37\lib\test\test_regrtest.py", line 602, in test_pcbuild_rt
    self.run_batch(script, *rt_args, *self.regrtest_args, *self.tests)
  File "C:\Users\vstinner\AppData\Local\Programs\Python\Python37\lib\test\test_regrtest.py", line 577, in run_batch
    proc = self.run_command(args)
  File "C:\Users\vstinner\AppData\Local\Programs\Python\Python37\lib\test\test_regrtest.py", line 475, in run_command
    **kw)
  File "C:\Users\vstinner\AppData\Local\Programs\Python\Python37\lib\subprocess.py", line 453, in run
    with Popen(*popenargs, **kwargs) as process:
  File "C:\Users\vstinner\AppData\Local\Programs\Python\Python37\lib\subprocess.py", line 756, in __init__
    restore_signals, start_new_session)
  File "C:\Users\vstinner\AppData\Local\Programs\Python\Python37\lib\subprocess.py", line 1155, in _execute_child
    startupinfo)
FileNotFoundError: [WinError 2] The system cannot find the file specified


We may just skip the test if the script is not installed.

----------
components: Tests, Windows
messages: 315730
nosy: paul.moore, steve.dower, tim.golden, vstinner, zach.ware
priority: normal
severity: normal
status: open
title: Windows: test_regrtest fails on installed Python
versions: Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33352>
_______________________________________

From report at bugs.python.org  Wed Apr 25 10:08:30 2018
From: report at bugs.python.org (STINNER Victor)
Date: Wed, 25 Apr 2018 14:08:30 +0000
Subject: [New-bugs-announce] [issue33353] test_asyncio:
 test_sock_sendfile_mix_with_regular_send() hangs on FreeBSD
Message-ID: <1524665310.44.0.682650639539.issue33353@psf.upfronthosting.co.za>


New submission from STINNER Victor <vstinner at redhat.com>:

Example with a timeout of 10 seconds:

vstinner at freebsd$ ./python -m test -v test_asyncio -m test_sock_sendfile_mix_with_regular_send -v --timeout=10
== CPython 3.8.0a0 (heads/master:0977091dca, Apr 25 2018, 14:02:04) [Clang 4.0.0 (tags/RELEASE_400/final 297347)]
== FreeBSD-11.1-RELEASE-amd64-64bit-ELF little-endian
== cwd: /usr/home/vstinner/prog/python/master/build/test_python_8317
== CPU count: 2
== encodings: locale=UTF-8, FS=utf-8
Run tests sequentially
0:00:00 load avg: 0.59 [1/1] test_asyncio
test_sock_sendfile_mix_with_regular_send (test.test_asyncio.test_events.KqueueEventLoopTests) ... Timeout (0:00:10)!
Thread 0x0000000801a16000 (most recent call first):
  File "/usr/home/vstinner/prog/python/master/Lib/asyncio/selector_events.py", line 456 in _sock_sendall
  File "/usr/home/vstinner/prog/python/master/Lib/asyncio/events.py", line 88 in _run
  File "/usr/home/vstinner/prog/python/master/Lib/asyncio/base_events.py", line 1738 in _run_once
  File "/usr/home/vstinner/prog/python/master/Lib/asyncio/base_events.py", line 521 in run_forever
  File "/usr/home/vstinner/prog/python/master/Lib/asyncio/base_events.py", line 553 in run_until_complete
  File "/usr/home/vstinner/prog/python/master/Lib/test/test_asyncio/test_events.py", line 2117 in run_loop
  File "/usr/home/vstinner/prog/python/master/Lib/test/test_asyncio/test_events.py", line 2213 in test_sock_sendfile_mix_with_regular_send
  (...)

The test has been introduced in by the commit a19fb3c6aaa7632410d1d9dcb395d7101d124da4, bpo-32622.

The FreeBSD 3.7 and 3.x buildbots are broken, likely since this commit.

----------
components: FreeBSD, asyncio
messages: 315733
nosy: asvetlov, koobs, vstinner, yselivanov
priority: normal
severity: normal
status: open
title: test_asyncio: test_sock_sendfile_mix_with_regular_send() hangs on FreeBSD
versions: Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33353>
_______________________________________

From report at bugs.python.org  Wed Apr 25 10:14:15 2018
From: report at bugs.python.org (STINNER Victor)
Date: Wed, 25 Apr 2018 14:14:15 +0000
Subject: [New-bugs-announce] [issue33354] Python2,
 FreeBSD: test_ssl fails on non-ASCII path
Message-ID: <1524665655.9.0.682650639539.issue33354@psf.upfronthosting.co.za>


New submission from STINNER Victor <vstinner at redhat.com>:

The FreeBSD 2.7 buildbot is failing because of test_ssl:

test test_ssl failed -- Traceback (most recent call last):
  File "/usr/home/buildbot/python/2.7.koobs-freebsd10/build/Lib/test/test_ssl.py", line 1005, in test_load_dh_params
    shutil.copy(DHFILE, fname)
  File "/usr/home/buildbot/python/2.7.koobs-freebsd10/build/Lib/shutil.py", line 131, in copy
    if os.path.isdir(dst):
  File "/usr/home/buildbot/python/2.7.koobs-freebsd10/build/Lib/genericpath.py", line 49, in isdir
    st = os.stat(s)
UnicodeEncodeError: 'ascii' codec can't encode character u'\xe4' in position 18: ordinal not in range(128)

======================================================================
ERROR: test_load_dh_params (test.test_ssl.ContextTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/home/buildbot/python/2.7.koobs-freebsd10/build/Lib/test/test_ssl.py", line 1005, in test_load_dh_params
    shutil.copy(DHFILE, fname)
  File "/usr/home/buildbot/python/2.7.koobs-freebsd10/build/Lib/shutil.py", line 131, in copy
    if os.path.isdir(dst):
  File "/usr/home/buildbot/python/2.7.koobs-freebsd10/build/Lib/genericpath.py", line 49, in isdir
    st = os.stat(s)
UnicodeEncodeError: 'ascii' codec can't encode character u'\xe4' in position 18: ordinal not in range(128)


test.pythoninfo:

* os.cwd: /usr/home/buildbot/python/2.7.koobs-freebsd10/build
* sys.filesystem_encoding: US-ASCII


The failure is likely related to the commit 6e8f395001b026daea047cf225dcca5a973ae824, bpo-25404:
---
commit 6e8f395001b026daea047cf225dcca5a973ae824
Author: Christian Heimes <christian at python.org>
Date:   Sun Feb 25 09:48:02 2018 +0100

    bpo-25404: SSLContext.load_dh_params() non-ASCII path (GH-3459)
    
    SSLContext.load_dh_params() now supports non-ASCII path.
    
    Signed-off-by: Christian Heimes <christian at python.org>
---


Maybe the test should be skipped if the filesystem encoding cannot encode the path?

----------
assignee: christian.heimes
components: FreeBSD, SSL
messages: 315734
nosy: christian.heimes, koobs, vstinner
priority: normal
severity: normal
status: open
title: Python2, FreeBSD: test_ssl fails on non-ASCII path
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33354>
_______________________________________

From report at bugs.python.org  Wed Apr 25 10:24:56 2018
From: report at bugs.python.org (STINNER Victor)
Date: Wed, 25 Apr 2018 14:24:56 +0000
Subject: [New-bugs-announce] [issue33355] Windows 10 buildbot: 15 min
 timeout on test_mmap.test_large_filesize()
Message-ID: <1524666296.0.0.682650639539.issue33355@psf.upfronthosting.co.za>


New submission from STINNER Victor <vstinner at redhat.com>:

Windows10 3.x:

http://buildbot.python.org/all/#/builders/3/builds/817

Timeout (0:15:00)!
Thread 0x00001e5c (most recent call first):
  File "D:\buildarea\3.x.bolen-windows10\build\lib\test\test_mmap.py", line 754 in _make_test_file
  File "D:\buildarea\3.x.bolen-windows10\build\lib\test\test_mmap.py", line 769 in test_large_filesize
  ...

----------
components: Tests
messages: 315736
nosy: vstinner
priority: normal
severity: normal
status: open
title: Windows 10 buildbot: 15 min timeout on test_mmap.test_large_filesize()
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33355>
_______________________________________

From report at bugs.python.org  Wed Apr 25 10:26:40 2018
From: report at bugs.python.org (STINNER Victor)
Date: Wed, 25 Apr 2018 14:26:40 +0000
Subject: [New-bugs-announce] [issue33356] Windows 10 buildbot:
 test__xxsubinterpreters.test_already_running() fails randomly
Message-ID: <1524666400.61.0.682650639539.issue33356@psf.upfronthosting.co.za>


New submission from STINNER Victor <vstinner at redhat.com>:

Windows10 3.x: test_already_running() failed once, but passed when run again:

http://buildbot.python.org/all/#/builders/3/builds/817/steps/3/logs/stdio

======================================================================
FAIL: test_already_running (test.test__xxsubinterpreters.RunStringTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "D:\buildarea\3.x.bolen-windows10\build\lib\test\test__xxsubinterpreters.py", line 561, in test_already_running
    interpreters.run_string(self.id, 'print("spam")')
AssertionError: RuntimeError not raised

(...)
Re-running test 'test__xxsubinterpreters' in verbose mode
(...)
Ran 94 tests in 14.582s
OK (skipped=3)

----------
components: Tests
messages: 315737
nosy: eric.snow, vstinner
priority: normal
severity: normal
status: open
title: Windows 10 buildbot: test__xxsubinterpreters.test_already_running() fails randomly
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33356>
_______________________________________

From report at bugs.python.org  Wed Apr 25 10:39:14 2018
From: report at bugs.python.org (STINNER Victor)
Date: Wed, 25 Apr 2018 14:39:14 +0000
Subject: [New-bugs-announce] [issue33357] [EASY C]
 test_posix.test_posix_spawn_file_actions() leaks memory
Message-ID: <1524667154.63.0.682650639539.issue33357@psf.upfronthosting.co.za>


New submission from STINNER Victor <vstinner at redhat.com>:

vstinner at apu$ ./python -m test test_posix -R 3:3 -m test.test_posix.PosixTester.test_posix_spawn_file_actions
Run tests sequentially
0:00:00 load avg: 0.58 [1/1] test_posix
beginning 6 repetitions
123456
......
test_posix leaked [7, 7, 7] references, sum=21
test_posix leaked [1, 2, 1] memory blocks, sum=4
test_posix failed

1 test failed:
    test_posix

Total duration: 315 ms
Tests result: FAILURE


I bet that the leak was introduced by:

commit 6c6ddf97c402709713d668d0ed53836a7749ba99
Author: Pablo Galindo <Pablogsal at gmail.com>
Date:   Mon Jan 29 01:56:10 2018 +0000

    bpo-20104: Expose `posix_spawn` in the os module (GH-5109)
    
    Add os.posix_spawn to wrap the low level POSIX API of the same name.
    
    Contributed by Pablo Galindo.

--

IMHO this issue is easy to fix, so I suggest to leave it to new contributors, or the author of the change.

----------
components: Tests
messages: 315740
nosy: vstinner
priority: normal
severity: normal
status: open
title: [EASY C] test_posix.test_posix_spawn_file_actions() leaks memory
versions: Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33357>
_______________________________________

From report at bugs.python.org  Wed Apr 25 10:41:16 2018
From: report at bugs.python.org (STINNER Victor)
Date: Wed, 25 Apr 2018 14:41:16 +0000
Subject: [New-bugs-announce] [issue33358] x86 Ubuntu Shared 3.x:
 test_embed.test_pre_initialization_sys_options() fails
Message-ID: <1524667276.72.0.682650639539.issue33358@psf.upfronthosting.co.za>


New submission from STINNER Victor <vstinner at redhat.com>:

Re-running test 'test_embed' in verbose mode
test_bpo20891 (test.test_embed.EmbeddingTests) ... ok
test_forced_io_encoding (test.test_embed.EmbeddingTests) ... ok
test_pre_initialization_api (test.test_embed.EmbeddingTests) ... ok
test_pre_initialization_sys_options (test.test_embed.EmbeddingTests) ... --- ['/srv/buildbot/buildarea/3.x.bolen-ubuntu/build/Programs/_testembed', 'pre_initialization_sys_options'] failed ---
stdout:
stderr:
/srv/buildbot/buildarea/3.x.bolen-ubuntu/build/Programs/_testembed: error while loading shared libraries: libpython3.8dm.so.1.0: cannot open shared object file: No such file or directory
------
FAIL
test_subinterps_different_ids (test.test_embed.EmbeddingTests) ... ok
test_subinterps_distinct_state (test.test_embed.EmbeddingTests) ... ok
test test_embed failed
test_subinterps_main (test.test_embed.EmbeddingTests) ... ok
======================================================================
FAIL: test_pre_initialization_sys_options (test.test_embed.EmbeddingTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/srv/buildbot/buildarea/3.x.bolen-ubuntu/build/Lib/test/test_embed.py", line 213, in test_pre_initialization_sys_options
    "pre_initialization_sys_options", env=env)
  File "/srv/buildbot/buildarea/3.x.bolen-ubuntu/build/Lib/test/test_embed.py", line 59, in run_embedded_interpreter
    (p.returncode, err))
AssertionError: 127 != 0 : bad returncode 127, stderr is '/srv/buildbot/buildarea/3.x.bolen-ubuntu/build/Programs/_testembed: error while loading shared libraries: libpython3.8dm.so.1.0: cannot open shared object file: No such file or directory\n'
----------------------------------------------------------------------
Ran 7 tests in 7.218s
FAILED (failures=1)
1 test failed again:
    test_embed

----------
components: Tests
messages: 315741
nosy: vstinner
priority: normal
severity: normal
status: open
title: x86 Ubuntu Shared 3.x: test_embed.test_pre_initialization_sys_options() fails
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33358>
_______________________________________

From report at bugs.python.org  Wed Apr 25 14:13:44 2018
From: report at bugs.python.org (Uzume)
Date: Wed, 25 Apr 2018 18:13:44 +0000
Subject: [New-bugs-announce] [issue33359] curses.has_key failure
Message-ID: <1524680024.26.0.682650639539.issue33359@psf.upfronthosting.co.za>


New submission from Uzume <uzume at users.sourceforge.net>:

The following change is broken and should be reverted (for the one file anyway):

https://github.com/python/cpython/commit/6e3dbbdf39f3b4eb6f18c0165e446df17218b7dc#diff-c322497c7350a91fc1d78da0bb76b5b5

This does not seem to affect master but I did not check every Python version.

This causes the following broken behavior (as found on Python 2.7):
$ python -m curses.has_key
yields the exception
"TypeError: argument of type 'module' is not iterable"

----------
components: Library (Lib)
messages: 315752
nosy: benjamin.peterson, uzume
priority: normal
severity: normal
status: open
title: curses.has_key failure
type: behavior
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33359>
_______________________________________

From report at bugs.python.org  Wed Apr 25 18:58:37 2018
From: report at bugs.python.org (Juan Postlbauer)
Date: Wed, 25 Apr 2018 22:58:37 +0000
Subject: [New-bugs-announce] [issue33360] ALternative recipe for password
 using secrets
Message-ID: <1524697117.91.0.682650639539.issue33360@psf.upfronthosting.co.za>


New submission from Juan Postlbauer <jpc4242 at gmail.com>:

Chapter 15.3.4 shows a recipe with an infinite potential loop.
An alternative would be:

''.join(sorted([choice(string.ascii_lowercase) for i in range(1)]+[choice(string.ascii_uppercase) for i in range(1)]+[choice(string.digits) for i in range(3)]+[choice(string.ascii_letters+string.digits) for i in range(10-(1+1+3))],key=lambda x:randbelow(4096)))

Can we assume secrets.SystemRandom is cryptographically strong but has all the methods of random??
If so it can be done in a more understandable way by using choices and shuffle. (see 2 examples below)

def generate_password_random(totalchars=10,minlower=1,minupper=1,mindigits=3):
    restcount= totalchars-(minlower+minupper+mindigits)
    if restcount<0:
        raise ValueError("Impossible conditions")
    lowerchars=random.choices(string.ascii_lowercase,k=minlower)
    upperchars=random.choices(string.ascii_uppercase,k=minupper)
    digitchars=random.choices(string.digits,k=mindigits)
    restchars=random.choices(string.ascii_letters+string.digits,k=restcount)
    allchars=lowerchars+upperchars+digitchars+restchars
    random.shuffle(allchars)
    password=''.join(allchars)
    return password

def generate_password_secrets(totalchars=10,minlower=1,minupper=1,mindigits=3):
    restcount= totalchars- (minlower+minupper+mindigits)
    if restcount<0:
        raise ValueError("Impossible conditions")
    lowerchars=[secrets.choice(string.ascii_lowercase) for _ in range(minlower)]
    upperchars=[secrets.choice(string.ascii_uppercase) for _ in range(minupper)]
    digitchars=[secrets.choice(string.digits) for _ in range (mindigits)]
    restchars=[secrets.choice(string.ascii_letters+string.digits) for _ in range (restcount)]
    allchars=lowerchars+upperchars+digitchars+restchars
    allchars.sort(key=lambda x:secrets.randbelow(4096))
    password=''.join(allchars)
    return password

----------
assignee: docs at python
components: Documentation
messages: 315763
nosy: docs at python, jpc4242
priority: normal
severity: normal
status: open
title: ALternative recipe for password using secrets
type: enhancement
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33360>
_______________________________________

From report at bugs.python.org  Wed Apr 25 21:06:03 2018
From: report at bugs.python.org (Diego Argueta)
Date: Thu, 26 Apr 2018 01:06:03 +0000
Subject: [New-bugs-announce] [issue33361] readline() + seek() on
 io.EncodedFile breaks next readline()
Message-ID: <1524704763.78.0.682650639539.issue33361@psf.upfronthosting.co.za>


New submission from Diego Argueta <diego.argueta at gmail.com>:

It appears that calling readline() on a codecs.EncodedFile stream breaks seeking and causes subsequent attempts to iterate over the lines or call readline() to backtrack and return already consumed lines.

A minimal example:

```
from __future__ import print_function

import codecs
import io


def run(stream):
    offset = stream.tell()
    try:
        stream.seek(0)
        header_row = stream.readline()
    finally:
        stream.seek(offset)

    print('Got header: %r' % header_row)

    if stream.tell() == 0:
        print('Skipping the header: %r' % stream.readline())

    for index, line in enumerate(stream, start=2):
        print('Line %d: %r' % (index, line))


b = io.BytesIO(u'a,b\r\n"asdf","jkl;"\r\n'.encode('utf-16-le'))
s = codecs.EncodedFile(b, 'utf-8', 'utf-16-le')

run(s)
```

Output:

```
Got header: 'a,b\r\n'
Skipping the header: '"asdf","jkl;"\r\n'    <-- this is line 2
Line 2: 'a,b\r\n'                           <-- this is line 1
Line 3: '"asdf","jkl;"\r\n'                 <-- now we're back to line 2
```

As you can see, the line being skipped is actually the second line, and when we try reading from the stream again, the iterator starts from the beginning of the file.

Even weirder, adding a second call to readline() to skip the second line shows it's going **backwards**:

```
Got header: 'a,b\r\n'
Skipping the header: '"asdf","jkl;"\r\n'    <-- this is actually line 2
Skipping the second line: 'a,b\r\n'         <-- this is line 1
Line 2: '"asdf","jkl;"\r\n'                 <-- this is now correct
```

The expected output shows that we got a header, skipped it, and then read one data line.

```
Got header: 'a,b'
Skipping the header: 'a,b\r\n'
Line 2: '"asdf","jkl;"\r\n'
```

I'm sure this is related to the implementation of readline() because if we change this:

```
header_row = stream.readline()
```

to this:

```
header_row = stream.read().splitlines()[0]
```

then we get the expected output. If on the other hand we comment out the seek() in the finally clause, we also get the expected output (minus the "skipping the header") code.

----------
components: IO, Library (Lib)
messages: 315768
nosy: da
priority: normal
severity: normal
status: open
title: readline() + seek() on io.EncodedFile breaks next readline()
type: behavior
versions: Python 2.7, Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33361>
_______________________________________

From report at bugs.python.org  Wed Apr 25 22:50:46 2018
From: report at bugs.python.org (David Ghiurco)
Date: Thu, 26 Apr 2018 02:50:46 +0000
Subject: [New-bugs-announce] [issue33362] string strip() strips extra
 characters that it shouldn't
Message-ID: <1524711046.09.0.682650639539.issue33362@psf.upfronthosting.co.za>


New submission from David Ghiurco <dm.ghiurco at gmail.com>:

No PR is attached since the issue is pretty trivial to reproduce but important nonetheless:

string = "LDA/AIC2/1919uc1b354363457"
print(string)
print(string.lstrip("LDA/"))

the stripped string should be "AIC2/1919uc1b354363457"
but is instead "IC2/1919uc1b354363457"

notice the leading "A" missing.

I've noticed this happens when the letter immediately following the slash is the same letter as the one immediately before the slash. Never contributed to python so I'm not exactly sure how to, but if anyone knowledgeable could take a look, this will probably be an easy fix.

Note: I am experiencing this on Python 3.6.4, specifically the Anaconda distribution. I have not tried another version.

----------
messages: 315771
nosy: rahvan
priority: normal
severity: normal
status: open
title: string strip() strips extra characters that it shouldn't
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33362>
_______________________________________

From report at bugs.python.org  Thu Apr 26 01:18:50 2018
From: report at bugs.python.org (Zsolt Dollenstein)
Date: Thu, 26 Apr 2018 05:18:50 +0000
Subject: [New-bugs-announce] [issue33363] async for statement is not a
 syntax error in sync context
Message-ID: <1524719930.25.0.682650639539.issue33363@psf.upfronthosting.co.za>


New submission from Zsolt Dollenstein <zsol.zsol at gmail.com>:

Since 3.7, async generators can be used in non-async functions. The way this is implemented allows the following to be valid syntax:

def f():
    async for i in aiterable:
        pass
    return 1

The return value of the above function is a Future that will never be done().

----------
components: Interpreter Core
messages: 315775
nosy: zsol
priority: normal
severity: normal
status: open
title: async for statement is not a syntax error in sync context
type: behavior
versions: Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33363>
_______________________________________

From report at bugs.python.org  Thu Apr 26 03:02:07 2018
From: report at bugs.python.org (Juan Enrique Segebre Zaghmout)
Date: Thu, 26 Apr 2018 07:02:07 +0000
Subject: [New-bugs-announce] [issue33364] Conditionals not evaluating
 propertly
Message-ID: <1524726127.25.0.682650639539.issue33364@psf.upfronthosting.co.za>


New submission from Juan Enrique Segebre Zaghmout <juansegebre at gmail.com>:

The following code generates False when it should generate true:

True == False < 20

Either way the order of operation is taken should result in True but the answer still results in False. To further confirm this issue I did a simple test in which we would group the operations, the test uses print statements to visualize results. The short following code represents the above conditional and the two order of operations possibilities.

print(True == False < 20);
print((True == False) < 20);
print(True == (False < 20));

This yields the following output:

False
True
True

Proving the bug. To explain even further, the code shall be interpreted in either of the following two ways:
1. True == False < 20
   False < 20
   True

2. True == False < 20
   True == True
   True

In either case the result is True, yet python yields False.

----------
components: Interpreter Core
messages: 315779
nosy: Segebre
priority: normal
severity: normal
status: open
title: Conditionals not evaluating propertly
type: behavior
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33364>
_______________________________________

From report at bugs.python.org  Thu Apr 26 08:12:56 2018
From: report at bugs.python.org (Marco Strigl)
Date: Thu, 26 Apr 2018 12:12:56 +0000
Subject: [New-bugs-announce] [issue33365] http/client.py does not print
 correct headers in debug
Message-ID: <1524744776.83.0.682650639539.issue33365@psf.upfronthosting.co.za>


New submission from Marco Strigl <marco.strigl at googlemail.com>:

Consider the following script: 

try:
    from urllib import request
except ImportError:
    import urllib2 as request

handler = request.HTTPSHandler(debuglevel=1)
opener = request.build_opener(handler)
f = opener.open('https://httpbin.org/user-agent')


In python2.x this works: 

$ python2 http_client_bug.py
send: 'GET /user-agent HTTP/1.1\r\nAccept-Encoding: identity\r\nHost:
httpbin.org\r\nConnection: close\r\nUser-Agent: Python-urllib/2.7\r\n\r\n'
reply: 'HTTP/1.1 200 OK\r\n'
header: Connection: close
header: Server: gunicorn/19.7.1
header: Date: Thu, 26 Apr 2018 12:01:35 GMT
header: Content-Type: application/json
header: Access-Control-Allow-Origin: *
header: Access-Control-Allow-Credentials: true
header: X-Powered-By: Flask
header: X-Processed-Time: 0
header: Content-Length: 40
header: Via: 1.1 vegur


But in python3.x only the header keys are printed. Not the values (also a newline after each header will be nice):

$ python3 http_client_bug.py
send: b'GET /user-agent HTTP/1.1\r\nAccept-Encoding: identity\r\nHost:
httpbin.org\r\nUser-Agent: Python-urllib/3.6\r\nConnection: close\r\n\r\n'
reply: 'HTTP/1.1 200 OK\r\n'
header: Connection header: Server header: Date header: Content-Type
header: Access-Control-Allow-Origin header:
Access-Control-Allow-Credentials header: X-Powered-By header:
X-Processed-Time header: Content-Length header:

Patch for this is attached.

----------
components: Library (Lib)
files: complete_pretty_headers.patch
keywords: patch
messages: 315787
nosy: mstrigl
priority: normal
severity: normal
status: open
title: http/client.py does not print correct headers in debug
versions: Python 3.4, Python 3.5, Python 3.6, Python 3.7, Python 3.8
Added file: https://bugs.python.org/file47550/complete_pretty_headers.patch

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33365>
_______________________________________

From report at bugs.python.org  Thu Apr 26 08:47:52 2018
From: report at bugs.python.org (Tom Christie)
Date: Thu, 26 Apr 2018 12:47:52 +0000
Subject: [New-bugs-announce] [issue33366] `contextvars` documentation
 incorrectly refers to "non-local state".
Message-ID: <1524746872.41.0.682650639539.issue33366@psf.upfronthosting.co.za>


New submission from Tom Christie <tom at tomchristie.com>:

The `contextvars` documentation, at https://docs.python.org/3.7/library/contextvars.html starts with the following:

"This module provides APIs to manage, store, and access non-local state."

I assume that must be a documentation bug, right. The module isn't for managing non-local state, it's for managing state that *is* local.

I'd assume it ought to read...

"This module provides APIs to manage, store, and access context-local state."

(ie. for managing state that is transparently either thread-local or task-local depending on the current execution context.)

----------
assignee: docs at python
components: Documentation, asyncio
messages: 315792
nosy: asvetlov, docs at python, tomchristie, yselivanov
priority: normal
severity: normal
status: open
title: `contextvars` documentation incorrectly refers to "non-local state".
type: enhancement
versions: Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33366>
_______________________________________

From report at bugs.python.org  Thu Apr 26 11:25:51 2018
From: report at bugs.python.org (Soumyadipta Das)
Date: Thu, 26 Apr 2018 15:25:51 +0000
Subject: [New-bugs-announce] [issue33367] Multiprocessing Pool workers
 initiated with maxtasksperchild do not execute when sharing logging
Message-ID: <1524756351.27.0.682650639539.issue33367@psf.upfronthosting.co.za>


New submission from Soumyadipta Das <das.soumyadipta.81 at gmail.com>:

When Multiprocessing Pool is initiated with maxtasksperchild and is sharing logging with the root process some workers do not execute. This  is only on Python 2.7 and not on Python 3. Also bug is present only on linux and not on windows.

----------
files: bugdemo.py
messages: 315804
nosy: Soumyadipta Das
priority: normal
severity: normal
status: open
title: Multiprocessing Pool workers initiated with maxtasksperchild do not execute when sharing logging
type: behavior
versions: Python 2.7
Added file: https://bugs.python.org/file47551/bugdemo.py

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33367>
_______________________________________

From report at bugs.python.org  Thu Apr 26 15:08:57 2018
From: report at bugs.python.org (C. Y. Hollander)
Date: Thu, 26 Apr 2018 19:08:57 +0000
Subject: [New-bugs-announce] [issue33368] Inaccuracy in
 https://docs.python.org/3/library/re.html#re.match.end
Message-ID: <1524769737.79.0.682650639539.issue33368@psf.upfronthosting.co.za>


New submission from C. Y. Hollander <judah0 at gmail.com>:

The documentation states that match.end([group]) returns "the ind[ex] of the... end of the substring matched by group". In fact, it returns [said index] + 1, as demonstrated by the example below:

s = 'example'
sre = re.search('le', s)
s[sre.end()]

Incidentally, I don't see the logic of this behaviour, but in any case it should be correctly documented.

----------
assignee: docs at python
components: Documentation
messages: 315813
nosy: C. Y. Hollander, docs at python
priority: normal
severity: normal
status: open
title: Inaccuracy in https://docs.python.org/3/library/re.html#re.match.end
type: enhancement

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33368>
_______________________________________

From report at bugs.python.org  Thu Apr 26 18:40:56 2018
From: report at bugs.python.org (=?utf-8?q?Peter_Bostr=C3=B6m?=)
Date: Thu, 26 Apr 2018 22:40:56 +0000
Subject: [New-bugs-announce] [issue33369] Removing Popen log files in
 threads is racy on Windows
Message-ID: <1524782456.24.0.682650639539.issue33369@psf.upfronthosting.co.za>


New submission from Peter Bostr?m <peterbostrom at gmail.com>:

OS: Windows 10, version 1607 (OS Build 14393.2189)
Python --version: Python 2.7.3

Removing logfiles files just after Popen().wait() outside an open() block is racy on Windows. This seems to require multiple parallel workers to pull off, not sure where the synchronization issue lies (interpreter or os).

This report is spun off from the following pull request where we ran into the issue (we always log to log files instead of memory pipes but sometimes discard them after the worker has finished): https://github.com/google/gtest-parallel/pull/59#issuecomment-369551914

Attaching repro script, in it essentially all workers do this:

  command = ['ping', '-n', '1', '127.0.0.1']
  with open(file_name, 'w') as file_obj:
    task = subprocess.Popen(command, stdout=file_obj, stderr=file_obj)
    task.wait()

  print 'Removing %s\n' % file_name
  os.remove(file_name)

Here os.remove() fails fairly often with an exception even though there should be no handles left to the file: WindowsError: [Error 32] The process cannot access the file because it is being used by another process: '25436.log'

Run (this seems to repro pretty reliably, more than half of the time on my machine):

C:\src>del *.log

C:\src>python racy_windows.py
Logging to 24788.log
Logging to 1052.log


Logging to 32320.log
Logging to 29316.log


Removing 1052.log

Exception in thread Thread-2:
Traceback (most recent call last):
  File "C:\python_27_amd64\files\lib\threading.py", line 551, in __bootstrap_inner
    self.run()
  File "C:\python_27_amd64\files\lib\threading.py", line 504, in run
    self.__target(*self.__args, **self.__kwargs)
  File "racy_windows.py", line 19, in __call__
    os.remove(file_name)
WindowsError: [Error 32] The process cannot access the file because it is being used by another process: '1052.log'

Removing 24788.log

Exception in thread Thread-1:
Traceback (most recent call last):
  File "C:\python_27_amd64\files\lib\threading.py", line 551, in __bootstrap_inner
    self.run()
  File "C:\python_27_amd64\files\lib\threading.py", line 504, in run
    self.__target(*self.__args, **self.__kwargs)
  File "racy_windows.py", line 19, in __call__
    os.remove(file_name)
WindowsError: [Error 32] The process cannot access the file because it is being used by another process: '24788.log'

Removing 29316.log
Removing 32320.log

----------
components: Windows
files: racy_windows.py
messages: 315822
nosy: paul.moore, pbos, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: Removing Popen log files in threads is racy on Windows
type: behavior
versions: Python 2.7
Added file: https://bugs.python.org/file47552/racy_windows.py

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33369>
_______________________________________

From report at bugs.python.org  Thu Apr 26 19:27:40 2018
From: report at bugs.python.org (Ekin Dursun)
Date: Thu, 26 Apr 2018 23:27:40 +0000
Subject: [New-bugs-announce] [issue33370] Addition of mypy cache to gitignore
Message-ID: <1524785260.87.0.682650639539.issue33370@psf.upfronthosting.co.za>


New submission from Ekin Dursun <ekindursun at gmail.com>:

Since Python developers started to use mypy, I need we should add .mypy_cache directories to .gitignore.

----------
messages: 315823
nosy: onlined
priority: normal
severity: normal
status: open
title: Addition of mypy cache to gitignore
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33370>
_______________________________________

From report at bugs.python.org  Thu Apr 26 19:37:00 2018
From: report at bugs.python.org (Brian Curtin)
Date: Thu, 26 Apr 2018 23:37:00 +0000
Subject: [New-bugs-announce] [issue33371] Clarify the predicate parameter of
 inspect.getmembers
Message-ID: <1524785820.22.0.682650639539.issue33371@psf.upfronthosting.co.za>


New submission from Brian Curtin <brian at python.org>:

It would be helpful to clarify what the *predicate* callable will be called with. I had to look into the source in order to figure out what it was going to receive.

----------
assignee: brian.curtin
components: Documentation
messages: 315824
nosy: brian.curtin
priority: normal
severity: normal
stage: needs patch
status: open
title: Clarify the predicate parameter of inspect.getmembers

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33371>
_______________________________________

From report at bugs.python.org  Fri Apr 27 01:33:07 2018
From: report at bugs.python.org (an0n.r00t32)
Date: Fri, 27 Apr 2018 05:33:07 +0000
Subject: [New-bugs-announce] [issue33372] Wrong calculation
Message-ID: <1524807187.54.0.682650639539.issue33372@psf.upfronthosting.co.za>


New submission from an0n.r00t32 <an0n.r00t32 at gmail.com>:

I found that python calculate this wrong. Other calculators gave me 9997809307 not 9997809507 (python), even python3 calculated it correctly. Here is the computation expression which Python 2.7 calculated wrong.

https://pastebin.com/uKx1FYx3

----------
messages: 315826
nosy: an0n.r00t32
priority: normal
severity: normal
status: open
title: Wrong calculation
type: performance
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33372>
_______________________________________

From report at bugs.python.org  Fri Apr 27 10:28:23 2018
From: report at bugs.python.org (C.D. MacEachern)
Date: Fri, 27 Apr 2018 14:28:23 +0000
Subject: [New-bugs-announce] [issue33373] ttk modules Label class does not
 respect background config option
Message-ID: <1524839303.7.0.682650639539.issue33373@psf.upfronthosting.co.za>


New submission from C.D. MacEachern <craigmaceachern at fastmail.com>:

Python tkinter.ttk.Label instance is not respecting background configuration.

I'm running 3.6.5 64-bit python with Tk 8.6 on OS X 10.13.4.

Here is an output of an interactive session to demonstrate the steps. Notice that the ttk.Label keys() output shows that the ?background? config option is supported. Additionally, the ttk::label manual page (http://www.tcl.tk/man/tcl8.6/TkCmd/ttk_label.htm#M6) shows that the ?-background? option is an available ?widget-specific option?. 


[~/projects/parou/src (2.0)] $ python3
Python 3.6.5 (v3.6.5:f59c0932b4, Mar 28 2018, 05:52:31)
[GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.57)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import tkinter as tk
>>> import tkinter.ttk as ttk
>>> root = tk.Tk()
>>> tk_label = tk.Label(root)
>>> tk_label.keys()
['activebackground', 'activeforeground', 'anchor', 'background', 'bd', 'bg', 'bitmap', 'borderwidth', 'compound', 'cursor', 'disabledforeground', 'fg', 'font', 'foreground', 'height', 'highlightbackground', 'highlightcolor', 'highlightthickness', 'image', 'justify', 'padx', 'pady', 'relief', 'state', 'takefocus', 'text', 'textvariable', 'underline', 'width', 'wraplength']
>>> tk_label.config(text='Old style tkinter.Label instance', foreground='blue', background='red')
>>> tk_label.pack()
>>> new_ttk_label = ttk.Label(root)
>>> new_ttk_label.keys()
['background', 'foreground', 'font', 'borderwidth', 'relief', 'anchor', 'justify', 'wraplength', 'takefocus', 'text', 'textvariable', 'underline', 'width', 'image', 'compound', 'padding', 'state', 'cursor', 'style', 'class']
>>> new_ttk_label.config(text='New tkinter.ttk.Label instance', foreground='blue', background='blue')
>>> new_ttk_label.pack()
>>> tk_label.config('background')
('background', 'background', 'Background', <border object: 'White'>, 'red')
>>> new_ttk_label.config('background')
('background', 'frameColor', 'FrameColor', '', <border object: 'blue'>)
>>> new_ttk_label.config('foreground')
('foreground', 'textColor', 'TextColor', '', <color object: 'blue'>)
>>> root.mainloop()

I would expect the background to change colour on the ttk Label the same way it does when I change the background on the tkinter.Label instance.

It appears to not have worked as documented since 2014 for some, so my suggestion would be to remove 'background' as an option exposed in ttk.Label().keys().

Link to SO thread where it is discussed as well:

https://stackoverflow.com/questions/23750141/tkinter-ttk-widgets-ignoring-background-color/50064376#50064376

----------
components: Tkinter
messages: 315844
nosy: cmaceachern
priority: normal
severity: normal
status: open
title: ttk modules Label class does not respect background config option
type: behavior
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33373>
_______________________________________

From report at bugs.python.org  Fri Apr 27 11:45:47 2018
From: report at bugs.python.org (Piotr Dobrogost)
Date: Fri, 27 Apr 2018 15:45:47 +0000
Subject: [New-bugs-announce] [issue33374] generate-posix-vars failed when
 building Python 2.7.14 on Linux
Message-ID: <1524843947.56.0.682650639539.issue33374@psf.upfronthosting.co.za>


New submission from Piotr Dobrogost <pd at bugs.python.dobrogost.net>:

When building Python 2.7.14 on Fedora 28 I get the following error:

[piotr at demon]/tmp/Python-2.7.14% make
(?)
./python -E -S -m sysconfig --generate-posix-vars ;\
if test $? -ne 0 ; then \
 echo "generate-posix-vars failed" ; \
 rm -f ./pybuilddir.txt ; \
 exit 1 ; \
fi
/bin/sh: line 5: 25857 Segmentation fault      (core dumped) ./python -E -S -m sysconfig --generate-posix-vars
generate-posix-vars failed
make[2]: *** [Makefile:514: pybuilddir.txt] Error 1
make[2]: Leaving directory '/tmp/Python-2.7.14'
make[1]: *** [Makefile:444: build_all_generate_profile] Error 2
make[1]: Leaving directory '/tmp/Python-2.7.14'
make: *** [Makefile:429: profile-opt] Error 2


Running problematic command alone I get this:

[piotr at demon]/tmp/Python-2.7.14% ./python -E -S -m sysconfig --generate-posix-vars
Could not find platform dependent libraries <exec_prefix>
Consider setting $PYTHONHOME to <prefix>[:<exec_prefix>]
zsh: segmentation fault (core dumped)  ./python -E -S -m sysconfig --generate-posix-vars

I found somehow related https://bugs.python.org/issue21166 with excellent explanation by Ned Deily (https://bugs.python.org/msg225217) but that issue is supposedly fixed.

I have Python 3.6.5 installed at /usr/bin/python3 and Python 2.7.14 installed at /usr/bin/python{2} ? both from RPM packages.

I have Python 2.7.13 installed at /usr/local/bin/python{2} ? built from source.

Running `which python` returns "/usr/local/bin/python".

Trying to isolate configuration and build by prefixing commands with
PATH=/usr/bin:/usr/sbin LIBRARY_PATH= LD_LIBRARY_PATH= (?)
does not help.

----------
components: Build
messages: 315846
nosy: ned.deily, piotr.dobrogost
priority: normal
severity: normal
status: open
title: generate-posix-vars failed when building Python 2.7.14 on Linux
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33374>
_______________________________________

From report at bugs.python.org  Fri Apr 27 12:38:55 2018
From: report at bugs.python.org (Thomas Kluyver)
Date: Fri, 27 Apr 2018 16:38:55 +0000
Subject: [New-bugs-announce] [issue33375] warnings: get filename from
 frame.f_code.co_filename
Message-ID: <1524847135.79.0.682650639539.issue33375@psf.upfronthosting.co.za>


New submission from Thomas Kluyver <thomas at kluyver.me.uk>:

The warnings module tries to find and show the line of code which is responsible for a warning, for the same reasons that tracebacks show a line of code from each stack frame. However, they work in quite different ways.

Native tracebacks, the traceback module and pdb all do something like this:

frame.f_code.co_filename

But warnings does something like this (paraphrasing C code in _warnings.c):

frame.f_globals.get('__file__', sys.argv[0])

This causes problems for interactive interpreters like IPython, because there are multiple pieces of entered code which have to share the same global namespace. E.g. https://github.com/ipython/ipython/issues/11080

This was raised a long time ago in #1692664. Back then, the answer was that co_filename could be wrong if the path of a pyc file changed. However, that issue was fixed in #1180193. And it seems that the co_filename approach must largely work today, because tracebacks and pdb rely on it.

So I'm proposing to make warnings match how those other tools find filenames. I think this should also be a minor simplification of the code.

----------
components: Library (Lib)
messages: 315848
nosy: takluyver
priority: normal
severity: normal
status: open
title: warnings: get filename from frame.f_code.co_filename

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33375>
_______________________________________

From report at bugs.python.org  Fri Apr 27 20:15:11 2018
From: report at bugs.python.org (cary)
Date: Sat, 28 Apr 2018 00:15:11 +0000
Subject: [New-bugs-announce] [issue33376] [pysqlite] Duplicate rows can be
 returned after rolling back a transaction
Message-ID: <1524874511.54.0.682650639539.issue33376@psf.upfronthosting.co.za>


New submission from cary <cary at dropbox.com>:

# Description
Rolling back a transaction causes all statements associated with that transaction to be reset, which allows the statements to be used again from the pysqlite statement cache. This can interact with various methods on `Cursor` objects to cause these statements to be reset again, possibly when they are in use by a different cursor.

This appears to be very similar to issue10513 and issue23129.

# Impact
Duplicate rows will be returned. Exceptions can be raised.

# Repro steps
  - Cursor *A* executes query *X*
  - Rollback occurs
  - Cursor *B* executes query *X*
  - Any of the following (and there might be other cases too):
    - Cursor *A* is closed
    - Cursor *A* is deallocated
    - Cursor *A* executes any other query.
  - Result: Cursor *B* returns duplicate rows.
  - Furthermore: Executing query *X* again afterwards raises `sqlite3.InterfaceError`

# Possible solutions
  - Similar to the solution for issue10513 and issue23129, we could remove `pysqlite_do_all_statements(self, ACTION_RESET, 1)` from `pysqlite_connection_rollback`. This fixes the given issue, but I'm not sure what the implications are for the rest of the system.

  - Do not reset `self->statement` in `Cursor` if `self->reset`. This is the fix we've adopted for now (through a local patch to our Python), but it's worth noting that this is rather brittle, and only works because `pysqlite_do_all_statements` is always called with `reset_cursors = 1`, and `self->reset` is not modified in too many places.

# Example
```
import sqlite3 as sqlite

if __name__ == '__main__':
    conn = sqlite.connect(":memory:")
    conn.executescript("""
        CREATE TABLE t(c);
        INSERT INTO t VALUES(0);
        INSERT INTO t VALUES(1);
        INSERT INTO t VALUES(2);
    """)

    curs = conn.cursor()
    curs.execute("BEGIN TRANSACTION")
    curs.execute("SELECT c FROM t WHERE ?", (1,))
    conn.rollback()

    # Reusing the same statement from the statement cache, which has been
    # reset by the rollback above.
    gen = conn.execute("SELECT c FROM t WHERE ?", (1,))

    # Any of the following will cause a spurious reset of the statement.
    curs.close()
    # curs.execute("SELECT 1")
    # del curs

    # Expected output: [(0,), (1,), (2,)]
    # Observed output: [(0,), (0,), (1,), (2,)]
    print(list(gen))

    # Raises `sqlite3.InterfaceError: Error binding parameter 0 - probably unsupported type.`
    conn.execute("SELECT c FROM t WHERE ?", (1,))
```

----------
components: Extension Modules
messages: 315862
nosy: cary
priority: normal
severity: normal
status: open
title: [pysqlite] Duplicate rows can be returned after rolling back a transaction
type: behavior
versions: Python 2.7, Python 3.4, Python 3.5, Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33376>
_______________________________________

From report at bugs.python.org  Sat Apr 28 11:45:46 2018
From: report at bugs.python.org (Matthias Klose)
Date: Sat, 28 Apr 2018 15:45:46 +0000
Subject: [New-bugs-announce] [issue33377] add new triplets for mips r6 and
 riscv variants
Message-ID: <1524930346.47.0.682650639539.issue33377@psf.upfronthosting.co.za>


New submission from Matthias Klose <doko at debian.org>:

Late last year / early this year, Debian defined new triplets for some mips r6 and riscv variants.  Please still consider these for the 3.7 release.  These patches are in use in Debian for a while and show no regressions.

The new variants are documented at https://wiki.debian.org/Multiarch/Tuples

----------
components: Build
files: mips-r6.diff
keywords: patch
messages: 315864
nosy: doko, ned.deily
priority: normal
severity: normal
status: open
title: add new triplets for mips r6 and riscv variants
versions: Python 3.7, Python 3.8
Added file: https://bugs.python.org/file47553/mips-r6.diff

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33377>
_______________________________________

From report at bugs.python.org  Sat Apr 28 13:28:16 2018
From: report at bugs.python.org (Julien Palard)
Date: Sat, 28 Apr 2018 17:28:16 +0000
Subject: [New-bugs-announce] [issue33378] Add Korean to the language switcher
Message-ID: <1524936496.16.0.682650639539.issue33378@psf.upfronthosting.co.za>


Change by Julien Palard <julien+python at palard.fr>:


----------
assignee: mdk
components: Documentation
nosy: mdk
priority: normal
severity: normal
status: open
title: Add Korean to the language switcher

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33378>
_______________________________________

From report at bugs.python.org  Sat Apr 28 18:03:00 2018
From: report at bugs.python.org (Serhiy Storchaka)
Date: Sat, 28 Apr 2018 22:03:00 +0000
Subject: [New-bugs-announce] [issue33379] PyImport_Cleanup is called with
 builtins_copy == NULL in test_embed
Message-ID: <1524952980.52.0.682650639539.issue33379@psf.upfronthosting.co.za>


New submission from Serhiy Storchaka <storchaka+cpython at gmail.com>:

PyImport_Cleanup is called with builtins_copy == NULL in run_embedded_interpreter() in Lib/test/test_embed.py. This error was silenced by calling PyErr_Clear(), but it still looks as error to me. It was exposed by PR 6606.

----------
components: Interpreter Core
messages: 315868
nosy: brett.cannon, eric.snow, ncoghlan, serhiy.storchaka
priority: normal
severity: normal
status: open
title: PyImport_Cleanup is called with builtins_copy == NULL in test_embed
type: behavior
versions: Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33379>
_______________________________________

From report at bugs.python.org  Sat Apr 28 18:40:12 2018
From: report at bugs.python.org (Allan Feldman)
Date: Sat, 28 Apr 2018 22:40:12 +0000
Subject: [New-bugs-announce] [issue33380] Update module attribute on
 namedtuple methods for introspection.
Message-ID: <1524955212.89.0.682650639539.issue33380@psf.upfronthosting.co.za>


New submission from Allan Feldman <allan.d.feldman at gmail.com>:

Python 3.7 made several performance improvements to the namedtuple class as part of https://bugs.python.org/issue28638

Prior to the implementation of bpo-28638, the __module__ attribute for a namedtuple's methods (e.g. _asdict) would return the value 'namedtuple_%s' % typename (e.g. namedtuple_Point).

Due to the optimizations made, the __module__ attribute for a namedtuple's methods now returns 'collections'.

The proposed change as part of this issue is to report the more accurate derived module name for the namedtuple methods. Updating the __module__ attribute should help debug and introspection tools more accurately report the details of executing calls (in profilers for example).

Example from Python 3.6:
>>> from collections import namedtuple
>>> Point = namedtuple('Point', ('x', 'y'))
>>> p1 = Point(1,2)
>>> p1._asdict.__module__
'namedtuple_Point'

Example from Python 3.7.0b3:
>>> from collections import namedtuple
>>> Point = namedtuple('Point', ('x', 'y'))
>>> p1 = Point(1,2)
>>> p1._asdict.__module__
'collections'

Desired behavior:
>>> from collections import namedtuple
>>> Point = namedtuple('Point', ('x', 'y'))
>>> p1 = Point(1,2)
>>> p1._asdict.__module__
'__main__'

----------
components: Library (Lib)
messages: 315869
nosy: a-feld, rhettinger
priority: normal
severity: normal
status: open
title: Update module attribute on namedtuple methods for introspection.
type: enhancement
versions: Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33380>
_______________________________________

From report at bugs.python.org  Sat Apr 28 21:06:18 2018
From: report at bugs.python.org (Judy Wang)
Date: Sun, 29 Apr 2018 01:06:18 +0000
Subject: [New-bugs-announce] [issue33381] Incorrect documentation for
 strftime()/strptime() format code %f
Message-ID: <1524963978.65.0.682650639539.issue33381@psf.upfronthosting.co.za>


New submission from Judy Wang <jwang54 at gmail.com>:

According to https://docs.python.org/3/library/datetime.html, %f is zero-padded on the left. But actual Python 3 behavior is puts the zero padding on the right.

----------
files: Bug description.py
messages: 315873
nosy: jujuwoman
priority: normal
severity: normal
status: open
title: Incorrect documentation for strftime()/strptime() format code %f
type: behavior
versions: Python 3.6
Added file: https://bugs.python.org/file47556/Bug description.py

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33381>
_______________________________________

From report at bugs.python.org  Sat Apr 28 22:13:31 2018
From: report at bugs.python.org (Patrick Young)
Date: Sun, 29 Apr 2018 02:13:31 +0000
Subject: [New-bugs-announce] [issue33382] make [profile-opt] failde with
 --enable-optimizations
Message-ID: <1524968011.46.0.682650639539.issue33382@psf.upfronthosting.co.za>


New submission from Patrick Young <kmahyygyyg at gmail.com>:

0:14:07 load avg: 1.26 [406/406] test_zlib

Total duration: 14 min 9 sec
Tests result: SUCCESS
make[1]: Leaving directory '/home/kmahyyg/Desktop/py36/Python-3.6.5'
make build_all_merge_profile
make[1]: Entering directory '/home/kmahyyg/Desktop/py36/Python-3.6.5'
true
/bin/true: 1: /bin/true: Syntax error: "(" unexpected
Makefile:497: recipe for target 'build_all_merge_profile' failed
make[1]: *** [build_all_merge_profile] Error 2
make[1]: Leaving directory '/home/kmahyyg/Desktop/py36/Python-3.6.5'
Makefile:471: recipe for target 'profile-opt' failed
make: *** [profile-opt] Error 2

----------
components: Installation
messages: 315876
nosy: kmahyyg
priority: normal
severity: normal
status: open
title: make [profile-opt] failde with --enable-optimizations
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33382>
_______________________________________

From report at bugs.python.org  Sun Apr 29 04:09:38 2018
From: report at bugs.python.org (Serhiy Storchaka)
Date: Sun, 29 Apr 2018 08:09:38 +0000
Subject: [New-bugs-announce] [issue33383] Crash in the get() method a single
 argument in dbm.ndbm
Message-ID: <1524989378.13.0.682650639539.issue33383@psf.upfronthosting.co.za>


New submission from Serhiy Storchaka <storchaka+cpython at gmail.com>:

Calling the get() method of the dbm.ndbm database object with only single argument causes a crash because the default value is set to NULL.

The regression was introduce in 3.5.

The proposed patch fixes the crash and add tests for the get() method for all dbm implementations.

----------
components: Extension Modules
messages: 315883
nosy: serhiy.storchaka
priority: normal
severity: normal
status: open
title: Crash in the get() method a single argument in dbm.ndbm
type: crash
versions: Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33383>
_______________________________________

From report at bugs.python.org  Sun Apr 29 06:16:35 2018
From: report at bugs.python.org (Martin Husemann)
Date: Sun, 29 Apr 2018 10:16:35 +0000
Subject: [New-bugs-announce] [issue33384] Build does not work with closed
 stdin
Message-ID: <1524996995.63.0.682650639539.issue33384@psf.upfronthosting.co.za>


New submission from Martin Husemann <martin at NetBSD.org>:

When building python extensions in the background w/o stdin (and stderr and stdout redirected to a log file), the invocation of setup.py fails.

Normal build in a shell:

Example from pyexpat:
> /usr/pkg/bin/python3.6  setup.py  build
running build
running build_ext
building 'pyexpat' extension
creating build
creating build/temp.netbsd-8.99.14-amd64-3.6
creating build/temp.netbsd-8.99.14-amd64-3.6/Modules
gcc -DNDEBUG -O2 -pipe -D_FORTIFY_SOURCE=2 -I/usr/include -I/usr/pkg/include -O2 -pipe -D_FORTIFY_SOURCE=2 -I/usr/include -I/usr/pkg/include/python3.6 -I/usr/include -I/usr/pkg/include/python3.6 -fPIC -DHAVE_EXPAT_H -I/work/pkgobj/textproc/py-expat/work/.buildlink/include -I/usr/pkg/include/python3.6 -c Modules/pyexpat.c -o build/temp.netbsd-8.99.14-amd64-3.6/Modules/pyexpat.o
creating build/lib.netbsd-8.99.14-amd64-3.6
gcc -pthread -shared -L. -L/usr/lib -Wl,-R/usr/lib -L/usr/pkg/lib -Wl,-R/usr/pkg/lib -L/usr/X11R7/lib -Wl,-R/usr/X11R7/lib -L/usr/lib -Wl,-R/usr/lib -L/usr/pkg/lib -Wl,-R/usr/pkg/lib -L/usr/X11R7/lib -Wl,-R/usr/X11R7/lib -L/usr/lib -Wl,-R/usr/lib -Wl,-R/usr/pkg/lib -O2 -pipe -D_FORTIFY_SOURCE=2 -I/usr/include -I/usr/pkg/include/python3.6 -I/usr/include -I/usr/pkg/include/python3.6 build/temp.netbsd-8.99.14-amd64-3.6/Modules/pyexpat.o -L/work/pkgobj/textproc/py-expat/work/.buildlink/lib -L/usr/pkg/lib -Wl,-R/work/pkgobj/textproc/py-expat/work/.buildlink/lib -lexpat -lpython3.6 -o build/lib.netbsd-8.99.14-amd64-3.6/pyexpat.so


But if invoked as 

> ( sleep 15; /usr/pkg/bin/python3.6  setup.py  build ) >/tmp/log 2>&1 &
> ^D

and then watching /tmp/log from another session:

Fatal Python error: Py_Initialize: can't initialize sys standard streams
OSError: [Errno 9] Bad file descriptor

Current thread 0x000075b4e4cee800 (most recent call first):
[1]   Abort trap (core dumped) /usr/pkg/bin/pyt...

----------
components: Build
messages: 315886
nosy: MartinHusemann
priority: normal
severity: normal
status: open
title: Build does not work with closed stdin
type: enhancement
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33384>
_______________________________________

From report at bugs.python.org  Sun Apr 29 10:00:31 2018
From: report at bugs.python.org (Serhiy Storchaka)
Date: Sun, 29 Apr 2018 14:00:31 +0000
Subject: [New-bugs-announce] [issue33385] setdefault() with a single
 argument doesn't work for dbm.gdbm and dmb.ndbm objects
Message-ID: <1525010431.96.0.682650639539.issue33385@psf.upfronthosting.co.za>


New submission from Serhiy Storchaka <storchaka+cpython at gmail.com>:

setdefault() is not implemented directly in dbm.gdbm and dmb.ndbm database classes. It is inherited from MutableMapping:

    def setdefault(self, key, default=None):
        try:
            return self[key]
        except KeyError:
            self[key] = default
        return default

But since assigning is supported only for bytes and str, setdefault(key) fails if the key was not set before. It works only if the key was set or with the second argument. d.setdefault(key) is equivalent to d[key] except that it raises a weird TypeError instead of KeyError.

There are two ways of solving this problem:

1. Reimplement setdefault() for dbm.gdbm and dmb.ndbm database classes with default=b'' by default.

2. Make the second argument mandatory.

In both cases this violates the MutableMapping interface.

----------
components: Library (Lib)
messages: 315898
nosy: serhiy.storchaka
priority: normal
severity: normal
status: open
title: setdefault() with a single argument doesn't work for dbm.gdbm and dmb.ndbm objects
type: enhancement
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33385>
_______________________________________

From report at bugs.python.org  Sun Apr 29 12:46:31 2018
From: report at bugs.python.org (Dan Snider)
Date: Sun, 29 Apr 2018 16:46:31 +0000
Subject: [New-bugs-announce] [issue33386] Double clicking to select an
 identifier in IDLE does not work with some unicode characters
Message-ID: <1525020391.31.0.682650639539.issue33386@psf.upfronthosting.co.za>


New submission from Dan Snider <mr.assume.away at gmail.com>:

https://i.imgur.com/61sHBRR.png

At least with "?", it behaves as if it's one of the ascii characters for which id.isidentifier() returns False, as in https://i.imgur.com/XbEW0ZC.png.

----------
assignee: terry.reedy
components: IDLE
messages: 315901
nosy: bup, terry.reedy
priority: normal
severity: normal
status: open
title: Double clicking to select an identifier in IDLE does not work with some unicode characters
type: behavior
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33386>
_______________________________________

From report at bugs.python.org  Sun Apr 29 14:28:42 2018
From: report at bugs.python.org (Mark Shannon)
Date: Sun, 29 Apr 2018 18:28:42 +0000
Subject: [New-bugs-announce] [issue33387] Simplify bytecodes for try-finally,
 try-except and with blocks.
Message-ID: <1525026522.22.0.682650639539.issue33387@psf.upfronthosting.co.za>


New submission from Mark Shannon <mark at hotpy.org>:

The six complex bytecodes currently used for implementing 'with' and 'try' statements can be replaced with just two simpler bytecodes.
The six bytecodes are WITH_CLEANUP_START, WITH_CLEANUP_FINISH,
BEGIN_FINALLY, END_FINALLY, CALL_FINALLY and POP_FINALLY.
They can be replaced with RERAISE and WITH_EXCEPT_FINISH.

See https://bugs.python.org/issue32949 for more details of the new bytecodes and how they are used in the 'with' statement.

The try-finally statement can be implemented broadly as 
   SETUP_FINALLY except
   try-body
   POP_BLOCK
   finally-body
   JUMP exit
except:
   finally-body
exit:

----------
components: Interpreter Core
messages: 315905
nosy: Mark.Shannon
priority: normal
severity: normal
status: open
title: Simplify bytecodes for try-finally, try-except and with blocks.
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33387>
_______________________________________

From report at bugs.python.org  Sun Apr 29 14:51:59 2018
From: report at bugs.python.org (Robert Bricheno)
Date: Sun, 29 Apr 2018 18:51:59 +0000
Subject: [New-bugs-announce] [issue33388] Support PEP 566 metadata in dist.py
Message-ID: <1525027919.12.0.682650639539.issue33388@psf.upfronthosting.co.za>


New submission from Robert Bricheno <python at ambrosia.org.uk>:

PEP 566 added the metadata fields 'Description-Content-Type' and 'Provides-Extra':

https://www.python.org/dev/peps/pep-0566/
http://setuptools.readthedocs.io/en/latest/setuptools.html#metadata

Currently dist.py in CPython distutils will warn if they are set, e.g.:

`dist.py:261: UserWarning: Unknown distribution option: 'long_description_content_type'`

The warnings are emitted when using setuptools to build a wheel (say) if either of the fields are defined. They seem spurious (I can still build wheels even if I see this warning...). But they looked a bit scary to me as a new user.

There are a few possible ways to fix this. Long-term they should be fully supported by distutils I suppose. In the short term, there are a few different ways to make these warnings go away.

My preferred option would be to simply declare fields for these variables in dist.py (as per setuptools). By declaring these variables, the warnings are no longer displayed. long_description_content_type, at least, is being used in the wild since:

pypa/sampleproject at 5be0970

----------
components: Distutils
messages: 315907
nosy: dstufft, eric.araujo, rbricheno
priority: normal
severity: normal
status: open
title: Support PEP 566 metadata in dist.py
type: enhancement
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33388>
_______________________________________

From report at bugs.python.org  Sun Apr 29 15:38:21 2018
From: report at bugs.python.org (Stefan Seefeld)
Date: Sun, 29 Apr 2018 19:38:21 +0000
Subject: [New-bugs-announce] [issue33389] argparse redundant help string
Message-ID: <1525030701.58.0.682650639539.issue33389@psf.upfronthosting.co.za>


New submission from Stefan Seefeld <stefan at seefeld.name>:

I'm using Python's `argparse` module to define optional arguments.
I'm calling the `argparse.add_argument` method to add both short and long arguments, but I notice that the generated help message lists some information twice. For example:
```
argparse.add_argument('-s', '--service',...)
```
will generate

```
-s SERVICE, --service SERVICE
```
and when I add a `choices` argument, even the choices list is repeated. I think it would be more useful to suppress the repetition to produce output such as
```
-s|--service SERVICE ...
```
instead with both the meta var as well as choices etc. printed only once.

----------
components: Library (Lib)
messages: 315917
nosy: stefan
priority: normal
severity: normal
status: open
title: argparse redundant help string
type: enhancement

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33389>
_______________________________________

From report at bugs.python.org  Sun Apr 29 15:49:56 2018
From: report at bugs.python.org (fabrice salvaire)
Date: Sun, 29 Apr 2018 19:49:56 +0000
Subject: [New-bugs-announce] [issue33390] matmul @ operator precedence
Message-ID: <1525031396.69.0.682650639539.issue33390@psf.upfronthosting.co.za>


New submission from fabrice salvaire <fabrice.salvaire at orange.fr>:

I use the new matmul @ operator to implement units, for example 1 at u_s for 1 second ( see alpha state implementation at https://github.com/FabriceSalvaire/PySpice/tree/master/PySpice/Unit ).

It looks cool, but unfortunately 10 at u_s / 2 at u_s is actually interpreted as (10 at u_s / 2)@u_s instead of (10 at u_s) / (2 at u_s) due to operator precedence + <<< - <<< * <<< @ <<< / <<< // <<< % https://docs.python.org/3/reference/expressions.html#operator-precedence

Since Python is widely used for scientific applications, would it be possible to investigate this topic ?

----------
components: Interpreter Core
messages: 315918
nosy: FabriceSalvaire
priority: normal
severity: normal
status: open
title: matmul @ operator precedence
type: behavior
versions: Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33390>
_______________________________________

From report at bugs.python.org  Mon Apr 30 05:42:42 2018
From: report at bugs.python.org (lekma)
Date: Mon, 30 Apr 2018 09:42:42 +0000
Subject: [New-bugs-announce] [issue33391] leak in set_symmetric_difference?
Message-ID: <1525081362.63.0.682650639539.issue33391@psf.upfronthosting.co.za>


New submission from lekma <lekmalek at gmail.com>:

shouldn't otherset be decrefed before returning on error in set_symmetric_difference? if not, what am I missing?

----------
components: Interpreter Core
files: setobject.c.diff
keywords: patch
messages: 315945
nosy: lekma
priority: normal
severity: normal
status: open
title: leak in set_symmetric_difference?
versions: Python 3.8
Added file: https://bugs.python.org/file47557/setobject.c.diff

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33391>
_______________________________________

From report at bugs.python.org  Mon Apr 30 09:17:33 2018
From: report at bugs.python.org (robbuckley)
Date: Mon, 30 Apr 2018 13:17:33 +0000
Subject: [New-bugs-announce] [issue33392] pathlib .glob('*/') returns files
 as well as directories
Message-ID: <1525094253.43.0.682650639539.issue33392@psf.upfronthosting.co.za>


New submission from robbuckley <rob at buckleynet.uk>:

Path.cwd().glob('/*') seems to yield all files and folders in cwd, the same as .glob('*').

I believe that glob('*/') should yield only directories, and glob('*') all files and directories. this behaviour isnt documented one way or the other. But it would be consistent with glob.glob()

console dump follows:

Python 3.6.5 |Anaconda custom (64-bit)| (default, Mar 29 2018, 13:14:23) 
Type 'copyright', 'credits' or 'license' for more information
IPython 6.3.1 -- An enhanced Interactive Python. Type '?' for help.

In [1]: import os

In [2]: from pathlib import Path

In [6]: import glob

In [13]: list(p.glob('*/'))
Out[13]: 
[PosixPath('/Users/robertbuckley/python-scripts/pathlib_bug/f1.txt'),
 PosixPath('/Users/robertbuckley/python-scripts/pathlib_bug/f3.txt'),
 PosixPath('/Users/robertbuckley/python-scripts/pathlib_bug/d1'),
 PosixPath('/Users/robertbuckley/python-scripts/pathlib_bug/f2.txt'),
 PosixPath('/Users/robertbuckley/python-scripts/pathlib_bug/d2')]

In [14]: list(p.glob('*'))
Out[14]: 
[PosixPath('/Users/robertbuckley/python-scripts/pathlib_bug/f1.txt'),
 PosixPath('/Users/robertbuckley/python-scripts/pathlib_bug/f3.txt'),
 PosixPath('/Users/robertbuckley/python-scripts/pathlib_bug/d1'),
 PosixPath('/Users/robertbuckley/python-scripts/pathlib_bug/f2.txt'),
 PosixPath('/Users/robertbuckley/python-scripts/pathlib_bug/d2')]

In [15]: glob.glob(os.path.join(str(p), '*/'))
Out[15]: 
['/Users/robertbuckley/python-scripts/pathlib_bug/d1/',
 '/Users/robertbuckley/python-scripts/pathlib_bug/d2/']

In [16]: glob.glob(os.path.join(str(p), '*'))
Out[16]: 
['/Users/robertbuckley/python-scripts/pathlib_bug/f1.txt',
 '/Users/robertbuckley/python-scripts/pathlib_bug/f3.txt',
 '/Users/robertbuckley/python-scripts/pathlib_bug/d1',
 '/Users/robertbuckley/python-scripts/pathlib_bug/f2.txt',
 '/Users/robertbuckley/python-scripts/pathlib_bug/d2']

----------
messages: 315949
nosy: robbuckley
priority: normal
severity: normal
status: open
title: pathlib .glob('*/') returns files as well as directories
type: behavior
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33392>
_______________________________________

From report at bugs.python.org  Mon Apr 30 13:14:44 2018
From: report at bugs.python.org (Matthias Klose)
Date: Mon, 30 Apr 2018 17:14:44 +0000
Subject: [New-bugs-announce] [issue33393] update config.guess and config.sub
Message-ID: <1525108484.46.0.682650639539.issue33393@psf.upfronthosting.co.za>


New submission from Matthias Klose <doko at debian.org>:

update config.guess and config.sub from http://git.savannah.gnu.org/

----------
assignee: doko
components: Build
messages: 315954
nosy: doko
priority: normal
severity: normal
status: open
title: update config.guess and config.sub
versions: Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33393>
_______________________________________

From report at bugs.python.org  Mon Apr 30 13:29:31 2018
From: report at bugs.python.org (Matthias Klose)
Date: Mon, 30 Apr 2018 17:29:31 +0000
Subject: [New-bugs-announce] [issue33394] the build of the shared modules is
 quiet/non-visible when GNU make gets passed macros
Message-ID: <1525109371.22.0.682650639539.issue33394@psf.upfronthosting.co.za>


New submission from Matthias Klose <doko at debian.org>:

the build of the shared modules is quiet/non-visible when GNU make gets passed macros. This is was introduced by the change to support BSD make.

GNU make adds all make macros passed on the command line to MAKEFLAGS. If one of these macros contains -s, the build of the extensions get quiet.

Also print out how setup.py is called.

----------
components: Build
files: verbose.diff
keywords: patch
messages: 315956
nosy: doko
priority: normal
severity: normal
status: open
title: the build of the shared modules is quiet/non-visible when GNU make gets passed macros
Added file: https://bugs.python.org/file47558/verbose.diff

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33394>
_______________________________________

From report at bugs.python.org  Mon Apr 30 16:39:51 2018
From: report at bugs.python.org (Siming Yuan)
Date: Mon, 30 Apr 2018 20:39:51 +0000
Subject: [New-bugs-announce] [issue33395] TypeError: unhashable type:
 'instancemethod'
Message-ID: <1525120791.11.0.682650639539.issue33395@psf.upfronthosting.co.za>


New submission from Siming Yuan <siming85 at gmail.com>:

in Python 3.5 it the pprint.PrettyPrinter mechanism got an overhaul, relying on PrettyPrinter._dispatch dict-lookup based on obj.__repr__ type.

This breaks any Cythonized 3rd party libraries that used to be pretty-printable in Python3.4.

type(object).__repr__
<instancemethod __repr__ at 0x10cf2a618>

since instancemethod_hash function has been commented out:
https://github.com/python/cpython/blob/c30098c8c6014f3340a369a31df9c74bdbacc269/Objects/classobject.c#L569


oddly the behavior is different between Linux and Mac.

The same object in Linux returns cyfunction, and is hashable,
where as under the same CPython version in Mac, it returns instancemethod, rendering it unhashable.
(based on Cython 0.27.3)

note that this isn't exactly something related directly to the implementation of Cython.

the old logic in Python <3.4 pprint was not pretty (pun not intended), but relied solely on type checking,
where as the new implementation depends on hashing, which introduces this bug.

----------
messages: 315964
nosy: siming85
priority: normal
severity: normal
status: open
title: TypeError: unhashable type: 'instancemethod'
type: crash
versions: Python 3.5, Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33395>
_______________________________________