From report at bugs.python.org  Wed Aug  1 01:09:03 2018
From: report at bugs.python.org (Sergey Fedoseev)
Date: Wed, 01 Aug 2018 05:09:03 +0000
Subject: [New-bugs-announce] [issue34303] micro-optimizations in
 functools.reduce()
Message-ID: <1533100143.91.0.56676864532.issue34303@psf.upfronthosting.co.za>


New submission from Sergey Fedoseev <fedoseev.sergey at gmail.com>:

`PyTuple_SetItem()` can be replaced with macro version and `PyObject_Call()` can be used instead of `PyEval_CallObject()`.

Here's benchmark results:

$ python -m perf timeit --compare-to ~/tmp/cpython-master-venv/bin/python -s "from functools import reduce; from operator import is_; r = range(50000)" "reduce(is_, r)"
/home/sergey/tmp/cpython-master-venv/bin/python: ..................... 1.64 ms +- 0.01 ms
/home/sergey/tmp/cpython-venv/bin/python: ..................... 1.40 ms +- 0.00 ms

Mean +- std dev: [/home/sergey/tmp/cpython-master-venv/bin/python] 1.64 ms +- 0.01 ms -> [/home/sergey/tmp/cpython-venv/bin/python] 1.40 ms +- 0.00 ms: 1.17x faster (-15%)

----------
components: Extension Modules
messages: 322841
nosy: sir-sigurd
priority: normal
severity: normal
status: open
title: micro-optimizations in functools.reduce()
type: enhancement

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34303>
_______________________________________

From report at bugs.python.org  Wed Aug  1 01:13:39 2018
From: report at bugs.python.org (Saba Kauser)
Date: Wed, 01 Aug 2018 05:13:39 +0000
Subject: [New-bugs-announce] [issue34304] clarification on escaping \d in
 regular expressions
Message-ID: <1533100419.92.0.56676864532.issue34304@psf.upfronthosting.co.za>


New submission from Saba Kauser <skauseribmdb at gmail.com>:

Hello,

I have a program that works well upto python 3.6 but fails with python 3.7.

import re

pattern="DBMS_NAME: string(%d) %s"
sym = ['\[','\]','\(','\)']
for chr in sym:
  pattern = re.sub(chr, '\\' + chr, pattern)
  print(pattern)
  
pattern=re.sub('%s','.*?',pattern)
print(pattern)
pattern = re.sub('%d', '\\d+', pattern) 
print(pattern)
result=re.match(pattern, "DBMS_NAME: string(8) \"DB2/NT64\" ")
print(result)
result=re.match("DBMS_NAME python4: string\(\d+\) .*?", "DBMS_NAME python4: string(8) \"DB2/NT64\" ")
print(result)

expected output:
DBMS_NAME: string(%d) %s
DBMS_NAME: string(%d) %s
DBMS_NAME: string\(%d) %s
DBMS_NAME: string\(%d\) %s
DBMS_NAME: string\(%d\) .*?
DBMS_NAME: string\(\d+\) .*?
<re.Match object; span=(0, 21), match='DBMS_NAME: string(8) '>
<re.Match object; span=(0, 29), match='DBMS_NAME python4: string(8) '>

However, the below statement execution fails with python 3.7:
pattern = re.sub('%d', '\\d+', pattern) 

DBMS_NAME: string(%d) %s
DBMS_NAME: string(%d) %s
DBMS_NAME: string\(%d) %s
DBMS_NAME: string\(%d\) %s
DBMS_NAME: string\(%d\) .*?
Traceback (most recent call last):
  File "c:\users\skauser\appdata\local\programs\python\python37\lib\sre_parse.py", line 1021, in parse_template
    this = chr(ESCAPES[this][1])
KeyError: '\\d'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "pattern.txt", line 11, in <module>
    pattern = re.sub('%d', '\\d+', pattern)
  File "c:\users\skauser\appdata\local\programs\python\python37\lib\re.py", line 192, in sub
    return _compile(pattern, flags).sub(repl, string, count)
  File "c:\users\skauser\appdata\local\programs\python\python37\lib\re.py", line 309, in _subx
    template = _compile_repl(template, pattern)
  File "c:\users\skauser\appdata\local\programs\python\python37\lib\re.py", line 300, in _compile_repl
    return sre_parse.parse_template(repl, pattern)
  File "c:\users\skauser\appdata\local\programs\python\python37\lib\sre_parse.py", line 1024, in parse_template
    raise s.error('bad escape %s' % this, len(this))
re.error: bad escape \d at position 0

if I change the statement to have 3 backslash like 
pattern = re.sub('%d', '\\\d+', pattern) 

I can correctly generate correct regular expression.

Can you please comment if this has changed in python 3.7 and we need to escape 'd' in '\d' as well ?

Thank you!

----------
components: Regular Expressions
messages: 322842
nosy: ezio.melotti, mrabarnett, sabakauser
priority: normal
severity: normal
status: open
title: clarification on escaping \d in regular expressions
type: behavior
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34304>
_______________________________________

From report at bugs.python.org  Wed Aug  1 03:20:53 2018
From: report at bugs.python.org (Eric Wieser)
Date: Wed, 01 Aug 2018 07:20:53 +0000
Subject: [New-bugs-announce] [issue34305] inspect.getsourcefile and
 inspect.getcomments do not work with decorated functions
Message-ID: <1533108053.89.0.56676864532.issue34305@psf.upfronthosting.co.za>


New submission from Eric Wieser <wieser.eric+pybug at gmail.com>:

Following on from https://bugs.python.org/issue1764286 - inspect.getsourcefile and inspect.getcomments fall afoul of the same problem.

----------
components: Library (Lib)
messages: 322847
nosy: Eric.Wieser
priority: normal
severity: normal
status: open
title: inspect.getsourcefile and inspect.getcomments do not work with decorated functions

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34305>
_______________________________________

From report at bugs.python.org  Wed Aug  1 09:32:29 2018
From: report at bugs.python.org (Victor Porton)
Date: Wed, 01 Aug 2018 13:32:29 +0000
Subject: [New-bugs-announce] [issue34306] minidom: wrong processing of xmlns
 attributes
Message-ID: <1533130349.62.0.56676864532.issue34306@psf.upfronthosting.co.za>


New submission from Victor Porton <porton at narod.ru>:

The below script prints

http://www.w3.org/2000/xmlns/
aa:aa

It should print

None
aa:aa

because xmlns:z is NOT an attribute of xmlns namespace.

~~~
import xml.dom.minidom

x = xml.dom.minidom.parseString("<x xmlns:z='aa:aa' z:q='1'/>")

for i in x.documentElement.attributes.values():
    print(i.namespaceURI)
~~~

----------
components: XML
messages: 322860
nosy: porton
priority: normal
severity: normal
status: open
title: minidom: wrong processing of xmlns attributes
versions: Python 2.7, Python 3.6, Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34306>
_______________________________________

From report at bugs.python.org  Wed Aug  1 09:34:15 2018
From: report at bugs.python.org (Romuald Brunet)
Date: Wed, 01 Aug 2018 13:34:15 +0000
Subject: [New-bugs-announce] [issue34307] NullHandler init refusing arguments
Message-ID: <1533130455.97.0.56676864532.issue34307@psf.upfronthosting.co.za>


New submission from Romuald Brunet <romuald.brunet at gmail.com>:

Context:

I'm using a custom Handler that relies on a third-party library that may not be present

try:
  from acme.logging import CustomHandler as BaseHandler
except ImportError:
  import logging.NullHandler as BaseHandler

class MoreCustomHandler(BaseHandler):
  def handle(self):
    self.stuff()

The logging configuration has some arguments for the CustomHandler. The problem is that the NullHandler does not accept them.

I expected it to be a "dummy" handler that might accept any argument

----------
components: Library (Lib)
messages: 322861
nosy: Romuald
priority: normal
severity: normal
status: open
title: NullHandler init refusing arguments
type: behavior
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34307>
_______________________________________

From report at bugs.python.org  Wed Aug  1 10:51:51 2018
From: report at bugs.python.org (Chantal Ackermann)
Date: Wed, 01 Aug 2018 14:51:51 +0000
Subject: [New-bugs-announce] [issue34308] shutil.copystat fails in dockered
 Linux (Debian) on Azure (Windows)
Message-ID: <1533135111.37.0.56676864532.issue34308@psf.upfronthosting.co.za>


New submission from Chantal Ackermann <chantal.ackermann at web.de>:

root:/media/documents# df -h
Filesystem                Size      Used Available Use% Mounted on
overlay                  29.0G     24.0G      5.0G  83% /
tmpfs                     1.7G         0      1.7G   0% /dev
tmpfs                     1.7G         0      1.7G   0% /sys/fs/cgroup
//storage01.file.core.windows.net/kubernetes-dynamic-pvc-526aa3da-6993-11e8-8c6f-0a58ac1f00ad
                          1.0G    384.0K   1023.6M   0% /media
root:/media/documents# ls -al
insgesamt 267
drwxrwxrwx 2 1000 1000      0 Jul 31 15:29 .
drwxrwxrwx 2 1000 1000      0 Jul 31 15:29 ..
-rwxrwxrwx 1 1000 1000 136479 Jul 31 16:48 orig.pdf
-rwxrwxrwx 1 1000 1000 136479 Jul 31 15:29 testfile
root:/media/documents# lsattr 
--S-----c-jI------- ./orig.pdf
--S-----c-jI------- ./testfile
root:/media/documents# python
Python 3.6.6 (default, Jul 17 2018, 11:12:33) 
[GCC 6.3.0 20170516] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import shutil
>>> shutil.copystat('orig.pdf', 'testfile')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.6/shutil.py", line 225, in copystat
    _copyxattr(src, dst, follow_symlinks=follow)
  File "/usr/local/lib/python3.6/shutil.py", line 157, in _copyxattr
    names = os.listxattr(src, follow_symlinks=follow_symlinks)
OSError: [Errno 38] Function not implemented: 'orig.pdf'
>>> shutil.copystat('orig.pdf', 'testfile', follow_symlinks=False)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.6/shutil.py", line 225, in copystat
    _copyxattr(src, dst, follow_symlinks=follow)
  File "/usr/local/lib/python3.6/shutil.py", line 157, in _copyxattr
    names = os.listxattr(src, follow_symlinks=follow_symlinks)
OSError: [Errno 38] Function not implemented: 'orig.pdf'
>>> 

shutil.py#164ff:

if hasattr(os, 'listxattr'):
    def _copyxattr(src, dst, *, follow_symlinks=True):
        try:
            names = os.listxattr(src, follow_symlinks=follow_symlinks)
        except OSError as e:
            if e.errno not in (errno.ENOTSUP, errno.ENODATA):
                raise
            return

=> This is true inside the docker container. However, calling os.listxattr fails with errno.ENOSYS (Error 38).

The documentation states that "copystat() never returns failure."

StackOverflow link: https://stackoverflow.com/questions/51616058/shutil-copystat-fails-inside-docker-on-azure#51635427

I can provide a pull request, if it is OK to add errno.ENOSYS to the list of ignored errors. (Would be a similar fix to the one provided for https://bugs.python.org/issue24564 (https://github.com/python/cpython/pull/8601).)

----------
components: Library (Lib)
messages: 322866
nosy: nuarhu
priority: normal
severity: normal
status: open
title: shutil.copystat fails in dockered Linux (Debian) on Azure (Windows)
type: behavior
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34308>
_______________________________________

From report at bugs.python.org  Wed Aug  1 13:36:13 2018
From: report at bugs.python.org (chris)
Date: Wed, 01 Aug 2018 17:36:13 +0000
Subject: [New-bugs-announce] [issue34309] Embedding Python;
 Py_Initialize / Py_Finalize cycles
Message-ID: <1533144973.44.0.56676864532.issue34309@psf.upfronthosting.co.za>


New submission from chris <christoph.wiedemann at hs-weingarten.de>:

I'm linking an issue from numpy here: https://github.com/numpy/numpy/issues/8097

Embedding python suffers from a possibility to reliably reset the state of the python interpreter. For my use case, I noticed that when using numpy with Py_Initialize() and Py_Finalize():

Py_Initialize()
// call code importing numpy
Py_Finalize()
Py_Initialize()
// call same code again

The above code will result in a crash.

One of the comments in the referenced issue is that Py_Finalize doesn't unload loaded DLL's or shared objects. Doing that would probably fix the issues.

As of now, embedding python is fundamentally broken for applications which want to embed non-trivial scientific python scripts involving user-adapted python code, because

a) Py_Finalize cannot be used reliably
b) There is no possibility to reliably reset the python interpreter otherwise (because the sub-interpreters are also not working reliably, which is stated in the documentation)
c) manually reloading changed modules via importlib.reload is not a feasible solution

The possibility to reset an embedded python interpreter to an initial state is a strong requirement for such applications.

----------
components: Extension Modules, Interpreter Core
messages: 322876
nosy: christoph.wiedemann at hs-weingarten.de
priority: normal
severity: normal
status: open
title: Embedding Python; Py_Initialize / Py_Finalize cycles
type: crash
versions: Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34309>
_______________________________________

From report at bugs.python.org  Wed Aug  1 13:38:45 2018
From: report at bugs.python.org (Wu Yu)
Date: Wed, 01 Aug 2018 17:38:45 +0000
Subject: [New-bugs-announce] [issue34310] Build error with option
 "--with-pydebug" on Mac OS 10.13.6
Message-ID: <1533145125.37.0.56676864532.issue34310@psf.upfronthosting.co.za>


New submission from Wu Yu <chaconnewu at gmail.com>:

I'm encountering a build error from the master branch of CPython (heads/master-dirty:d17fe275a3) on Mac OS 10.13.6, with "--with-pydebug" turned on. 

The error message:

./python.exe -E -S -m sysconfig --generate-posix-vars ;\
	if test $? -ne 0 ; then \
		echo "generate-posix-vars failed" ; \
		rm -f ./pybuilddir.txt ; \
		exit 1 ; \
	fi
Assertion failed: (TYPE(ch) == test || TYPE(ch) == test_nocond || TYPE(ch) == star_expr), function seq_for_testlist, file Python/ast.c, line 1205.
/bin/sh: line 1: 34527 Abort trap: 6           ./python.exe -E -S -m sysconfig --generate-posix-vars
generate-posix-vars failed
make: *** [pybuilddir.txt] Error 1

----------
components: Build
messages: 322877
nosy: wuyu
priority: normal
severity: normal
status: open
title: Build error with option "--with-pydebug" on Mac OS 10.13.6
type: compile error
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34310>
_______________________________________

From report at bugs.python.org  Wed Aug  1 16:22:51 2018
From: report at bugs.python.org (James Emerton)
Date: Wed, 01 Aug 2018 20:22:51 +0000
Subject: [New-bugs-announce] [issue34311] locale.format() and
 locale.format_string() cast Decimals to float
Message-ID: <1533154971.69.0.56676864532.issue34311@psf.upfronthosting.co.za>


New submission from James Emerton <james at emerton.info>:

We use locale.format('%.2f', x, True) to convert Decimal values to strings for display. Unfortunately, the locale module is using %-formatting to generate the initial string before applying locale specific formatting. As a result, any value which cannot be accurately represented as a float will produce incorrect results.

I've built some formatting that uses new-style string formatting (and some internal locale functions) which corrects the problem.

Unfortunately, making this change in the locale module would require converting the input format string to the new syntax, so '%.2f' would become '{:.2f}'.

See also #33731

----------
components: Library (Lib)
messages: 322885
nosy: jemerton
priority: normal
severity: normal
status: open
title: locale.format() and locale.format_string() cast Decimals to float
versions: Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34311>
_______________________________________

From report at bugs.python.org  Wed Aug  1 16:47:42 2018
From: report at bugs.python.org (Brett Cannon)
Date: Wed, 01 Aug 2018 20:47:42 +0000
Subject: [New-bugs-announce] [issue34312] Allow str.endswith and
 str.startswith to accept an iterable
Message-ID: <1533156462.08.0.56676864532.issue34312@psf.upfronthosting.co.za>


New submission from Brett Cannon <brett at python.org>:

str.endswith() and str.startswith() only takes str or a tuple of str. It might be nice to make this str or an iterable of str (and that order must be kept so that a string isn't treated as an iterable of length-1 str).

----------
components: Interpreter Core
messages: 322887
nosy: brett.cannon
priority: low
severity: normal
status: open
title: Allow str.endswith and str.startswith to accept an iterable
type: enhancement
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34312>
_______________________________________

From report at bugs.python.org  Wed Aug  1 16:49:42 2018
From: report at bugs.python.org (Tal Einat)
Date: Wed, 01 Aug 2018 20:49:42 +0000
Subject: [New-bugs-announce] [issue34313] IDLE crashes with Tk-related error
 on macOS with ActiveTcl 8.6
Message-ID: <1533156582.15.0.56676864532.issue34313@psf.upfronthosting.co.za>


New submission from Tal Einat <taleinat at gmail.com>:

On macOS 10.13.5 (the latest available) with ActiveTcl 8.6.7 (the latest available), I'm building Python from latest master (to be 3.8.0).  I'm consistently having IDLE crash almost immediately, just by opening any file for editing.  The crash results in the following error:

$ root/bin/python3 -m idlelib
objc[50759]: Invalid or prematurely-freed autorelease pool 0x7fb74091f050.
[1]    50759 abort      root/bin/python3 -m idlelib

In issue34120, @vtudorache reported a similar issue (also on macOS).

With Python 3.7.0 from python.org, everything works fine.

Strangely, replacing ActiveTcl with version 8.5.18.0 resolves this issue.

I'm not sure whether this is specific to IDLE; I'll try to provide a minimal way to reproduce.

----------
assignee: terry.reedy
components: IDLE, Tkinter
messages: 322888
nosy: gpolo, serhiy.storchaka, taleinat, terry.reedy, vtudorache
priority: normal
severity: normal
status: open
title: IDLE crashes with Tk-related error on macOS with ActiveTcl 8.6
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34313>
_______________________________________

From report at bugs.python.org  Wed Aug  1 23:02:06 2018
From: report at bugs.python.org (Dan Snider)
Date: Thu, 02 Aug 2018 03:02:06 +0000
Subject: [New-bugs-announce] [issue34314] Like __hash__,
 allow setting MyClass.__init__ to None to prevent it being called
 by type.__call__
Message-ID: <1533178926.29.0.56676864532.issue34314@psf.upfronthosting.co.za>


New submission from Dan Snider <mr.assume.away at gmail.com>:

Right now, you really gotta jump through hoops in some cases if you only want to use __new__ and don't care about __init__ (is there ever a time where you'd use both?). The problem originates in type.__call__. I'm attaching a full Python implementation of type_call/object_init at the end, but here's a tl;dr representation of the issue i type_call (where the whole thing can be basically summarized as this):

def type_call(type, *args, **kws):
   r = type.__new__(type, *args, **kws)
   if not issubclass(r, type):
       type(r).__init__(r, *args, **kws)
   return r

So if type(r).__init__ is object.__init__ or some other method with an incompatible signature to the metaclass's, errors are raised, which leads to having to implement really annoying workarounds. The annoyingness is compounded further by the fact that all object_init does is make sure it didn't receive any arguments. 

All of that can be avoided by setting __init__ in the class to None, which would have the same effect as setting tp_init to NULL on an extension type. Perhaps with a caveat that the only current reachable __init__ is object.__init__?

I don't really find myself in situations involving the need for __new__ too often, but wouldn't this have a profoundly positive impact on pickle as well? One thing I can think of is there can be a universal pickle implementation for extension types with Py_TPFLAGS_BASETYPE set and a nonzero dict offset since if it detected a NULL init all it has to do is memcpy the base struct and pickle the instance dict.

Anyway, the C code is pretty easy to understand but here's a Python implementation almost exactly equivalent that shows a case where a metaclass can be used as a callable that accepts an arbitrary number of arguments instead of a forced three and returns a class:


PyType_Type = type
Py_TYPE = type
PyType_IsSubtype = issubclass
PyTuple_Check = tuple.__instancecheck__
PyDict_Check = dict.__instancecheck__
PyDict_GET_SIZE = dict.__len__
PyTUple_GET_SIZE = tuple.__len__

NULL = 0
ALLOW_NULL_INIT = False

def excess_args(args, kws):
    return args or kws

class Object:

    def __init__(self, *args, **kws):
        # object_init literally does nothing but check for zero len args/kws
        # (bit of a tangent but why is excess_args not a macro...?)
        if ALLOW_NULL_INIT:
            return
        tp = Py_TYPE(self)
        print('Object.__init__, type is:', tp.__name__)
        if excess_args(args, kws):
            if tp.__init__ is object.__init__:
                raise TypeError("object.__init__() takes no arguments")
            raise TypeError(f'{tp.__name__}.__init__() takes no arguments')

def fake_init(*args, **kws):
    pass

class Type(type):

    def __getattribute__(self, attr):
        value = type.__getattribute__(self, attr)
        if attr == '__init__':
            if ALLOW_NULL_INIT and value is None:
                return fake_init
            return super(type(self).__mro__[1], self).__init__
        return value
    
    def __call__(tp, *args, **kws):
        
        if getattr(tp, '__new__', 0) == NULL:
            raise TypeError(f"cannot create {tp.__name__} instances")
        
        obj = tp.__new__(tp, *args, **kws)
        if (tp is type and
            PyTuple_Check(args) and PyTuple_GET_SIZE(args)== 1 and
            (kws == NULL or
             (PyDict_Check(kws) and PyDict_GET_SIZE(kws)==0))):
            return obj
        
        if not PyType_IsSubtype(Py_TYPE(obj), tp):
            return obj
        
        tp = Py_TYPE(obj)
        
        # Here's where the problem is. What could possibly be negatively
        # affected by this?
        if getattr(tp, '__init__', 0):
            res = tp.__init__(obj, *args, **kws)
            
        return obj

def autoname(arg):
    return '_'.join(arg.split())

def autobase(opt):
    return () if not opt else (opt,)

class MetaBase(Object, type):
    pass

class Mixin1:
    pass

class Meta(MetaBase, metaclass=Type):

    __init__ = None
        
    def __new__(metacls, name, opt=None):
        return super().__new__(metacls, autoname(name), autobase(opt), {})

if __name__ == '__main__':
    try:
        foobar = Meta('foo bar', Mixin1)
    except TypeError as er:
        print(f'could not make foobar because: {er}')
    print('setting ALLOW_NULL_INIT to True;')
    ALLOW_NULL_INIT = True
    foobar = Meta('foo bar', Mixin1)
    print('foobar.__name__ is:', foobar.__name__)
    print('foobar__mro__ is:', foobar.__mro__)

----------
components: Interpreter Core
messages: 322907
nosy: bup
priority: normal
severity: normal
status: open
title: Like __hash__, allow setting MyClass.__init__ to None to prevent it being called by type.__call__
type: enhancement
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34314>
_______________________________________

From report at bugs.python.org  Thu Aug  2 01:52:59 2018
From: report at bugs.python.org (Raman)
Date: Thu, 02 Aug 2018 05:52:59 +0000
Subject: [New-bugs-announce] [issue34315] Regex not evalauated correctly
Message-ID: <1533189179.81.0.56676864532.issue34315@psf.upfronthosting.co.za>


New submission from Raman <s_raman002 at yahoo.com>:

Sample code below

import re

regex = r'DELETE\s*(?P<table_alias>[a-zA-z_0-9]*)\s*FROM\s*(?P<table_name>[a-zA-z_0-9]+)\s*([a-zA-Z0-9_]*)\s*(?P<where_statement>WHERE){0,1}(\s.)*?'

test_str = 'DELETE FROM my_table1 t_ WHERE id in (1,2,3)'

matches = re.finditer(regex, test_str, re.MULTILINE)

print([m.groupdict() for m in matches])

Below is the expected output.

[{'table_alias': '', 'table_name': 'my_table1', 'where_statement': 'WHERE'}]

But in Win Server 2012 R2, the output is
[{'table_alias': '', 'table_name': 'mytable1', 'where_statement': None}]

Using 3.7 in Win Server 2012 R2 also the output is not as expected. But in Win 10 and other linux variants, expected output is obtained.

----------
components: Regular Expressions
messages: 322916
nosy: ezio.melotti, mrabarnett, ram, serhiy.storchaka
priority: normal
severity: normal
status: open
title: Regex not evalauated correctly
type: behavior
versions: Python 3.6, Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34315>
_______________________________________

From report at bugs.python.org  Thu Aug  2 06:11:39 2018
From: report at bugs.python.org (Pablo Galindo Salgado)
Date: Thu, 02 Aug 2018 10:11:39 +0000
Subject: [New-bugs-announce] [issue34316] test_socket timeouts in AMD64
 Windows10 3.x buildbots
Message-ID: <1533204699.35.0.56676864532.issue34316@psf.upfronthosting.co.za>


New submission from Pablo Galindo Salgado <pablogsal at gmail.com>:

Some tests in test_socket and test_asyncio are receiving timeouts in AMD64 Windows10 3.x buildbots: 

test_asyncio: https://buildbot.python.org/all/#/builders/3/builds/1192
test_socket: https://buildbot.python.org/all/#/builders/3/builds/1191

Sample error messages:

test_not_implemented (test.test_asyncio.test_events.AbstractEventLoopTests) ... ok
Timeout (0:15:00)!
Thread 0x00001a74 (most recent call first):
  File "D:\buildarea\3.x.bolen-windows10\build\lib\socket.py", line 212 in accept
  File "D:\buildarea\3.x.bolen-windows10\build\lib\socket.py", line 528 in socketpair
  File "D:\buildarea\3.x.bolen-windows10\build\lib\asyncio\selector_events.py", line 113 in _make_self_pipe
  File "D:\buildarea\3.x.bolen-windows10\build\lib\asyncio\selector_events.py", line 66 in __init__
  File "D:\buildarea\3.x.bolen-windows10\build\lib\asyncio\events.py", line 660 in new_event_loop
  File "D:\buildarea\3.x.bolen-windows10\build\lib\asyncio\events.py", line 762 in new_event_loop
  File "D:\buildarea\3.x.bolen-windows10\build\lib\test\test_asyncio\test_events.py", line 3061 in test_not_implemented_async
  File "D:\buildarea\3.x.bolen-windows10\build\lib\unittest\case.py", line 615 in run
  File "D:\buildarea\3.x.bolen-windows10\build\lib\unittest\case.py", line 663 in __call__
  File "D:\buildarea\3.x.bolen-windows10\build\lib\unittest\suite.py", line 122 in run
  File "D:\buildarea\3.x.bolen-windows10\build\lib\unittest\suite.py", line 84 in __call__
  File "D:\buildarea\3.x.bolen-windows10\build\lib\unittest\suite.py", line 122 in run
  File "D:\buildarea\3.x.bolen-windows10\build\lib\unittest\suite.py", line 84 in __call__
  File "D:\buildarea\3.x.bolen-windows10\build\lib\unittest\suite.py", line 122 in run
  File "D:\buildarea\3.x.bolen-windows10\build\lib\unittest\suite.py", line 84 in __call__
  File "D:\buildarea\3.x.bolen-windows10\build\lib\unittest\suite.py", line 122 in run
  File "D:\buildarea\3.x.bolen-windows10\build\lib\unittest\suite.py", line 84 in __call__
  File "D:\buildarea\3.x.bolen-windows10\build\lib\unittest\suite.py", line 122 in run
  File "D:\buildarea\3.x.bolen-windows10\build\lib\unittest\suite.py", line 84 in __call__
  File "D:\buildarea\3.x.bolen-windows10\build\lib\unittest\runner.py", line 176 in run
  File "D:\buildarea\3.x.bolen-windows10\build\lib\test\support\__init__.py", line 1883 in _run_suite
  File "D:\buildarea\3.x.bolen-windows10\build\lib\test\support\__init__.py", line 1973 in run_unittest
  File "D:\buildarea\3.x.bolen-windows10\build\lib\test\libregrtest\runtest.py", line 175 in test_runner
  File "D:\buildarea\3.x.bolen-windows10\build\lib\test\libregrtest\runtest.py", line 179 in runtest_inner
  File "D:\buildarea\3.x.bolen-windows10\build\lib\test\libregrtest\runtest.py", line 140 in runtest
  File "D:\buildarea\3.x.bolen-windows10\build\lib\test\libregrtest\main.py", line 286 in rerun_failed_tests
  File "D:\buildarea\3.x.bolen-windows10\build\lib\test\libregrtest\main.py", line 570 in _main
  File "D:\buildarea\3.x.bolen-windows10\build\lib\test\libregrtest\main.py", line 531 in main
  File "D:\buildarea\3.x.bolen-windows10\build\lib\test\libregrtest\main.py", line 584 in main
  File "D:\buildarea\3.x.bolen-windows10\build\lib\test\__main__.py", line 2 in <module>
  File "D:\buildarea\3.x.bolen-windows10\build\lib\runpy.py", line 85 in _run_code
  File "D:\buildarea\3.x.bolen-windows10\build\lib\runpy.py", line 193 in _run_module_as_main
program finished with exit code 1
elapsedTime=5350.801000
test_not_implemented_async (test.test_asyncio.test_events.AbstractEventLoopTests) ... 

and 

testTimeoutDefault (test.test_socket.NetworkConnectionAttributesTest) ... ok
Unhandled exception in thread started by <bound method ThreadableTest.clientRun of <test.test_socket.NetworkConnectionAttributesTest testMethod=testTimeoutNone>>
Traceback (most recent call last):
  File "D:\buildarea\3.x.bolen-windows10\build\lib\test\test_socket.py", line 336, in clientRun
    self.clientTearDown()
  File "D:\buildarea\3.x.bolen-windows10\build\lib\test\test_socket.py", line 4752, in clientTearDown
    self.cli.close()
AttributeError: 'NetworkConnectionAttributesTest' object has no attribute 'cli'
Timeout (0:15:00)!
Thread 0x0000184c (most recent call first):
  File "D:\buildarea\3.x.bolen-windows10\build\lib\socket.py", line 212 in accept
  File "D:\buildarea\3.x.bolen-windows10\build\lib\test\test_socket.py", line 4757 in _justAccept
  File "D:\buildarea\3.x.bolen-windows10\build\lib\unittest\case.py", line 615 in run
  File "D:\buildarea\3.x.bolen-windows10\build\lib\unittest\case.py", line 663 in __call__
  File "D:\buildarea\3.x.bolen-windows10\build\lib\unittest\suite.py", line 122 in run
  File "D:\buildarea\3.x.bolen-windows10\build\lib\unittest\suite.py", line 84 in __call__
  File "D:\buildarea\3.x.bolen-windows10\build\lib\unittest\suite.py", line 122 in run
  File "D:\buildarea\3.x.bolen-windows10\build\lib\unittest\suite.py", line 84 in __call__
  File "D:\buildarea\3.x.bolen-windows10\build\lib\unittest\runner.py", line 176 in run
  File "D:\buildarea\3.x.bolen-windows10\build\lib\test\support\__init__.py", line 1883 in _run_suite
  File "D:\buildarea\3.x.bolen-windows10\build\lib\test\support\__init__.py", line 1973 in run_unittest
  File "D:\buildarea\3.x.bolen-windows10\build\lib\test\test_socket.py", line 6032 in test_main
  File "D:\buildarea\3.x.bolen-windows10\build\lib\test\libregrtest\runtest.py", line 179 in runtest_inner
  File "D:\buildarea\3.x.bolen-windows10\build\lib\test\libregrtest\runtest.py", line 140 in runtest
  File "D:\buildarea\3.x.bolen-windows10\build\lib\test\libregrtest\main.py", line 286 in rerun_failed_tests
  File "D:\buildarea\3.x.bolen-windows10\build\lib\test\libregrtest\main.py", line 570 in _main
  File "D:\buildarea\3.x.bolen-windows10\build\lib\test\libregrtest\main.py", line 531 in main
  File "D:\buildarea\3.x.bolen-windows10\build\lib\test\libregrtest\main.py", line 584 in main
  File "D:\buildarea\3.x.bolen-windows10\build\lib\test\__main__.py", line 2 in <module>
  File "D:\buildarea\3.x.bolen-windows10\build\lib\runpy.py", line 85 in _run_code
  File "D:\buildarea\3.x.bolen-windows10\build\lib\runpy.py", line 193 in _run_module_as_main
program finished with exit code 1
elapsedTime=4059.474000

----------
components: Tests
messages: 322937
nosy: pablogsal
priority: normal
severity: normal
status: open
title: test_socket timeouts in AMD64 Windows10 3.x buildbots
type: behavior
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34316>
_______________________________________

From report at bugs.python.org  Thu Aug  2 06:53:50 2018
From: report at bugs.python.org (sakamoto aya)
Date: Thu, 02 Aug 2018 10:53:50 +0000
Subject: [New-bugs-announce] [issue34317] Improve docstring Environment
 variables in Windows NT
Message-ID: <1533207230.71.0.56676864532.issue34317@psf.upfronthosting.co.za>


New submission from sakamoto aya <aya.sakamoto at outlook.com>:

Expired link in windows.rst:  
https://support.microsoft.com/en-us/help/100843/environment-variables-in-windows-nt

May I make a suggestion?

The new link:https://www.microsoft.com/en-us/wdsi/help/folder-variables

----------
assignee: docs at python
components: Documentation
messages: 322944
nosy: HiyashiChuka, docs at python
priority: normal
severity: normal
status: open
title: Improve docstring   Environment variables in Windows NT
type: behavior
versions: Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34317>
_______________________________________

From report at bugs.python.org  Thu Aug  2 07:39:54 2018
From: report at bugs.python.org (Serhiy Storchaka)
Date: Thu, 02 Aug 2018 11:39:54 +0000
Subject: [New-bugs-announce] [issue34318] Convert deprecated behavior of
 assertRaises() etc into errors
Message-ID: <1533209994.63.0.56676864532.issue34318@psf.upfronthosting.co.za>


New submission from Serhiy Storchaka <storchaka+cpython at gmail.com>:

Currently assertRaises(), assertRaisesRegex(), assertWarns() and assertWarnsRegex() have some weird behavior.

    # always success if the callable is None
    self.assertRaises(SomeException, None)

    # keyword arguments except "msg" are ignored
    with self.assertRaises(SomeException, foobar=123):
        ...

    # always success because keyword arguments are ignored
    self.assertRaises(SomeException, callable=func)

Hardly any user code uses these "features" intentionally. More likely such examples are hidden bugs (see for example [1]). A DeprecationWarning is raised in these cases since 3.5 (issue24134), and it is time to make them errors.

[1] https://mail.python.org/pipermail/python-list/2018-July/736363.html

----------
components: Tests
messages: 322946
nosy: ezio.melotti, michael.foord, rbcollins, serhiy.storchaka
priority: normal
severity: normal
status: open
title: Convert deprecated behavior of assertRaises() etc into errors
type: enhancement
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34318>
_______________________________________

From report at bugs.python.org  Thu Aug  2 07:52:33 2018
From: report at bugs.python.org (Thomas Nyberg)
Date: Thu, 02 Aug 2018 11:52:33 +0000
Subject: [New-bugs-announce] [issue34319] Clarify
 pathlib.Path("filepath").read_text()
Message-ID: <1533210753.86.0.56676864532.issue34319@psf.upfronthosting.co.za>


New submission from Thomas Nyberg <tomnyberg at gmail.com>:

This came out of the following posts:

https://mail.python.org/pipermail/python-ideas/2018-August/052549.html
https://mail.python.org/pipermail/python-ideas/2018-August/052553.html

Basically my request would be to change the documentation here:

https://docs.python.org/3.7/library/pathlib.html#pathlib.Path.read_text

I would like to add a note that the underlying file object itself is closed after the read_text() method is called. Maybe I'm just a little dense and it should be obvious that the functionality here would be different than open("filepath").read(), but given that thread I linked, I don't believe I'm the only one.

----------
assignee: docs at python
components: Documentation
messages: 322947
nosy: docs at python, thomas.nyberg
priority: normal
severity: normal
status: open
title: Clarify pathlib.Path("filepath").read_text()

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34319>
_______________________________________

From report at bugs.python.org  Thu Aug  2 09:12:23 2018
From: report at bugs.python.org (Serhiy Storchaka)
Date: Thu, 02 Aug 2018 13:12:23 +0000
Subject: [New-bugs-announce] [issue34320] Creating dict from OrderedDict
 doesn't preserve order
Message-ID: <1533215543.91.0.56676864532.issue34320@psf.upfronthosting.co.za>


New submission from Serhiy Storchaka <storchaka+cpython at gmail.com>:

>>> from collections import OrderedDict
>>> od = OrderedDict([('a', 1), ('b', 2)])
>>> od.move_to_end('a')
>>> od
OrderedDict([('b', 2), ('a', 1)])
>>> dict(od)
{'a': 1, 'b': 2}

This affects also PEP 468.

>>> def f(**kwargs): return kwargs
... 
>>> f(**od)
{'a': 1, 'b': 2}

And PEP 520.

>>> type('A', (), od).__dict__
mappingproxy({'a': 1, 'b': 2, '__module__': '__main__', '__dict__': <attribute '__dict__' of 'A' objects>, '__weakref__': <attribute '__weakref__' of 'A' objects>, '__doc__': None})

----------
components: Interpreter Core
messages: 322951
nosy: Rosuav, eric.snow, inada.naoki, rhettinger, serhiy.storchaka, steve.dower
priority: normal
severity: normal
status: open
title: Creating dict from OrderedDict doesn't preserve order
type: behavior
versions: Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34320>
_______________________________________

From report at bugs.python.org  Thu Aug  2 09:22:03 2018
From: report at bugs.python.org (Manuel)
Date: Thu, 02 Aug 2018 13:22:03 +0000
Subject: [New-bugs-announce] [issue34321] mmap.mmap() should not necessarily
 clone the file descriptor
Message-ID: <1533216123.68.0.56676864532.issue34321@psf.upfronthosting.co.za>


New submission from Manuel <manuel.schoelling at gmx.de>:

mmap.mmap(fileno, length, flags, prot, access, offset) always clones the file descriptor that should be used [1].

The cloning of the file descriptor seems to be done to ensure that the file cannot be closed behind mmap's back, but if you are mmap()'ing a lot of memory regions of a file this can cause a 'Too many open files' error.

I would suggest to add an option to mmap.mmap() that tells it not to clone the file descriptor. This can cause an issue if the file is closed before accessing the mmapped region, so this fact should also be pointed out in the documentation.

[1] https://github.com/python/cpython/blob/master/Modules/mmapmodule.c#L1159

----------
components: Library (Lib)
messages: 322953
nosy: manuels
priority: normal
severity: normal
status: open
title: mmap.mmap() should not necessarily clone the file descriptor
type: enhancement
versions: Python 2.7, Python 3.4, Python 3.5, Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34321>
_______________________________________

From report at bugs.python.org  Thu Aug  2 10:31:18 2018
From: report at bugs.python.org (Michael Schnaitter)
Date: Thu, 02 Aug 2018 14:31:18 +0000
Subject: [New-bugs-announce] [issue34322] modification to
 Lib/distutils/ccompiler.py to simplify handling of compile arguments by
 subclasses
Message-ID: <1533220278.22.0.56676864532.issue34322@psf.upfronthosting.co.za>


New submission from Michael Schnaitter <schnaitterm at knights.ucf.edu>:

Discussion on details in the referenced PR is needed.

----------
components: Library (Lib)
messages: 322959
nosy: schnaitterm
priority: normal
pull_requests: 8132
severity: normal
status: open
title: modification to Lib/distutils/ccompiler.py to simplify handling of compile arguments by subclasses
type: enhancement

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34322>
_______________________________________

From report at bugs.python.org  Thu Aug  2 18:50:13 2018
From: report at bugs.python.org (John Nelson)
Date: Thu, 02 Aug 2018 22:50:13 +0000
Subject: [New-bugs-announce] [issue34323] False timeout log message on
 proactor close
Message-ID: <1533250213.71.0.56676864532.issue34323@psf.upfronthosting.co.za>


New submission from John Nelson <john at jnelson.ca>:

Repro:
 1. Run the attached script ('repro.py')

Expected output:

2018-07-31 16:39:57,069 - asyncio - DEBUG - Using proactor: IocpProactor
2018-07-31 16:39:57,084 - root - INFO - Starting
2018-07-31 16:39:58,100 - root - INFO - Sleep done
2018-07-31 16:39:58,100 - root - INFO - Loop closed

Actual output:

2018-07-31 16:37:58,583 - asyncio - DEBUG - Using proactor: IocpProactor
2018-07-31 16:37:58,599 - root - INFO - Starting
2018-07-31 16:37:59,614 - root - INFO - Sleep done
2018-07-31 16:37:59,614 - asyncio - DEBUG - taking long time to close proactor
2018-07-31 16:37:59,614 - root - INFO - Loop closed

----------
components: asyncio
files: repro.py
messages: 322994
nosy: asvetlov, johnboy2, yselivanov
priority: normal
severity: normal
status: open
title: False timeout log message on proactor close
type: behavior
versions: Python 3.5
Added file: https://bugs.python.org/file47730/repro.py

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34323>
_______________________________________

From report at bugs.python.org  Thu Aug  2 19:19:01 2018
From: report at bugs.python.org (Lysandros Nikolaou)
Date: Thu, 02 Aug 2018 23:19:01 +0000
Subject: [New-bugs-announce] [issue34324] Doc README wrong directory name
 for vent
Message-ID: <1533251941.31.0.56676864532.issue34324@psf.upfronthosting.co.za>


New submission from Lysandros Nikolaou <lisandrosnik at gmail.com>:

In the Doc README there is the following sentence after the make venv command:
Assuming the virtual environment was created in the env directory (the default; configurable with the VENVDIR variable)

Should't it be venv directory instead? In the makefile the VENVDIR variable is set to venv by default.

----------
assignee: docs at python
components: Documentation
messages: 322997
nosy: docs at python, lys.nikolaou
priority: normal
severity: normal
status: open
title: Doc README wrong directory name for vent
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34324>
_______________________________________

From report at bugs.python.org  Thu Aug  2 19:52:26 2018
From: report at bugs.python.org (Pablo Galindo Salgado)
Date: Thu, 02 Aug 2018 23:52:26 +0000
Subject: [New-bugs-announce] [issue34325] test_zipfile gets OverflowError in
 multiple buildbots
Message-ID: <1533253946.81.0.56676864532.issue34325@psf.upfronthosting.co.za>


New submission from Pablo Galindo Salgado <pablogsal at gmail.com>:

test_zipfile gets OverflowError in multiple buildbots:

https://buildbot.python.org/all/#/builders/106/builds/1374
https://buildbot.python.org/all/#/builders/106/builds/1375
https://buildbot.python.org/all/#/builders/106/builds/1376

Example error log:


======================================================================
ERROR: test_add_file_after_2107 (test.test_zipfile.StoredTestsWithSourceFile)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/ssd/buildbot/buildarea/3.x.gps-ubuntu-exynos5-armv7l/build/Lib/test/test_zipfile.py", line 559, in test_add_file_after_2107
    os.utime(TESTFN, (4386268800, 4386268800))
OverflowError: timestamp out of range for platform time_t
----------------------------------------------------------------------
Ran 208 tests in 39.261s
FAILED (errors=1, skipped=1)
1 test failed again:

----------
components: Tests
messages: 322999
nosy: pablogsal
priority: normal
severity: normal
status: open
title: test_zipfile gets OverflowError in multiple buildbots
type: behavior
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34325>
_______________________________________

From report at bugs.python.org  Thu Aug  2 19:57:08 2018
From: report at bugs.python.org (Pablo Galindo Salgado)
Date: Thu, 02 Aug 2018 23:57:08 +0000
Subject: [New-bugs-announce] [issue34326]
 test_subprocess.POSIXProcessTestCase fails in AMD64 Ubuntu 3.x
Message-ID: <1533254228.03.0.56676864532.issue34326@psf.upfronthosting.co.za>


New submission from Pablo Galindo Salgado <pablogsal at gmail.com>:

test_subprocess.POSIXProcessTestCase fails in AMD64 Ubuntu 3.x buildbots:

https://buildbot.python.org/all/#/builders/154/builds/104

It seems that it has some problems with (unclosed?) file descriptors:

test_terminate_dead (test.test_subprocess.Win32ProcessTestCase) ... skipped 'Windows specific tests'
======================================================================
FAIL: test_close_fds (test.test_subprocess.POSIXProcessTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.x.einat-ubuntu/build/Lib/test/test_subprocess.py", line 2378, in test_close_fds
    "Some fds were left open")
AssertionError: {3} is not false : Some fds were left open
======================================================================
FAIL: test_close_fds_after_preexec (test.test_subprocess.POSIXProcessTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.x.einat-ubuntu/build/Lib/test/test_subprocess.py", line 2653, in test_close_fds_after_preexec
    self.assertNotIn(fd, remaining_fds)
AssertionError: 3 unexpectedly found in {0, 1, 2, 3}
======================================================================
FAIL: test_pass_fds (test.test_subprocess.POSIXProcessTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.x.einat-ubuntu/build/Lib/test/support/__init__.py", line 615, in wrapper
    return func(*args, **kw)
  File "/home/buildbot/buildarea/3.x.einat-ubuntu/build/Lib/test/test_subprocess.py", line 2503, in test_pass_fds
    "fd to be closed passed")
AssertionError: {4} is not false : fd to be closed passed
----------------------------------------------------------------------
Ran 285 tests in 68.129s
FAILED (failures=3, skipped=28)
1 test failed again:
    test_subprocess

----------
components: Tests
messages: 323001
nosy: pablogsal
priority: normal
severity: normal
status: open
title: test_subprocess.POSIXProcessTestCase fails in AMD64 Ubuntu 3.x
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34326>
_______________________________________

From report at bugs.python.org  Thu Aug  2 20:55:31 2018
From: report at bugs.python.org (Asher Mancinelli)
Date: Fri, 03 Aug 2018 00:55:31 +0000
Subject: [New-bugs-announce] [issue34327] CPython make test crash
Message-ID: <1533257731.89.0.56676864532.issue34327@psf.upfronthosting.co.za>


New submission from Asher Mancinelli <ashermancinelli at gmail.com>:

Cloned master (v3.8) from github. os: Arch x86_64, kernel: 4.14.56-1-lts. successfully ran `./configure --enable-optimizations` and `make`, but failed with this error message when running `make test` 

```
======================================================================
FAIL: test_close_fds (test.test_subprocess.POSIXProcessTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/asher/sources/cpython3.8/Lib/test/test_subprocess.py", line 2378, in test_close_fds
    "Some fds were left open")
AssertionError: {3} is not false : Some fds were left open

======================================================================
FAIL: test_close_fds_after_preexec (test.test_subprocess.POSIXProcessTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/asher/sources/cpython3.8/Lib/test/test_subprocess.py", line 2653, in test_close_fds_after_preexec
    self.assertNotIn(fd, remaining_fds)
AssertionError: 3 unexpectedly found in {0, 1, 2, 3}

======================================================================
FAIL: test_pass_fds (test.test_subprocess.POSIXProcessTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/asher/sources/cpython3.8/Lib/test/support/__init__.py", line 615, in wrapper
    return func(*args, **kw)
  File "/home/asher/sources/cpython3.8/Lib/test/test_subprocess.py", line 2503, in test_pass_fds
    "fd to be closed passed")
AssertionError: {4} is not false : fd to be closed passed

----------------------------------------------------------------------
Ran 285 tests in 23.861s

FAILED (failures=3, skipped=28)
test test_subprocess failed
1 test failed again:
    test_subprocess

== Tests result: FAILURE then FAILURE ==

404 tests OK.

1 test failed:
    test_subprocess

13 tests skipped:
    test_devpoll test_gdb test_kqueue test_msilib test_ossaudiodev
    test_startfile test_tix test_tk test_ttk_guionly test_winconsoleio
    test_winreg test_winsound test_zipfile64

1 re-run test:
    test_subprocess

Total duration: 3 min 59 sec
Tests result: FAILURE then FAILURE
make: *** [Makefile:1058: test] Error 2
```

----------
components: Build
messages: 323007
nosy: ashermancinelli
priority: normal
severity: normal
status: open
title: CPython make test crash
type: crash
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34327>
_______________________________________

From report at bugs.python.org  Fri Aug  3 01:21:37 2018
From: report at bugs.python.org (sakamotoaya)
Date: Fri, 03 Aug 2018 05:21:37 +0000
Subject: [New-bugs-announce] [issue34328] Question Incorrect URL for the
 Visual C++ Build Tools
Message-ID: <1533273697.75.0.56676864532.issue34328@psf.upfronthosting.co.za>


New submission from sakamotoaya <aya.sakamoto at outlook.com>:

I try "python setup.py". 
The following issue has occurred.

Excerpt from Error
?????????????????????
reading manifest template 'MANIFEST.in'
writing manifest file 'scikit_learn.egg-info\SOURCES.txt'
running build_ext
No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from
distutils
customize MSVCCompiler
Missing compiler_cxx fix for MSVCCompiler
customize MSVCCompiler using build_clib
building 'libsvm-skl' library
compiling C sources
error: Microsoft Visual C++ 14.0 is required. Get it with "Microsoft Visual C++
Build Tools": http://landinghub.visualstudio.com/visual-cpp-build-tools
?????????????????????

Expired link:  
http://landinghub.visualstudio.com/visual-cpp-build-tools

I think The new link:
https://download.microsoft.com/download/5/F/7/5F7ACAEB-8363-451F-9425-68A90F98B238/visualcppbuildtools_full.exe


Please give me advice on how I should correct those error message.

----------
components: Windows
messages: 323024
nosy: HiyashiChuka, docs at python, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: Question Incorrect URL for the Visual C++ Build Tools
type: behavior
versions: Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34328>
_______________________________________

From report at bugs.python.org  Fri Aug  3 05:58:05 2018
From: report at bugs.python.org (Stefan Otte)
Date: Fri, 03 Aug 2018 09:58:05 +0000
Subject: [New-bugs-announce] [issue34329] Document how to remove a suffix
 with pathlib.Path.with_suffix
Message-ID: <1533290285.58.0.56676864532.issue34329@psf.upfronthosting.co.za>


New submission from Stefan Otte <stefan.otte at gmail.com>:

Hello folks!

I didn't realize that you can remove a suffix with the `with_suffix` function of the `Path` class of `pathlib` and I always used a little utility function that I wrote.
I wanted to add that functionality (removing of a suffix) and submit a PR but then I saw that `with_suffix` has you covered already. I'm just documenting this feature with the PR I'll submit.

----------
assignee: docs at python
components: Documentation
messages: 323042
nosy: docs at python, sotte
priority: normal
severity: normal
status: open
title: Document how to remove a suffix with pathlib.Path.with_suffix
type: enhancement

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34329>
_______________________________________

From report at bugs.python.org  Fri Aug  3 10:40:05 2018
From: report at bugs.python.org (=?utf-8?b?0JLQu9Cw0LQg0Jo=?=)
Date: Fri, 03 Aug 2018 14:40:05 +0000
Subject: [New-bugs-announce] [issue34330] The decode_header function does
 not always work correctly.
Message-ID: <1533307205.91.0.56676864532.issue34330@psf.upfronthosting.co.za>


New submission from ???? ? <kisly.va at gmail.com>:

The decode_header function does not always work correctly.

me at Serv:~$ python3
Python 3.6.5 (default, Apr  1 2018, 05:46:30)
[GCC 7.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from email.header import decode_header
>>> s = '=?windows-1251?B?zeDp5Oj46ujtINDu7ODtIMLr4OTo7Ojw7uLo9w==?=\r\n\t<Roman at mail.ru>'
>>> decode_header(s)
[(b'\xcd\xe0\xe9\xe4\xe8\xf8\xea\xe8\xed \xd0\xee\xec\xe0\xed \xc2\xeb\xe0\xe4\xe8\xec\xe8\xf0\xee\xe2\xe8\xf7', 'windows                                                                                                             -1251'), (b'<Roman at mail.ru>', None)]
>>> exit()

me at Serv:~$ python
Python 2.7.15rc1 (default, Apr 15 2018, 21:51:34)
[GCC 7.3.0] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from email.header import decode_header
>>> s = '=?windows-1251?B?zeDp5Oj46ujtINDu7ODtIMLr4OTo7Ojw7uLo9w==?=\r\n\t<Roman at mail.ru>'
>>> decode_header(s)
[('=?windows-1251?B?zeDp5Oj46ujtINDu7ODtIMLr4OTo7Ojw7uLo9w==?=\r\n\t<Roman at mail.ru>', None)]
>>> exit()

----------
components: email
messages: 323056
nosy: barry, r.david.murray, ???? ?2
priority: normal
severity: normal
status: open
title: The decode_header function does not always work correctly.
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34330>
_______________________________________

From report at bugs.python.org  Fri Aug  3 11:56:31 2018
From: report at bugs.python.org (ppperry)
Date: Fri, 03 Aug 2018 15:56:31 +0000
Subject: [New-bugs-announce] [issue34331] Incorrectly pluralized abstract
 class error message
Message-ID: <1533311791.76.0.56676864532.issue34331@psf.upfronthosting.co.za>


New submission from ppperry <mapreader at olum.org>:

>>> class abstract(abc.ABC):
...     @abc.abstractmethod
...     def meth():
...         pass
...
>>> x()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: Can't instantiate abstract class x with abstract methods meth

Error should be "Can't instantiate abstract class abstract with abstract method meth" in the singular, because there is only one abstract method.

----------
components: Interpreter Core, Library (Lib)
messages: 323059
nosy: Anjali Bansal, ppperry, rhettinger, serhiy.storchaka, terry.reedy, xtreak
priority: normal
severity: normal
status: open
title: Incorrectly pluralized abstract class error message
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34331>
_______________________________________

From report at bugs.python.org  Fri Aug  3 15:28:12 2018
From: report at bugs.python.org (Happy Fakeboulder)
Date: Fri, 03 Aug 2018 19:28:12 +0000
Subject: [New-bugs-announce] [issue34332] Suggestion for a new loop type
Message-ID: <1533324492.86.0.56676864532.issue34332@psf.upfronthosting.co.za>


New submission from Happy Fakeboulder <weird1152 at gmail.com>:

A "while-except" loop, in which it acts like a do-while loop from other languages, except the condition is "did an exception occur during the execution of the iteration". One could specify a type of exception at the top, similar to an except block header. (sorry, I'm an idiot)
Example (the keyword might be different):

# asks for a number until the user gives one properly, then
# calculates the square of it.
whexc ValueError:
    num = int(input("Enter a number: "))
print("The square of your number is " + str(num * num)) # assuming whexc works, this will not cause an error

----------
messages: 323065
nosy: Happy Fakeboulder
priority: normal
severity: normal
status: open
title: Suggestion for a new loop type
type: enhancement
versions: Python 3.4, Python 3.5, Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34332>
_______________________________________

From report at bugs.python.org  Fri Aug  3 17:03:07 2018
From: report at bugs.python.org (Berker Peksag)
Date: Fri, 03 Aug 2018 21:03:07 +0000
Subject: [New-bugs-announce] [issue34333] Path.with_suffix() raises
 TypeError when doing %-formatting
Message-ID: <1533330187.01.0.56676864532.issue34333@psf.upfronthosting.co.za>


New submission from Berker Peksag <berker.peksag at gmail.com>:

import pathlib

path = pathlib.Path('TR')
with_suffix = path.with_suffix(('/', '.md'))

The snippet I shared above will raise:

    TypeError: not all arguments converted during string formatting

While we are relying on duck typing, I think this can be considered as a bug in this case.

----------
assignee: berker.peksag
components: Library (Lib)
messages: 323078
nosy: berker.peksag, pitrou
priority: normal
severity: normal
stage: needs patch
status: open
title: Path.with_suffix() raises TypeError when doing %-formatting
type: behavior
versions: Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34333>
_______________________________________

From report at bugs.python.org  Sat Aug  4 06:08:13 2018
From: report at bugs.python.org (Adrian Dries)
Date: Sat, 04 Aug 2018 10:08:13 +0000
Subject: [New-bugs-announce] [issue34334] QueueHandler logs exc_info twice
Message-ID: <1533377293.21.0.56676864532.issue34334@psf.upfronthosting.co.za>


New submission from Adrian Dries <adries at gmail.com>:

Since Python 3.7 logging.handlers.QueueHandler logs tracebacks twice::


>>> import logging
>>> from logging.handlers import QueueHandler, QueueListener
>>> from queue import Queue
>>> q = Queue()
>>> logging.getLogger().addHandler(QueueHandler(q))
>>> listener = QueueListener(q, logging.StreamHandler())
>>> listener.start()
>>> try: 1/0
... except: logging.exception('Look out!')
... 
Look out!
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ZeroDivisionError: division by zero
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ZeroDivisionError: division by zero


Patching QueueHandler.prepare() to set exc_text to None seems to fix this.

----------
components: Library (Lib)
messages: 323100
nosy: avdd
priority: normal
severity: normal
status: open
title: QueueHandler logs exc_info twice
type: behavior
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34334>
_______________________________________

From report at bugs.python.org  Sat Aug  4 18:56:22 2018
From: report at bugs.python.org (Mikhail Terekhov)
Date: Sat, 04 Aug 2018 22:56:22 +0000
Subject: [New-bugs-announce] [issue34335] Fix examples in asyncio docs
 (suppliment to bpo-32258)
Message-ID: <1533423382.45.0.56676864532.issue34335@psf.upfronthosting.co.za>


New submission from Mikhail Terekhov <termim at gmail.com>:

Couple of examples in in asyncio documentation mix @asyncio.coroutine decorator and await.

----------
components: asyncio
messages: 323121
nosy: asvetlov, termim, yselivanov
priority: normal
severity: normal
status: open
title: Fix examples in asyncio docs (suppliment to bpo-32258)
versions: Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34335>
_______________________________________

From report at bugs.python.org  Sun Aug  5 02:51:25 2018
From: report at bugs.python.org (=?utf-8?q?Ville_Skytt=C3=A4?=)
Date: Sun, 05 Aug 2018 06:51:25 +0000
Subject: [New-bugs-announce] [issue34336] Don't promote possibility to omit
 Optional on optional/defaulted arguments
Message-ID: <1533451885.05.0.56676864532.issue34336@psf.upfronthosting.co.za>


New submission from Ville Skytt? <ville.skytta at iki.fi>:

The documentation of typing.Optional seems to promote practices for not specifying Optional that are recommended against in PEP 484:

https://www.python.org/dev/peps/pep-0484/#union-types -- end of chapter recommends against inferring Optional.

https://docs.python.org/3.8/library/typing.html#typing.Optional -- second paragraph promotes the possibility of leaving out Optional.

I'm not sure how to improve the typing.Optional doc wording, perhaps it would be better to just remove the second paragraph altogether.

----------
assignee: docs at python
components: Documentation
messages: 323132
nosy: docs at python, scop
priority: normal
severity: normal
status: open
title: Don't promote possibility to omit Optional on optional/defaulted arguments
type: enhancement
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34336>
_______________________________________

From report at bugs.python.org  Sun Aug  5 03:44:46 2018
From: report at bugs.python.org (Yiwei Guo)
Date: Sun, 05 Aug 2018 07:44:46 +0000
Subject: [New-bugs-announce] [issue34337] Fail to get a right answer for
 1.2%0.2
Message-ID: <1533455086.01.0.56676864532.issue34337@psf.upfronthosting.co.za>


Change by Yiwei Guo <yguo at eiu.edu>:


----------
components: Tests
nosy: qq100460045
priority: normal
severity: normal
status: open
title: Fail to get a right answer for 1.2%0.2
type: compile error
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34337>
_______________________________________

From report at bugs.python.org  Sun Aug  5 12:29:46 2018
From: report at bugs.python.org (Michael Hooreman)
Date: Sun, 05 Aug 2018 16:29:46 +0000
Subject: [New-bugs-announce] [issue34338] abstractmethod can run on classes
Message-ID: <1533486586.96.0.56676864532.issue34338@psf.upfronthosting.co.za>


New submission from Michael Hooreman <mhooreman at gmail.com>:

Hello,

When I decorate a class method with abc.abstractmethod, and I call it from the class (not the instance), the call is successful.

It looks like ID 5867 which was closed years ago.

See https://stackoverflow.com/questions/51669362/python-3-6-abc-abstracmethod-on-classmethod-no-check-on-class-level-call

Thanks!

----------
components: Library (Lib)
messages: 323157
nosy: Michael Hooreman
priority: normal
severity: normal
status: open
title: abstractmethod can run on classes
type: behavior
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34338>
_______________________________________

From report at bugs.python.org  Sun Aug  5 12:37:04 2018
From: report at bugs.python.org (danijar)
Date: Sun, 05 Aug 2018 16:37:04 +0000
Subject: [New-bugs-announce] [issue34339] Argument unpacking syntax for
 lambdas
Message-ID: <1533487024.74.0.56676864532.issue34339@psf.upfronthosting.co.za>


New submission from danijar <mail at danijar.com>:

It would be great to support argument unpacking for lambdas. Example:

  lambda x, (y, z): x + y + z

instead of

  lambda x, y_and_z: x + y_and_z[0] + y_and_z[1]

Similar unpacking syntax is available for normal functions:

  def foo(x, y_and_z):
    y, z = y_and_z
    return x + y + z

This would greatly increase readability in some cases.

----------
components: Interpreter Core
messages: 323158
nosy: benjamin.peterson, brett.cannon, danijar, yselivanov
priority: normal
severity: normal
status: open
title: Argument unpacking syntax for lambdas

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34339>
_______________________________________

From report at bugs.python.org  Sun Aug  5 12:48:48 2018
From: report at bugs.python.org (bpypy)
Date: Sun, 05 Aug 2018 16:48:48 +0000
Subject: [New-bugs-announce] [issue34340] mimetypes libmagic compatibility
Message-ID: <1533487728.36.0.56676864532.issue34340@psf.upfronthosting.co.za>


New submission from bpypy <python at bzx2.com>:

An obvious use case for mimetypes would be to check if file has correct mimetype, by getting the type with some of libmagic wrappers, and then comparing to guess_type result.

But it seems that they don't always match. One example:
applicattion/rar vs application/x-rar
https://github.com/file/file/blob/b9e60f088847f885b5c9fde61ff8fc9645843506/magic/Magdir/archive#L986

Kills half the usefullness of the module.

----------
components: Extension Modules
messages: 323160
nosy: bpypy
priority: normal
severity: normal
status: open
title: mimetypes libmagic compatibility
type: enhancement

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34340>
_______________________________________

From report at bugs.python.org  Mon Aug  6 03:57:50 2018
From: report at bugs.python.org (Serhiy Storchaka)
Date: Mon, 06 Aug 2018 07:57:50 +0000
Subject: [New-bugs-announce] [issue34341] Appending to ZIP archive blows up
 existing Central Directory entries
Message-ID: <1533542270.27.0.56676864532.issue34341@psf.upfronthosting.co.za>


New submission from Serhiy Storchaka <storchaka+cpython at gmail.com>:

When the ZIP archive is opened for appending, it reads the Central Directory at opening and writes existing entries back at closing. The Zip64 Extra Field is appended to existing Extra Fields if necessary. This leads to increasing the size of the Extra Fields data every time when append to the ZIP archive. Since the total size of Extra Fields is limited by 0xffff bytes, this can cause the failure.

The proposed PR removes the Zip64 Extra Field before adding a new Zip64 Extra Field.

----------
assignee: serhiy.storchaka
components: Library (Lib)
messages: 323181
nosy: alanmcintyre, serhiy.storchaka, twouters
priority: normal
severity: normal
status: open
title: Appending to ZIP archive blows up existing Central Directory entries
type: behavior
versions: Python 2.7, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34341>
_______________________________________

From report at bugs.python.org  Mon Aug  6 03:59:21 2018
From: report at bugs.python.org (Shahriyar Rzayev)
Date: Mon, 06 Aug 2018 07:59:21 +0000
Subject: [New-bugs-announce] [issue34342] Fix the broken links in
 CPythonVmInternals wiki page
Message-ID: <1533542361.92.0.56676864532.issue34342@psf.upfronthosting.co.za>


New submission from Shahriyar Rzayev <rzayev.sehriyar at gmail.com>:

Please see:
https://wiki.python.org/moin/CPythonVmInternals

There are some links under "Particularly useful pieces of documentation: " which are broken and should be updated respectively as follows:

[Execution Model] -> https://docs.python.org/3/reference/executionmodel.html

[GIL] -> https://docs.python.org/3/c-api/init.html#thread-state-and-the-global-interpreter-lock or this one https://wiki.python.org/moin/GlobalInterpreterLock

[Bytecodes] -> Which is now can be found in https://docs.python.org/3/library/dis.html

[Disassembler] -> same here https://docs.python.org/3/library/dis.html

----------
assignee: docs at python
components: Documentation
messages: 323182
nosy: docs at python, shako
priority: normal
severity: normal
status: open
title: Fix the broken links in CPythonVmInternals wiki page
type: enhancement

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34342>
_______________________________________

From report at bugs.python.org  Mon Aug  6 05:24:31 2018
From: report at bugs.python.org (Vlad Tudorache)
Date: Mon, 06 Aug 2018 09:24:31 +0000
Subject: [New-bugs-announce] [issue34343] Why is turtle still present in
 python embedded for Windows?
Message-ID: <1533547471.89.0.56676864532.issue34343@psf.upfronthosting.co.za>


New submission from Vlad Tudorache <tudorache.vlad at gmail.com>:

I've just started using the embedded Python 3.6 for Windows and I find turtle.py in the archive, knowing that tkinter isn't present. There's no large space loss because of it, but it's presence may be confusing.

----------
components: Library (Lib), Windows
messages: 323188
nosy: paul.moore, steve.dower, tim.golden, vtudorache, zach.ware
priority: normal
severity: normal
status: open
title: Why is turtle still present in python embedded for Windows?
type: resource usage
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34343>
_______________________________________

From report at bugs.python.org  Mon Aug  6 06:04:47 2018
From: report at bugs.python.org (Vlad Starostin)
Date: Mon, 06 Aug 2018 10:04:47 +0000
Subject: [New-bugs-announce] [issue34344] Fix the docstring for
 AbstractEventLoopPolicy.get_event_loop
Message-ID: <1533549887.39.0.56676864532.issue34344@psf.upfronthosting.co.za>


New submission from Vlad Starostin <drtyrsa at yandex.ru>:

The docstring says "This may be None or an instance of EventLoop". But docs explicitly state that get_event_loop "must never return None". The same docstring is also in the example in docs: https://docs.python.org/3.6/library/asyncio-eventloops.html#customizing-the-event-loop-policy

I propose changing it to:

def get_event_loop(self):
    """Get the event loop.

    Returns an instance of EventLoop or raises an exception.
    """

If the wording is ok, I'll make a PR.

----------
assignee: docs at python
components: Documentation, asyncio
messages: 323190
nosy: asvetlov, docs at python, drtyrsa, yselivanov
priority: normal
severity: normal
status: open
title: Fix the docstring for AbstractEventLoopPolicy.get_event_loop
versions: Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34344>
_______________________________________

From report at bugs.python.org  Mon Aug  6 08:08:19 2018
From: report at bugs.python.org (Serhiy Storchaka)
Date: Mon, 06 Aug 2018 12:08:19 +0000
Subject: [New-bugs-announce] [issue34345] Add tests for PEP 468 and PEP 520
Message-ID: <1533557299.17.0.56676864532.issue34345@psf.upfronthosting.co.za>


New submission from Serhiy Storchaka <storchaka+cpython at gmail.com>:

AFAIK there no purposed tests for PEP 468 and PEP 520. If making dict not preserving order, noting related to these PEPs is failed.

I think we need explicit tests for PEP 468 and PEP 520. At least in 3.6 the dict order is not guaranteed. And even in 3.7+ preserving the dict order doesn't guarantee implementing PEP 468 and PEP 520. The order can be broken in a meanwhile (for example by iterating a list of keys in the reversed order).

----------
components: Tests
messages: 323199
nosy: eric.snow, serhiy.storchaka
priority: normal
severity: normal
status: open
title: Add tests for PEP 468 and PEP 520
type: enhancement
versions: Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34345>
_______________________________________

From report at bugs.python.org  Mon Aug  6 09:57:49 2018
From: report at bugs.python.org (Chris Fuller)
Date: Mon, 06 Aug 2018 13:57:49 +0000
Subject: [New-bugs-announce] [issue34346] dir() hangs interpreter
Message-ID: <1533563869.83.0.56676864532.issue34346@psf.upfronthosting.co.za>


New submission from Chris Fuller <cfuller732 at thinkingplanet.net>:

It's a little obscure. Nothing to get alarmed about, probably. It involves recursion and __getattr__() during object creation.

Only present with 2.7 new-style classes. Old-style 2.7 did not exhibit this behavior nor did 3.x. Checked on Linux/Cygwin/Windows.

----------
files: splat.py
messages: 323208
nosy: sfaleron
priority: normal
severity: normal
status: open
title: dir() hangs interpreter
type: crash
versions: Python 2.7
Added file: https://bugs.python.org/file47731/splat.py

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34346>
_______________________________________

From report at bugs.python.org  Mon Aug  6 12:06:50 2018
From: report at bugs.python.org (Michael Felt)
Date: Mon, 06 Aug 2018 16:06:50 +0000
Subject: [New-bugs-announce] [issue34347] AIX: test_utf8_mode.test_cmd_line
 fails
Message-ID: <1533571610.9.0.56676864532.issue34347@psf.upfronthosting.co.za>


New submission from Michael Felt <michael at felt.demon.nl>:

The test fails because

byte_str.decode('ascii', 'surragateescape')

is not what ascii(byte_str) - returns when called from the commandline.

Assumption: since " check('utf8', [arg_utf8])" succeeds I assume the parsing of the command-line is correct.

DETAILS
>>> arg = 'h\xe9\u20ac'.encode('utf-8')
>>> arg
b'h\xc3\xa9\xe2\x82\xac'

>>> arg.decode('ascii', 'surrogateescape')
'h\udcc3\udca9\udce2\udc82\udcac'


I am having a difficult time getting the syntax correct for all the "escapes", so I added a print statement in the check routine:

test_cmd_line (test.test_utf8_mode.UTF8ModeTests) ...
code:import locale, sys; print("%s:%s" % (locale.getpreferredencoding(), ascii(sys.argv[1:]))) arg:b'h\xc3\xa9\xe2\x82\xac'
out:UTF-8:['h\xe9\u20ac']

code:import locale, sys; print("%s:%s" % (locale.getpreferredencoding(), ascii(sys.argv[1:]))) arg:b'h\xc3\xa9\xe2\x82\xac'
out:ISO8859-1:['h\xc3\xa9\xe2\x82\xac']

test code with my debug statement (to generate above):

    def test_cmd_line(self):
        arg = 'h\xe9\u20ac'.encode('utf-8')
        arg_utf8 = arg.decode('utf-8')
        arg_ascii = arg.decode('ascii', 'surrogateescape')
        code = 'import locale, sys; print("%s:%s" % (locale.getpreferredencoding(), ascii(sys.argv[1:])))'

        def check(utf8_opt, expected, **kw):
            out = self.get_output('-X', utf8_opt, '-c', code, arg, **kw)
            print("\ncode:%s arg:%s\nout:%s" % (code, arg, out))
            args = out.partition(':')[2].rstrip()
            self.assertEqual(args, ascii(expected), out)

        check('utf8', [arg_utf8])
        if sys.platform == 'darwin' or support.is_android:
            c_arg = arg_utf8
        else:
            c_arg = arg_ascii
        check('utf8=0', [c_arg], LC_ALL='C')

So the first check succeeds:

        check('utf8', [arg_utf8])

But the second does not:

FAIL: test_cmd_line (test.test_utf8_mode.UTF8ModeTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/data/prj/python/src/python3-3.7.0/Lib/test/test_utf8_mode.py", line 225, in test_cmd_line
    check('utf8=0', [c_arg], LC_ALL='C')
  File "/data/prj/python/src/python3-3.7.0/Lib/test/test_utf8_mode.py", line 218, in check
    self.assertEqual(args, ascii(expected), out)
AssertionError: "['h\\xc3\\xa9\\xe2\\x82\\xac']" != "['h\\udcc3\\udca9\\udce2\\udc82\\udcac']"
- ['h\xc3\xa9\xe2\x82\xac']
+ ['h\udcc3\udca9\udce2\udc82\udcac']
 : ISO8859-1:['h\xc3\xa9\xe2\x82\xac']

I tried saying the "expected" is arg, but arg is still a byte object, the cmd_line result is not (printed as such).

AssertionError: "['h\\xc3\\xa9\\xe2\\x82\\xac']" != "[b'h\\xc3\\xa9\\xe2\\x82\\xac']"
- ['h\xc3\xa9\xe2\x82\xac']
+ [b'h\xc3\xa9\xe2\x82\xac']
?  +
 : ISO8859-1:['h\xc3\xa9\xe2\x82\xac']

----------
components: Interpreter Core, Tests
messages: 323214
nosy: Michael.Felt
priority: normal
severity: normal
status: open
title: AIX: test_utf8_mode.test_cmd_line fails
type: behavior
versions: Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34347>
_______________________________________

From report at bugs.python.org  Mon Aug  6 14:38:14 2018
From: report at bugs.python.org (Abhishek Reddy)
Date: Mon, 06 Aug 2018 18:38:14 +0000
Subject: [New-bugs-announce] [issue34348] Python 3.7 - Issues Installing
 Scikit Learn
Message-ID: <1533580694.5.0.56676864532.issue34348@psf.upfronthosting.co.za>


New submission from Abhishek Reddy <abhishek.chamakura at bluestembrands.com>:

Hi

I am currently encountering below issues when trying to install any version of Scikit Learn (0.19.0 or 0.19.1 or 0.19.2)

Python Version - 3.7

/usr/local/bsb-python37 - Custom Location in which I configured and installed Python 3.7

I have installed all the prerequisite - OS packages - bias-devel , lapack-devel , atlas-devel. 

Earlier when I installed Python 2.7 I didn't run into any issues. Now when I re-try to install the scikit learn under Python 2.7 I am running into the same issue and the earlier successful installed version of scikit learn is corrupted.

Error
# /usr/local/bsb-python37/bin/python3.7 setup.py install --prefix=/usr/local/bsb-python37
Partial import of sklearn during the build process.
blas_opt_info:
blas_mkl_info:
customize UnixCCompiler
  libraries mkl_rt not found in ['/usr/local/bsb-python37/lib', '/usr/local/lib64', '/usr/local/lib', '/usr/lib64', '/usr/lib', '/usr/lib/']
  NOT AVAILABLE

blis_info:
customize UnixCCompiler
  libraries blis not found in ['/usr/local/bsb-python37/lib', '/usr/local/lib64', '/usr/local/lib', '/usr/lib64', '/usr/lib', '/usr/lib/']
  NOT AVAILABLE

  error: Command "g++ -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -I/usr/local/bsb-python37/lib/python3.7/site-packages/numpy-1.14.5-py3.7-linux-x86_64.egg/numpy/core/include -I/usr/local/bsb-python37/lib/python3.7/site-packages/numpy-1.14.5-py3.7-linux-x86_64.egg/numpy/core/include -I/usr/local/bsb-python37/include/python3.7m -c sklearn/cluster/_dbscan_inner.cpp -o build/temp.linux-x86_64-3.7/sklearn/cluster/_dbscan_inner.o -MMD -MF build/temp.linux-x86_64-3.7/sklearn/cluster/_dbscan_inner.o.d" failed with exit status 1

Any help is greatly appreciated. I tried to google around and did all that I could try. No luck.

Thanks,
Abhishek

----------
components: Installation
messages: 323215
nosy: abhishekreddyc
priority: normal
severity: normal
status: open
title: Python 3.7 - Issues Installing Scikit Learn
type: compile error
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34348>
_______________________________________

From report at bugs.python.org  Mon Aug  6 15:41:06 2018
From: report at bugs.python.org (Nic Watson)
Date: Mon, 06 Aug 2018 19:41:06 +0000
Subject: [New-bugs-announce] [issue34349] asyncio.wait should accept
 generator of tasks as first argument
Message-ID: <1533584466.72.0.56676864532.issue34349@psf.upfronthosting.co.za>


New submission from Nic Watson <pythonbugs at nicwatson.org>:

Currently, passing a generator of coroutines or futures as the first parameter of asyncio.wait raises a TypeError.  This is in conflict with the documentation calling the first parameter a "sequence".

Line in question. https://github.com/python/cpython/blob/3.7/Lib/asyncio/tasks.py#L347

Generators are indeed coroutines, so the check to validate that the first parameter is not a coroutine or a future is too specific.

I'd suggest replacing that line with a check that the passed-in parameter is iterable, i.e. hasattr(futures, __iter__).

----------
components: asyncio
messages: 323217
nosy: asvetlov, jnwatson, yselivanov
priority: normal
severity: normal
status: open
title: asyncio.wait should accept generator of tasks as first argument
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34349>
_______________________________________

From report at bugs.python.org  Mon Aug  6 16:41:27 2018
From: report at bugs.python.org (Sebastian)
Date: Mon, 06 Aug 2018 20:41:27 +0000
Subject: [New-bugs-announce] [issue34350] Non obvious logging handler
 behaviour
Message-ID: <1533588087.55.0.56676864532.issue34350@psf.upfronthosting.co.za>


New submission from Sebastian <seb.schaetz at gmail.com>:

In Python 3.6.3 I can do:

import logging
logger = logging.getLogger()
logger.setLevel(logging.INFO)
logger.info("this does not work")
logging.info("PARTY")
logger.info("this works")

And it outputs:

INFO:root:PARTY
INFO:root:this works


The line 
logging.info("PARTY") 
seems to add a handler which makes the last line work. This is very confusing behavior as it is not obvious that a call to "logging.info" mutates the state of the logging subsystem and affects subsequent logging calls.

----------
components: Library (Lib)
messages: 323224
nosy: oneofthose
priority: normal
severity: normal
status: open
title: Non obvious logging handler behaviour
versions: Python 3.6, Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34350>
_______________________________________

From report at bugs.python.org  Tue Aug  7 10:55:20 2018
From: report at bugs.python.org (Abhishek Reddy)
Date: Tue, 07 Aug 2018 14:55:20 +0000
Subject: [New-bugs-announce] [issue34351] Python 3.7 - Issues Installing
 Scikit Learn
Message-ID: <1533653720.64.0.56676864532.issue34351@psf.upfronthosting.co.za>


New submission from Abhishek Reddy <abhishek.chamakura at bluestembrands.com>:

Hi

I am currently encountering below issues when trying to install any version of Scikit Learn (0.19.0 or 0.19.1 or 0.19.2) on Linux - RHEL / Centos 7 OS. 

Few months back I could successfully install scikit-learn under python 2.7 without issues. When I re-run the installation of scikit-learn package under python 2.7 its failing with the same below error.

I have installed all the required dependencies of OS and Python packages for scikit-learn

Python Version - 3.7

/usr/local/bsb-python37 - Custom Location in which I configured and installed Python 3.7

I have installed all the prerequisite - OS packages - bias-devel , lapack-devel , atlas-devel. 

Earlier when I installed Python 2.7 I didn't run into any issues. Now when I re-try to install the scikit learn under Python 2.7 I am running into the same issue and the earlier successful installed version of scikit learn is corrupted.

Error
# /usr/local/bsb-python37/bin/python3.7 setup.py install --prefix=/usr/local/bsb-python37
Partial import of sklearn during the build process.
blas_opt_info:
blas_mkl_info:
customize UnixCCompiler
  libraries mkl_rt not found in ['/usr/local/bsb-python37/lib', '/usr/local/lib64', '/usr/local/lib', '/usr/lib64', '/usr/lib', '/usr/lib/']
  NOT AVAILABLE

blis_info:
customize UnixCCompiler
  libraries blis not found in ['/usr/local/bsb-python37/lib', '/usr/local/lib64', '/usr/local/lib', '/usr/lib64', '/usr/lib', '/usr/lib/']
  NOT AVAILABLE

  error: Command "g++ -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -I/usr/local/bsb-python37/lib/python3.7/site-packages/numpy-1.14.5-py3.7-linux-x86_64.egg/numpy/core/include -I/usr/local/bsb-python37/lib/python3.7/site-packages/numpy-1.14.5-py3.7-linux-x86_64.egg/numpy/core/include -I/usr/local/bsb-python37/include/python3.7m -c sklearn/cluster/_dbscan_inner.cpp -o build/temp.linux-x86_64-3.7/sklearn/cluster/_dbscan_inner.o -MMD -MF build/temp.linux-x86_64-3.7/sklearn/cluster/_dbscan_inner.o.d" failed with exit status 1

Any help is greatly appreciated. I tried to google around and did all that I could try. No luck.

Thanks,
Abhishek

----------
components: Library (Lib)
messages: 323243
nosy: abhishekreddyc
priority: normal
severity: normal
status: open
title: Python 3.7 - Issues Installing Scikit Learn
type: compile error
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34351>
_______________________________________

From report at bugs.python.org  Tue Aug  7 12:52:40 2018
From: report at bugs.python.org (Anthony Guevara)
Date: Tue, 07 Aug 2018 16:52:40 +0000
Subject: [New-bugs-announce] [issue34352] Using tailf with python3.4
Message-ID: <1533660760.63.0.56676864532.issue34352@psf.upfronthosting.co.za>


New submission from Anthony Guevara <amboxer21 at gmail.com>:

When using tailf with Python2.7 there are no issues. When using tailf with Python3.4 some print statements use the old version 2 style. Python3 also complains about an implicit conversion. I have included a patch.

----------
components: Distutils
files: tailf.patch
keywords: patch
messages: 323246
nosy: amboxer21, dstufft, eric.araujo
priority: normal
severity: normal
status: open
title: Using tailf with python3.4
type: crash
versions: Python 3.4
Added file: https://bugs.python.org/file47734/tailf.patch

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34352>
_______________________________________

From report at bugs.python.org  Tue Aug  7 13:23:14 2018
From: report at bugs.python.org (G)
Date: Tue, 07 Aug 2018 17:23:14 +0000
Subject: [New-bugs-announce] [issue34353] stat's python implementation of
 filemode (fallback) doesn't behave like the C implementation
Message-ID: <1533662594.31.0.56676864532.issue34353@psf.upfronthosting.co.za>


New submission from G <gpery at protonmail.com>:

stat.py (Lib/stat.py)'s implementation of filemode doesn't account for socket-type files, while _stat (Modules/_stat.c) does, using S_IFSOCK.

----------
components: Library (Lib)
messages: 323248
nosy: gpery
priority: normal
severity: normal
status: open
title: stat's python implementation of filemode (fallback) doesn't behave like the C implementation
type: behavior
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34353>
_______________________________________

From report at bugs.python.org  Wed Aug  8 01:21:01 2018
From: report at bugs.python.org (Vinicius Pacheco)
Date: Wed, 08 Aug 2018 05:21:01 +0000
Subject: [New-bugs-announce] [issue34354] Memory leak on _testCongestion
Message-ID: <1533705661.31.0.56676864532.issue34354@psf.upfronthosting.co.za>


New submission from Vinicius Pacheco <vfpweb at gmail.com>:

Following this document https://devguide.python.org/ to compile the python I've got a memory leak on the test _testCongestion. This test is on the file Lib/test/test_socket.py.

The line that show me the issue is:

while True:
    self.cli.sendto(self.data, 0, (HOST, self.port))

The process enters on the "while" and never finish. I'm using a Fedora 28 on this test.

----------
components: Interpreter Core
messages: 323262
nosy: Vinicius Pacheco
priority: normal
severity: normal
status: open
title: Memory leak on _testCongestion
type: behavior
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34354>
_______________________________________

From report at bugs.python.org  Wed Aug  8 02:47:22 2018
From: report at bugs.python.org (Yohan Boniface)
Date: Wed, 08 Aug 2018 06:47:22 +0000
Subject: [New-bugs-announce] [issue34355] SIGSEGV (Address boundary error)
Message-ID: <1533710842.16.0.56676864532.issue34355@psf.upfronthosting.co.za>


New submission from Yohan Boniface <yohanboniface at free.fr>:

Hi!

Just installed 3.7 (ArchLinux) and I've a SIGSEGV on one of my projects.
I've a hard time reducing to a minimal testcase, because it seems whatever random piece of code I remove the crash disappears at some point.

Here is the repository:

https://framagit.org/ybon/trefle

To reproduce, install the project in a 3.7 venv with `python setup.py develop` then run `python trefle/bin.py` (or even `python -c 'from trefle import routine'`).

Here is the output I have:

```
Initializing config
Done initializing config
fish: ?python trefle/bin.py? terminated by signal SIGSEGV (Address boundary error)
```

Here are some elements:

- if I run the code with PYTHONMALLOC=debug, I have no crash
- the project is using quite a lot of unicode (French content written in config files), even in some file names
- the project is using asyncio (but it does not seem directly related at first look)
- it is running without issue as is on python 3.6

Here is a gdb backtrace:

```
$ gdb python                                                                                                                                GNU gdb (GDB) 8.1
Copyright (C) 2018 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-pc-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from python...(no debugging symbols found)...done.
(gdb) run trefle/bin.py 
Starting program: /home/ybon/.virtualenvs/trefle/bin/python trefle/bin.py
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/usr/lib/libthread_db.so.1".
Initializing config
Done initializing config

Program received signal SIGSEGV, Segmentation fault.
0x00007ffff791a9ce in PyObject_Malloc () from /usr/lib/libpython3.7m.so.1.0
(gdb) backtrace 
#0  0x00007ffff791a9ce in PyObject_Malloc () from /usr/lib/libpython3.7m.so.1.0
#1  0x00007ffff79fec6e in ?? () from /usr/lib/libpython3.7m.so.1.0
#2  0x00007ffff7a05874 in PyParser_ASTFromStringObject () from /usr/lib/libpython3.7m.so.1.0
#3  0x00007ffff7a693f2 in Py_CompileStringObject () from /usr/lib/libpython3.7m.so.1.0
#4  0x00007ffff7a695c3 in ?? () from /usr/lib/libpython3.7m.so.1.0
#5  0x00007ffff795963f in _PyMethodDef_RawFastCallDict () from /usr/lib/libpython3.7m.so.1.0
#6  0x00007ffff79597d1 in _PyCFunction_FastCallDict () from /usr/lib/libpython3.7m.so.1.0
#7  0x00007ffff79f7e16 in _PyEval_EvalFrameDefault () from /usr/lib/libpython3.7m.so.1.0
#8  0x00007ffff7939069 in _PyEval_EvalCodeWithName () from /usr/lib/libpython3.7m.so.1.0
#9  0x00007ffff7980982 in _PyFunction_FastCallKeywords () from /usr/lib/libpython3.7m.so.1.0
#10 0x00007ffff79f3142 in _PyEval_EvalFrameDefault () from /usr/lib/libpython3.7m.so.1.0
#11 0x00007ffff7939069 in _PyEval_EvalCodeWithName () from /usr/lib/libpython3.7m.so.1.0
#12 0x00007ffff7980982 in _PyFunction_FastCallKeywords () from /usr/lib/libpython3.7m.so.1.0
#13 0x00007ffff79f2225 in _PyEval_EvalFrameDefault () from /usr/lib/libpython3.7m.so.1.0
#14 0x00007ffff79807db in _PyFunction_FastCallKeywords () from /usr/lib/libpython3.7m.so.1.0
#15 0x00007ffff79f2225 in _PyEval_EvalFrameDefault () from /usr/lib/libpython3.7m.so.1.0
#16 0x00007ffff79807db in _PyFunction_FastCallKeywords () from /usr/lib/libpython3.7m.so.1.0
#17 0x00007ffff79f2225 in _PyEval_EvalFrameDefault () from /usr/lib/libpython3.7m.so.1.0
#18 0x00007ffff79807db in _PyFunction_FastCallKeywords () from /usr/lib/libpython3.7m.so.1.0
#19 0x00007ffff79f23cd in _PyEval_EvalFrameDefault () from /usr/lib/libpython3.7m.so.1.0
#20 0x00007ffff79807db in _PyFunction_FastCallKeywords () from /usr/lib/libpython3.7m.so.1.0
#21 0x00007ffff79f23cd in _PyEval_EvalFrameDefault () from /usr/lib/libpython3.7m.so.1.0
#22 0x00007ffff793a08b in _PyFunction_FastCallDict () from /usr/lib/libpython3.7m.so.1.0
#23 0x00007ffff7949888 in ?? () from /usr/lib/libpython3.7m.so.1.0
#24 0x00007ffff79b71b9 in _PyObject_CallMethodIdObjArgs () from /usr/lib/libpython3.7m.so.1.0
#25 0x00007ffff792e285 in PyImport_ImportModuleLevelObject () from /usr/lib/libpython3.7m.so.1.0
#26 0x00007ffff79f4434 in _PyEval_EvalFrameDefault () from /usr/lib/libpython3.7m.so.1.0
#27 0x00007ffff7939069 in _PyEval_EvalCodeWithName () from /usr/lib/libpython3.7m.so.1.0
#28 0x00007ffff7939f34 in PyEval_EvalCodeEx () from /usr/lib/libpython3.7m.so.1.0
#29 0x00007ffff7939f5c in PyEval_EvalCode () from /usr/lib/libpython3.7m.so.1.0
#30 0x00007ffff7a05a64 in ?? () from /usr/lib/libpython3.7m.so.1.0
#31 0x00007ffff7959709 in _PyMethodDef_RawFastCallDict () from /usr/lib/libpython3.7m.so.1.0
#32 0x00007ffff79597d1 in _PyCFunction_FastCallDict () from /usr/lib/libpython3.7m.so.1.0
#33 0x00007ffff79f7e16 in _PyEval_EvalFrameDefault () from /usr/lib/libpython3.7m.so.1.0
#34 0x00007ffff7939069 in _PyEval_EvalCodeWithName () from /usr/lib/libpython3.7m.so.1.0
#35 0x00007ffff7980982 in _PyFunction_FastCallKeywords () from /usr/lib/libpython3.7m.so.1.0
#36 0x00007ffff79f6933 in _PyEval_EvalFrameDefault () from /usr/lib/libpython3.7m.so.1.0
#37 0x00007ffff79807db in _PyFunction_FastCallKeywords () from /usr/lib/libpython3.7m.so.1.0
#38 0x00007ffff79f2225 in _PyEval_EvalFrameDefault () from /usr/lib/libpython3.7m.so.1.0
#39 0x00007ffff79807db in _PyFunction_FastCallKeywords () from /usr/lib/libpython3.7m.so.1.0
#40 0x00007ffff79f23cd in _PyEval_EvalFrameDefault () from /usr/lib/libpython3.7m.so.1.0
#41 0x00007ffff79807db in _PyFunction_FastCallKeywords () from /usr/lib/libpython3.7m.so.1.0
#42 0x00007ffff79f23cd in _PyEval_EvalFrameDefault () from /usr/lib/libpython3.7m.so.1.0
#43 0x00007ffff793a08b in _PyFunction_FastCallDict () from /usr/lib/libpython3.7m.so.1.0
#44 0x00007ffff7949888 in ?? () from /usr/lib/libpython3.7m.so.1.0
#45 0x00007ffff79b71b9 in _PyObject_CallMethodIdObjArgs () from /usr/lib/libpython3.7m.so.1.0
#46 0x00007ffff792e285 in PyImport_ImportModuleLevelObject () from /usr/lib/libpython3.7m.so.1.0
#47 0x00007ffff79f4434 in _PyEval_EvalFrameDefault () from /usr/lib/libpython3.7m.so.1.0
#48 0x00007ffff7939069 in _PyEval_EvalCodeWithName () from /usr/lib/libpython3.7m.so.1.0
#49 0x00007ffff7939f34 in PyEval_EvalCodeEx () from /usr/lib/libpython3.7m.so.1.0
#50 0x00007ffff7939f5c in PyEval_EvalCode () from /usr/lib/libpython3.7m.so.1.0
#51 0x00007ffff7a68770 in ?? () from /usr/lib/libpython3.7m.so.1.0
#52 0x00007ffff7a6a54a in PyRun_FileExFlags () from /usr/lib/libpython3.7m.so.1.0
#53 0x00007ffff7a6bac5 in PyRun_SimpleFileExFlags () from /usr/lib/libpython3.7m.so.1.0
#54 0x00007ffff7a6da8f in ?? () from /usr/lib/libpython3.7m.so.1.0
#55 0x00007ffff7a6e420 in _Py_UnixMain () from /usr/lib/libpython3.7m.so.1.0
#56 0x00007ffff7dc9003 in __libc_start_main () from /usr/lib/libc.so.6
#57 0x000055555555477a in _start ()
```

Thanks for your help on tracking this! :)

Yohan

----------
components: asyncio
messages: 323263
nosy: asvetlov, ybon, yselivanov
priority: normal
severity: normal
status: open
title: SIGSEGV (Address boundary error)
type: crash
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34355>
_______________________________________

From report at bugs.python.org  Wed Aug  8 04:02:58 2018
From: report at bugs.python.org (Xavier Hardy)
Date: Wed, 08 Aug 2018 08:02:58 +0000
Subject: [New-bugs-announce] [issue34356] Add support for args and kwargs in
 logging.conf
Message-ID: <1533715378.01.0.56676864532.issue34356@psf.upfronthosting.co.za>


New submission from Xavier Hardy <hardyxavier at gmail.com>:

The behavior of formatters and handlers in logging.conf files is inconsistent.

With handlers, it is possible to add custom (keyword and non-keyword) arguments, but it is not possible for formatters (only class, format, datefmt, style).

https://github.com/python/cpython/blob/master/Lib/logging/config.py#L112
https://github.com/python/cpython/blob/master/Lib/logging/config.py#L141

I'll try to submit a pull request as soon as possible.

----------
messages: 323269
nosy: xavier.hardy
priority: normal
severity: normal
status: open
title: Add support for args and kwargs in logging.conf
type: enhancement
versions: Python 2.7, Python 3.4, Python 3.5, Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34356>
_______________________________________

From report at bugs.python.org  Wed Aug  8 08:17:07 2018
From: report at bugs.python.org (David)
Date: Wed, 08 Aug 2018 12:17:07 +0000
Subject: [New-bugs-announce] [issue34357] situation where urllib3 works,
 but urllib does not work
Message-ID: <1533730627.83.0.56676864532.issue34357@psf.upfronthosting.co.za>


New submission from David <deivid.rodriguez at riseup.net>:

Hello!

Newbie to python here. I run into an issue with one desktop library, Cinnamon. Specifically this one: https://github.com/linuxmint/Cinnamon/issues/5926#issuecomment-411232144. This library uses the urllib in the standard library to download some json. But for some reason, it does not work for me. If however, I use [https://github.com/urllib3/urllib3](urllib3), it just works. It sounds like something the standard library could do better, so I'm reporting it here in case it's helpful.

A minimal example would be:


```python
from urllib.request import urlopen
 
data = urlopen("https://cinnamon-spices.linuxmint.com/json/applets.json").read()
 
print(data)
```

which just hangs for me. If I pass a specific number of bytes (less than ~65000), it works, but only downloads parts of the file.

Using the equivalent code in urllib3 works just fine:

```python
import urllib3

http = urllib3.PoolManager()
response = http.request('GET', 'https://cinnamon-spices.linuxmint.com/json/applets.json')
print(response.data)

```

This is on

```
Python 3.7.0 (default, Aug  7 2018, 23:24:26) 
[GCC 5.5.0 20171010] on linux
```

Any help troubleshooting this would be appreciated!

----------
messages: 323275
nosy: deivid
priority: normal
severity: normal
status: open
title: situation where urllib3 works, but urllib does not work

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34357>
_______________________________________

From report at bugs.python.org  Wed Aug  8 08:26:39 2018
From: report at bugs.python.org (Sandu)
Date: Wed, 08 Aug 2018 12:26:39 +0000
Subject: [New-bugs-announce] [issue34358] round() combined with product
 outputs ugly result
Message-ID: <1533731199.22.0.56676864532.issue34358@psf.upfronthosting.co.za>


New submission from Sandu <sandu.ursu at gmail.com>:

`from scipy import stats`

`a = stats.norm.cdf(2.1882658846227234)`

`round(a,4)` gives:
`0.9857`

`round(a,4)*100` one would expect to output 98.57.
It returns:
`98.57000000000001`

For comparison, if I multiply with 10 or 1000 the output is as expected.

----------
messages: 323276
nosy: ursus
priority: normal
severity: normal
status: open
title: round() combined with product outputs ugly result
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34358>
_______________________________________

From report at bugs.python.org  Wed Aug  8 10:30:16 2018
From: report at bugs.python.org (Sascha Fuhrmann)
Date: Wed, 08 Aug 2018 14:30:16 +0000
Subject: [New-bugs-announce] [issue34359] Wrong virtual environment found
Message-ID: <1533738616.85.0.56676864532.issue34359@psf.upfronthosting.co.za>


New submission from Sascha Fuhrmann <saschaf300578 at googlemail.com>:

For my python development I have several directories on my Windows system:

common:
F:\python\scripts -> one-file-scripts based on python 2
F:\python\scripts3 -> one-file-scripts base on python 3

projects:
F:\python\projects\timetracking
...

Each directory has its own virtual environment (managed with pipenv):
F:\python\scripts\.venv
F:\python\scripts3\.venv
F:\python\projects\timetracking\.venv

Because I want to be able to call the scripts from everywhere, I added the directories to the path-environment variable.

[...]F:\Python\scripts3;F:\Python\projects\timetracking[...]

Let's have a look at the timetracking project. The main script (timetracking.py) has the following shebang-line:
#! /usr/bin/env .venv/scripts/python.exe

My current directory is the root of C

When I call 'timetracking.py' I get an error that some modules have not been installed.

With activating the debug mode (set PYLAUNCH_DEBUG=1) I get some more information:

***
C:\>timetracking.py
launcher build: 32bit
launcher executable: Console
File 'C:\Users\itsme\AppData\Local\py.ini' non-existent
File 'C:\Windows\py.ini' non-existent
Called with command line: "F:\Python\projects\timetracking\timetracking.py"
maybe_handle_shebang: read 256 bytes
maybe_handle_shebang: BOM not found, using UTF-8
parse_shebang: found command: F:\Python\scripts3\.venv\scripts\python.exe
run_child: about to run 'F:\Python\scripts3\.venv\scripts\python.exe "F:\Python\projects\timetracking\timetracking.py" '
Traceback (most recent call last):
  File "F:\Python\projects\timetracking\timetracking.py", line 18, in <module>
    import menu_definitions
  File "F:\Python\projects\timetracking\menu_definitions.py", line 15, in <module>
    import menu_edit_item_functions
  File "F:\Python\projects\timetracking\menu_edit_item_functions.py", line 15, in <module>
    import tools
  File "F:\Python\projects\timetracking\tools.py", line 19, in <module>
    from texttable import Texttable
ModuleNotFoundError: No module named 'texttable'
child process exit code: 1
***

As you can see, the pylauncher found a shebang - but not that one I expected ('scripts3\.venv' instead of 'timetracking\.venv') *confused*
It seems that the pylauncher uses the first .venv it found within the path variable...

Any ideas?

----------
components: Windows
messages: 323278
nosy: Sascha Fuhrmann, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: Wrong virtual environment found
type: behavior
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34359>
_______________________________________

From report at bugs.python.org  Wed Aug  8 12:02:35 2018
From: report at bugs.python.org (Florian Bruhin)
Date: Wed, 08 Aug 2018 16:02:35 +0000
Subject: [New-bugs-announce] [issue34360] urllib.parse doesn't fail with
 multiple unmatching square brackets
Message-ID: <1533744155.42.0.56676864532.issue34360@psf.upfronthosting.co.za>


New submission from Florian Bruhin <python.org at the-compiler.org>:

Since bpo-29651, the urllib.parse docs say:

> Unmatched square brackets in the netloc attribute will raise a ValueError.

However, when there are at least one [ and ], but they don't match, there's somewhat inconsistent behavior:

>>> urllib.parse.urlparse('http://[::1]]').hostname
'::1'
>>> urllib.parse.urlparse('http://[[::1]').hostname
'[::1'

----------
components: Library (Lib)
messages: 323292
nosy: Howie Benefiel, The Compiler, orsenthil
priority: normal
severity: normal
status: open
title: urllib.parse doesn't fail with multiple unmatching square brackets
type: behavior

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34360>
_______________________________________

From report at bugs.python.org  Wed Aug  8 14:16:09 2018
From: report at bugs.python.org (Sanyam Khurana)
Date: Wed, 08 Aug 2018 18:16:09 +0000
Subject: [New-bugs-announce] [issue34361] An error should be returned when
 there are spaces in between function name and parameters
Message-ID: <1533752169.63.0.56676864532.issue34361@psf.upfronthosting.co.za>


New submission from Sanyam Khurana <sanyam.khurana01 at gmail.com>:

I noticed this while reviewing the code.

The print function works like:

```
print       (5)
```

This works with user-defined function too.

Ideally, this is a function call and we should return an error stating that there shouldn't be any spaces between the function name and the parameters listed.

This essentially is not good for people who're new to programming and learning to code with Python as their first language.

----------
components: Library (Lib)
messages: 323294
nosy: CuriousLearner
priority: normal
severity: normal
stage: needs patch
status: open
title: An error should be returned when there are spaces in between function name and parameters
type: behavior
versions: Python 2.7, Python 3.4, Python 3.5, Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34361>
_______________________________________

From report at bugs.python.org  Wed Aug  8 15:07:33 2018
From: report at bugs.python.org (ppperry)
Date: Wed, 08 Aug 2018 19:07:33 +0000
Subject: [New-bugs-announce] [issue34362] User-created types with wrong
 __new__ can be instantiated
Message-ID: <1533755253.45.0.56676864532.issue34362@psf.upfronthosting.co.za>


New submission from ppperry <mapreader at olum.org>:

If you have a class that defines __new__ to the __new__ of another builtin type that it isn't a subclass of:
>>> class X:
...    __new__ = tuple.__new__
Instantiating this class should produce an error because `tuple.__new__` can't handle non-tuples, but instead it succeeds:
>>> X()
<__main__.X object at 0x00000000032C3F98>

(related: issue34284)

----------
components: Interpreter Core
messages: 323297
nosy: Vadim Pushtaev, ncoghlan, ppperry, serhiy.storchaka
priority: normal
severity: normal
status: open
title: User-created types with wrong __new__ can be instantiated
type: behavior
versions: Python 2.7, Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34362>
_______________________________________

From report at bugs.python.org  Wed Aug  8 15:41:25 2018
From: report at bugs.python.org (Alex DeLorenzo)
Date: Wed, 08 Aug 2018 19:41:25 +0000
Subject: [New-bugs-announce] [issue34363] dataclasses.asdict() mishandles
 dataclass instance attributes that are instances of subclassed
 typing.NamedTuple
Message-ID: <1533757285.89.0.56676864532.issue34363@psf.upfronthosting.co.za>


New submission from Alex DeLorenzo <alex.delorenzo at gmail.com>:

Example:

from typing import NamedTuple
from dataclasses import dataclass, asdict

class NamedTupleAttribute(NamedTuple):
    example: bool = True

@dataclass
class Data:
    attr1: bool
    attr2: NamedTupleAttribute

data = Data(True, NamedTupleAttribute(example=True))
namedtuple_attr = asdict(data)['attr2']
print(type(namedtuple_attr.example))
>>> generator

One would expect that the printed type would be of type bool.

----------
components: Interpreter Core
messages: 323298
nosy: alexdelorenzo
priority: normal
severity: normal
status: open
title: dataclasses.asdict() mishandles dataclass instance attributes that are instances of subclassed typing.NamedTuple
type: behavior
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34363>
_______________________________________

From report at bugs.python.org  Wed Aug  8 16:42:15 2018
From: report at bugs.python.org (bob gailer)
Date: Wed, 08 Aug 2018 20:42:15 +0000
Subject: [New-bugs-announce] [issue34364] problem with traceback for syntax
 error in f-string
Message-ID: <31c748a7-5e57-009a-08e5-c18171489bc0@gmail.com>


New submission from bob gailer <bgailer at gmail.com>:

Inconsistent tracebacks. Note that the traceback for bug.py does not 
reference the
module file and line number.

# bug.py
def f():
 ? f'''
 ? {d e}'''
a=b

import bug

Traceback (most recent call last):
 ? File "<fstring>", line 1
 ??? (d e)
 ?????? ^
SyntaxError: invalid syntax

----------------------------------------

# bug2.py
def f():
 ? f'''
 ? {de}'''
a=b

import bug2
bug2.f()

Traceback (most recent call last):
 ? File "C:\python\bug2.py", line 5, in <module>
 ??? a=b
NameError: name 'b' is not defined

----------
messages: 323301
nosy: bgailer
priority: normal
severity: normal
status: open
title: problem with traceback for syntax error in f-string

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34364>
_______________________________________

From report at bugs.python.org  Thu Aug  9 01:03:49 2018
From: report at bugs.python.org (Kevin Norris)
Date: Thu, 09 Aug 2018 05:03:49 +0000
Subject: [New-bugs-announce] [issue34365] datetime's documentation refers to
 "comparison [...] falling back to the default scheme of comparing object
 addresses"
Message-ID: <1533791029.87.0.56676864532.issue34365@psf.upfronthosting.co.za>


New submission from Kevin Norris <nykevin.norris at gmail.com>:

The 3.x datetime documentation contains the following footnote:

> In other words, date1 < date2 if and only if date1.toordinal() < date2.toordinal(). In order to stop comparison from falling back to the default scheme of comparing object addresses, date comparison normally raises TypeError if the other comparand isn?t also a date object. However, NotImplemented is returned instead if the other comparand has a timetuple() attribute. This hook gives other kinds of date objects a chance at implementing mixed-type comparison. If not, when a date object is compared to an object of a different type, TypeError is raised unless the comparison is == or !=. The latter cases return False or True, respectively.

But in 3.x, comparison no longer falls back to comparing object addresses.  Also, some of the comments on issue 8005 seem to suggest that this footnote is not actually true in 3.x (aside from the first sentence, of course).  But regardless, the footnote should not refer to a long dead interpreter behavior as if it were still around.

----------
assignee: docs at python
components: Documentation
messages: 323314
nosy: Kevin.Norris, docs at python
priority: normal
severity: normal
status: open
title: datetime's documentation refers to "comparison [...] falling back to the default scheme of comparing object addresses"
versions: Python 3.4, Python 3.5, Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34365>
_______________________________________

From report at bugs.python.org  Thu Aug  9 02:48:30 2018
From: report at bugs.python.org (=?utf-8?b?TWljaGHFgiBHw7Nybnk=?=)
Date: Thu, 09 Aug 2018 06:48:30 +0000
Subject: [New-bugs-announce] [issue34366] _uuid module fails to compile on
 FreeBSD when libuuid is installed
Message-ID: <1533797310.78.0.56676864532.issue34366@psf.upfronthosting.co.za>


New submission from Micha? G?rny <mgorny at gentoo.org>:

The _uuid extension fails to build on my Gentoo/FreeBSD system:

building '_uuid' extension
x86_64-gentoo-freebsd11.1-gcc -pthread -fPIC -Wno-unused-result -Wsign-compare -DNDEBUG -O2 -pipe -march=native -fwrapv -std=c99 -Wextra -Wno-unused-result -Wno-unused-parameter -Wno-missing-field-initializers -Werror=implicit-function-declaration -I./Include -I. -I/var/tmp/portage/dev-lang/python-3.7.0/work/Python-3.7.0/Include -I/var/tmp/portage/dev-lang/python-3.7.0/work/Python-3.7.0 -c /var/tmp/portage/dev-lang/python-3.7.0/work/Python-3.7.0/Modules/_uuidmodule.c -o build/temp.freebsd-11.1-RELEASE-amd64-3.7/var/tmp/portage/dev-lang/python-3.7.0/work/Python-3.7.0/Modules/_uuidmodule.o
In file included from /usr/include/uuid.h:34:0,
                 from /var/tmp/portage/dev-lang/python-3.7.0/work/Python-3.7.0/Modules/_uuidmodule.c:8:
/usr/include/sys/uuid.h:77:21: error: conflicting types for 'uuid_t'
 typedef struct uuid uuid_t;
                     ^~~~~~
In file included from /var/tmp/portage/dev-lang/python-3.7.0/work/Python-3.7.0/Modules/_uuidmodule.c:5:0:
/usr/include/uuid/uuid.h:44:23: note: previous declaration of 'uuid_t' was here
 typedef unsigned char uuid_t[16];
                       ^~~~~~
[...]


Apparently the cause is that it includes headers belonging to the system uuid library and libuuid (from util-linux) simultaneously.  The whole module seems to be written with the assumption that the two different implementations will not be present simultaneously.  However, some software supports libuuid only, so we're forced to have both.

Attaching the complete build log.  I will submit a PR soonish.

----------
components: Extension Modules
files: dev-lang:python-3.7.0:20180809-062928.log
messages: 323317
nosy: mgorny
priority: normal
severity: normal
status: open
title: _uuid module fails to compile on FreeBSD when libuuid is installed
type: compile error
versions: Python 3.7, Python 3.8
Added file: https://bugs.python.org/file47735/dev-lang:python-3.7.0:20180809-062928.log

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34366>
_______________________________________

From report at bugs.python.org  Thu Aug  9 10:17:49 2018
From: report at bugs.python.org (Alexander Tsvetkov)
Date: Thu, 09 Aug 2018 14:17:49 +0000
Subject: [New-bugs-announce] [issue34367] AsyncResult.get() only notifies
 one thread
Message-ID: <1533824269.46.0.56676864532.issue34367@psf.upfronthosting.co.za>


New submission from Alexander Tsvetkov <xflower at gmail.com>:

If more than one thread is waiting for the multiprocessing.pool.AsyncResult (aka ApplyResult) returned from apply_async, only one thread will be notified once the result arrives.

It happens because AsyncResult._set calls notify upon the result arrival, not notify_all.

threading.Event used in Python 3 uses notify_all.

Reproduction script is attached.
Expected outcome:
1
1
is printed to STDOUT.

Observed outcome:
The script hangs.

----------
components: Library (Lib)
files: async_result_demo.py
messages: 323323
nosy: AlexWithBeard
priority: normal
severity: normal
status: open
title: AsyncResult.get() only notifies one thread
versions: Python 2.7
Added file: https://bugs.python.org/file47736/async_result_demo.py

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34367>
_______________________________________

From report at bugs.python.org  Fri Aug 10 00:57:34 2018
From: report at bugs.python.org (H-ZeX)
Date: Fri, 10 Aug 2018 04:57:34 +0000
Subject: [New-bugs-announce] [issue34368] ftplib __init__ function can't
 handle 120 or 4xy reply when connect to the server
Message-ID: <1533877054.01.0.56676864532.issue34368@psf.upfronthosting.co.za>


Change by H-ZeX <hzx20112012 at gmail.com>:


----------
components: XML
nosy: H-ZeX
priority: normal
severity: normal
status: open
title: ftplib __init__ function can't handle 120 or 4xy reply when connect to the server
type: behavior
versions: Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34368>
_______________________________________

From report at bugs.python.org  Fri Aug 10 02:20:16 2018
From: report at bugs.python.org (Toshio Kuratomi)
Date: Fri, 10 Aug 2018 06:20:16 +0000
Subject: [New-bugs-announce] [issue34369] kqueue.control() documentation and
 implementation mismatch
Message-ID: <1533882016.89.0.56676864532.issue34369@psf.upfronthosting.co.za>


New submission from Toshio Kuratomi <a.badger at gmail.com>:

The current kqueue documentation specifies that timeout is a keyword argument but it can only be passed as a positional argument right now:

>>> import select
>>> ko = select.kqueue()
>>> ko.control([1], 0, timeout=10)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: control() takes no keyword arguments
>>> help(ko.control)
Help on built-in function control:

control(...) method of select.kqueue instance
    control(changelist, max_events[, timeout=None]) -> eventlist
    
    Calls the kernel kevent function.
    - changelist must be an iterable of kevent objects describing the changes
      to be made to the kernel's watch list or None.
    - max_events lets you specify the maximum number of events that the
      kernel will return.
    - timeout is the maximum time to wait in seconds, or else None,
      to wait forever. timeout accepts floats for smaller timeouts, too.

This may be related to https://bugs.python.org/issue3852 in which the max_events argument used to be documented as optional but the code made it mandatory.

----------
components: Library (Lib)
messages: 323357
nosy: a.badger
priority: normal
severity: normal
status: open
title: kqueue.control() documentation and implementation mismatch
type: behavior
versions: Python 2.7, Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34369>
_______________________________________

From report at bugs.python.org  Fri Aug 10 06:07:16 2018
From: report at bugs.python.org (Vlad Tudorache)
Date: Fri, 10 Aug 2018 10:07:16 +0000
Subject: [New-bugs-announce] [issue34370] Tkinter scroll issues on macOS
Message-ID: <1533895636.89.0.56676864532.issue34370@psf.upfronthosting.co.za>


New submission from Vlad Tudorache <tudorache.vlad at gmail.com>:

Run the python script below.

import tkinter

root = tkinter.Tk()

text = tkinter.Text(root)
vbar = tkinter.Scrollbar(root)

vbar.pack(side=tkinter.RIGHT, fill=tkinter.Y)
text.pack(side=tkinter.LEFT, fill=tkinter.BOTH, expand=1)

text.config(yscrollcommand=vbar.set)
vbar.config(command=text.yview)

lines = ['This is the line number %d.\n' % i for i in range(256)]
text.insert(tkinter.END, ''.join(lines))
 
def click_trace(event):
    text.insert('%d.%d' % (1, 0), 'Clicked at (%d,%d) on %s.\n' % (event.x, event.y, vbar.identify(event.x, event.y)))

vbar.bind('<Button-1>', click_trace)

root.mainloop()

When the slider is at the top of the scrollbar, clicking on its upper half doesn't allow dragging. The little script shows that the Scrollbar considers it as being on "through1" - in Tk the zone between the upper arrow and the "slider" - and not on "slider". Another consequence is that one can drag (up and down) the slider when clicking a little bit below the slider, on "through2" region. This issue doesn't manifest on Windows, where the scrollbar's arrows are shown.
I've seen this issue when trying to explore the reasons of another (34047, fixed) concerning the slider locked at the end of the scrollbar in IDLE.

----------
components: Tkinter
messages: 323363
nosy: vtudorache
priority: normal
severity: normal
status: open
title: Tkinter scroll issues on macOS
type: behavior
versions: Python 3.6, Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34370>
_______________________________________

From report at bugs.python.org  Fri Aug 10 07:43:14 2018
From: report at bugs.python.org (Sverrir Berg)
Date: Fri, 10 Aug 2018 11:43:14 +0000
Subject: [New-bugs-announce] [issue34371] File reading gets stuck if you
 read at eof on macos
Message-ID: <1533901394.11.0.56676864532.issue34371@psf.upfronthosting.co.za>


New submission from Sverrir Berg <sab at keilir.com>:

Reading a file that is at eof causes the file reading to halt indefinitely.
1) open() a file and read() everything from it.
2) call read() a second time - reading nothing (this causes the issue).
3) append to the file (using another file handle or manually).
4) try to read it again - read() always returns nothing.

Attached is a a test causing this - works correctly on Ubuntu 2.7.12/3.5.2, macos 3.7.0, macos PyPy 5.10.0 but fails on macos 2.7.10/2.7.15(brew)

I assume the expected behaviour is the same as on the other versions where you can continue to read from the file when it has been appended to?

----------
components: IO
files: readwrite.py
messages: 323365
nosy: sverrirab
priority: normal
severity: normal
status: open
title: File reading gets stuck if you read at eof on macos
type: behavior
versions: Python 2.7
Added file: https://bugs.python.org/file47739/readwrite.py

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34371>
_______________________________________

From report at bugs.python.org  Fri Aug 10 11:02:24 2018
From: report at bugs.python.org (Arusekk)
Date: Fri, 10 Aug 2018 15:02:24 +0000
Subject: [New-bugs-announce] [issue34372] Compiler could output more
 accurate line numbers
Message-ID: <1533913344.46.0.56676864532.issue34372@psf.upfronthosting.co.za>


New submission from Arusekk <arek_koz at o2.pl>:

If this is a duplicate, please excuse me.

In particular, the most noticeable inaccuracy happens when the postfix if-else expression is involved. Maybe there are more of them.
The problem is quite self-explaining. The module named 'dis' will be helpful to reproduce the issue.

>>> import dis
>>> code = """(
... [
...     call1(),
...     call2()
... ]
... + call3()
... * call4()
... )"""
>>> dis.dis(code)
  3           0 LOAD_NAME                0 (call1)
              3 CALL_FUNCTION            0 (0 positional, 0 keyword pair)

  4           6 LOAD_NAME                1 (call2)
              9 CALL_FUNCTION            0 (0 positional, 0 keyword pair)
             12 BUILD_LIST               2

  6          15 LOAD_NAME                2 (call3)
             18 CALL_FUNCTION            0 (0 positional, 0 keyword pair)

  7          21 LOAD_NAME                3 (call4)
             24 CALL_FUNCTION            0 (0 positional, 0 keyword pair)
             27 BINARY_MULTIPLY
             28 BINARY_ADD
             29 RETURN_VALUE
>>> dis.dis(code.replace("+", "if").replace("*", "else"))
  6           0 LOAD_NAME                0 (call3)
              3 CALL_FUNCTION            0 (0 positional, 0 keyword pair)
              6 POP_JUMP_IF_FALSE       25
              9 LOAD_NAME                1 (call1)
             12 CALL_FUNCTION            0 (0 positional, 0 keyword pair)
             15 LOAD_NAME                2 (call2)
             18 CALL_FUNCTION            0 (0 positional, 0 keyword pair)
             21 BUILD_LIST               2
             24 RETURN_VALUE

  7     >>   25 LOAD_NAME                3 (call4)
             28 CALL_FUNCTION            0 (0 positional, 0 keyword pair)
             31 RETURN_VALUE

I used this code to show the difference between if-else and some arithmetics.

AFAICT the feature is possible to implement, as lnotab can contain negative line differences.

I don't know whether it is just a bug or a fully intended feature, but it would be quite an enhancement to have better line number tracking, useful for debugging.

If this is implemented, it may be worth further backporting.

Possible reasons in the upstream Python/compile.c (using < instead of !=):

https://github.com/python/cpython/blob/077059e0f086cf8c8b7fb9d1f053e38ddc743f59/Python/compile.c#L4092

https://github.com/python/cpython/blob/077059e0f086cf8c8b7fb9d1f053e38ddc743f59/Python/compile.c#L4438

----------
components: Interpreter Core
messages: 323371
nosy: Arusekk
priority: normal
severity: normal
status: open
title: Compiler could output more accurate line numbers
type: behavior
versions: Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34372>
_______________________________________

From report at bugs.python.org  Fri Aug 10 12:15:51 2018
From: report at bugs.python.org (Michael Felt)
Date: Fri, 10 Aug 2018 16:15:51 +0000
Subject: [New-bugs-announce] [issue34373] test_time errors on AIX
Message-ID: <1533917751.88.0.56676864532.issue34373@psf.upfronthosting.co.za>


New submission from Michael Felt <aixtools at felt.demon.nl>:

32-bit:

======================================================================
ERROR: test_mktime (test.test_time.TimeTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/data/prj/python/src/python3-3.7.0/Lib/test/test_time.py", line 446, in test_mktime
    self.assertEqual(time.mktime(tt), t)
OverflowError: mktime argument out of range

======================================================================
FAIL: test_pthread_getcpuclockid (test.test_time.TimeTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/data/prj/python/src/python3-3.7.0/Lib/test/test_time.py", line 129, in test_pthread_getcpuclockid
    self.assertNotEqual(clk_id, time.CLOCK_THREAD_CPUTIME_ID)
AssertionError: 12 == 12

----------------------------------------------------------------------

64-bit:
======================================================================
ERROR: test_mktime (test.test_time.TimeTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/opt/lib/python3.7/test/test_time.py", line 446, in test_mktime
    self.assertEqual(time.mktime(tt), t)
OverflowError: mktime argument out of range

======================================================================
ERROR: test_pthread_getcpuclockid (test.test_time.TimeTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/opt/lib/python3.7/test/test_time.py", line 130, in test_pthread_getcpuclockid
    t1 = time.clock_gettime(clk_id)
OverflowError: signed integer is greater than maximum

The test_mktime failed because
time.mktime(time.localtime(-2)) -- actually any negative value in localtime - fails

The patch in the file Modules/timemodule.c fixes this.

The test_pthread_getcpuclockid is a bit more complex as the result is different
depending on 32 or 64-bit mode.

In 32-bit mode AIX always responds with the constant CLOCK_THREAD_CPUTIME_ID.
The patch in Lib/test/test_time.py makes the test aware of this.

In AIX clockid_t is actually long long. However, changing the type to parse to "L"
broke many other things. It seems to work as a long ("l") when the ABI is 64-bit.

See additional change in Modules/timemodule.c 
static PyObject *
time_clock_gettime

Finally,
added some (additional) range checking for 
_PyTime_localtime for AIX. Not having this complicated testing for above.

----------
components: Interpreter Core, Tests
messages: 323372
nosy: Michael.Felt
priority: normal
severity: normal
status: open
title: test_time errors on AIX
versions: Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34373>
_______________________________________

From report at bugs.python.org  Fri Aug 10 13:36:47 2018
From: report at bugs.python.org (theger)
Date: Fri, 10 Aug 2018 17:36:47 +0000
Subject: [New-bugs-announce] [issue34374] Shouldn't shutil.copyfile replace
 link in dst with follow_symlinks set?
Message-ID: <1533922607.61.0.56676864532.issue34374@psf.upfronthosting.co.za>


New submission from theger <heger.tomas at gmail.com>:

This might not be an issue at all and maybe I'm just missing something but this confused me a bit and hence the question in title.

Python's shutil.copyfile() has a follow_symlinks argument but it only affects the source, not the destination - i.e. when src is a symlink, it makes a difference whether the flag is set (and then it will copy the destination of that src symlink) or not (and then it will just create a new symlink in dst pointing to the src symlink's destination). I hope it's clear so far.

When dst is a link though, the flag doesn't change anything and it will always copy that file to the dst symlink's destination.

My question is... If dst is a symlink and follow_symlinks flag is not set, wouldn't it be more logical if the dst symlink was just replaced by a copy of src instead of copying src to dst's destination?

Thanks and sorry if this is just noise.

----------
components: Library (Lib)
messages: 323373
nosy: theger
priority: normal
severity: normal
status: open
title: Shouldn't shutil.copyfile replace link in dst with follow_symlinks set?
type: behavior
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34374>
_______________________________________

From report at bugs.python.org  Fri Aug 10 13:56:26 2018
From: report at bugs.python.org (Victor Porton)
Date: Fri, 10 Aug 2018 17:56:26 +0000
Subject: [New-bugs-announce] [issue34375] Subtests (unittest)
Message-ID: <1533923786.85.0.56676864532.issue34375@psf.upfronthosting.co.za>


New submission from Victor Porton <porton at narod.ru>:

The following is a fragment of a real code:

~~~
    def test_run_xinlude(self):
        # stub_stdin(self, Global.get_resource_bytes("tests/core/data/xml/xinclude.xml"))
        for next_script_mode in ['doc1', 'doc2']:
            for order in ['breadth', 'depth']:
                with capture_stdin_and_stdout():
                    command_line.main(['-r',
                                       order,
                                       'chain',
                                       Global.get_filename("tests/core/data/xml/xinclude.xml"),
                                       '-u',
                                       'http://portonvictor.org/ns/trans/precedence-include',
                                       '-s',
                                       next_script_mode])
                    self.assertEqual(sys.stdout.buffer.getvalue(), TestUtility.XInclude_output,
                                     "for next_script=%s, order=%s" % (next_script_mode, order))
~~~

I wrote it in one test method instead of four similar methods.

It has the deficiency that if the first test fails, the three remaining tests are skipped.

I propose to add `subtest` context manager to use it like:

~~~
            with subtest():
                with capture_stdin_and_stdout():
                    command_line.main(['-r',
                                       order,
                                       'chain',
                                       Global.get_filename("tests/core/data/xml/xinclude.xml"),
                                       '-u',
                                       'http://portonvictor.org/ns/trans/precedence-include',
                                       '-s',
                                       next_script_mode])
                    self.assertEqual(sys.stdout.buffer.getvalue(), TestUtility.XInclude_output,
                                     "for next_script=%s, order=%s" % (next_script_mode, order))
~~~

which would split our test into four independent tests.

----------
components: Library (Lib)
messages: 323375
nosy: porton
priority: normal
severity: normal
status: open
title: Subtests (unittest)
type: enhancement
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34375>
_______________________________________

From report at bugs.python.org  Fri Aug 10 14:19:58 2018
From: report at bugs.python.org (Raymond Hettinger)
Date: Fri, 10 Aug 2018 18:19:58 +0000
Subject: [New-bugs-announce] [issue34376] Improve accuracy of math.hypot()
 and math.dist()
Message-ID: <1533925198.56.0.56676864532.issue34376@psf.upfronthosting.co.za>


New submission from Raymond Hettinger <raymond.hettinger at gmail.com>:

Apply two accuracy improvements that work well together and that are both cheap (less than 5% difference in speed).  

1. Moving the max value to the end saves an iteration and can improve accuracy (especially in the case where len(args) <= 3 where the summation order is always optimal).

2. Use Kahan summation which works especially well because all the inputs are non-negative.

The patched version generally gets the error down to within 1 ULP (tested with 100,000 trials using 100 arguments to hypot() where the arguments are generated using random.expovariate(1.0) and arranged in the least favorable order, largest-to-smallest):

Patched:
 [(0.0, 67276),
  (1.0, 16700),
  (-0.5, 14702),
  (-1.0, 1322)]

Baseline:
[(1.0, 30364),
 (0.0, 25328),
 (-0.5, 17407),
 (-1.0, 10554),
 (2.0, 6274),
 (-1.5, 4890),
 (-2.0, 3027),
 (-2.5, 897),
 (3.0, 752),
 (-3.0, 380),
 (-3.5, 61),
 (4.0, 51),
 (-4.0, 13),
 (-4.5, 1),
 (5.0, 1)]

The impact on performance is minimal (well under 5%).

Patched:
$ py -c 'import sys; print(sys.version)'
3.8.0a0 (heads/hypot-kahan-late-swap-2:e1d89184f0, Aug 10 2018, 11:06:21)
[Clang 9.1.0 (clang-902.0.39.2)]
$ py -m timeit -r 7 -s 'from math import hypot' 'hypot(15.0, 14.0, 13.0, 12.0, 11.0, 10.0, 9.0, 8.0, 7.0, 6.0, 5.0, 4.0, 3.0, 2.0, 1.0)'
1000000 loops, best of 7: 230 nsec per loop
$ py -m timeit -r 7 -s 'from math import hypot' 'hypot(10.0, 9.0, 8.0, 7.0, 6.0, 5.0, 4.0, 3.0, 2.0, 1.0)'
2000000 loops, best of 7: 170 nsec per loop
$ py -m timeit -r 7 -s 'from math import hypot' 'hypot(5.0, 4.0, 3.0, 2.0, 1.0)'
2000000 loops, best of 7: 119 nsec per loop
$ py -m timeit -r 7 -s 'from math import hypot' 'hypot(3.0, 2.0, 1.0)'
5000000 loops, best of 7: 95.7 nsec per loop
$ py -m timeit -r 7 -s 'from math import hypot' 'hypot(2.0, 1.0)'
5000000 loops, best of 7: 81.5 nsec per loop
$ py -m timeit -r 7 -s 'from math import hypot' 'hypot(1.0)'
5000000 loops, best of 7: 67.4 nsec per loop

Baseline:
$ py -c 'import sys; print(sys.version)'
3.8.0a0 (heads/master:077059e0f0, Aug 10 2018, 11:02:47)
[Clang 9.1.0 (clang-902.0.39.2)]
$ py -m timeit -r 7 -s 'from math import hypot' 'hypot(15.0, 14.0, 13.0, 12.0, 11.0, 10.0, 9.0, 8.0, 7.0, 6.0, 5.0, 4.0, 3.0, 2.0, 1.0)'
1000000 loops, best of 7: 237 nsec per loop
$ py -m timeit -r 7 -s 'from math import hypot' 'hypot(10.0, 9.0, 8.0, 7.0, 6.0, 5.0, 4.0, 3.0, 2.0, 1.0)'
2000000 loops, best of 7: 173 nsec per loop
$ py -m timeit -r 7 -s 'from math import hypot' 'hypot(5.0, 4.0, 3.0, 2.0, 1.0)'
2000000 loops, best of 7: 120 nsec per loop
$ py -m timeit -r 7 -s 'from math import hypot' 'hypot(3.0, 2.0, 1.0)'
5000000 loops, best of 7: 94.8 nsec per loop
$ py -m timeit -r 7 -s 'from math import hypot' 'hypot(2.0, 1.0)'
5000000 loops, best of 7: 80.3 nsec per loop
$ py -m timeit -r 7 -s 'from math import hypot' 'hypot(1.0)'
5000000 loops, best of 7: 67.1 nsec per loop

----------
assignee: rhettinger
components: Library (Lib)
files: measure_hypot_alternatives.py
messages: 323378
nosy: mark.dickinson, rhettinger, serhiy.storchaka, tim.peters
priority: low
severity: normal
status: open
title: Improve accuracy of math.hypot() and math.dist()
type: enhancement
versions: Python 3.8
Added file: https://bugs.python.org/file47741/measure_hypot_alternatives.py

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34376>
_______________________________________

From report at bugs.python.org  Fri Aug 10 14:54:25 2018
From: report at bugs.python.org (Paul Price)
Date: Fri, 10 Aug 2018 18:54:25 +0000
Subject: [New-bugs-announce] [issue34377] Update valgrind suppressions
Message-ID: <1533927265.92.0.56676864532.issue34377@psf.upfronthosting.co.za>


New submission from Paul Price <paul.antony.price at gmail.com>:

I've found that the valgrind suppressions don't work for me with cpython 3.6.2. Renaming PyObject_Free/PyObject_Realloc to _PyObject_Free/_PyObject_Realloc in the suppressions file works.

I've got a patch that I'll put in via a GitHub PR.

----------
components: Demos and Tools
messages: 323385
nosy: Paul Price
priority: normal
severity: normal
status: open
title: Update valgrind suppressions
type: enhancement
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34377>
_______________________________________

From report at bugs.python.org  Fri Aug 10 17:28:50 2018
From: report at bugs.python.org (Karan)
Date: Fri, 10 Aug 2018 21:28:50 +0000
Subject: [New-bugs-announce] [issue34378] Documentation 3.5+
Message-ID: <1533936530.23.0.56676864532.issue34378@psf.upfronthosting.co.za>


New submission from Karan <karan.svnit at gmail.com>:

The documentation link for 3.5+ is having an issue with the left sidebar where the index is present. In 2.7 if I scroll the main doc below to mid of the document the sidebar still shows me the index. The same is not present in the 3.5+. 

I know this is a very minor thing but for a new learner like me who is constantly reading these docs, this very small thing irritates a lot. 

Thanks

----------
assignee: docs at python
components: Documentation
files: imgonline-com-ua-twotoone-eLRSfZftmcsz.jpg
messages: 323393
nosy: Kapple, docs at python
priority: normal
severity: normal
status: open
title: Documentation 3.5+
type: behavior
versions: Python 3.5, Python 3.6, Python 3.7, Python 3.8
Added file: https://bugs.python.org/file47745/imgonline-com-ua-twotoone-eLRSfZftmcsz.jpg

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34378>
_______________________________________

From report at bugs.python.org  Fri Aug 10 18:10:19 2018
From: report at bugs.python.org (Evan Allrich)
Date: Fri, 10 Aug 2018 22:10:19 +0000
Subject: [New-bugs-announce] [issue34379] Move note about repeated calls to
 json.dump using the same fp to the json.dump section
Message-ID: <1533939019.03.0.56676864532.issue34379@psf.upfronthosting.co.za>


New submission from Evan Allrich <psf at allri.ch>:

At present the note [0] appears under the documentation for `json.dumps` (which does not use the `fp` argument). It seems the note would be better placed with the documentation for the affected function.

[0] 
> Note
> 
> Unlike pickle and marshal, JSON is not a framed protocol, so trying to > serialize multiple objects with repeated calls to dump() using the same > fp will result in an invalid JSON file.

----------
assignee: docs at python
components: Documentation
messages: 323396
nosy: docs at python, eallrich
priority: normal
severity: normal
status: open
title: Move note about repeated calls to json.dump using the same fp to the json.dump section
type: enhancement
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34379>
_______________________________________

From report at bugs.python.org  Sat Aug 11 05:22:13 2018
From: report at bugs.python.org (Steven Silvester)
Date: Sat, 11 Aug 2018 09:22:13 +0000
Subject: [New-bugs-announce] [issue34380] Relax Restrictions on await?
Message-ID: <1533979333.92.0.56676864532.issue34380@psf.upfronthosting.co.za>


New submission from Steven Silvester <steven.silvester at gmail.com>:

When writing an `async` function, we often want to `await` a method call that may or may not be `async`.  For instance, it may be synchronous in the base class, but asynchronous in the subclass on the instance we have been given.  It would be nice for `await foo` to be a no-op if `foo` does not have an `__await__` method.  For instance, the following works in EMCAScript 2017:  `async function foo () { let bar = await 1; console.log(bar); }`.  As a workaround, we have been using `await force_async(foo.bar)`, where `force_async` is the following:

```python
async def force_async(obj):
    """Force an object to be asynchronous"""
    if hasattr(obj, '__await__'):
        return await obj
    async def inner():
        return obj
    return await inner()
```

This functionality is roughly equivalent to `gen.maybe_future` for coroutines, that is now deprecated in Tornado.  cf http://www.tornadoweb.org/en/stable/gen.html#tornado.gen.maybe_future

----------
components: asyncio
messages: 323410
nosy: Steven Silvester, asvetlov, yselivanov
priority: normal
severity: normal
status: open
title: Relax Restrictions on await?
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34380>
_______________________________________

From report at bugs.python.org  Sat Aug 11 05:43:13 2018
From: report at bugs.python.org (Michael Osipov)
Date: Sat, 11 Aug 2018 09:43:13 +0000
Subject: [New-bugs-announce] [issue34381] Make tests with outbound
 connection optional
Message-ID: <1533980593.8.0.56676864532.issue34381@psf.upfronthosting.co.za>


New submission from Michael Osipov <1983-01-06 at gmx.net>:

I am currently trying investigate tests failures on HP-UX to port some engineering applications away from Fortran to Python, but a bunch of tests require outbound connection which I do not have. I do have an HTTP proxy only.

Please add something to TESTOPTS, e.g., '-o' (offline) to skip test. I'd like to focus on the real test failures.

----------
components: Tests
messages: 323413
nosy: michael-o
priority: normal
severity: normal
status: open
title: Make tests with outbound connection optional
type: enhancement
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34381>
_______________________________________

From report at bugs.python.org  Sat Aug 11 08:31:06 2018
From: report at bugs.python.org (Michael Felt)
Date: Sat, 11 Aug 2018 12:31:06 +0000
Subject: [New-bugs-announce] [issue34382] test_os.test_mode fails when
 directory base directory has g+s set
Message-ID: <1533990666.21.0.56676864532.issue34382@psf.upfronthosting.co.za>


New submission from Michael Felt <aixtools at felt.demon.nl>:

test_mode assumes that the SGID bit is not set in the parent directory. If it is set the assertEqual() tests fail.

This PR checks the mode of 'base' to see if the SGID bit is set, or not, and compares results accordingly.

Back-porting requested.

----------
components: Tests
messages: 323414
nosy: Michael.Felt
priority: normal
severity: normal
status: open
title: test_os.test_mode fails when directory base directory has g+s set
type: behavior
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34382>
_______________________________________

From report at bugs.python.org  Sat Aug 11 15:28:45 2018
From: report at bugs.python.org (Andrei)
Date: Sat, 11 Aug 2018 19:28:45 +0000
Subject: [New-bugs-announce] [issue34383] asyncio passes SSL certificate
 error to loop.call_exception_handler()
Message-ID: <1534015725.83.0.56676864532.issue34383@psf.upfronthosting.co.za>


New submission from Andrei <danandrei at gmail.com>:

I'm using 3.7, but it's an issue that I've found in the old github repo for 3.6.

When using asyncio, SSL certificate errors don't get suppressed.

I've tried:
- with contextlib.suppress(Exception)
- except & pass

Original ticket:
https://github.com/python/asyncio/issues/404

----------
components: asyncio
messages: 323421
nosy: DanAndrei, asvetlov, yselivanov
priority: normal
severity: normal
status: open
title: asyncio passes SSL certificate error to loop.call_exception_handler()
type: behavior
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34383>
_______________________________________

From report at bugs.python.org  Sat Aug 11 23:41:11 2018
From: report at bugs.python.org (Girts Folkmanis)
Date: Sun, 12 Aug 2018 03:41:11 +0000
Subject: [New-bugs-announce] [issue34384] os.readlink does not accept
 pathlib.Path on Windows
Message-ID: <1534045271.73.0.56676864532.issue34384@psf.upfronthosting.co.za>


New submission from Girts Folkmanis <girts.folkmanis at gmail.com>:

Documentation for os.readlink says "Changed in version 3.6: Accepts a path-like object.". This works fine on macOS, but blows up on Windows:

Python 3.7.0 (v3.7.0:1bf9cc5093, Jun 27 2018, 04:06:47) [MSC v.1914 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> import pathlib
>>> os.readlink(pathlib.Path('foo'))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: readlink() argument 1 must be str, not WindowsPath

----------
components: Library (Lib)
messages: 323427
nosy: girtsf
priority: normal
severity: normal
status: open
title: os.readlink does not accept pathlib.Path on Windows
type: behavior
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34384>
_______________________________________

From report at bugs.python.org  Sun Aug 12 01:10:35 2018
From: report at bugs.python.org (LarBob Doomer)
Date: Sun, 12 Aug 2018 05:10:35 +0000
Subject: [New-bugs-announce] [issue34385] Failed to build these modules:
 _ctypes on Solaris 10
Message-ID: <1534050635.26.0.56676864532.issue34385@psf.upfronthosting.co.za>


New submission from LarBob Doomer <larbobdoomer at gmail.com>:

I am compiling with GNU binutils 2.24, GCC 5.5.0, GNU make 4.2.1. 

When attempting to build 3.6.x, the build is successful except for the fact that the _ctypes module fails to build. This of course causes the install to fail.

Failed to build these modules:
_ctypes

I do have libffi-dev installed through OpenCSW.

----------
components: ctypes
messages: 323429
nosy: LarBob Doomer
priority: normal
severity: normal
status: open
title: Failed to build these modules: _ctypes on Solaris 10
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34385>
_______________________________________

From report at bugs.python.org  Sun Aug 12 05:36:13 2018
From: report at bugs.python.org (Artem Golubin)
Date: Sun, 12 Aug 2018 09:36:13 +0000
Subject: [New-bugs-announce] [issue34386] Expose a dictionary of interned
 strings in sys module
Message-ID: <1534066573.72.0.56676864532.issue34386@psf.upfronthosting.co.za>


New submission from Artem Golubin <me at rushter.com>:

Python provides an ability to intern strings (sys.intern). It would be useful to expose a read-only dictionary of interned strings to the Python users so we can see what kind of strings are interned. 

It takes minimal changes since internally it's just a dictionary. Is this worth adding to the sys module?

----------
components: Interpreter Core
messages: 323437
nosy: rushter
priority: normal
severity: normal
status: open
title: Expose a dictionary of interned strings in sys module
type: enhancement
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34386>
_______________________________________

From report at bugs.python.org  Sun Aug 12 07:55:20 2018
From: report at bugs.python.org (Kamyar Inanloo)
Date: Sun, 12 Aug 2018 11:55:20 +0000
Subject: [New-bugs-announce] [issue34387] Deletion of attributes in
 dataclass is buggy!
Message-ID: <1534074920.67.0.56676864532.issue34387@psf.upfronthosting.co.za>


New submission from Kamyar Inanloo <kamyar1979 at gmail.com>:

When using del or delattr on an instance of dataclass, attribute does not get deleted truly! using vars or getattr still returns the deleted attribute just using delattr again raises error!

----------
components: Argument Clinic
messages: 323442
nosy: Kamyar Inanloo, larry
priority: normal
severity: normal
status: open
title: Deletion of attributes in dataclass is buggy!
type: behavior
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34387>
_______________________________________

From report at bugs.python.org  Sun Aug 12 11:58:55 2018
From: report at bugs.python.org (Pablo Galindo Salgado)
Date: Sun, 12 Aug 2018 15:58:55 +0000
Subject: [New-bugs-announce] [issue34388] collect_gdb fails for
 test.pythoninfo in several buildbots
Message-ID: <1534089535.5.0.56676864532.issue34388@psf.upfronthosting.co.za>


New submission from Pablo Galindo Salgado <pablogsal at gmail.com>:

collect_gdb fails for the test.pythoninfo step in several buildbots:

https://buildbot.python.org/all/#/builders/102/builds/55
https://buildbot.python.org/all/#/builders/79/builds/237
https://buildbot.python.org/all/#/builders/112/builds/161
https://buildbot.python.org/all/#/builders/118/builds/164
https://buildbot.python.org/all/#/builders/56/builds/105
https://buildbot.python.org/all/#/builders/18/builds/109

Example error:

./python -m test.pythoninfo
ERROR: collect_gdb() failed
Traceback (most recent call last):
  File "/usr/home/buildbot/python/2.7.koobs-freebsd-current.nondebug/build/Lib/test/pythoninfo.py", line 561, in collect_info
    collect_func(info_add)
  File "/usr/home/buildbot/python/2.7.koobs-freebsd-current.nondebug/build/Lib/test/pythoninfo.py", line 300, in collect_gdb
    version = version.splitlines()[0]
IndexError: list index out of range

----------
components: Build, Tests
messages: 323446
nosy: pablogsal
priority: normal
severity: normal
status: open
title: collect_gdb fails for test.pythoninfo in several buildbots
type: crash
versions: Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34388>
_______________________________________

From report at bugs.python.org  Sun Aug 12 15:32:17 2018
From: report at bugs.python.org (Antony Lee)
Date: Sun, 12 Aug 2018 19:32:17 +0000
Subject: [New-bugs-announce] [issue34389] CPython may fail to build in the
 presence of a ~/.pydistutils.cfg
Message-ID: <1534102337.9.0.56676864532.issue34389@psf.upfronthosting.co.za>


New submission from Antony Lee <anntzer.lee at gmail.com>:

I have a ~/.pydistutils.cfg with the following contents:

    [build_ext]
    force = true
    inplace = true

(--force is useful because sometimes just comparing timestamps is insufficient to know whether a package needs to be rebuilt, e.g. in the presence of external dependencies -- and I use ccache anyways to avoid paying for excessive rebuilds; --inplace is useful as I have quite a few packages with extension modules that are editably-installed).

With this ~/.pydistutils.cfg, cpython fails to build.  For example, having checked out v3.7.0 from a git clone (otherwise clean git repo, per `git clean -xfd`),

    mkdir build && cd build && ../configure && make

ultimately results in

    <elided>

    gcc -pthread -shared build/temp.linux-x86_64-3.7/home/antony/src/extern/cpython/Modules/_ctypes/_ctypes.o build/temp.linux-x86_64-3.7/home/antony/src/extern/cpython/Modules/_ctypes/callbacks.o build/temp.linux-x86_64-3.7/home/antony/src/extern/cpython/Modules/_ctypes/callproc.o build/temp.linux-x86_64-3.7/home/antony/src/extern/cpython/Modules/_ctypes/stgdict.o build/temp.linux-x86_64-3.7/home/antony/src/extern/cpython/Modules/_ctypes/cfield.o -L/usr/local/lib -lffi -ldl -o /home/antony/src/extern/cpython/build/_ctypes.cpython-37m-x86_64-linux-gnu.so
    *** WARNING: renaming "_struct" since importing it failed: build/lib.linux-x86_64-3.7/_struct.cpython-37m-x86_64-linux-gnu.so: cannot open shared object file: No such file or directory
    Traceback (most recent call last):
    File "../setup.py", line 442, in check_extension_import
        importlib._bootstrap._load(spec)
    File "<frozen importlib._bootstrap>", line 696, in _load
    File "<frozen importlib._bootstrap>", line 670, in _load_unlocked
    File "<frozen importlib._bootstrap>", line 583, in module_from_spec
    File "<frozen importlib._bootstrap_external>", line 1043, in create_module
    File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
    ImportError: build/lib.linux-x86_64-3.7/_struct.cpython-37m-x86_64-linux-gnu.so: cannot open shared object file: No such file or directory

    During handling of the above exception, another exception occurred:

    Traceback (most recent call last):
    File "../setup.py", line 2363, in <module>
        main()
    File "../setup.py", line 2358, in main
        "Tools/scripts/2to3", "Tools/scripts/pyvenv"]
    File "/home/antony/src/extern/cpython/Lib/distutils/core.py", line 148, in setup
        dist.run_commands()
    File "/home/antony/src/extern/cpython/Lib/distutils/dist.py", line 966, in run_commands
        self.run_command(cmd)
    File "/home/antony/src/extern/cpython/Lib/distutils/dist.py", line 985, in run_command
        cmd_obj.run()
    File "/home/antony/src/extern/cpython/Lib/distutils/command/build.py", line 135, in run
        self.run_command(cmd_name)
    File "/home/antony/src/extern/cpython/Lib/distutils/cmd.py", line 313, in run_command
        self.distribution.run_command(command)
    File "/home/antony/src/extern/cpython/Lib/distutils/dist.py", line 985, in run_command
        cmd_obj.run()
    File "/home/antony/src/extern/cpython/Lib/distutils/command/build_ext.py", line 339, in run
        self.build_extensions()
    File "../setup.py", line 308, in build_extensions
        self.check_extension_import(ext)
    File "../setup.py", line 447, in check_extension_import
        assert not self.inplace
    AssertionError
    make: *** [Makefile:618: sharedmods] Error 1

Removing the ~/.pydistutils.cfg fixes the issue and leads to a successful build.

I think(?) CPython's build system should essentially make sure that distutils behaves as if `--no-user-cfg` was in effect.  Or at least the limitation should be documented, but it's not very practical to have to temporarily rename an existing ~/.pydistutils.cfg whenever building CPython.  See also https://bugs.python.org/issue9309, perhaps.

----------
components: Build
messages: 323452
nosy: Antony.Lee
priority: normal
severity: normal
status: open
title: CPython may fail to build in the presence of a ~/.pydistutils.cfg
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34389>
_______________________________________

From report at bugs.python.org  Sun Aug 12 23:25:05 2018
From: report at bugs.python.org (Matthias Fripp)
Date: Mon, 13 Aug 2018 03:25:05 +0000
Subject: [New-bugs-announce] [issue34390] arparse.ArgumentParser misparses
 list arguments followed by undefined arguments
Message-ID: <1534130705.52.0.56676864532.issue34390@psf.upfronthosting.co.za>


New submission from Matthias Fripp <mfripp at gmail.com>:

The code below demonstrates this bug.

import argparse
parser = argparse.ArgumentParser()
parser.add_argument('--list-arg', nargs='+', default=[])
parser.parse_known_args(['--list-arg', 'a', '--text-arg=hello world'])

The result should be (Namespace(list_arg=['a']), ['--text-arg=hello world']), but is actually (Namespace(list_arg=['a', '--text-arg=hello world']), []). i.e., --list-arg consumes the next argument if that argument hasn't been defined and uses an equal sign and has a space in the assigned value.

Note that both of the following work correctly:

parser.parse_known_args(['--list-arg', 'a', '--text-arg', 'hello world'])
parser.parse_known_args(['--list-arg', 'a', '--text-arg=hello'])

Further, the next line should cause an error, but doesn't, due to the behavior noted above:

parser.parse_args(['--list-arg', 'a', '--text-arg=hello world'])

----------
components: Library (Lib)
messages: 323458
nosy: Matthias Fripp
priority: normal
severity: normal
status: open
title: arparse.ArgumentParser misparses list arguments followed by undefined arguments
type: behavior
versions: Python 2.7, Python 3.4, Python 3.5, Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34390>
_______________________________________

From report at bugs.python.org  Mon Aug 13 05:50:35 2018
From: report at bugs.python.org (Christian Heimes)
Date: Mon, 13 Aug 2018 09:50:35 +0000
Subject: [New-bugs-announce] [issue34391] test_ftplib is failing with TLS 1.3
Message-ID: <1534153835.38.0.56676864532.issue34391@psf.upfronthosting.co.za>


New submission from Christian Heimes <lists at cheimes.de>:

Related to #32947

Four ftplib tests are failing with OpenSSL 1.1.1-pre8 and TLS 1.3 enabled. All failing tests use a separate data connection and transfer data on the server. For store operations, the client never calls recv(). This breaks bidirectional shutdown. I assume there are session tickets stuck on the wire.

======================================================================
ERROR: test_storbinary (test.test_ftplib.TestTLS_FTPClassMixin)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/heimes/dev/python/cpython/Lib/test/test_ftplib.py", line 591, in test_storbinary
    self.client.storbinary('stor', f)
  File "/home/heimes/dev/python/cpython/Lib/ftplib.py", line 514, in storbinary
    conn.unwrap()
  File "/home/heimes/dev/python/cpython/Lib/ssl.py", line 1091, in unwrap
    s = self._sslobj.shutdown()
OSError: [Errno 0] Error

======================================================================
ERROR: test_storbinary_rest (test.test_ftplib.TestTLS_FTPClassMixin)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/heimes/dev/python/cpython/Lib/test/test_ftplib.py", line 603, in test_storbinary_rest
    self.client.storbinary('stor', f, rest=r)
  File "/home/heimes/dev/python/cpython/Lib/ftplib.py", line 514, in storbinary
    conn.unwrap()
  File "/home/heimes/dev/python/cpython/Lib/ssl.py", line 1091, in unwrap
    s = self._sslobj.shutdown()
OSError: [Errno 0] Error

======================================================================
ERROR: test_storlines (test.test_ftplib.TestTLS_FTPClassMixin)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/heimes/dev/python/cpython/Lib/test/test_ftplib.py", line 608, in test_storlines
    self.client.storlines('stor', f)
  File "/home/heimes/dev/python/cpython/Lib/ftplib.py", line 545, in storlines
    conn.unwrap()
  File "/home/heimes/dev/python/cpython/Lib/ssl.py", line 1091, in unwrap
    s = self._sslobj.shutdown()
OSError: [Errno 0] Error

======================================================================
ERROR: test_data_connection (test.test_ftplib.TestTLS_FTPClass)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/heimes/dev/python/cpython/Lib/test/test_ftplib.py", line 889, in test_data_connection
    self.assertEqual(self.client.voidresp(), "226 transfer complete")
  File "/home/heimes/dev/python/cpython/Lib/ftplib.py", line 251, in voidresp
    resp = self.getresp()
  File "/home/heimes/dev/python/cpython/Lib/ftplib.py", line 236, in getresp
    resp = self.getmultiline()
  File "/home/heimes/dev/python/cpython/Lib/ftplib.py", line 222, in getmultiline
    line = self.getline()
  File "/home/heimes/dev/python/cpython/Lib/ftplib.py", line 204, in getline
    line = self.file.readline(self.maxline + 1)
  File "/home/heimes/dev/python/cpython/Lib/socket.py", line 589, in readinto
    return self._sock.recv_into(b)
socket.timeout: timed out

----------------------------------------------------------------------
Ran 88 tests in 9.402s

FAILED (errors=4, skipped=1)

----------
messages: 323472
nosy: christian.heimes
priority: normal
severity: normal
status: open
title: test_ftplib is failing with TLS 1.3

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34391>
_______________________________________

From report at bugs.python.org  Mon Aug 13 06:24:33 2018
From: report at bugs.python.org (Serhiy Storchaka)
Date: Mon, 13 Aug 2018 10:24:33 +0000
Subject: [New-bugs-announce] [issue34392] Add sys.isinterned()
Message-ID: <1534155873.32.0.56676864532.issue34392@psf.upfronthosting.co.za>


New submission from Serhiy Storchaka <storchaka+cpython at gmail.com>:

I need to test whether the string is interned for Python implementations of marshal.dumps(). It is easy to do in C, and I suggest to expose this functionality to Python.

Currently you can test if the string is interned using the following function:

    def isinterned(s):
        return sys.intern(s) is s

But it has a side effect -- it interns a string if it is not equal to an already interned string.

----------
components: Interpreter Core
messages: 323473
nosy: rhettinger, ronaldoussoren, serhiy.storchaka
priority: normal
severity: normal
status: open
title: Add sys.isinterned()
type: enhancement
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34392>
_______________________________________

From report at bugs.python.org  Mon Aug 13 07:34:23 2018
From: report at bugs.python.org (liad)
Date: Mon, 13 Aug 2018 11:34:23 +0000
Subject: [New-bugs-announce] [issue34393] json.dumps - allow compression
Message-ID: <1534160063.78.0.56676864532.issue34393@psf.upfronthosting.co.za>


New submission from liad <liad at mailinator.com>:

The list of arguments of json.dump() can be seen here: https://docs.python.org/2/library/json.html

Notice that there is no way to make compression.

For example pandas allows you to do:
        df.to_csv(path_or_buf=file_name, index=False, encoding='utf-8',
                  compression='gzip',
                  quoting=QUOTE_NONNUMERIC)

I want to be able to compress when I do:
    with open('products.json', 'w') as outfile:
        json.dump(data, outfile, sort_keys=True)


Please add the ability to compress using json.dump()

----------
messages: 323475
nosy: liad100
priority: normal
severity: normal
status: open
title: json.dumps - allow compression
type: enhancement
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34393>
_______________________________________

From report at bugs.python.org  Mon Aug 13 08:36:26 2018
From: report at bugs.python.org (Semyon)
Date: Mon, 13 Aug 2018 12:36:26 +0000
Subject: [New-bugs-announce] [issue34394] Descriptors HowTo doesn't mention
 __set_name__
Message-ID: <1534163786.32.0.56676864532.issue34394@psf.upfronthosting.co.za>


New submission from Semyon <simeon+bpo at maryasin.name>:

There is a great HowTo document for descriptors https://github.com/python/cpython/blob/master/Doc/howto/descriptor.rst
But it doesn't even mention the __set_name__ method which was added in py3. And it lists the descriptor protocol without that method as if it is the full protocol. The only way to know about that method is to go to the link for any other method and then you'll see that there is a __set_name__.
I think the guide sholud be updated to include at least information about existence of __set_name__.

----------
assignee: docs at python
components: Documentation
messages: 323479
nosy: MarSoft, docs at python
priority: normal
severity: normal
status: open
title: Descriptors HowTo doesn't mention __set_name__
versions: Python 3.4, Python 3.5, Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34394>
_______________________________________

From report at bugs.python.org  Mon Aug 13 12:53:22 2018
From: report at bugs.python.org (Sergey Fedoseev)
Date: Mon, 13 Aug 2018 16:53:22 +0000
Subject: [New-bugs-announce] [issue34395] memory leaks in error handling in
 csv and pickle modules
Message-ID: <1534179202.53.0.56676864532.issue34395@psf.upfronthosting.co.za>


New submission from Sergey Fedoseev <fedoseev.sergey at gmail.com>:

There are memory leaks in csv and pickle modules caused by incautious usage of PyMem_Resize().

See GitHub PR.

----------
components: Extension Modules
messages: 323487
nosy: sir-sigurd
priority: normal
severity: normal
status: open
title: memory leaks in error handling in csv and pickle modules

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34395>
_______________________________________

From report at bugs.python.org  Mon Aug 13 15:36:41 2018
From: report at bugs.python.org (Dan Snider)
Date: Mon, 13 Aug 2018 19:36:41 +0000
Subject: [New-bugs-announce] [issue34396] Certain methods that heap
 allocated subtypes inherit suffer a 50-80% performance penalty
Message-ID: <1534189001.92.0.56676864532.issue34396@psf.upfronthosting.co.za>


New submission from Dan Snider <mr.assume.away at gmail.com>:

The ones I know of are list.__getitem__, dict __contains__ & __getitem__, and (frozen)set.__contains__ but from what I can tell so far it seems like dict.__getitem__ takes the worst hit. For dicts, I've spent a few hours trying to figure out what's getting put into the heap type's mp_subscript slot to slow it down so drastically but I just can't figure it out. 

So I just forced `dict_subscript` into the heap type's mp_subscript slot to confirm the rather obvious impossibility of it breaking anything. Here's a "little" timeit script to demonstrate how horribly inefficient `dict.__getitem__` is when called on anything other than an extension type:

if __name__ == '__main__':

    import timeit
    from collections import OrderedDict as FastDictSub
    from ctypes import *
    
    class mappingmethods(Structure):
        _fields_ = [
            ('mp_length', c_void_p),
            ('mp_subscript', PYFUNCTYPE(py_object,py_object,py_object)),
            ('mp_ass_subscript', c_void_p)]
        
    class _type_object(Structure):
        _fields_ = [('junk', (c_uint8*56)), # <- brevity
                    ('tp_as_mapping', POINTER(mappingmethods))]
    py_type = POINTER(_type_object)
    
    class SlowDictSub(dict):
        pass

    assert SlowDictSub.__base__ is dict and FastDictSub.__base__ is dict
    assert SlowDictSub.__getitem__ is dict.__getitem__
    assert FastDictSub.__getitem__ is dict.__getitem__

    mp = dict.fromkeys(range(15))
    setup = 'from __main__ import mp, %s as Dict; d=Dict(mp)'
    print(f'Comparing execution time of heap allocated dict subtype '
          f'versus PyODict_Type (who also inherits dict.__getitem__)...\n')
    slown, slowt = timeit.Timer('d[10]', setup%'SlowDictSub').autorange()
    print(f'avg. exec {slown/slowt:_.0f} SlowDictSub[x] statements per second')
    fastn, fastt = timeit.Timer('d[10]', setup%'FastDictSub').autorange()
    print(f'avg. exec {fastn/fastt:_.0f} OrderedDict[x] statements per second')
    print()
    print(f'SlowDictSub was {1/(fastt/slowt):.2f}x slower than OrderedDict... '
          f"Let's see what happens when we fix SlowDictSub's 'broken' "
          "mp_subscript slot:\n")
    slow_tp = cast(id(SlowDictSub), py_type).contents
    fast_tp = cast(id(FastDictSub), py_type).contents
    
    slow_tp.tp_as_mapping.contents.mp_subscript = (
        fast_tp.tp_as_mapping.contents.mp_subscript)
    postn, postt = timeit.Timer('d[10]', setup%'SlowDictSub').autorange()
    print(f'avg. exec {postn/postt:_.0f} SlowDictSub[x] after slot change')
    print(f'which is now {1/(fastt/postt):.2}x the speed of dict_item')


Comparing execution time of heap allocated dict subtype versus PyODict_Type (who also inherits dict.__getitem__)...

avg. exec 6_220_709 SlowDictSub[x] statements per second
avg. exec 21_894_971 OrderedDict[x] statements per second

SlowDictSub was 3.52x slower than OrderedDict... Let's see what happens when we fix SlowDictSub's 'broken' mp_subscript slot:

avg. exec 21_665_595 SlowDictSub[x] after slot change
which is now 1.0x the speed of dict_item

----------
components: Interpreter Core
messages: 323490
nosy: bup
priority: normal
severity: normal
status: open
title: Certain methods that heap allocated subtypes inherit suffer a 50-80% performance penalty
type: performance
versions: Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34396>
_______________________________________

From report at bugs.python.org  Mon Aug 13 15:38:54 2018
From: report at bugs.python.org (Sergey Fedoseev)
Date: Mon, 13 Aug 2018 19:38:54 +0000
Subject: [New-bugs-announce] [issue34397] remove redundant overflow checks
 in tuple and list implementations
Message-ID: <1534189134.26.0.56676864532.issue34397@psf.upfronthosting.co.za>


New submission from Sergey Fedoseev <fedoseev.sergey at gmail.com>:

Max size of list and tuples is limited by PY_SSIZE_T_MAX / sizeof(PyObject*), so the sum of any two list/tuples sizes always <= PY_SSIZE_T_MAX if sizeof(PyObject*) > 1, which seems true for all supported (existing?) platforms.
It means that overflow checks in app1, ins1, list_concat and tupleconcat are redundant and can be safely removed.

----------
components: Interpreter Core
messages: 323491
nosy: sir-sigurd
priority: normal
severity: normal
status: open
title: remove redundant overflow checks in tuple and list implementations
type: enhancement
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34397>
_______________________________________

From report at bugs.python.org  Tue Aug 14 03:52:28 2018
From: report at bugs.python.org (Jonathan Fine)
Date: Tue, 14 Aug 2018 07:52:28 +0000
Subject: [New-bugs-announce] [issue34398] Docs search does not index glossary
Message-ID: <1534233148.48.0.56676864532.issue34398@psf.upfronthosting.co.za>


New submission from Jonathan Fine <jfine2358 at gmail.com>:

The docs contain a very useful page https://docs.python.org/3.5/glossary.html. However, the search feature does not index the glossary.

Thus, the search https://docs.python.org/3.5/search.html?q=iterable does not produce the helpful glossary entry
===
iterable
An object capable of returning its members one at a time. Examples of iterables include all sequence types (such as list, str, and tuple) and some non-sequence types like dict, file objects, and objects of any  [...]
===

#788509 is the only docs issue I could find, whose title contains glossary. It gives insight into the thoughts then about the tutorial. In msg44460 Skip Montaro says (1) that the glossary is "for the tutorial", and (2) he'd like to improve links into the tutorial. 

I suggest that part of the fix for this issue is on the home page page Glossary in the first grouping "Parts of the documentation."

----------
assignee: docs at python
components: Documentation
messages: 323503
nosy: docs at python, jfine2358
priority: normal
severity: normal
status: open
title: Docs search does not index glossary
type: behavior
versions: Python 2.7, Python 3.4, Python 3.5, Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34398>
_______________________________________

From report at bugs.python.org  Tue Aug 14 04:46:29 2018
From: report at bugs.python.org (Christian Heimes)
Date: Tue, 14 Aug 2018 08:46:29 +0000
Subject: [New-bugs-announce] [issue34399] [TLS] Update test keys to >=
 2048bit
Message-ID: <1534236389.74.0.56676864532.issue34399@psf.upfronthosting.co.za>


New submission from Christian Heimes <lists at cheimes.de>:

Downstream vendors have started to tighten security. 1024 bits RSA and DH params are no longer considered as secure. Python 3.7 and master already use 2048 bits RSA keys for some tests (bpo-32602). 3.6 and 2.7 don't have 1024bit keys. DH params and some other certs are still 1024 bits.

I'm going to update all keys and parameters to 2048.

----------
assignee: christian.heimes
components: SSL, Tests
messages: 323504
nosy: christian.heimes
priority: normal
severity: normal
status: open
title: [TLS] Update test keys to >= 2048bit
type: behavior
versions: Python 2.7, Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34399>
_______________________________________

From report at bugs.python.org  Tue Aug 14 05:46:55 2018
From: report at bugs.python.org (Zackery Spytz)
Date: Tue, 14 Aug 2018 09:46:55 +0000
Subject: [New-bugs-announce] [issue34400] Undefined behavior in
 Parser/parsetok.c
Message-ID: <1534240015.53.0.56676864532.issue34400@psf.upfronthosting.co.za>


New submission from Zackery Spytz <zspytz at gmail.com>:

In parsetok(), null pointers are used in pointer arithmetic.

----------
components: Interpreter Core
messages: 323505
nosy: ZackerySpytz
priority: normal
severity: normal
status: open
title: Undefined behavior in Parser/parsetok.c
type: behavior
versions: Python 2.7, Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34400>
_______________________________________

From report at bugs.python.org  Tue Aug 14 07:54:16 2018
From: report at bugs.python.org (Michael Osipov)
Date: Tue, 14 Aug 2018 11:54:16 +0000
Subject: [New-bugs-announce] [issue34401] Make test_gdb work on HP-UX
Message-ID: <1534247656.08.0.56676864532.issue34401@psf.upfronthosting.co.za>


New submission from Michael Osipov <1983-01-06 at gmx.net>:

Regex in test_gdb.py needs to be changed and test can continue, though will be skipped due to old version.

----------
components: Tests
files: test_gdb.patch
keywords: patch
messages: 323508
nosy: michael-o
priority: normal
severity: normal
status: open
title: Make test_gdb work on HP-UX
versions: Python 3.7
Added file: https://bugs.python.org/file47749/test_gdb.patch

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34401>
_______________________________________

From report at bugs.python.org  Tue Aug 14 08:36:57 2018
From: report at bugs.python.org (Michael Osipov)
Date: Tue, 14 Aug 2018 12:36:57 +0000
Subject: [New-bugs-announce] [issue34402] [SOLUTION] strftime fails on HP-UX
Message-ID: <1534250217.83.0.56676864532.issue34402@psf.upfronthosting.co.za>


New submission from Michael Osipov <1983-01-06 at gmx.net>:

strftime() fails on HP-UX. It is mapped to wcsftime(3). It has a quirk on HP-UX that is does not conform to POSIX. To enable POSIX compat one has to do (excerpt from manpage):
> APPLICATION USAGE
>       The "Unix Standards Only" prototype of wcsftime() is available to
>       applications if they are:
>            a. c99 conformant.
> 
>            b. Compiled with -D_XOPEN_SOURCE macro with a value >=500.
> 
>            c. Compiled with -D_POSIX_C_SOURCE macro with a value >= 200112.
> 
>       Also the application must be compiled with the environment variable
>       UNIX_STD set to the value 98 or above and exported.

b and c are satasfied according to config.log. Let's get to a). The manpage of aCC says:
>  EXTERNAL INFLUENCES
>     Environment Variables
>       [...]
>       UNIX95 or UNIX_STD specify the XPG4 behavior for c89(1) and c99(1).
>       -D_XOPEN_UNIX is also set.  UNIX_STD must be set to 1995, 1998 or
>       2003.  Both these variables cause an appropriate object file to be
>       linked into executables in order to customize libc to match ISO/IEC
>       9899:1990/Amendment 1:1995 (1995) and the C++ and C99 Standards for
>       the various wide char (1998) or other C99 libc functions (2003).
>       NOTE: 2003 is only available on HP-UX 11.31.

So one must at least do "export UNIX_STD=1998".

I am quite certain that other parts of Python are affected too.

The safest bet would be actually to test wcsftime at configure time wether the output is fine and have this env var set in the Makefile, unfortunately, this will only work for GNU make. Alternatively, a warning at the very end of the configure run has to be printed to export this env var.

If done, test_strftime and test_teststrptime will pass flawlessly.

----------
components: Library (Lib)
messages: 323512
nosy: michael-o
priority: normal
severity: normal
status: open
title: [SOLUTION] strftime fails on HP-UX
type: crash
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34402>
_______________________________________

From report at bugs.python.org  Tue Aug 14 08:57:12 2018
From: report at bugs.python.org (Michael Osipov)
Date: Tue, 14 Aug 2018 12:57:12 +0000
Subject: [New-bugs-announce] [issue34403] test_utf8_mode.test_cmd_line()
 fails on HP-UX due to false assumptions
Message-ID: <1534251432.68.0.56676864532.issue34403@psf.upfronthosting.co.za>


New submission from Michael Osipov <1983-01-06 at gmx.net>:

Running from 3.7 branch on HP-UX 11.31 ia64, 32 bit, big endian.
The test output is:
> Re-running failed tests in verbose mode
> Re-running test 'test_utf8_mode' in verbose mode
> test_cmd_line (test.test_utf8_mode.UTF8ModeTests) ... FAIL
> test_env_var (test.test_utf8_mode.UTF8ModeTests) ... ok
> test_filesystemencoding (test.test_utf8_mode.UTF8ModeTests) ... ok
> test_io (test.test_utf8_mode.UTF8ModeTests) ... ok
> test_io_encoding (test.test_utf8_mode.UTF8ModeTests) ... ok
> test_locale_getpreferredencoding (test.test_utf8_mode.UTF8ModeTests) ... ok
> test_optim_level (test.test_utf8_mode.UTF8ModeTests) ... ok
> test_posix_locale (test.test_utf8_mode.UTF8ModeTests) ... ok
> test_stdio (test.test_utf8_mode.UTF8ModeTests) ... ok
> test_xoption (test.test_utf8_mode.UTF8ModeTests) ... ok
> 
> ======================================================================
> FAIL: test_cmd_line (test.test_utf8_mode.UTF8ModeTests)
> ----------------------------------------------------------------------
> Traceback (most recent call last):
>   File "/var/osipovmi/cpython/Lib/test/test_utf8_mode.py", line 230, in test_cmd_line
>     check('utf8=0', [c_arg], LC_ALL='C')
>   File "/var/osipovmi/cpython/Lib/test/test_utf8_mode.py", line 223, in check
>     self.assertEqual(args, ascii(expected), out)
> AssertionError: "['h\\xc3\\xa9\\xe2\\x82\\xac']" != "['h\\udcc3\\udca9\\udce2\\udc82\\udcac']"
> - ['h\xc3\xa9\xe2\x82\xac']
> + ['h\udcc3\udca9\udce2\udc82\udcac']
>  : roman8:['h\xc3\xa9\xe2\x82\xac']
> 
> ----------------------------------------------------------------------
> Ran 10 tests in 2.595s
> 
> FAILED (failures=1)
> test test_utf8_mode failed
> 1 test failed again:
>     test_utf8_mode
> 
> == Tests result: FAILURE then FAILURE ==
> 
> 1 test failed:
>     test_utf8_mode
> 
> 1 re-run test:
>     test_utf8_mode
> 
> Total duration: 7 sec 265 ms
> Tests result: FAILURE then FAILURE
> Makefile:1066: recipe for target 'test' failed
> gmake: *** [test] Error 2
> 

I tried to understand the issue, but my Python knowledge is too low, especially I do not understand by a byte array "arg = 'h\xe9\u20ac'.encode('utf-8')" is passed as one arg to the forked process.

I highly assume that this is related to the non-standard, default character encoding on HP-UX: https://en.wikipedia.org/wiki/HP_Roman#HP_Roman-8 (roman8).

A stupid 8 bit encoding. The very same snippet on FreeBSD says:
> $ LC_ALL=C python3.6 test_utf8.py
> US-ASCII:[]

Willing to test and modify if someone tells what to do.

----------
components: Library (Lib), Tests
messages: 323516
nosy: michael-o
priority: normal
severity: normal
status: open
title: test_utf8_mode.test_cmd_line() fails on HP-UX due to false assumptions
type: behavior
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34403>
_______________________________________

From report at bugs.python.org  Tue Aug 14 09:36:49 2018
From: report at bugs.python.org (Michael Osipov)
Date: Tue, 14 Aug 2018 13:36:49 +0000
Subject: [New-bugs-announce] [issue34404] test_time incorrectly defined
Message-ID: <1534253809.16.0.56676864532.issue34404@psf.upfronthosting.co.za>


New submission from Michael Osipov <1983-01-06 at gmx.net>:

I see a test failure on HP-UX:
> test_negative (test.test_time.TestStrftime4dyear) ... FAIL
> ======================================================================
> FAIL: test_negative (test.test_time.TestStrftime4dyear)
> ----------------------------------------------------------------------
> Traceback (most recent call last):
>   File "/var/osipovmi/cpython/Lib/test/test_time.py", line 687, in test_negative
>     return super().test_negative()
>   File "/var/osipovmi/cpython/Lib/test/test_time.py", line 713, in test_negative
>     self.assertEqual(self.yearstr(-1), self._format % -1)
> AssertionError: '-0001' != '-001'
> - -0001
> ?  -
> + -001

The bug is in the test class, not cause by HP-UX.
Lib/test/test_time.py:713 says "self.assertEqual(self.yearstr(-1), self._format % -1)" where self._format is set in line 654 to "_format = '%04d'" because
> Python 3.7.0+ (heads/3.7-dirty:ea8835fb30, Aug 14 2018, 14:44:19) [C] on hp-ux11
> Type "help", "copyright", "credits" or "license" for more information.
> >>> import time
> >>> time.strftime('%Y', (1,) + (0,) * 8)
> '0001'

Unfortunately, the zero padding in printf() applies to the entire string length. So
> >>> "%+04d" % -1
> '-001'

on HP-UX, Linux and FreeBSD.

The format string must add an additional zero for negative numbers.

----------
components: Tests
messages: 323519
nosy: michael-o
priority: normal
severity: normal
status: open
title: test_time incorrectly defined
type: behavior
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34404>
_______________________________________

From report at bugs.python.org  Tue Aug 14 09:52:47 2018
From: report at bugs.python.org (Christian Heimes)
Date: Tue, 14 Aug 2018 13:52:47 +0000
Subject: [New-bugs-announce] [issue34405] Upgrade to OpenSSL 1.1.0i / 1.0.2p
Message-ID: <1534254767.16.0.56676864532.issue34405@psf.upfronthosting.co.za>


New submission from Christian Heimes <lists at cheimes.de>:

1.0.2p and 1.1.0i were released today. Please update the Windows and macOS installers.

----------
components: Windows, macOS
messages: 323521
nosy: christian.heimes, ned.deily, paul.moore, ronaldoussoren, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: Upgrade to OpenSSL 1.1.0i / 1.0.2p
type: security
versions: Python 2.7, Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34405>
_______________________________________

From report at bugs.python.org  Tue Aug 14 11:04:48 2018
From: report at bugs.python.org (Alisue Lambda)
Date: Tue, 14 Aug 2018 15:04:48 +0000
Subject: [New-bugs-announce] [issue34406] Type in documentation
Message-ID: <1534259088.23.0.56676864532.issue34406@psf.upfronthosting.co.za>


New submission from Alisue Lambda <lambdalisue at hashnote.net>:

https://docs.python.org/3.8/using/windows.html#removing-the-max-path-limitation

Current
> HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\FileSystem at LongPathsEnabled

Should be
> HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\FileSystem\LongPathsEnabled

----------
assignee: docs at python
components: Documentation
messages: 323527
nosy: Alisue Lambda, docs at python
priority: normal
severity: normal
status: open
title: Type in documentation
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34406>
_______________________________________

From report at bugs.python.org  Tue Aug 14 14:18:04 2018
From: report at bugs.python.org (Maksym Shalenyi (Enkidulan))
Date: Tue, 14 Aug 2018 18:18:04 +0000
Subject: [New-bugs-announce] [issue34407] datetime.time.isoformat function
 has inconsistent behavior with timezone
Message-ID: <1534270684.14.0.56676864532.issue34407@psf.upfronthosting.co.za>


New submission from Maksym Shalenyi (Enkidulan) <supamaxy at gmail.com>:

In some cases datetime.time.isoformat shows timezone info, but in some does not. Consider the example below.

import datetime
import pytz

t = dict(hour=12, minute=31, second=21, microsecond=213456)

# `datetime.time.isoformat` has inconsistent behavior. Some of printed has timezone, but others does not.
print(datetime.time(tzinfo=pytz.timezone('Asia/Seoul'), **t).isoformat())
print(datetime.time(tzinfo=pytz.timezone('Etc/GMT-9'), **t).isoformat())
print(datetime.time(tzinfo=pytz.timezone('Australia/Sydney'), **t).isoformat())
print(datetime.time(tzinfo=pytz.timezone('Etc/UTC'), **t).isoformat())
# output:
# 12:31:21.213456
# 12:31:21.213456+09:00
# 12:31:21.213456
# 12:31:21.213456+00:00



# `datetime.time.isoformat` is inconsistent with `datetime.datetime.isoformat`. `datetime` objects always shows tz information when tz is present.
d = dict(year=2018, month=2, day=2, **t)
print(datetime.datetime(tzinfo=pytz.timezone('Asia/Seoul'), **d).isoformat())
print(datetime.datetime(tzinfo=pytz.timezone('Etc/GMT-9'), **d).isoformat())
print(datetime.datetime(tzinfo=pytz.timezone('Australia/Sydney'), **d).isoformat())
print(datetime.datetime(tzinfo=pytz.timezone('Etc/UTC'), **d).isoformat())
# output:
# 2018-02-02T12:31:21.213456+08:28
# 2018-02-02T12:31:21.213456+09:00
# 2018-02-02T12:31:21.213456+10:05
# 2018-02-02T12:31:21.213456+00:00

----------
components: ctypes
messages: 323531
nosy: Maksym Shalenyi (Enkidulan)
priority: normal
severity: normal
status: open
title: datetime.time.isoformat function has inconsistent behavior with timezone
type: behavior
versions: Python 2.7, Python 3.4, Python 3.5, Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34407>
_______________________________________

From report at bugs.python.org  Tue Aug 14 16:09:38 2018
From: report at bugs.python.org (Pablo Galindo Salgado)
Date: Tue, 14 Aug 2018 20:09:38 +0000
Subject: [New-bugs-announce] [issue34408] possible null pointer dereference
 in pystate.c
Message-ID: <1534277378.73.0.56676864532.issue34408@psf.upfronthosting.co.za>


New submission from Pablo Galindo Salgado <pablogsal at gmail.com>:

The problem occurs here:

https://github.com/python/cpython/blob/master/Python/pystate.c#L185

If _PyRuntime.interpreters.next_id < 0 then interp is set to NULL and it will be dereferenced later:

interp

----------
components: Interpreter Core
messages: 323538
nosy: pablogsal
priority: normal
severity: normal
status: open
title: possible null pointer dereference in pystate.c
type: crash
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34408>
_______________________________________

From report at bugs.python.org  Tue Aug 14 19:11:20 2018
From: report at bugs.python.org (mkurnikov)
Date: Tue, 14 Aug 2018 23:11:20 +0000
Subject: [New-bugs-announce] [issue34409] Add a way to customize iteration
 over fields in asdict() for the nested dataclasses
Message-ID: <1534288280.15.0.56676864532.issue34409@psf.upfronthosting.co.za>


New submission from mkurnikov <maxim.kurnikov at gmail.com>:

Suppose I have two dataclasses:

@dataclass
class NestedDataclass(object):
    name: str
    options: Dict[str, Any] = field(default_factory=dict)

@dataclass
class RootDataclass(object):
    nested_list: List[NestedDataclass]

I want a dict under the key "options" to be merged in the NestedDataclass dict in the dataclasses.asdict(root_dcls_instance). 

For that, according to docs, I need to specify dict_factory= for dataclasses.asdict() function. 

The problem is that, according to the implementation, when this function "meets" dataclass, there's no way to customize how result dict will be built. Dataclass itself is never passed to the function. 

    if _is_dataclass_instance(obj):
        result = []
        for f in fields(obj):
            value = _asdict_inner(getattr(obj, f.name), dict_factory)
            result.append((f.name, value))
        return dict_factory(result)

Yes, I can catch "result" obj (what I did in the end):

def root_dataclass_dict_factory(obj):
    if isinstance(obj, list):
        dataclass_dict = dict(obj)
        if 'options' in dataclass_dict:
            dataclass_dict.update(dataclass_dict.pop('options'))

    return dict(obj)

The problem is that type of the dataclass is lost for the list, and if by any chance later I'll have "options" key in the RootDataclass, there's no way for me to distinguish between them cleanly. 

Other solution is to iterate over the RootDataclass dictionary, follow the path to the NestedDataclass and change dictionary there, but it even uglier. 

Would be nice to be able to somehow hook into the field traversal of the dataclass instance.

----------
components: Library (Lib)
messages: 323542
nosy: mkurnikov
priority: normal
severity: normal
status: open
title: Add a way to customize iteration over fields in asdict() for the nested dataclasses
type: enhancement
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34409>
_______________________________________

From report at bugs.python.org  Tue Aug 14 22:08:28 2018
From: report at bugs.python.org (Carlo Rosati)
Date: Wed, 15 Aug 2018 02:08:28 +0000
Subject: [New-bugs-announce] [issue34410] Segfault/TimeoutError:
 itertools.tee of multiprocessing.pool.imap_unordered
Message-ID: <1534298908.58.0.56676864532.issue34410@psf.upfronthosting.co.za>


New submission from Carlo Rosati <crosati777 at icloud.com>:

Hello, 

When I run the attached code, I encounter a segmentation fault.

Thanks,
Carlo

----------
files: 3.py
messages: 323546
nosy: carlorosati
priority: normal
severity: normal
status: open
title: Segfault/TimeoutError: itertools.tee of multiprocessing.pool.imap_unordered
type: crash
versions: Python 2.7, Python 3.7
Added file: https://bugs.python.org/file47750/3.py

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34410>
_______________________________________

From report at bugs.python.org  Wed Aug 15 11:56:16 2018
From: report at bugs.python.org (ysangkok+launchpad)
Date: Wed, 15 Aug 2018 15:56:16 +0000
Subject: [New-bugs-announce] [issue34411] ProactorEventLoop should use
 implement GetAddrInfo, GetNameInfo
Message-ID: <1534348576.75.0.56676864532.issue34411@psf.upfronthosting.co.za>


New submission from ysangkok+launchpad <ysangkok at gmail.com>:

Since socket.getaddrinfo and socket.getnameinfo take a lock on Windows, it would be nice to have their corresponding methods in asyncio.ProactorEventLoop use IOCP, so that they wouldn't block the whole event loop.

Overlapped I/O flags are available, so I am wondering how hard this would be: https://docs.microsoft.com/en-us/windows/desktop/api/ws2tcpip/nf-ws2tcpip-getaddrinfoexa

----------
components: asyncio
messages: 323572
nosy: asvetlov, ysangkok+launchpad, yselivanov
priority: normal
severity: normal
status: open
title: ProactorEventLoop should use implement GetAddrInfo, GetNameInfo
type: enhancement
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34411>
_______________________________________

From report at bugs.python.org  Thu Aug 16 10:47:52 2018
From: report at bugs.python.org (Michael Osipov)
Date: Thu, 16 Aug 2018 14:47:52 +0000
Subject: [New-bugs-announce] [issue34412] strsignal(3) does not exist on
 HP-UX
Message-ID: <1534430872.24.0.56676864532.issue34412@psf.upfronthosting.co.za>


New submission from Michael Osipov <1983-01-06 at gmx.net>:

HP-UX does not provide any mappings from signals to strings. The proper approach is to map just like for Windows. Alternatively, one could simply return the singal as an int.

----------
components: Library (Lib)
messages: 323600
nosy: michael-o
priority: normal
severity: normal
status: open
title: strsignal(3) does not exist on HP-UX
type: compile error
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34412>
_______________________________________

From report at bugs.python.org  Thu Aug 16 11:46:13 2018
From: report at bugs.python.org (Eric Snow)
Date: Thu, 16 Aug 2018 15:46:13 +0000
Subject: [New-bugs-announce] [issue34413] Porting section of 3.5 "What's
 New" doc does not mention bytecode changes.
Message-ID: <1534434373.24.0.56676864532.issue34413@psf.upfronthosting.co.za>


New submission from Eric Snow <ericsnowcurrently at gmail.com>:

In the 3.5 "What's New" doc, the porting section [1] does not mention the bytecode changes resulting from the PEP 448 implementation. [2][3]  It should.

I've marked this issue for the versions past 3.5 because each branch has its own "What's New" doc for 3.5.

[1] https://docs.python.org/3/whatsnew/3.5.html#porting-to-python-3-5
[2] bpo-33216
[3] https://github.com/python/cpython/pull/8783#discussion_r210642202

----------
assignee: docs at python
components: Documentation
messages: 323605
nosy: docs at python, eric.snow, serhiy.storchaka
priority: low
severity: normal
stage: needs patch
status: open
title: Porting section of 3.5 "What's New" doc does not mention bytecode changes.
type: enhancement
versions: Python 3.5, Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34413>
_______________________________________

From report at bugs.python.org  Thu Aug 16 12:45:25 2018
From: report at bugs.python.org (Jonathan Hadida)
Date: Thu, 16 Aug 2018 16:45:25 +0000
Subject: [New-bugs-announce] [issue34414] Absolute imports conflict with
 local files
Message-ID: <1534437925.07.0.56676864532.issue34414@psf.upfronthosting.co.za>


New submission from Jonathan Hadida <ariel.hadida at gmail.com>:

This submission follows a post on StackOverflow: https://stackoverflow.com/q/51878397/472610

I have reproduced the unexpected behaviour with multiple python versions, either with a Homebrew install, or using Anaconda/Miniconda. Note that comments to the post on StackOverflow indicate that this behaviour could only be reproduced on Windows and Linux using conda with late versions of Python.


THE ISSUE
---------

Absolute imports seem to conflict with local files (not in any module). 
This is at odds with the documented behaviour (https://docs.python.org/3/tutorial/modules.html#the-module-search-path):

"When a module named spam is imported, the interpreter first searches for a built-in module with that name."


STEPS TO REPRODUCE
------------------

STEP 1:
On OSX, use either:

 - A Homebrew-installed version (e.g. /usr/local/bin/python2 or 3)
 - A Miniconda2 or 3 installation
 - An Anaconda3 v5.2.0 installation (not tested with other versions, nor Anaconda2)

NOTE: in cases 1 and 2, you need to install numpy manually.


STEP 2:
Create a directory structure as follows:

  .
  ??? foo
      ??? a.py
      ??? math.py

  1 directory, 2 files

where a.py contains "import numpy", and math.py contains "x++" (intentionally invalid).
For example, the following bash code sets this up in a temporary folder:

  D=$(mktemp -d)
  mkdir "$D/foo"
  echo "import numpy" >| "$D/foo/a.py"
  echo "x++" >| "$D/foo/math.py"


STEP 3:
Go to that directory (the one containing folder "foo"), and run:

  <PythonExecutableFromStep1> foo/a.py

The previous command causes the following error, for example using /usr/local/bin/python3 (Homebrew):

Traceback (most recent call last):
  File "foo/a.py", line 1, in <module>
    import numpy
  File "/usr/local/lib/python3.6/site-packages/numpy/__init__.py", line 142, in <module>
    from . import add_newdocs
  File "/usr/local/lib/python3.6/site-packages/numpy/add_newdocs.py", line 13, in <module>
    from numpy.lib import add_newdoc
  File "/usr/local/lib/python3.6/site-packages/numpy/lib/__init__.py", line 3, in <module>
    import math
  File "/private/var/folders/j7/kd8mc69j25j0yw50q_08wmlm0000gt/T/tmp.FfJzdVuG/foo/math.py", line 1
    x++
      ^
SyntaxError: invalid syntax


PROBLEM:
The statement "import math" in numpy/lib/__init__.py should not resolve to foo/math.py, but rather, it should find the standard module "math".


ADDITIONAL INFO
---------------

Although I do not know what this list should look like, I would expect the list of builtin modules to be larger than this:

> python3 -c "import sys; print(sys.builtin_module_names)"
('_ast', '_codecs', '_collections', '_functools', '_imp', '_io', '_locale', '_operator', '_signal', '_sre', '_stat', '_string', '_symtable', '_thread', '_tracemalloc', '_warnings', '_weakref', 'atexit', 'builtins', 'errno', 'faulthandler', 'gc', 'itertools', 'marshal', 'posix', 'pwd', 'sys', 'time', 'xxsubtype', 'zipimport')

> python2 -c "import sys; print sys.builtin_module_names"
('__builtin__', '__main__', '_ast', '_codecs', '_sre', '_symtable', '_warnings', '_weakref', 'errno', 'exceptions', 'gc', 'imp', 'marshal', 'posix', 'pwd', 'signal', 'sys', 'thread', 'xxsubtype', 'zipimport')

----------
components: macOS
messages: 323609
nosy: jhadida, ned.deily, ronaldoussoren
priority: normal
severity: normal
status: open
title: Absolute imports conflict with local files
type: behavior
versions: Python 2.7, Python 3.4, Python 3.5, Python 3.6, Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34414>
_______________________________________

From report at bugs.python.org  Thu Aug 16 15:52:26 2018
From: report at bugs.python.org (Semyon)
Date: Thu, 16 Aug 2018 19:52:26 +0000
Subject: [New-bugs-announce] [issue34415] Typo in logging.Formatter docstring
Message-ID: <1534449146.03.0.56676864532.issue34415@psf.upfronthosting.co.za>


New submission from Semyon <simeon+bpo at maryasin.name>:

There is a typo in the docstring for logging.Formatter:

> default value of "%s(message)\\n" is used.

I am sure it should be different (and in sync with the actual value):

> default value of "%(message)s\\n" is used.

The problem is in py2.7, py3.7 and most likely other versions.

----------
components: Library (Lib)
messages: 323618
nosy: MarSoft
priority: normal
severity: normal
status: open
title: Typo in logging.Formatter docstring
versions: Python 2.7, Python 3.4, Python 3.5, Python 3.6, Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34415>
_______________________________________

From report at bugs.python.org  Thu Aug 16 17:15:03 2018
From: report at bugs.python.org (Paul Ganssle)
Date: Thu, 16 Aug 2018 21:15:03 +0000
Subject: [New-bugs-announce] [issue34416] PyCapsule_Import seems to release
 the GIL without acquiring it
Message-ID: <1534454103.63.0.56676864532.issue34416@psf.upfronthosting.co.za>


New submission from Paul Ganssle <p.ganssle at gmail.com>:

I was recently debugging some multithreaded Rust code that was deadlocking, and I tracked it down to what I'm fairly certain is a bug somewhere in PyCapsule_Import, where it seems that PyCapsule_Import releases the GIL without first acquiring it.

I've attached a MWE of a multi-threaded application that is able to simultaneously acquire the GIL twice. The relevant portion is here:

void *acquire_gil(void *arg) {
    bool import = ((arg_t *)arg)->import;
    int n = ((arg_t *)arg)->id;

    printf("Waiting for GIL (%d)\n", n);
    int gstate = PyGILState_Ensure();
    printf("Gil acquired! (%d)\n", n);
    usleep(125000);
    if (import) {
        PyCapsule_Import(CAPSULE_NAME, 0);
    }
    usleep(125000);
    PyGILState_Release(gstate);
    printf("Gil released! (%d)\n", n);
    return NULL;
}

If you run it with `./gil` such that the PyCapsule_Import call is never reached, you get:

    Waiting for GIL (0)
    Gil acquired! (0)
    Waiting for GIL (1)
    Gil released! (0)
    Gil acquired! (1)
    Gil released! (1)

However, if you run with `./gil import` such that PyCapsule_Import is reached, you get (emphasis mine):

    Waiting for GIL (0)
    Gil acquired! (0)
    Waiting for GIL (1)
    **Gil acquired! (1)**
    **Gil released! (1)**
    Gil released! (0)

For convenience sake, I have created a small repo with a make file for the PoC: https://github.com/pganssle/capsule-gil-poc

I have tested this on version 3.6.6 and 3.7.0. The makefile works in a virtualenv, but you have to manually tweak the PY_MAJOR and PY_MINOR variables in Makefile because I didn't get too fancy with it.

----------
files: main.c
messages: 323620
nosy: p-ganssle, pablogsal
priority: normal
severity: normal
status: open
title: PyCapsule_Import seems to release the GIL without acquiring it
versions: Python 3.6, Python 3.7
Added file: https://bugs.python.org/file47753/main.c

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34416>
_______________________________________

From report at bugs.python.org  Thu Aug 16 17:51:44 2018
From: report at bugs.python.org (Phillip M. Feldman)
Date: Thu, 16 Aug 2018 21:51:44 +0000
Subject: [New-bugs-announce] [issue34417] imp.find_module reacts badly to
 iterator
Message-ID: <1534456304.96.0.56676864532.issue34417@psf.upfronthosting.co.za>


New submission from Phillip M. Feldman <Phillip.M.Feldman at gmail.com>:

`imp.find_module` goes down in flames if one tries to pass an iterator rather than a list of folders.  Firstly, the message that it produces is somewhat misleading:

   RuntimeError: sys.path must be a list of directory names

Secondly, it would be helpful if one could pass an iterator. I'm thinking in particular of the situation where one wants to import something from a large folder tree, and the module in question is likely to be found early in the search process, so that it is more efficient to explore the folder tree incrementally.

----------
components: Library (Lib)
messages: 323623
nosy: Phillip.M.Feldman at gmail.com
priority: normal
severity: normal
status: open
title: imp.find_module reacts badly to iterator
type: behavior
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34417>
_______________________________________

From report at bugs.python.org  Fri Aug 17 03:45:47 2018
From: report at bugs.python.org (Sebastian Rittau)
Date: Fri, 17 Aug 2018 07:45:47 +0000
Subject: [New-bugs-announce] [issue34418] Arguments missing from
 documentation for HTTPErrorProcessor.http(s)_response()
Message-ID: <1534491947.25.0.56676864532.issue34418@psf.upfronthosting.co.za>


New submission from Sebastian Rittau <srittau at rittau.biz>:

The documentation at https://docs.python.org/3/library/urllib.request.html#urllib.request.HTTPErrorProcessor does not list the two arguments "request" and "response" that HTTPErrorProcessor.http_response() and https_response() actually require. (I checked the implementation in Python 3.5 and 3.7.)

----------
assignee: docs at python
components: Documentation
messages: 323638
nosy: docs at python, srittau
priority: normal
severity: normal
status: open
title: Arguments missing from documentation for HTTPErrorProcessor.http(s)_response()
versions: Python 3.5, Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34418>
_______________________________________

From report at bugs.python.org  Fri Aug 17 05:28:09 2018
From: report at bugs.python.org (Michael Osipov)
Date: Fri, 17 Aug 2018 09:28:09 +0000
Subject: [New-bugs-announce] [issue34419] selectmodule.c does not compile on
 HP-UX due to bpo-31938/6dc57e2a20c5beb99e8bf5eb04cc836d53fa9aee
Message-ID: <1534498089.92.0.56676864532.issue34419@psf.upfronthosting.co.za>


New submission from Michael Osipov <1983-01-06 at gmx.net>:

It seems like an oversight of the author, though it is not clear why this happily compiles with clang 3.4.1 on FreeBSD 10.4-STABLE and GCC on RHEL6.

The error is (HP-UX 11.31 and HP C/aC++ B3910B A.06.28.03 [Dec 13 2016]):

> /opt/aCC/bin/cc -Ae +z -O -I./Include -I. -I/usr/local/include -I/var/osipovmi/cpython/Include -I/var/osipovmi/cpython -c /var/osipovmi/cpython/Modules/selectmodule.c -o build/temp.hp-ux-B.11.31-ia64-3.8-pydebug/var/osipovmi/cpython/Modules/selectmodule.o
> "/var/osipovmi/cpython/Modules/selectmodule.c", line 412: warning #4232-D:
>           conversion from "PyObject *" to a more strictly aligned type
>           "PyDictObject *" may cause misaligned access
>       self->ufd_len = PyDict_GET_SIZE(self->dict);
>                       ^
> 
> "/var/osipovmi/cpython/Modules/selectmodule.c", line 849: error #2130:
>           expected a "{"
>   static PyObject *
>   ^
> 
> "/var/osipovmi/cpython/Modules/selectmodule.c", line 898: error #2101:
>           "timeout_obj" has already been declared in the current scope
>       PyObject *result_list = NULL, *timeout_obj = NULL;
>                                      ^
> 
> 2 errors detected in the compilation of "/var/osipovmi/cpython/Modules/selectmodule.c".


caused by:

> +static PyObject *
> +select_devpoll_modify_impl(devpollObject *self, int fd,
> +                           unsigned short eventmask)
> +/*[clinic end generated code: output=bc2e6d23aaff98b4 input=f0d7de3889cc55fb]*/
>  static PyObject *
>  devpoll_modify(devpollObject *self, PyObject *args)
>  {
> -    return internal_devpoll_register(self, args, 1);
> +    return internal_devpoll_register(self, fd, eventmask, 1);
>  }

Signature change, but prototype has not been changed.

>  static PyObject *
> -poll_poll(pollObject *self, PyObject *args)
> +select_poll_poll_impl(pollObject *self, PyObject *timeout_obj)
> +/*[clinic end generated code: output=876e837d193ed7e4 input=7a446ed45189e894]*/
>  {
> -    PyObject *result_list = NULL, *timeout_obj = NULL;
> +    PyObject *result_list = NULL;
>      int poll_result, i, j;
>      PyObject *value = NULL, *num = NULL;
>      _PyTime_t timeout = -1, ms = -1, deadline = 0;
>      int async_err = 0;
> 
> -    if (!PyArg_ParseTuple(args, "|O:poll", &timeout_obj)) {
> -        return NULL;
> -    }
> -
> -    if (timeout_obj != NULL && timeout_obj != Py_None) {
> +    if (timeout_obj != Py_None) {

timeout_obj has been added to the function signature, but the internal NULL assignment has not been removed.

----------
components: Build, Library (Lib)
messages: 323640
nosy: michael-o
priority: normal
severity: normal
status: open
title: selectmodule.c does not compile on HP-UX due to bpo-31938/6dc57e2a20c5beb99e8bf5eb04cc836d53fa9aee
type: compile error
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34419>
_______________________________________

From report at bugs.python.org  Fri Aug 17 09:56:19 2018
From: report at bugs.python.org (sai arjun)
Date: Fri, 17 Aug 2018 13:56:19 +0000
Subject: [New-bugs-announce] [issue34420] Complete your registration to
 Python tracker -- keysnSwaZe6PtikiEZf4bdIXIiuwFyFZFxp
In-Reply-To: <20180816113813.9CDD8117A17@psf.upfronthosting.co.za>
Message-ID: <1534514179.66.0.094306209913.issueNone@psf.upfronthosting.co.za>


New submission from sai arjun <venkkarjun at yahoo.com>:

I have received the email.

Sent from Mail for Windows 10

From: Python tracker
Sent: 16 August 2018 17:08
To: venkkarjun at yahoo.com
Subject: Complete your registration to Python tracker -- keysnSwaZe6PtikiEZf4bdIXIiuwFyFZFxp

To complete your registration of the user "arjun" with
Python tracker, please do one of the following:

- send a reply to report at bugs.python.org and maintain the subject line as is (the
reply's additional "Re:" is ok),

- or visit the following URL:

https://bugs.python.org/?@action=confrego&otk=snSwaZe6PtikiEZf4bdIXIiuwFyFZFxp

----------
messages: 323657
nosy: arjun
priority: normal
severity: normal
status: open
title: Complete your registration to Python tracker -- keysnSwaZe6PtikiEZf4bdIXIiuwFyFZFxp

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34420>
_______________________________________

From report at bugs.python.org  Fri Aug 17 13:06:15 2018
From: report at bugs.python.org (Julien Malard)
Date: Fri, 17 Aug 2018 17:06:15 +0000
Subject: [New-bugs-announce] [issue34421] Cannot install package with
 unicode module names on Windows
Message-ID: <1534525575.04.0.56676864532.issue34421@psf.upfronthosting.co.za>


Change by Julien Malard <julien.malard at mail.mcgill.ca>:


----------
components: Distutils
nosy: dstufft, eric.araujo, julien.malard
priority: normal
severity: normal
status: open
title: Cannot install package with unicode module names on Windows
type: crash
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34421>
_______________________________________

From report at bugs.python.org  Fri Aug 17 13:21:06 2018
From: report at bugs.python.org (Gabriel Tremblay)
Date: Fri, 17 Aug 2018 17:21:06 +0000
Subject: [New-bugs-announce] [issue34422] __name__ not available for classes
 in typing module
Message-ID: <1534526466.89.0.56676864532.issue34422@psf.upfronthosting.co.za>


New submission from Gabriel Tremblay <gabriel at delvelabs.ca>:

Types under the typing module used to behave like other python classes regarding the __name__ attribute before Python 3.7. The behavior seems to have changed to a non-standard way.

Py3.6 
>>> from typing import List
>>> dir(List)
['__abstractmethods__', '__add__', '__args__', '__class__', '__contains__', '__delattr__', '__delitem__', 
'__dir__', '__doc__', '__eq__', '__extra__', '__format__', '__ge__', '__getattribute__', '__getitem__', '_
_gt__', '__hash__', '__iadd__', '__imul__', '__init__', '__init_subclass__', '__iter__', '__le__', '__len_
_', '__lt__', '__module__', '__mul__', '__ne__', '__new__', '__next_in_mro__', '__orig_bases__', '__origin
__', '__parameters__', '__reduce__', '__reduce_ex__', '__repr__', '__reversed__', '__rmul__', '__setattr__
', '__setitem__', '__sizeof__', '__slots__', '__str__', '__subclasshook__', '__tree_hash__', '_abc_cache',
 '_abc_negative_cache', '_abc_negative_cache_version', '_abc_registry', 'append', 'clear', 'copy', 'count'
, 'extend', 'index', 'insert', 'pop', 'remove', 'reverse', 'sort']
>>> List.__name__
'List'


Py3.7:
>>> from typing import List
>>> dir(List)
['__args__', '__call__', '__class__', '__delattr__', '__dict__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattr__', '__getattribute__', '__getitem__', '__gt__', '__hash__', '__init__', '__init_subclass__', '__instancecheck__', '__le__', '__lt__', '__module__', '__mro_entries__', '__ne__', '__new__', '__origin__', '__parameters__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__slots__', '__str__', '__subclasscheck__', '__subclasshook__', '__weakref__', '_inst', '_name', '_special', 'copy_with']
>>> List.__name__
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python3.7/typing.py", line 699, in __getattr__
    raise AttributeError(attr)
AttributeError: __name__
>>> List._name
'List'

----------
components: Interpreter Core
messages: 323663
nosy: Gabriel Tremblay
priority: normal
severity: normal
status: open
title: __name__ not available for classes in typing module
type: behavior
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34422>
_______________________________________

From report at bugs.python.org  Fri Aug 17 17:56:49 2018
From: report at bugs.python.org (=?utf-8?b?TWljaGHFgiBSYWR3YcWEc2tp?=)
Date: Fri, 17 Aug 2018 21:56:49 +0000
Subject: [New-bugs-announce] [issue34423] Overflow when casting from double
 to time_t, and_PyTime_t.
Message-ID: <1534543009.21.0.56676864532.issue34423@psf.upfronthosting.co.za>


New submission from Micha? Radwa?ski <lisklusownik at gmail.com>:

Code triggering bug:

import time
time.sleep(2**63 / 10**9)

Result:

ValueError: sleep length must be non-negative

The problem is with the macro that checks the range of provided double:

Line 228 of Include/pymath.h:
#define _Py_InIntegralTypeRange(type, v) (_Py_IntegralTypeMin(type) <= v && v <= _Py_IntegralTypeMax(type))

The type here is _PyTime_t, which is a typedef to int64_t. 

Proposed solution is to change <= into <, since float(2**63-1) == float(2**63), and that means that none double can ever be equal to 2**63-1.

----------
components: Interpreter Core
messages: 323676
nosy: belopolsky, enedil, lemburg, vstinner
priority: normal
severity: normal
status: open
title: Overflow when casting from double to time_t, and_PyTime_t.
type: behavior
versions: Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34423>
_______________________________________

From report at bugs.python.org  Fri Aug 17 19:02:02 2018
From: report at bugs.python.org (Jens Troeger)
Date: Fri, 17 Aug 2018 23:02:02 +0000
Subject: [New-bugs-announce] [issue34424] Unicode names break email header
Message-ID: <1534546922.5.0.56676864532.issue34424@psf.upfronthosting.co.za>


New submission from Jens Troeger <jens.troeger at gmail.com>:

See also this comment and ensuing conversation: https://bugs.python.org/issue24218?#msg322761

Consider an email message with the following:

message = EmailMessage()
message["From"] = Address(addr_spec="bar at foo.com", display_name="Jens Troeger")
message["To"] = Address(addr_spec="foo at bar.com", display_name="Mart?n C?rdoba")

It?s important here that the email itself is `ascii` encodable, but the names are not. Flattening the object (https://github.com/python/cpython/blob/master/Lib/smtplib.py#L964) incorrectly inserts multiple linefeeds, thus breaking the email header, thus mangling the entire email:

flatmsg: b'From: Jens Troeger <bar at foo.com>\r\nTo: Fernando =?utf-8?q?Mart=C3=ADn_C=C3=B3rdoba?= <foo at bar.com>\r\r\r\r\r\nSubject:\r\n Confirmation: ?\r\n?'

After an initial investigation into the BytesGenerator (used to flatten an EmailMessage object), here is what?s happening.

Flattening the body and attachments of the EmailMessage object works, and eventually _write_headers() is called to flatten the headers which happens entry by entry (https://github.com/python/cpython/blob/master/Lib/email/generator.py#L417-L418). Flattening a header entry is a recursive process over the parse tree of the entry, which builds the flattened and encoded final string by descending into the parse tree and encoding & concatenating the individual ?parts? (tokens of the header entry).

Given the parse tree for a header entry like "Mart?n C?rdoba <foo at bar.com>" eventually results in the correct flattened string:

    '=?utf-8?q?Mart=C3=ADn_C=C3=B3rdoba?= <foo at bar.com>\r\n'

at the bottom of the recursion for this ?Mailbox? part. The recursive callstack is then:

    _refold_parse_tree _header_value_parser.py:2687
    fold [Mailbox] _header_value_parser.py:144
    _refold_parse_tree _header_value_parser.py:2630
    fold [Address] _header_value_parser.py:144
    _refold_parse_tree _header_value_parser.py:2630
    fold [AddressList] _header_value_parser.py:144
    _refold_parse_tree _header_value_parser.py:2630
    fold [Header] _header_value_parser.py:144
    fold [_UniqueAddressHeader] headerregistry.py:258
    _fold [EmailPolicy] policy.py:205
    fold_binary [EmailPolicy] policy.py:199
    _write_headers [BytesGenerator] generator.py:418
    _write [BytesGenerator] generator.py:195

The problem now arises from the interplay of 

    # https://github.com/python/cpython/blob/master/Lib/email/_header_value_parser.py#L2629
    encoded_part = part.fold(policy=policy)[:-1] # strip nl

which strips the '\n' from the returned string, and

    # https://github.com/python/cpython/blob/master/Lib/email/_header_value_parser.py#L2686
    return policy.linesep.join(lines) + policy.linesep

which adds the policy?s line separation string linesep="\r\n" to the end of the flattened string upon unrolling the recursion.

I am not sure about a proper fix here, but considering that the linesep policy can be any string length (in this case len("\r\n") == 2) a fixed truncation of one character [:-1] seems wrong. Instead, using:

    encoded_part = part.fold(policy=policy)[:-len(policy.linesep)] # strip nl

seems to work for entries with and without Unicode characters in their display names.

----------
components: email
messages: 323686
nosy: _savage, barry, r.david.murray
priority: normal
severity: normal
status: open
title: Unicode names break email header
type: behavior
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34424>
_______________________________________

From report at bugs.python.org  Fri Aug 17 20:18:56 2018
From: report at bugs.python.org (Jason Spencer)
Date: Sat, 18 Aug 2018 00:18:56 +0000
Subject: [New-bugs-announce] [issue34425] :s formatting broken for objects
 without __format__
Message-ID: <1534551536.12.0.56676864532.issue34425@psf.upfronthosting.co.za>


New submission from Jason Spencer <jasonspencer at google.com>:

Objects with __str__ but WITHOUT __format__ are string-convertible and default-to-str formattable.  But the explicit use of '{:s}' as a format string fails to format these objects as expected.  Either it is no longer the case that '{}' and '{:s}' are equivalent format strings, or formatting for {:s} is broken.  Users would not expect the following to be true.

(Tested in 3.5.3 and 3.6.  Maybe be the same in later releases.)

Python 3.5.3 (default, Jan 19 2017, 14:11:04) 
[GCC 6.3.0 20170118] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> class Coord:
...   def __init__(self, a, b):
...     self.a=a
...     self.b=b
...   def __str__(self):
...     return '{:d}:{:d}'.format(self.a, self.b)
... 
>>> c = Coord(3,4)
>>> str(c)
'3:4'
>>> '{:s}'.format(c)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: unsupported format string passed to Coord.__format__
>>> '{}'.format(c)
'3:4'
>>> '{!s:s}'.format(c)
'3:4'
>>>

----------
components: Interpreter Core
messages: 323688
nosy: Jason Spencer
priority: normal
severity: normal
status: open
title: :s formatting broken for objects without __format__
type: behavior
versions: Python 3.5, Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34425>
_______________________________________

From report at bugs.python.org  Sat Aug 18 01:35:11 2018
From: report at bugs.python.org (Nojima Takahide)
Date: Sat, 18 Aug 2018 05:35:11 +0000
Subject: [New-bugs-announce] [issue34426] "__lltrace__"  seems to be wrong ,
 "__ltrace__" is correct in Misc/SpecialBuilds.txt
Message-ID: <1534570511.08.0.56676864532.issue34426@psf.upfronthosting.co.za>


New submission from Nojima Takahide <nozzy123nozzy at gmail.com>:

In the Section "LLTRACE" of $(Python3.6-source)/Misc/SpecialBuilds.txt , it describes variable name is "__lltrace__",however it seems to be wrong, "__ltrace__" is correct. Would someone correct this document?

----------
assignee: docs at python
components: Documentation
messages: 323692
nosy: Nojima Takahide, docs at python
priority: normal
severity: normal
status: open
title: "__lltrace__"  seems to be wrong , "__ltrace__" is correct in Misc/SpecialBuilds.txt
type: behavior
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34426>
_______________________________________

From report at bugs.python.org  Sat Aug 18 05:58:31 2018
From: report at bugs.python.org (Naris R)
Date: Sat, 18 Aug 2018 09:58:31 +0000
Subject: [New-bugs-announce] [issue34427] calling MutableSequence.extend on
 self produces infinite loop
Message-ID: <1534586311.51.0.56676864532.issue34427@psf.upfronthosting.co.za>


New submission from Naris R <nariscatboy at gmail.com>:

Example:

```
from typing import MutableSequence, TypeVar

CliffordGate = TypeVar('CliffordGate')

class QCircuit(MutableSequence[CliffordGate]):
    def __init__(self, gates):
        self.gates = list(gates)

    def __repr__(self):
        return f'{self.__class__.__name__}({self.gates})'
   
    def __getitem__(self, key):
        return self.gates[key]

    def __setitem__(self, key, item):
        self.gates[key] = item

    def __delitem__(self, key):
        del self.gates[key]

    def insert(self, key, item):
        self.gates.insert(key, item)

a = QCircuit(['H0', 'S2'])
a += a
```

----------
components: Library (Lib)
messages: 323696
nosy: Naris R
priority: normal
severity: normal
status: open
title: calling MutableSequence.extend on self produces infinite loop
type: behavior
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34427>
_______________________________________

From report at bugs.python.org  Sat Aug 18 06:29:27 2018
From: report at bugs.python.org (Luke Kenneth Casson Leighton)
Date: Sat, 18 Aug 2018 10:29:27 +0000
Subject: [New-bugs-announce] [issue34428] tokenize
Message-ID: <1534588167.35.0.56676864532.issue34428@psf.upfronthosting.co.za>


New submission from Luke Kenneth Casson Leighton <lkcl at lkcl.net>:

https://github.com/hhatto/autopep8/issues/414

the following two lines of code are not parseable by tokenize.py:

co = re.compile(
            "\(")

the combination of:
* being split on two lines
* having a backslash inside quotes
* having a bracket inside quotes

is an edge-case that _tokenize cannot cope with.

----------
components: Library (Lib)
messages: 323698
nosy: lkcl
priority: normal
severity: normal
status: open
title: tokenize
type: crash
versions: Python 3.6, Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34428>
_______________________________________

From report at bugs.python.org  Sat Aug 18 07:00:17 2018
From: report at bugs.python.org (dwich)
Date: Sat, 18 Aug 2018 11:00:17 +0000
Subject: [New-bugs-announce] [issue34429] On Windows tempfile.TemporaryFile
 behaves like NamedTemporaryFile
Message-ID: <1534590017.91.0.56676864532.issue34429@psf.upfronthosting.co.za>


New submission from dwich <michal.konvalinka at gmail.com>:

On Windows tempfile.TemporaryFile() accepts delete attribute. On Linux TemporaryFile() raises TypeError if delete attribute is used.

In tempfile.py source is the code below which means that on Windows TemporaryFile behaves like NamedTemporaryFile. I suppose the code should not be changed but the behaviour should be mentioned in documentation.

if _os.name != 'posix' or _os.sys.platform == 'cygwin':
    # On non-POSIX and Cygwin systems, assume that we cannot unlink a file
    # while it is open.
    TemporaryFile = NamedTemporaryFile

Steps to reproduce:
>>> import tempfile
>>> tf = tempfile.TemporaryFile(delete=False)

On Linux Python throws TypeError:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: TemporaryFile() got an unexpected keyword argument 'delete'

On Windows Python does not throw TypeError because on Windows TemporaryFile is in fact NamedTemporaryFile which accepts delete attribute.

Tested on all these versions: 3.7.0 64 bit, 3.6.5 32 bit, 3.5.4 32 bit, 3.4.4 32 bit, 2.7.15 32 bit 


Proposed text to tempfile.TemporaryFile documentation:

On non POSIX or Cygwin platforms TemporaryFile behaves exacly like NamedTemporaryFile including the fact that TemporaryFile accepts delete attribute and does not raise
TypeError: TemporaryFile() got an unexpected keyword argument 'delete'.

----------
assignee: docs at python
components: Documentation
messages: 323702
nosy: docs at python, dwich
priority: normal
severity: normal
status: open
title: On Windows tempfile.TemporaryFile behaves like NamedTemporaryFile
versions: Python 2.7, Python 3.4, Python 3.5, Python 3.6, Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34429>
_______________________________________

From report at bugs.python.org  Sat Aug 18 07:02:20 2018
From: report at bugs.python.org (Alfred Sawaya)
Date: Sat, 18 Aug 2018 11:02:20 +0000
Subject: [New-bugs-announce] [issue34430] Double chaining futures in
 asyncio.future.wrap_future
Message-ID: <1534590140.17.0.56676864532.issue34430@psf.upfronthosting.co.za>


New submission from Alfred Sawaya <alfred at huji.fr>:

asyncio.future.wrap_future is used to wrap a concurrent.future.Future in a asyncio.future.Future. 

The actual implementation as the following behaviours : 

 1) When the concurrent.future.Future gets a result, the asyncio.future.Future gets the same result,
 2) When the asyncio.future.Future is cancelled, the concurrent.future.Future is cancelled

I wonder why the futures synchronisation is not symmetrical ?
I propose to add the following behaviours : 

 3) When the asyncio.future.Future gets a result, the concurrent.future.Future gets the same result,
 4) When the concurrent.future.Future is cancelled, the asyncio.future.Future is cancelled

I have also posted a request pull that implements the proposed behaviours, in case of acceptation.

If there is good reasons to not implement the proposed behaviours, I would be glad to know.

Thank you !

----------
components: asyncio
messages: 323703
nosy: asvetlov, huji, yselivanov
priority: normal
pull_requests: 8280
severity: normal
status: open
title: Double chaining futures in asyncio.future.wrap_future
type: enhancement
versions: Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34430>
_______________________________________

From report at bugs.python.org  Sat Aug 18 12:03:19 2018
From: report at bugs.python.org (Jonathan Fine)
Date: Sat, 18 Aug 2018 16:03:19 +0000
Subject: [New-bugs-announce] [issue34431] Docs does not eval allows code
 object as argument
Message-ID: <1534608199.83.0.56676864532.issue34431@psf.upfronthosting.co.za>


New submission from Jonathan Fine <jfine2358 at gmail.com>:

See https://docs.python.org/3.6/library/functions.html#eval
This says the following won't happen. But it does.

Python 3.6.2 (default, Jul 29 2017, 00:00:00) 
[GCC 4.8.4] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> def fn(): print(3); return 4
... 
>>> eval(fn.__code__)
3
4

Compare with https://docs.python.org/3.6/library/functions.html#exec

Search for eval in title of open issues bring up related

#22057 (The doc say all globals are copied on eval(), but only __builtins__ is copied)
#25810 (Python 3 documentation for eval is incorrect)

----------
assignee: docs at python
components: Documentation
messages: 323716
nosy: docs at python, jfine2358
priority: normal
severity: normal
status: open
title: Docs does not eval allows code object as argument
versions: Python 2.7, Python 3.4, Python 3.5, Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34431>
_______________________________________

From report at bugs.python.org  Sat Aug 18 13:05:23 2018
From: report at bugs.python.org (=?utf-8?q?Andr=C3=A9s_Delfino?=)
Date: Sat, 18 Aug 2018 17:05:23 +0000
Subject: [New-bugs-announce] [issue34432] doc Mention complex and
 decimal.Decimal on str.format note about locales
Message-ID: <1534611923.59.0.56676864532.issue34432@psf.upfronthosting.co.za>


New submission from Andr?s Delfino <adelfino at gmail.com>:

On str.format method, the note about how locales are dealt with reads:

"When formatting a number (int, float, float and subclasses)..."

PR removes the second float, and mentions complex and decimal.Decimal.

The bug which introduced this note is: #31900

----------
assignee: docs at python
components: Documentation
messages: 323718
nosy: adelfino, docs at python
priority: normal
severity: normal
status: open
title: doc Mention complex and decimal.Decimal on str.format note about locales
type: enhancement
versions: Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34432>
_______________________________________

From report at bugs.python.org  Sun Aug 19 09:13:40 2018
From: report at bugs.python.org (tzongw)
Date: Sun, 19 Aug 2018 13:13:40 +0000
Subject: [New-bugs-announce] [issue34433] cancel all other pending child
 futures
Message-ID: <1534684420.56.0.56676864532.issue34433@psf.upfronthosting.co.za>


New submission from tzongw <tzongw at 163.com>:

In `tasks.gather`, when a child future throws an exception and `return_exceptions` is False, outer future will call `set_exception` while other child futures is still running. In this case, outer future call `_GatheringFuture.cancel' first to cancel all other pending child futures for efficiency.

----------
components: asyncio
messages: 323755
nosy: asvetlov, tzongw, yselivanov
priority: normal
severity: normal
status: open
title: cancel all other pending child futures
type: performance
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34433>
_______________________________________

From report at bugs.python.org  Sun Aug 19 09:37:10 2018
From: report at bugs.python.org (Andy Maier)
Date: Sun, 19 Aug 2018 13:37:10 +0000
Subject: [New-bugs-announce] [issue34434] Removal of kwargs for int() etc
 not described as change
Message-ID: <1534685830.81.0.56676864532.issue34434@psf.upfronthosting.co.za>


New submission from Andy Maier <andreas.r.maier at gmx.de>:

Python 3.7 removed support for passing the argument to the built-in functions int(), bool(), float(), list() and tuple() as a keyword argument.

This change is described in the "What's New" for 3.7 (https://docs.python.org/3/whatsnew/3.7.html) in section "API and Feature Removals":

    Functions bool(), float(), list() and tuple() no longer take
    keyword arguments. The first argument of int() can now be passed
    only as positional argument.

The issue is that this change is not described in the documentation of these built-in functions.

----------
messages: 323756
nosy: andymaier
priority: normal
severity: normal
status: open
title: Removal of kwargs for int() etc not described as change
type: enhancement
versions: Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34434>
_______________________________________

From report at bugs.python.org  Sun Aug 19 13:42:35 2018
From: report at bugs.python.org (Alexey Izbyshev)
Date: Sun, 19 Aug 2018 17:42:35 +0000
Subject: [New-bugs-announce] [issue34435] Missing NULL check in
 unicode_encode_ucs1()
Message-ID: <1534700555.39.0.56676864532.issue34435@psf.upfronthosting.co.za>


New submission from Alexey Izbyshev <izbyshev at ispras.ru>:

A possible NULL return on out-of-memory condition at Objects/unicodeobject.c:6835 is not handled.

Reported by Svace static analyzer.

----------
components: Interpreter Core
messages: 323762
nosy: inada.naoki, izbyshev, serhiy.storchaka, vstinner
priority: normal
severity: normal
status: open
title: Missing NULL check in unicode_encode_ucs1()
versions: Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34435>
_______________________________________

From report at bugs.python.org  Sun Aug 19 14:51:43 2018
From: report at bugs.python.org (Alexey Izbyshev)
Date: Sun, 19 Aug 2018 18:51:43 +0000
Subject: [New-bugs-announce] [issue34436] Overallocation is never disabled
 in _PyBytes_FormatEx()
Message-ID: <1534704703.42.0.56676864532.issue34436@psf.upfronthosting.co.za>


New submission from Alexey Izbyshev <izbyshev at ispras.ru>:

The condition for disabling overallocation at 225b055/Objects/bytesobject.c:822 is always false.

Reported by Svace static analyzer.

----------
components: Interpreter Core
messages: 323763
nosy: inada.naoki, izbyshev, serhiy.storchaka, vstinner
priority: normal
severity: normal
status: open
title: Overallocation is never disabled in _PyBytes_FormatEx()
type: performance
versions: Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34436>
_______________________________________

From report at bugs.python.org  Sun Aug 19 19:24:40 2018
From: report at bugs.python.org (Nathan Benson)
Date: Sun, 19 Aug 2018 23:24:40 +0000
Subject: [New-bugs-announce] [issue34437] print statement using \x results
 in improper and extra bytes
Message-ID: <1534721080.82.0.56676864532.issue34437@psf.upfronthosting.co.za>


New submission from Nathan Benson <jfa at packetdamage.com>:

While writing some shellcode I uncovered an unusual bug where Python 3 seems to print out incorrect (and extra) hex bytes using the print statement with \x. Needless to say I was pulling my hair out trying to figure out why my shellcode wasn?t working.  Python 2 behaves as expected.

I haven't tested the latest version of Python 3, but all the versions prior to that seem to have the bug.  I?ve been able to reproduce the bug in Ubuntu Linux and on my Mac.

An example printing "\xfd\x84\x04\x08" I expect to get back "fd 84 04 08", but Python 3 seems to add bytes beginning with c2 and c3 and tosses in random bytes.

For the purpose of these demonstrations:

  Akame:~ jfa$ python2 --version
  Python 2.7.15

  Akame:~ jfa$ python3 --version
  Python 3.7.0


Here is Python 2 operating as expected:

Akame:~ jfa$ python2 -c 'print("\xfd\x84\x04\x08")' | hexdump -C
00000000  fd 84 04 08 0a                                    |.....|
00000005


Here is Python 3 with the exact same print statement:

Akame:~ jfa$ python3 -c 'print("\xfd\x84\x04\x08")' | hexdump -C
00000000  c3 bd c2 84 04 08 0a                              |.......|
00000007

There are 6 bytes not 4 and where did the c3, bd, and c2 come from?

Playing around with it a little bit more it seems like the problem arises when you are printing bytes that start with a-f or 8 or 9:

Here is a-f:

Akame:~ jfa$ for b in {a..f}; do echo "\x${b}0"; python3 -c "print(\"\x${b}0\")" | hexdump -C; done
\xa0
00000000  c2 a0 0a                                          |...|
00000003
\xb0
00000000  c2 b0 0a                                          |...|
00000003
\xc0
00000000  c3 80 0a                                          |...|
00000003
\xd0
00000000  c3 90 0a                                          |...|
00000003
\xe0
00000000  c3 a0 0a                                          |...|
00000003
\xf0
00000000  c3 b0 0a                                          |...|
00000003


Here is 0-9 (notice everything is fine until 8):

Akame:~ jfa$ for b in {0..9}; do echo "\x${b}0"; python3 -c "print(\"\x${b}0\")" | hexdump -C; done
\x00
00000000  00 0a                                             |..|
00000002
\x10
00000000  10 0a                                             |..|
00000002
\x20
00000000  20 0a                                             | .|
00000002
\x30
00000000  30 0a                                             |0.|
00000002
\x40
00000000  40 0a                                             |@.|
00000002
\x50
00000000  50 0a                                             |P.|
00000002
\x60
00000000  60 0a                                             |`.|
00000002
\x70
00000000  70 0a                                             |p.|
00000002
\x80
00000000  c2 80 0a                                          |...|
00000003
\x90
00000000  c2 90 0a                                          |...|
00000003



Here are the same tests with Python 2:

Akame:~ jfa$ for b in {a..f}; do echo "\x${b}0"; python2 -c "print(\"\x${b}0\")" | hexdump -C; done
\xa0
00000000  a0 0a                                             |..|
00000002
\xb0
00000000  b0 0a                                             |..|
00000002
\xc0
00000000  c0 0a                                             |..|
00000002
\xd0
00000000  d0 0a                                             |..|
00000002
\xe0
00000000  e0 0a                                             |..|
00000002
\xf0
00000000  f0 0a                                             |..|
00000002


Akame:~ jfa$ for b in {0..9}; do echo "\x${b}0"; python2 -c "print(\"\x${b}0\")" | hexdump -C; done
\x00
00000000  00 0a                                             |..|
00000002
\x10
00000000  10 0a                                             |..|
00000002
\x20
00000000  20 0a                                             | .|
00000002
\x30
00000000  30 0a                                             |0.|
00000002
\x40
00000000  40 0a                                             |@.|
00000002
\x50
00000000  50 0a                                             |P.|
00000002
\x60
00000000  60 0a                                             |`.|
00000002
\x70
00000000  70 0a                                             |p.|
00000002
\x80
00000000  80 0a                                             |..|
00000002
\x90
00000000  90 0a                                             |..|
00000002


As you can see Python 2 works as expected and Python 3, when printing using \x[a-f08], seem to cause the byte to be replaced with a c2 or c3 and another byte of data.

----------
messages: 323773
nosy: Nathan Benson
priority: normal
severity: normal
status: open
title: print statement using \x results in improper and extra bytes
type: behavior
versions: Python 3.4, Python 3.5, Python 3.6, Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34437>
_______________________________________

From report at bugs.python.org  Sun Aug 19 23:25:35 2018
From: report at bugs.python.org (David)
Date: Mon, 20 Aug 2018 03:25:35 +0000
Subject: [New-bugs-announce] [issue34438] do_handshake stuck in ssl.py
Message-ID: <1534735535.4.0.56676864532.issue34438@psf.upfronthosting.co.za>


New submission from David <davidwho at gmail.com>:

Sometimes, rarely, API calls will get stuck indefinitely on ssl.py. Stack trace:

  response = json.loads(urllib.request.urlopen(req).read())
  File "/usr/lib/python3.6/urllib/request.py", line 223, in urlopen
    return opener.open(url, data, timeout)
  File "/usr/lib/python3.6/urllib/request.py", line 526, in open
    response = self._open(req, data)
  File "/usr/lib/python3.6/urllib/request.py", line 544, in _open
    '_open', req)
  File "/usr/lib/python3.6/urllib/request.py", line 504, in _call_chain
    result = func(*args)
  File "/usr/lib/python3.6/urllib/request.py", line 1361, in https_open
    context=self._context, check_hostname=self._check_hostname)
  File "/usr/lib/python3.6/urllib/request.py", line 1318, in do_open
    encode_chunked=req.has_header('Transfer-encoding'))
  File "/usr/lib/python3.6/http/client.py", line 1239, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/usr/lib/python3.6/http/client.py", line 1285, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/usr/lib/python3.6/http/client.py", line 1234, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/usr/lib/python3.6/http/client.py", line 1026, in _send_output
    self.send(msg)
  File "/usr/lib/python3.6/http/client.py", line 964, in send
    self.connect()
  File "/usr/lib/python3.6/http/client.py", line 1400, in connect
    server_hostname=server_hostname)
  File "/usr/lib/python3.6/ssl.py", line 407, in wrap_socket
    _context=self, _session=session)
  File "/usr/lib/python3.6/ssl.py", line 814, in __init__
    self.do_handshake()
  File "/usr/lib/python3.6/ssl.py", line 1068, in do_handshake
    self._sslobj.do_handshake()
  File "/usr/lib/python3.6/ssl.py", line 689, in do_handshake
    self._sslobj.do_handshake()


Almost identical issue as https://stackoverflow.com/questions/19938593/web-app-hangs-for-several-hours-in-ssl-py-at-self-sslobj-do-handshake
But using python 3.6

----------
assignee: christian.heimes
components: SSL
messages: 323775
nosy: christian.heimes, david987
priority: normal
severity: normal
status: open
title: do_handshake stuck in ssl.py
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34438>
_______________________________________

From report at bugs.python.org  Mon Aug 20 06:20:50 2018
From: report at bugs.python.org (Tomer Keren)
Date: Mon, 20 Aug 2018 10:20:50 +0000
Subject: [New-bugs-announce] [issue34439] Expose venv --prompt value to an
 environment value
Message-ID: <1534760450.03.0.56676864532.issue34439@psf.upfronthosting.co.za>


New submission from Tomer Keren <tomer.keren.dev at gmail.com>:

In the same way the $VIRTUAL_ENV variable tells the virtual environment directory, A variable such as $VENV_NAME or $VENV_PROMPT should tell the value given to the venv `--prompt` option (Introduced in Issue22829).

An individual could override the EnvBuilder class like the docs say, but that wouldn't insure any user will have the variable set. This is crucial for usage in custom libraries that want to display the given prompt.

----------
components: Library (Lib)
messages: 323782
nosy: Tomer
priority: normal
severity: normal
status: open
title: Expose venv --prompt value to an environment value
type: behavior
versions: Python 3.5, Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34439>
_______________________________________

From report at bugs.python.org  Mon Aug 20 07:22:01 2018
From: report at bugs.python.org (Lennart Grahl)
Date: Mon, 20 Aug 2018 11:22:01 +0000
Subject: [New-bugs-announce] [issue34440] Certificate verify failed (works
 fine in 3.6)
Message-ID: <1534764121.45.0.56676864532.issue34440@psf.upfronthosting.co.za>


New submission from Lennart Grahl <lennart.grahl at gmail.com>:

When running the attached script with the attached cert, Python 3.7 raises an exception (see https://paste.pound-python.org/show/VLr84Yn2Fnz6RSKEq3ui/). In Python 3.6, the certificate is being accepted.

I don't see anything wrong with the self-signed certificate.

You can (hopefully) reproduce this by running minimal_server.py

----------
assignee: christian.heimes
components: SSL
files: minimal_server.zip
messages: 323783
nosy: Lennart Grahl, christian.heimes
priority: normal
severity: normal
status: open
title: Certificate verify failed (works fine in 3.6)
versions: Python 3.7
Added file: https://bugs.python.org/file47754/minimal_server.zip

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34440>
_______________________________________

From report at bugs.python.org  Mon Aug 20 09:41:11 2018
From: report at bugs.python.org (Alexey Izbyshev)
Date: Mon, 20 Aug 2018 13:41:11 +0000
Subject: [New-bugs-announce] [issue34441] NULL dereference when issubclass()
 is called on a class with bogus __subclasses__
Message-ID: <1534772471.92.0.56676864532.issue34441@psf.upfronthosting.co.za>


New submission from Alexey Izbyshev <izbyshev at ispras.ru>:

>>> from abc import ABCMeta
>>> class S(metaclass=ABCMeta):
...   __subclasses__ = None
... 
>>> issubclass(int, S)
Segmentation fault (core dumped)

This is the result of missing NULL check for 'subclasses' in _abc__abc_subclasscheck_impl (Modules/_abc.c):

    /* 6. Check if it's a subclass of a subclass (recursive). */
    subclasses = PyObject_CallMethod(self, "__subclasses__", NULL);
    if (!PyList_Check(subclasses)) {
        PyErr_SetString(PyExc_TypeError, "__subclasses__() must return a list");
        goto end;
    }

Reported by Svace static analyzer.

----------
components: Extension Modules
messages: 323789
nosy: inada.naoki, izbyshev, levkivskyi, serhiy.storchaka
priority: normal
severity: normal
status: open
title: NULL dereference when issubclass() is called on a class with bogus __subclasses__
type: crash
versions: Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34441>
_______________________________________

From report at bugs.python.org  Mon Aug 20 10:40:14 2018
From: report at bugs.python.org (Stephen Kelly)
Date: Mon, 20 Aug 2018 14:40:14 +0000
Subject: [New-bugs-announce] [issue34442] zlib module not built on windows
Message-ID: <1534776014.27.0.56676864532.issue34442@psf.upfronthosting.co.za>


New submission from Stephen Kelly <steveire at gmail.com>:

I tried to build python 3.7.0 from source.

I ran the PCBuild/build.bat script. That downloaded zlib to the external/ directory.

However, it does not seem to be built. When running I get:

python\3.7.0\lib\zipfile.py", line 646, in _check_compression                                                                                                                       "Compression requires the (missing) zlib module")                                                                                                                                                                RuntimeError: Compression requires the (missing) zlib module  

I examined PCBuild/pcbuild.proj. It makes no mention of zlib, though it mentions other externals.

I notice that Python 3.6.1 contained zlib in Modules/zlib instead of coming from external/.

Presumably that source was moved, but the Windows-related scripts were not updated.

----------
components: Extension Modules
messages: 323791
nosy: steveire
priority: normal
severity: normal
status: open
title: zlib module not built on windows
type: compile error
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34442>
_______________________________________

From report at bugs.python.org  Mon Aug 20 12:34:28 2018
From: report at bugs.python.org (=?utf-8?q?Walter_D=C3=B6rwald?=)
Date: Mon, 20 Aug 2018 16:34:28 +0000
Subject: [New-bugs-announce] [issue34443] enum repr should use __qualname__
Message-ID: <1534782868.03.0.56676864532.issue34443@psf.upfronthosting.co.za>


New submission from Walter D?rwald <walter at livinglogic.de>:

The __repr__ output of an enum class should use __qualname__ instead of __name__. The following example shows the problem:

import enum

class X:
   class I:
      pass

class Y:
   class I(enum.Enum):
      pass

print(X.I)
print(Y.I)

This prints:

<class '__main__.X.I'>
<enum 'I'>

I would have expected it to print

<class '__main__.X.I'>
<enum 'Y.I'>

or even for maximum consistency

<class '__main__.X.I'>
<enum '__main__.Y.I'>

----------
components: Library (Lib)
messages: 323799
nosy: doerwalter
priority: normal
severity: normal
status: open
title: enum repr should use __qualname__
type: enhancement

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34443>
_______________________________________

From report at bugs.python.org  Mon Aug 20 12:57:16 2018
From: report at bugs.python.org (daniel hahler)
Date: Mon, 20 Aug 2018 16:57:16 +0000
Subject: [New-bugs-announce] [issue34444] Module's __file__ should be
 absolute always ("." in sys.path)
Message-ID: <1534784236.56.0.56676864532.issue34444@psf.upfronthosting.co.za>


New submission from daniel hahler <python-bugs at thequod.de>:

With "." in sys.path the "__file__" attribute will be a relative path, and therefore cannot be used after "chdir".

This likely affects relative paths in general, but have not tested it.

```
import os
import sys

sys.path.insert(0, '.')

# Importing it before chdir already causes failure.
import imported

os.chdir('/')
print(imported.__file__)  # ./imported.py
assert imported.__file__ == os.path.abspath(imported.__file__)
```

It works for "" in sys.path (https://bugs.python.org/issue18416).

----------
components: Interpreter Core
messages: 323800
nosy: blueyed
priority: normal
severity: normal
status: open
title: Module's __file__ should be absolute always ("." in sys.path)
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34444>
_______________________________________

From report at bugs.python.org  Mon Aug 20 13:30:39 2018
From: report at bugs.python.org (Stefan Tjarks)
Date: Mon, 20 Aug 2018 17:30:39 +0000
Subject: [New-bugs-announce] [issue34445] asyncio support in doctests
Message-ID: <1534786239.59.0.56676864532.issue34445@psf.upfronthosting.co.za>


New submission from Stefan Tjarks <stefan.tjarks at eventbase.com>:

When writing a docstring for an async function I wrote a doctest for it.

```
async def hello_world():
    """
    Will great the world with a friendly hello.

    >>> await hello_world()
    "hello world"
    """
    return "hello world"
```

I kind of expected an error that no event loop is running but actually get a SyntaxError.

```
**********************************************************************
File "asyncdoctest.py", line 5, in __main__.hello_world
Failed example:
    await hello_world()
Exception raised:
    Traceback (most recent call last):
      File "/usr/lib/python3.6/doctest.py", line 1330, in __run
        compileflags, 1), test.globs)
      File "<doctest __main__.hello_world[0]>", line 1
        await hello_world()
                        ^
    SyntaxError: invalid syntax
**********************************************************************
1 items had failures:
   1 of   1 in __main__.hello_world
***Test Failed*** 1 failures.
```

Is the SyntaxError intentional or does doctest simply not support asyncio syntax?

----------
components: Library (Lib), asyncio
files: asyncdoctest.py
messages: 323803
nosy: Stefan Tjarks, asvetlov, yselivanov
priority: normal
severity: normal
status: open
title: asyncio support in doctests
type: behavior
versions: Python 3.7
Added file: https://bugs.python.org/file47756/asyncdoctest.py

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34445>
_______________________________________

From report at bugs.python.org  Mon Aug 20 15:47:35 2018
From: report at bugs.python.org (jcc2220)
Date: Mon, 20 Aug 2018 19:47:35 +0000
Subject: [New-bugs-announce] [issue34446] ambiguous _max_size parameter in
 SpooledTemporaryFile
Message-ID: <1534794455.51.0.56676864532.issue34446@psf.upfronthosting.co.za>


New submission from jcc2220 <jcc2220 at columbia.edu>:

When _max_size is set to 0, the _check method will never call rollover, as the conditional for a rollover is never satisfied.  However, in the truncate method, the _max_size is not checked against 0, and so a rollover could be called when it is 0.  This is more of an issue of consistency - should 0 mean that it will never rollover?  If so, truncate should be modified.  Should 0 be interpreted as 'always rollover'?  If so, the _check should be modified.  Personally, I'd rather have something like -1 mean never, 0 mean always, and >0 mean only when greater than specified size.


John

----------
components: Library (Lib)
messages: 323809
nosy: jcc2220
priority: normal
severity: normal
status: open
title: ambiguous _max_size parameter in SpooledTemporaryFile
type: behavior
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34446>
_______________________________________

From report at bugs.python.org  Tue Aug 21 00:49:03 2018
From: report at bugs.python.org (Andrew Barnert)
Date: Tue, 21 Aug 2018 04:49:03 +0000
Subject: [New-bugs-announce] [issue34447] ttk.TreeView (and maybe other
 functions) is overzealous in converting item values to ints
Message-ID: <1534826943.57.0.56676864532.issue34447@psf.upfronthosting.co.za>


New submission from Andrew Barnert <abarnert at yahoo.com>:

See this StackOverflow question for a repro case: https://stackoverflow.com/questions/51941260/

If you store a string that looks like an int in a TreeView's values, then call widget.item(row), the dict's 'values' value has that string converted to an int.

This seems reasonable, because it's legal to store an int in a TreeView, and if tkinter didn't convert the values, you'd store 123 and get back '123'.

However, I think it should only be converting strings that could be a valid Tcl int. If you store the int 123_456, Tcl of course gives you back '123456', not '123_456', so there's no reason for tkinter to convert '123_456' to an int. But it does, because ttk._tclobj_to_py just does a try: return int(s).

---

Maeanwhile, if you instead call widget.item(row, options='values'), the string is left alone. 

At first glance, that made sense. But looking into what tkinter is getting back from Tcl, there's really no difference here. The tuple ('180518-23', '23/06/18') is no more or less conversion-worthy than the same tuple inside the list (<option object: '-text'>, '', <option object: '-image'>, '', <option object: '-values'>, ('180518-23', '23/06/18'), <option object: '-open'>, 0, <option object: '-tags'>, ''). It's just that ttk._val_or_dict doesn't call _tclobj_to_py on the value when it gets back a key-value pair, and I don't see why it shouldn't.

However, that turns out to be a handy workaround for the Stack Overflow user who ran into this problem, so maybe it's better not to change that.

----------
components: Tkinter
messages: 323819
nosy: abarnert
priority: normal
severity: normal
status: open
title: ttk.TreeView (and maybe other functions) is overzealous in converting item values to ints
versions: Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34447>
_______________________________________

From report at bugs.python.org  Tue Aug 21 04:50:26 2018
From: report at bugs.python.org (Michael Osipov)
Date: Tue, 21 Aug 2018 08:50:26 +0000
Subject: [New-bugs-announce] [issue34448] incomplete/missing message from
 configure.ac for usable wchar_t
Message-ID: <1534841426.48.0.56676864532.issue34448@psf.upfronthosting.co.za>


New submission from Michael Osipov <1983-01-06 at gmx.net>:

If the wchar_t is usable on a system we get the following:

> checking for UCS-4 tcl... no
> checking whether wchar_t is signed... no
> yes
> checking whether byte ordering is bigendian... yes
> checking ABIFLAGS... dm
> checking SOABI... cpython-38dm

Yes -- that is a simple 'yes'. Dangling.

A PR is in preparation.

----------
components: Build
messages: 323826
nosy: michael-o
priority: normal
severity: normal
status: open
title: incomplete/missing message from configure.ac for usable wchar_t
type: enhancement
versions: Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34448>
_______________________________________

From report at bugs.python.org  Tue Aug 21 05:06:55 2018
From: report at bugs.python.org (Michael Osipov)
Date: Tue, 21 Aug 2018 09:06:55 +0000
Subject: [New-bugs-announce] [issue34449] HP aCC complains about invalid
 -fPIC on HP-UX
Message-ID: <1534842415.32.0.56676864532.issue34449@psf.upfronthosting.co.za>


New submission from Michael Osipov <1983-01-06 at gmx.net>:

The compiler (HP aCC) says:

> /opt/aCC/bin/cc -Ae -b -L/usr/local/lib/hpux32 build/temp.hp-ux-B.11.31-ia64-3.8-pydebug/var/osipovmi/cpython/Modules/_ctypes/_ctypes.o build/temp.hp-ux-B.11.31-ia64-3.8-pydebug/var/osipovmi/cpython/Modules/_ctypes/callbacks.o build/temp.hp-ux-B.11.31-ia64-3.8-pydebug/var/osipovmi/cpython/Modules/_ctypes/callproc.o build/temp.hp-ux-B.11.31-ia64-3.8-pydebug/var/osipovmi/cpython/Modules/_ctypes/stgdict.o build/temp.hp-ux-B.11.31-ia64-3.8-pydebug/var/osipovmi/cpython/Modules/_ctypes/cfield.o -L/usr/local/lib/hpux64 -L/usr/lib/hpux32 -L/usr/lib/hpux64 -L/usr/lib -L/lib -L/usr/local/lib/hpux32 -L/usr/local/lib -lffi -ldl -o build/lib.hp-ux-B.11.31-ia64-3.8-pydebug/_ctypes.so -fPIC
> cc: warning 901: unknown option: `-fPIC': use +help for online documentation.

On clang or GCC this is a compile time option, never a link option. For HP aCC objects must be compiled with +z -- which actually happens -- and linked with +b to a shared library -- which also happens here --. One can safely drop this switch from setup.py.

----------
components: Build
messages: 323827
nosy: michael-o
priority: normal
severity: normal
status: open
title: HP aCC complains about invalid -fPIC on HP-UX
type: behavior
versions: Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34449>
_______________________________________

From report at bugs.python.org  Tue Aug 21 06:58:56 2018
From: report at bugs.python.org (iMath)
Date: Tue, 21 Aug 2018 10:58:56 +0000
Subject: [New-bugs-announce] [issue34450] improve  shutil.make_archive
Message-ID: <1534849136.61.0.56676864532.issue34450@psf.upfronthosting.co.za>


New submission from iMath <redstone-cold at 163.com>:

The current behavior of shutil.make_archive caused many issues , the problem is mainly on the extracted archive directory hierarchy. These are the proofs:
https://stackoverflow.com/questions/51914467/directory-hierarchy-issue-when-using-shutil-make-archive

https://stackoverflow.com/questions/32640053/compressing-directory-using-shutil-make-archive-while-preserving-directory-str

https://stackoverflow.com/questions/41624800/shutil-make-archive-issue-dont-want-directories-included-in-zip-file

https://stackoverflow.com/questions/50156657/unexpected-file-using-shutil-make-archive-to-compress-file
For example , if I want to create a zip archive of the pip package (path specified by pip.__path__[0])), and need a directory named pip to hold the  files and folders which originally reside in the pip package when unpacking the archive, then I need give root_dirparameter of shutil.make_archive the parent directory of the pippackage path (root_dir=Path(pip.__path__[0]).parent), and then the base_dir parameter the final path component of the pip package path(base_dir=Path(pip.__path__[0]).name) , so it is os.path.join(root_dir, base_dir) that specified the directory to archive , so weird !!!
I suggest to change shutil.make_archive(base_name, format[, root_dir[, base_dir]]) to shutil.make_archive(base_name, format[, archived_dir[, archive_prfix]]) where archived_dirdenotes the path to be archived and archive_prfix denotes the common prefix of all files and directories in the archive (it is just a path component and we shouldn?t assume the existence of it on the file system).

If the current behavior of shutil.make_archive won?t be changed , I?d suggest improve its doc, because  so many people couldn?t grasp the use of shutil.make_archive even consulting the doc , these are the proofs:
https://stackoverflow.com/questions/45245079/python-how-to-use-shutil-make-archive
https://stackoverflow.com/questions/30049201/how-to-compress-a-file-with-shutil-make-archive-in-python

----------
messages: 323829
nosy: redstone-cold
priority: normal
severity: normal
status: open
title: improve  shutil.make_archive

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34450>
_______________________________________

From report at bugs.python.org  Tue Aug 21 14:50:18 2018
From: report at bugs.python.org (Jonathan Fine)
Date: Tue, 21 Aug 2018 18:50:18 +0000
Subject: [New-bugs-announce] [issue34451] docs: tutorial/introduction
 doesn't mention toggle of prompts
Message-ID: <1534877418.64.0.56676864532.issue34451@psf.upfronthosting.co.za>


New submission from Jonathan Fine <jfine2358 at gmail.com>:

Interactive code examples need the prompt to be stripped, before copy-and-paste. This is explained in https://docs.python.org/3/tutorial/introduction.html

But this page does not tell us about the [>>>] prompt-toggle at top of each interactive code example. This caused a user error, reported in  https://mail.python.org/pipermail/python-ideas/2018-August/052869.html.

The [>>>] toggle isn't in the Python 2.7 docs, so nothing to fix there!

----------
assignee: docs at python
components: Documentation
messages: 323839
nosy: docs at python, jfine2358
priority: normal
severity: normal
status: open
title: docs: tutorial/introduction doesn't mention toggle of prompts
versions: Python 3.4, Python 3.5, Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34451>
_______________________________________

From report at bugs.python.org  Tue Aug 21 15:01:13 2018
From: report at bugs.python.org (Jonathan Fine)
Date: Tue, 21 Aug 2018 19:01:13 +0000
Subject: [New-bugs-announce] [issue34452] docs: help text for [>>>] toggle
 always in English
Message-ID: <1534878073.37.0.56676864532.issue34452@psf.upfronthosting.co.za>


New submission from Jonathan Fine <jfine2358 at gmail.com>:

As subject. Background information in #34451.

See https://docs.python.org/fr/3/tutorial/introduction.html

The hover help text for the [>>>] is in English, not French. Korean and Japanese documentation has the same problem.

----------
assignee: docs at python
components: Documentation
messages: 323840
nosy: docs at python, jfine2358
priority: normal
severity: normal
status: open
title: docs: help text for [>>>] toggle always in English
type: behavior
versions: Python 3.4, Python 3.5, Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34452>
_______________________________________

From report at bugs.python.org  Tue Aug 21 16:22:58 2018
From: report at bugs.python.org (Luke Kenneth Casson Leighton)
Date: Tue, 21 Aug 2018 20:22:58 +0000
Subject: [New-bugs-announce] [issue34453] ipaddress module accepts some
 strange IPv6 addresses that it shouldn't
Message-ID: <1534882978.68.0.56676864532.issue34453@psf.upfronthosting.co.za>


New submission from Luke Kenneth Casson Leighton <lkcl at lkcl.net>:

adding some unit tests to some code being written,
searched randomly on the internet for an IPv6 test
suite and found one in php *shudder*
# https://github.com/gws/ipv6-address-test/blob/master/Tests/Ipv6TestCase.php

converted it to python, and was surprised to find that
there are two cases which the ipaddress.IPv6Address
module accepts, that it should not!

attached is a quick-and-dirty hacked together unit test
which shows the issue.  the full suite may be found
here (over 400 ipv6 addresses to test):

https://github.com/threefoldtech/jumpscale_core/blob/development_dynamic/tests/jumpscale_test/test10_j_data_types.py

----------
components: Tests
files: ipv6test.py
messages: 323843
nosy: lkcl
priority: normal
severity: normal
status: open
title: ipaddress module accepts some strange IPv6 addresses that it shouldn't
type: behavior
versions: Python 3.5, Python 3.6
Added file: https://bugs.python.org/file47757/ipv6test.py

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34453>
_______________________________________

From report at bugs.python.org  Tue Aug 21 17:38:29 2018
From: report at bugs.python.org (Alexey Izbyshev)
Date: Tue, 21 Aug 2018 21:38:29 +0000
Subject: [New-bugs-announce] [issue34454] datetime: NULL dereference in
 fromisoformat() on PyUnicode_AsUTF8AndSize() failure
Message-ID: <1534887509.78.0.56676864532.issue34454@psf.upfronthosting.co.za>


New submission from Alexey Izbyshev <izbyshev at ispras.ru>:

A failure of PyUnicode_AsUTF8AndSize() in various fromisoformat() functions in Modules/_datetimemodule.c leads to NULL dereference due to the missing check, e.g.:

>>> from datetime import date
>>> date.fromisoformat('\ud800')
Segmentation fault (core dumped)

This is similar to msg123474. The missing NULL check was reported by Svace static analyzer.

While preparing tests for this issue, I've discovered a deeper problem. The C datetime implementation uses PyUnicode_AsUTF8AndSize() in several places, making some functions reject strings containing surrogate code points (0xD800 - 0xDFFF) since they can't be encoded in UTF-8. On the other hand, the pure-Python datetime implementation doesn't have this restriction. For example:

>>> import sys
>>> sys.modules['_datetime'] = None # block C implementation
>>> from datetime import time
>>> time().strftime('\ud800')
'\ud800'
>>> del sys.modules['datetime']
>>> del sys.modules['_datetime']
>>> from datetime import time
>>> time().strftime('\ud800')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
UnicodeEncodeError: 'utf-8' codec can't encode character '\ud800' in position 0: surrogates not allowed

My PR (coming soon) doesn't address this difference but focuses on fixing the immediate problem instead. Suggestions are appreciated.

----------
components: Extension Modules
messages: 323844
nosy: belopolsky, izbyshev, p-ganssle, serhiy.storchaka
priority: normal
severity: normal
status: open
title: datetime: NULL dereference in fromisoformat() on PyUnicode_AsUTF8AndSize() failure
type: crash
versions: Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34454>
_______________________________________

From report at bugs.python.org  Tue Aug 21 18:20:10 2018
From: report at bugs.python.org (iPodClassic)
Date: Tue, 21 Aug 2018 22:20:10 +0000
Subject: [New-bugs-announce] [issue34455] IDLE crashing when pressing
 Command + ^ (OSX)
Message-ID: <1534890010.44.0.56676864532.issue34455@psf.upfronthosting.co.za>


New submission from iPodClassic <marianogedisman at gmail.com>:

IDLE crashes when pressing Command (?) + ^ 

I have attached the crash report just in case.

----------
assignee: terry.reedy
components: IDLE
files: crash report idle.txt
messages: 323848
nosy: iPodClassic, terry.reedy
priority: normal
severity: normal
status: open
title: IDLE crashing when pressing Command + ^  (OSX)
type: crash
versions: Python 3.7
Added file: https://bugs.python.org/file47758/crash report idle.txt

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34455>
_______________________________________

From report at bugs.python.org  Tue Aug 21 19:15:26 2018
From: report at bugs.python.org (Alexey Izbyshev)
Date: Tue, 21 Aug 2018 23:15:26 +0000
Subject: [New-bugs-announce] [issue34456] pickle: Missing NULL check in
 save_global()
Message-ID: <1534893326.11.0.56676864532.issue34456@psf.upfronthosting.co.za>


New submission from Alexey Izbyshev <izbyshev at ispras.ru>:

The return value of Py_BuildValue() is dereferenced without checking at https://github.com/python/cpython/blob/ef8861c112ed1dac9351958c121bc24ca4ecdb08/Modules/_pickle.c#L3457

Reported by Svace static analyzer.

----------
components: Extension Modules
messages: 323853
nosy: izbyshev, serhiy.storchaka
priority: normal
severity: normal
status: open
title: pickle: Missing NULL check in save_global()
type: behavior
versions: Python 3.5, Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34456>
_______________________________________

From report at bugs.python.org  Tue Aug 21 19:37:32 2018
From: report at bugs.python.org (Alexey Izbyshev)
Date: Tue, 21 Aug 2018 23:37:32 +0000
Subject: [New-bugs-announce] [issue34457] Missing NULL check in
 alias_for_import_name() from Python/ast.c
Message-ID: <1534894652.28.0.56676864532.issue34457@psf.upfronthosting.co.za>


New submission from Alexey Izbyshev <izbyshev at ispras.ru>:

The return value of PyUnicode_InternFromString() is dereferenced without checking at https://github.com/python/cpython/blob/ef8861c112ed1dac9351958c121bc24ca4ecdb08/Python/ast.c#L3256

Reported by Svace static analyzer.

----------
components: Interpreter Core
messages: 323854
nosy: izbyshev, serhiy.storchaka
priority: normal
severity: normal
status: open
title: Missing NULL check in alias_for_import_name() from Python/ast.c
type: behavior
versions: Python 2.7, Python 3.4, Python 3.5, Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34457>
_______________________________________

From report at bugs.python.org  Tue Aug 21 20:13:51 2018
From: report at bugs.python.org (Victor Porton)
Date: Wed, 22 Aug 2018 00:13:51 +0000
Subject: [New-bugs-announce] [issue34458] No way to alternate options
Message-ID: <1534896831.7.0.56676864532.issue34458@psf.upfronthosting.co.za>


New submission from Victor Porton <porton at narod.ru>:

I have:

parser.add_argument('-p', '--preload', help='preload asset', action='append', metavar='NAMESPACE')

I want also add:

parser.add_argument('-f', '--file', help='preload file', action='append', metavar='FILE', dest='preload')

This way I could specify -p and/or -f options in any order (like: first -p then -f then -p again) and store then in 'preload' field in the order which the user specified the options.

But I have no way to know if an option is -p or -f :-(

Please add something to argparse to solve this problem.

----------
components: Library (Lib)
messages: 323857
nosy: porton
priority: normal
severity: normal
status: open
title: No way to alternate options
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34458>
_______________________________________

From report at bugs.python.org  Wed Aug 22 05:00:20 2018
From: report at bugs.python.org (era)
Date: Wed, 22 Aug 2018 09:00:20 +0000
Subject: [New-bugs-announce] [issue34459] email.contentmanager should use
 IANA encoding
Message-ID: <1534928420.28.0.56676864532.issue34459@psf.upfronthosting.co.za>


New submission from era <era+python at iki.fi>:

https://github.com/python/cpython/blob/3.7/Lib/email/contentmanager.py#L64 currently contains the following code:

    def get_text_content(msg, errors='replace'):
        content = msg.get_payload(decode=True)
        charset = msg.get_param('charset', 'ASCII')
        return content.decode(charset, errors=errors)

This breaks when the IANA character set is not identical to the Python encoding name. For example, pass it a message with

    Content-type: text/plain; charset=cp-850

This breaks for two separate reasons (and I will report two separate bugs); the IANA character-set label should be looked up and converted to a Python codec name (that's this bug) and the character-set alias 'cp-850' is not defined in the lookup table in the place.

There are probably other places in contentmanager.py where a similar mapping should take place. 

I do not have a proper patch, but in general outline, the fix would look like

+    import email.charset
+
     def get_text_content(msg, errors='replace'):
        content = msg.get_payload(decode=True)
        charset = msg.get_param('charset', 'ASCII')
-       return content.decode(charset, errors=errors)
+       encoding = Charset(charset).output_charset()
+       return content.decode(encoding, errors=errors)

This was discovered in this Stack Overflow post: https://stackoverflow.com/a/51961225/874188

----------
components: email
messages: 323869
nosy: barry, era, r.david.murray
priority: normal
severity: normal
status: open
title: email.contentmanager should use IANA encoding
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34459>
_______________________________________

From report at bugs.python.org  Wed Aug 22 05:17:06 2018
From: report at bugs.python.org (era)
Date: Wed, 22 Aug 2018 09:17:06 +0000
Subject: [New-bugs-announce] [issue34460] email.charset: common IANA labels
 missing
Message-ID: <1534929426.63.0.56676864532.issue34460@psf.upfronthosting.co.za>


New submission from era <era+python at iki.fi>:

The email.charset module should contain common informal character-set identifiers even if they are not formally specified in a IANA RFC.

>From a quick grep of a pile of recent email, I find the following:

   46 "cp-850"
    6 "windows-874"

For scale, the same collection contained around 10,000 messages with "utf-8" and 2,000 with "iso-8859-1".  Still, the fact that there are multiple occurrences in a spool of recent messages indicates that they are fairly common.

Currently, the email module throws a traceback if you attempt to parse a message whose character set is not known to Python. This is not possible to prevent in the general case, but making it more robust with encodings which are reasonably prevalent in the wild would definitely be desirable.  

For what it's worth, "cp-850" is apparently an alias for IBM code page 850 which is defined with the name "cp850" in RFC1345.  "windows-874" is an official designation which is detailed in https://www.iana.org/assignments/charset-reg/windows-874 which is apparently equivalent to the Python codec "cp784".

----------
components: email
messages: 323870
nosy: barry, era, r.david.murray
priority: normal
severity: normal
status: open
title: email.charset: common IANA labels missing
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34460>
_______________________________________

From report at bugs.python.org  Wed Aug 22 06:41:28 2018
From: report at bugs.python.org (nilanjan roy)
Date: Wed, 22 Aug 2018 10:41:28 +0000
Subject: [New-bugs-announce] [issue34461] Availability of parsers in etree
 initializer
Message-ID: <1534934488.79.0.56676864532.issue34461@psf.upfronthosting.co.za>


New submission from nilanjan roy <roy.nilanjan at gmail.com>:

As xml package make the availability of separate global name-space by *__all__* so considerably *etree* should have included of all the parser files in its initialize(i.e. in *__init__.py*). 

So if any script consider as "from xml import *" then *etree* should have accessibility on all of it's parser files i.e. etree should access like *etree.ElementTree* which is not working as expected currently. Currently it's enforced to use as normal behavior as "import xml.etree.ElementTree"

----------
components: XML
messages: 323872
nosy: nilanjan roy
priority: normal
severity: normal
status: open
title: Availability of parsers in etree initializer
type: behavior
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34461>
_______________________________________

From report at bugs.python.org  Wed Aug 22 13:52:44 2018
From: report at bugs.python.org (Alexey Izbyshev)
Date: Wed, 22 Aug 2018 17:52:44 +0000
Subject: [New-bugs-announce] [issue34462] _xxsubinterpreters: Wrong NULL
 check in _copy_raw_string()
Message-ID: <1534960364.06.0.56676864532.issue34462@psf.upfronthosting.co.za>


New submission from Alexey Izbyshev <izbyshev at ispras.ru>:

The code at https://github.com/python/cpython/blob/28853a249b1d0c890b7e9ca345290bb8c1756446/Modules/_xxsubinterpretersmodule.c#L18 checks a wrong variable for NULL:

    const char *str = PyUnicode_AsUTF8(strobj);
    if (str == NULL) {
        return NULL;
    }
    char *copied = PyMem_Malloc(strlen(str)+1);
    if (str == NULL) {
        PyErr_NoMemory();
        return NULL;
    }

Reported by Svace static analyzer.

----------
components: Extension Modules
messages: 323897
nosy: eric.snow, izbyshev, serhiy.storchaka
priority: normal
severity: normal
status: open
title: _xxsubinterpreters: Wrong NULL check in _copy_raw_string()
type: behavior
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34462>
_______________________________________

From report at bugs.python.org  Wed Aug 22 15:01:06 2018
From: report at bugs.python.org (Timothy McCurrach)
Date: Wed, 22 Aug 2018 19:01:06 +0000
Subject: [New-bugs-announce] [issue34463] Discrepancy between
 traceback.print_exception and sys.__excepthook__
Message-ID: <1534964466.61.0.56676864532.issue34463@psf.upfronthosting.co.za>


New submission from Timothy McCurrach <tim.mccurrach at gmail.com>:

If you have set sys.excepthook to some function which calls traceback.print_exception, then I would expect to get identical traceback/exception messages. If you run raise SyntaxError("some message"), then print_exception has the extra line `File "<string>", line None`. 

This comes from lines 558-561 of traceback.py:

# It was a syntax error; show exactly where the problem was found.
filename = self.filename or "<string>"
lineno = str(self.lineno) or '?'
yield '  File "{}", line {}\n'.format(filename, lineno)

Is it expected behaviour that these two functions behave differently, or should there be something like:
if self.filename or self.lineno:
    etc.

Also, if self.lineno is None, then str(self.lineno) evaluates to "None" and so the ? is never used.

----------
components: Demos and Tools
messages: 323902
nosy: Timothy McCurrach
priority: normal
severity: normal
status: open
title: Discrepancy between traceback.print_exception and sys.__excepthook__
type: behavior

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34463>
_______________________________________

From report at bugs.python.org  Wed Aug 22 15:08:50 2018
From: report at bugs.python.org (Jonathan Fine)
Date: Wed, 22 Aug 2018 19:08:50 +0000
Subject: [New-bugs-announce] [issue34464] docs: keywords are special - eg
 constants.html
Message-ID: <1534964930.72.0.56676864532.issue34464@psf.upfronthosting.co.za>


New submission from Jonathan Fine <jfine2358 at gmail.com>:

The identifiers True, False, None and __debug__ are keywords in the language.  For example
>>> __debug__ = __debug__
SyntaxError: assignment to keyword


1. The page constants.html incorrectly says then are in the built-in namespace. Some of them were, once.

2. The list keyword.kwlist does not contain __debug__. (Problem in Lib/keyword.py.)

3. https://docs.python.org/3/reference/datamodel.html for None, NotImplemented, etc.

4. There may be other places that need looking at.

See also: https://github.com/jfine2358/py-jfine2358/blob/master/docs/none-is-special.md

Credit: The __debug__ problem arises from Steve D'Aprano's message

https://mail.python.org/pipermail/python-ideas/2018-August/052917.html
The std lib contains a test that this correctly raises SyntaxError:
def f(*, x=lambda __debug__:0): pass

----------
assignee: docs at python
components: Documentation
messages: 323904
nosy: docs at python, jfine2358
priority: normal
severity: normal
status: open
title: docs: keywords are special - eg constants.html
type: enhancement
versions: Python 2.7, Python 3.4, Python 3.5, Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34464>
_______________________________________

From report at bugs.python.org  Wed Aug 22 15:48:00 2018
From: report at bugs.python.org (=?utf-8?q?J=C3=B6rn_Heissler?=)
Date: Wed, 22 Aug 2018 19:48:00 +0000
Subject: [New-bugs-announce] [issue34465] ipaddress should accept bytearray
 in addition to bytes
Message-ID: <1534967280.44.0.56676864532.issue34465@psf.upfronthosting.co.za>


New submission from J?rn Heissler <launchpad at joern.heissler.de>:

Hi,

the ipaddress module accepts `bytes' objects in the constructors.
`bytearray' however is not supported, see paste below.

Should this be supported too?


>>> import ipaddress

>>> ipaddress.IPv4Address(bytes([127, 0, 0, 1]))
IPv4Address('127.0.0.1')

>>> ipaddress.IPv4Address(bytearray([127, 0, 0, 1]))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python3.7/ipaddress.py", line 1301, in __init__
    self._ip = self._ip_int_from_string(addr_str)
  File "/usr/lib/python3.7/ipaddress.py", line 1135, in _ip_int_from_string
    raise AddressValueError("Expected 4 octets in %r" % ip_str)
ipaddress.AddressValueError: Expected 4 octets in "bytearray(b'\\x7f\\x00\\x00\\x01')"

----------
components: Library (Lib)
messages: 323906
nosy: joernheissler
priority: normal
severity: normal
status: open
title: ipaddress should accept bytearray in addition to bytes
type: enhancement
versions: Python 3.4, Python 3.5, Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34465>
_______________________________________

From report at bugs.python.org  Wed Aug 22 17:04:11 2018
From: report at bugs.python.org (Fedor Korotkiy)
Date: Wed, 22 Aug 2018 21:04:11 +0000
Subject: [New-bugs-announce] [issue34466] socket.settimeout working
 incorrectly for connect() method of AF_UNIX socket
Message-ID: <1534971851.11.0.56676864532.issue34466@psf.upfronthosting.co.za>


New submission from Fedor Korotkiy <dartslon at gmail.com>:

Attached program creates listening server socket, fills backlog queue with 2 client socket and performs third connect attempt after setting socket timeout to 5 second.

I expect third connect() attempt to fail after specified timeout of 5 second. That is indeed what's happening for tcp socket (AF_INET).

But when underlying socket has type AF_UNIX, connect() fails immediately with 'Resource temporarily unavailable' error. I'm able to reproduce this issue on linux 4.17.12 and python 3.6.6.

The underlying issue seems to be different behavior of connect(3) system call for unix sockets. Instead of starting operation and returning EINPROGRESS error it fails immediately with EAGAIN error.

Thanks to Vladislav Bidzilya for discovering this issue.

----------
files: bug.py
messages: 323910
nosy: Slon
priority: normal
severity: normal
status: open
title: socket.settimeout working incorrectly for connect() method of AF_UNIX socket
versions: Python 3.6
Added file: https://bugs.python.org/file47759/bug.py

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34466>
_______________________________________

From report at bugs.python.org  Wed Aug 22 18:02:43 2018
From: report at bugs.python.org (Sheng Zhong)
Date: Wed, 22 Aug 2018 22:02:43 +0000
Subject: [New-bugs-announce] [issue34467] No mechanism to abort created
 coroutine or suppress not-awaited warning
Message-ID: <1534975363.12.0.56676864532.issue34467@psf.upfronthosting.co.za>


New submission from Sheng Zhong <zhsh at umich.edu>:

As far as I know, there is no mechanism for aborting a created coroutine instance before it's executed with an await (directly or indirectly). I have a function which takes in coroutine instances and conditionally creates a task for them that I track so that they can be later gathered. When the condition is false, I'd like to not execute the passed in coroutine.

Simply ignoring the coroutine works but reports a RuntimeWarning about coroutine not being awaited on. Checking the conditional before calling the function works but damages maintainability and is putting the responsibility on the wrong party.

Creating the task then immediately cancelling it does not work since execution for it will start and leads to other cancellation issues.

Is there a way to abort a coroutine instance that I'm not aware of?

----------
components: asyncio
messages: 323911
nosy: Sheng Zhong, asvetlov, yselivanov
priority: normal
severity: normal
status: open
title: No mechanism to abort created coroutine or suppress not-awaited warning
type: enhancement
versions: Python 3.6, Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34467>
_______________________________________

From report at bugs.python.org  Wed Aug 22 19:57:14 2018
From: report at bugs.python.org (Alexey Izbyshev)
Date: Wed, 22 Aug 2018 23:57:14 +0000
Subject: [New-bugs-announce] [issue34468] An always-false condition in
 range_repr() from Objects/rangeobject.c
Message-ID: <1534982234.29.0.56676864532.issue34468@psf.upfronthosting.co.za>


New submission from Alexey Izbyshev <izbyshev at ispras.ru>:

range_repr() contains the following snippet:

    /* Check for special case values for printing.  We don't always
       need the step value.  We don't care about errors
       (it means overflow), so clear the errors. */
    istep = PyNumber_AsSsize_t(r->step, NULL);
    if (istep != 1 || (istep == -1 && PyErr_Occurred())) {
        PyErr_Clear();
    }

The second condition in 'if' statement is always false (reported by Svace static analyzer). Moreover, the comment is incorrect. It implies that any error from PyNumber_AsSsize_t() means that an overflow occurred, but according to its implementation and documentation, any error is guaranteed *not* to be an OverflowError if the second parameter is NULL (the result is clamped on overflow).

In practice, 'range' invariants currently guarantee that no error can occur in this case because 'r->step' is a subtype of int (enforced in validate_step()). So the 'if' statement can be replaced either with an assert:

    assert(!(istep == -1 && PyErr_Occurred()))

or with a conservative check:

    if (istep == -1 && PyErr_Occurred())) {
        // may also add assert(!PyErr_ExceptionMatches( PyExc_OverflowError);
        return NULL;
    }

Suggestions are appreciated.

----------
components: Interpreter Core
messages: 323912
nosy: izbyshev, ncoghlan, serhiy.storchaka
priority: normal
severity: normal
status: open
title: An always-false condition in range_repr() from Objects/rangeobject.c
type: behavior
versions: Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34468>
_______________________________________

From report at bugs.python.org  Wed Aug 22 23:40:21 2018
From: report at bugs.python.org (Christopher Knorowski)
Date: Thu, 23 Aug 2018 03:40:21 +0000
Subject: [New-bugs-announce] [issue34469] windows msi in headless mode fails
 to install Script directory on reinstall if the python folder was deleted
 but not uninstalled
Message-ID: <1534995621.59.0.56676864532.issue34469@psf.upfronthosting.co.za>


New submission from Christopher Knorowski <cknorow at gmail.com>:

Steps to repeate (windows 10)

msiexec.exe /i python-2.7.15.amd64.msi TARGET_DIR=C:/Users/Public/python /qr

(fresh install works without issue)

rm -r C:/Users/Public/python

msiexec.exe /i python-2.7.15.amd64.msi TARGET_DIR=C:/Users/Public/python /qr

(Scripts folder and all site-packages are not copied ie. no Pip or easy_install)

----------
components: Installation
messages: 323918
nosy: cdknorow
priority: normal
severity: normal
status: open
title: windows msi in headless mode fails to install Script directory on reinstall if the python folder was deleted but not uninstalled
type: compile error
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34469>
_______________________________________

From report at bugs.python.org  Wed Aug 22 23:50:13 2018
From: report at bugs.python.org (Christopher Knorowski)
Date: Thu, 23 Aug 2018 03:50:13 +0000
Subject: [New-bugs-announce] [issue34470] windows msi in headless mode fails
 to install Script directory on reinstall if the python folder was deleted
 but not uninstalled
Message-ID: <1534996213.68.0.56676864532.issue34470@psf.upfronthosting.co.za>


New submission from Christopher Knorowski <cknorow at gmail.com>:

Steps to repeate (windows 10)

msiexec.exe /i python-2.7.15.amd64.msi ALL_USERS=0 TARGETDIR=C:\Users\Public\python /qr

(fresh install works without issue)

rm -r C:\Users\Public\python

(On reinstall scripts folder and all site-packages are not copied ie. no Pip or easy_install)

msiexec.exe /i python-2.7.15.amd64.msi ALL_USERS=0 TARGETDIR=C:\Users\Public\python /qr

----------
components: Installation
messages: 323921
nosy: cdknorow
priority: normal
severity: normal
status: open
title: windows msi in headless mode fails to install Script directory on reinstall if the python folder was deleted but not uninstalled
type: compile error
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34470>
_______________________________________

From report at bugs.python.org  Thu Aug 23 06:14:40 2018
From: report at bugs.python.org (Alexey Izbyshev)
Date: Thu, 23 Aug 2018 10:14:40 +0000
Subject: [New-bugs-announce] [issue34471] _datetime: Missing NULL check in
 tzinfo_from_isoformat_results()
Message-ID: <1535019280.96.0.56676864532.issue34471@psf.upfronthosting.co.za>


New submission from Alexey Izbyshev <izbyshev at ispras.ru>:

The return value of new_delta() is not checked for NULL at https://github.com/python/cpython/blob/ccd99752675042bd5f67d332c5b0ed85ba1f2da3/Modules/_datetimemodule.c#L1292 and then dereferenced.

Reported by Svace static analyzer.

----------
components: Extension Modules
messages: 323934
nosy: belopolsky, izbyshev, p-ganssle, serhiy.storchaka
priority: normal
severity: normal
status: open
title: _datetime: Missing NULL check in tzinfo_from_isoformat_results()
type: behavior
versions: Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34471>
_______________________________________

From report at bugs.python.org  Thu Aug 23 08:57:59 2018
From: report at bugs.python.org (Silas Sewell)
Date: Thu, 23 Aug 2018 12:57:59 +0000
Subject: [New-bugs-announce] [issue34472] zipfile: does not include optional
 descriptor signature
Message-ID: <1535029079.23.0.56676864532.issue34472@psf.upfronthosting.co.za>


New submission from Silas Sewell <silas at sewell.org>:

When adding a fileobj that is not seekable to a zip using the zipfile module, the optional signature value 0x08074b50 is not included. This breaks certain zip implementations, including the builtin MacOS Archive Utility.

See https://pkware.cachefly.net/webdocs/casestudies/APPNOTE.TXT sections 4.3.9.3 and 4.3.9.4

----------
components: Library (Lib)
files: zip_stream_example.py
messages: 323943
nosy: silas
priority: normal
severity: normal
status: open
title: zipfile: does not include optional descriptor signature
versions: Python 3.5, Python 3.6, Python 3.7, Python 3.8
Added file: https://bugs.python.org/file47760/zip_stream_example.py

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34472>
_______________________________________

From report at bugs.python.org  Thu Aug 23 09:36:22 2018
From: report at bugs.python.org (Matt Tuttle)
Date: Thu, 23 Aug 2018 13:36:22 +0000
Subject: [New-bugs-announce] [issue34473] pip is unusable on Windows 10 1803
 - SHA-1 certificates are untrusted
Message-ID: <1535031382.08.0.56676864532.issue34473@psf.upfronthosting.co.za>


New submission from Matt Tuttle <mtuttle at aha-net.org>:

Running pip install ANYTHING results in an SSL failure on Windows 10 1803 because SHA-1 certificates are untrusted.

----------
components: Installation, Windows
messages: 323945
nosy: mtuttle, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: pip is unusable on Windows 10 1803 - SHA-1 certificates are untrusted
type: behavior
versions: Python 2.7, Python 3.4, Python 3.5, Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34473>
_______________________________________

From report at bugs.python.org  Thu Aug 23 10:04:14 2018
From: report at bugs.python.org (Alexey Izbyshev)
Date: Thu, 23 Aug 2018 14:04:14 +0000
Subject: [New-bugs-announce] [issue34474] Python/bltinmodule.c: Missing NULL
 check in builtin_sum_impl()
Message-ID: <1535033054.32.0.56676864532.issue34474@psf.upfronthosting.co.za>


New submission from Alexey Izbyshev <izbyshev at ispras.ru>:

The return value of PyFloat_FromDouble() at https://github.com/python/cpython/blob/48ce4897f8f8d91d948ecd1241ffab002df2be9e/Python/bltinmodule.c#L2403 is not checked for NULL and then dereferenced.

Reported by Svace static analyzer.

A similar issue in an 'if' statement above was fixed in #18560. Note that 2.7 is affected by both issues, so I can fix both if needed.

----------
components: Interpreter Core
messages: 323946
nosy: berker.peksag, izbyshev, pitrou, serhiy.storchaka
priority: normal
severity: normal
status: open
title: Python/bltinmodule.c: Missing NULL check in builtin_sum_impl()
type: behavior
versions: Python 2.7, Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34474>
_______________________________________

From report at bugs.python.org  Thu Aug 23 10:25:59 2018
From: report at bugs.python.org (Chris Jerdonek)
Date: Thu, 23 Aug 2018 14:25:59 +0000
Subject: [New-bugs-announce] [issue34475] functools.partial objects have no
 __qualname__ attribute
Message-ID: <1535034359.11.0.56676864532.issue34475@psf.upfronthosting.co.za>


New submission from Chris Jerdonek <chris.jerdonek at gmail.com>:

functools.partial objects have no __qualname__ attribute. This means, for example, that code expecting a callable that logs the __qualname__ attribute can break when passed a functools.partial object.

Example:

>>> import functools
>>> int.__qualname__
'int'
>>> p = functools.partial(int)
>>> p.__qualname__
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'functools.partial' object has no attribute '__qualname__'

----------
components: Library (Lib)
messages: 323947
nosy: chris.jerdonek
priority: normal
severity: normal
status: open
title: functools.partial objects have no __qualname__ attribute
type: enhancement
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34475>
_______________________________________

From report at bugs.python.org  Thu Aug 23 10:41:27 2018
From: report at bugs.python.org (=?utf-8?b?SHJ2b2plIE5pa8WhacSH?=)
Date: Thu, 23 Aug 2018 14:41:27 +0000
Subject: [New-bugs-announce] [issue34476] asyncio.sleep(0) not documented
Message-ID: <1535035287.88.0.56676864532.issue34476@psf.upfronthosting.co.za>


New submission from Hrvoje Nik?i? <hniksic at gmail.com>:

Looking at the implementation and at the old issue at https://github.com/python/asyncio/issues/284 shows that asyncio.sleep special-cases asyncio.sleep(0) to mean "yield control to the event loop" without incurring additional overhead of sleeping. However, this is not documented at https://docs.python.org/3/library/asyncio-task.html#asyncio.sleep .

I propose to add to the documentation a statement along the lines of:

"""
A delay of 0 will relinquish control to the event loop, allowing other tasks and callbacks to run. The control will be returned to the coroutine as soon as they have run, without an explicit delay.
"""

----------
assignee: docs at python
components: Documentation, asyncio
messages: 323949
nosy: asvetlov, docs at python, hniksic, yselivanov
priority: normal
severity: normal
status: open
title: asyncio.sleep(0) not documented
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34476>
_______________________________________

From report at bugs.python.org  Thu Aug 23 10:52:19 2018
From: report at bugs.python.org (Alexey Izbyshev)
Date: Thu, 23 Aug 2018 14:52:19 +0000
Subject: [New-bugs-announce] [issue34477] Objects/typeobject.c: Missing NULL
 check in type_init()
Message-ID: <1535035939.28.0.56676864532.issue34477@psf.upfronthosting.co.za>


New submission from Alexey Izbyshev <izbyshev at ispras.ru>:

The return value of PyTuple_GetSlice() at https://github.com/python/cpython/blob/48ce4897f8f8d91d948ecd1241ffab002df2be9e/Objects/typeobject.c#L2297 is not checked for NULL and then dereferenced.

Reported by Svace static analyzer.

Currently, PyTuple_GetSlice() for a zero-length slice can fail only if it's the first zero-length tuple ever created. If there is an invariant that at least one zero-length tuple exists on any type_init() call, the check may be replaced with an assert.

----------
components: Interpreter Core
messages: 323951
nosy: berker.peksag, izbyshev, pitrou, serhiy.storchaka
priority: normal
severity: normal
status: open
title: Objects/typeobject.c: Missing NULL check in type_init()
type: behavior
versions: Python 2.7, Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34477>
_______________________________________

From report at bugs.python.org  Thu Aug 23 11:07:34 2018
From: report at bugs.python.org (Segev Finer)
Date: Thu, 23 Aug 2018 15:07:34 +0000
Subject: [New-bugs-announce] [issue34478] Possibly misleading/wrong
 documentation about PyModuleDef.m_free
Message-ID: <1535036854.66.0.56676864532.issue34478@psf.upfronthosting.co.za>


New submission from Segev Finer <segev208 at gmail.com>:

This section in the documentation https://docs.python.org/3.7/howto/cporting.html#module-initialization-and-state shows how to implement an extension module that works for both Python 2 and 3. It also shows how to use m_size to manage module state instead of using static globals. But it omits implementing m_free. This is also done in https://www.python.org/dev/peps/pep-3121/#example

To my understanding, m_clear is *only* called by the Python GC when the object is a part of a reference cycle. It won't be called for a normal free when the object wasn't a part of a reference cycle. This means that for such a case the objects referenced in the state structure will leak!

Looking around the standard library, it seems the extension modules there point m_free to calling the function used to implement m_clear.

The documentation of the structure is also quite unclear on the subject https://docs.python.org/3/c-api/module.html?highlight=pymoduledef#c.PyModuleDef.

P.S. There is also no freeing/DECREF-ing stuff in the state on error in the Python 2 case, but I think even the standard library is like this.

----------
assignee: docs at python
components: Documentation
messages: 323955
nosy: Segev Finer, docs at python
priority: normal
severity: normal
status: open
title: Possibly misleading/wrong documentation about PyModuleDef.m_free
type: enhancement
versions: Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34478>
_______________________________________

From report at bugs.python.org  Thu Aug 23 11:32:12 2018
From: report at bugs.python.org (Siming Yuan)
Date: Thu, 23 Aug 2018 15:32:12 +0000
Subject: [New-bugs-announce] [issue34479] ArgumentParser subparser error
 display at the wrong level
Message-ID: <1535038332.48.0.56676864532.issue34479@psf.upfronthosting.co.za>


New submission from Siming Yuan <siming85 at gmail.com>:

If you take the example from
https://docs.python.org/3/library/argparse.html#argparse.ArgumentParser.add_subparsers

# create the top-level parser
parser = argparse.ArgumentParser(prog='PROG')
parser.add_argument('--foo', action='store_true', help='foo help')
subparsers = parser.add_subparsers(help='sub-command help')
# create the parser for the "a" command
parser_a = subparsers.add_parser('a', help='a help')
parser_a.add_argument('-bar', type=int, help='bar help')
# create the parser for the "b" command
parser_b = subparsers.add_parser('b', help='b help')
parser_b.add_argument('--baz', choices='XYZ', help='baz help')


and run a subcommand but with an argument the subcommand doesn't understand

parser.parse_args(['a', '-x'])

the output doesn't help much - because the usage is coming from the main parser

usage: PROG [-h] [--foo] {a,b} ...
PROG: error: unrecognized arguments: -x


the reason for failure is because the error api being called in this case is
parser.error(msg)

not the subparser
parser_a.error(msg)

the proper error should've been

usage: PROG a [-h] [-bar BAR]
PROG a: error: unrecognized arguments: -x

----------
components: Library (Lib)
messages: 323956
nosy: siming85
priority: normal
severity: normal
status: open
title: ArgumentParser subparser error display at the wrong level
type: behavior
versions: Python 3.4, Python 3.5, Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34479>
_______________________________________

From report at bugs.python.org  Thu Aug 23 13:30:37 2018
From: report at bugs.python.org (Conrad)
Date: Thu, 23 Aug 2018 17:30:37 +0000
Subject: [New-bugs-announce] [issue34480] _markupbase.py fails with
 UnboundLocalError on invalid keyword in marked section
Message-ID: <1535045437.7.0.56676864532.issue34480@psf.upfronthosting.co.za>


New submission from Conrad <conrad.roche at gmail.com>:

$ pip freeze | grep beautifulsoup4
beautifulsoup4==4.6.3

$ python
Python 3.7.0 (default, Jul 23 2018, 20:24:19)
[Clang 9.0.0 (clang-900.0.39.2)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from bs4 import BeautifulSoup
>>> text = "<![hi world]>"
>>> BeautifulSoup(text, 'html.parser')
/Users/conradroche/solvvy/ml-pipeline/venv/lib/python3.7/site-packages/bs4/builder/_htmlparser.py:78: UserWarning: unknown status keyword 'hi ' in marked section
  warnings.warn(msg)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/conradroche/solvvy/ml-pipeline/venv/lib/python3.7/site-packages/bs4/__init__.py", line 282, in __init__
    self._feed()
  File "/Users/conradroche/solvvy/ml-pipeline/venv/lib/python3.7/site-packages/bs4/__init__.py", line 343, in _feed
    self.builder.feed(self.markup)
  File "/Users/conradroche/solvvy/ml-pipeline/venv/lib/python3.7/site-packages/bs4/builder/_htmlparser.py", line 247, in feed
    parser.feed(markup)
  File "/usr/local/Cellar/python/3.7.0/Frameworks/Python.framework/Versions/3.7/lib/python3.7/html/parser.py", line 111, in feed
    self.goahead(0)
  File "/usr/local/Cellar/python/3.7.0/Frameworks/Python.framework/Versions/3.7/lib/python3.7/html/parser.py", line 179, in goahead
    k = self.parse_html_declaration(i)
  File "/usr/local/Cellar/python/3.7.0/Frameworks/Python.framework/Versions/3.7/lib/python3.7/html/parser.py", line 264, in parse_html_declaration
    return self.parse_marked_section(i)
  File "/usr/local/Cellar/python/3.7.0/Frameworks/Python.framework/Versions/3.7/lib/python3.7/_markupbase.py", line 160, in parse_marked_section
    if not match:
UnboundLocalError: local variable 'match' referenced before assignment
>>>

----------
components: Library (Lib)
messages: 323962
nosy: kodial
priority: normal
severity: normal
status: open
title: _markupbase.py fails with UnboundLocalError on invalid keyword in marked section
type: behavior
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34480>
_______________________________________

From report at bugs.python.org  Thu Aug 23 13:40:57 2018
From: report at bugs.python.org (Alexey Izbyshev)
Date: Thu, 23 Aug 2018 17:40:57 +0000
Subject: [New-bugs-announce] [issue34481] Different behavior of C and Python
 impls of datetime.strftime with non-UTF-8-encodable strings
Message-ID: <1535046057.5.0.56676864532.issue34481@psf.upfronthosting.co.za>


New submission from Alexey Izbyshev <izbyshev at ispras.ru>:

The C datetime implementation uses PyUnicode_AsUTF8AndSize() in wrap_strftime() and rejects strings containing surrogate code points (0xD800 - 0xDFFF) since they can't be encoded in UTF-8. On the other hand, the pure-Python datetime implementation doesn't have this restriction:

>>> import sys
>>> sys.modules['_datetime'] = None # block C implementation
>>> from datetime import time
>>> time().strftime('\ud800')
'\ud800'
>>> del sys.modules['datetime']
>>> del sys.modules['_datetime']
>>> from datetime import time
>>> time().strftime('\ud800')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
UnicodeEncodeError: 'utf-8' codec can't encode character '\ud800' in position 0: surrogates not allowed

----------
components: Extension Modules
messages: 323963
nosy: belopolsky, izbyshev, pitrou, serhiy.storchaka, taleinat
priority: normal
severity: normal
status: open
title: Different behavior of C and Python impls of datetime.strftime with non-UTF-8-encodable strings
type: behavior
versions: Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34481>
_______________________________________

From report at bugs.python.org  Thu Aug 23 13:41:50 2018
From: report at bugs.python.org (Alexey Izbyshev)
Date: Thu, 23 Aug 2018 17:41:50 +0000
Subject: [New-bugs-announce] [issue34482] datetime: Tests for potential
 crashes due to non-UTF-8-encodable strings
Message-ID: <1535046110.83.0.56676864532.issue34482@psf.upfronthosting.co.za>


New submission from Alexey Izbyshev <izbyshev at ispras.ru>:

This is a follow-up of #34454. 'datetime' extension module attempts to encode input strings into UTF-8 in several places, which requires special care because some valid Python strings can't be represented in UTF-8. It makes sense to add more tests for methods dealing with strings.

Note that my PR doesn't attempt to deal with #34481. In cases where behavior differs between C and Python datetime impls the tests check only for  absence of crashes.

----------
components: Extension Modules
messages: 323964
nosy: belopolsky, izbyshev, p-ganssle, serhiy.storchaka, taleinat
priority: normal
severity: normal
status: open
title: datetime: Tests for potential crashes due to non-UTF-8-encodable strings
type: security
versions: Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34482>
_______________________________________

From report at bugs.python.org  Thu Aug 23 13:49:17 2018
From: report at bugs.python.org (Alex)
Date: Thu, 23 Aug 2018 17:49:17 +0000
Subject: [New-bugs-announce] [issue34483] eval() raises NameError: name
 '...' is not defined
Message-ID: <1535046557.48.0.56676864532.issue34483@psf.upfronthosting.co.za>


New submission from Alex <bojenko.alex at gmail.com>:

Builtin eval() function raises NameError on a valid expression:


--- example of bug on Python 3.4
Python 3.4.5 (default, May 29 2017, 15:17:55)
[GCC 4.8.5 20150623 (Red Hat 4.8.5-11)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> x = [1,2,3]
>>> [x for i in x]
[[1, 2, 3], [1, 2, 3], [1, 2, 3]]
>>> eval('[x for i in x]', {}, dict(x=x))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<string>", line 1, in <module>
  File "<string>", line 1, in <listcomp>
NameError: name 'x' is not defined


--- example on Python 2.7 (no bug)
Python 2.7.5 (default, May  3 2017, 07:55:04)
[GCC 4.8.5 20150623 (Red Hat 4.8.5-14)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> x = [1,2,3]
>>> [x for i in x]
[[1, 2, 3], [1, 2, 3], [1, 2, 3]]
>>> eval('[x for i in x]', {}, dict(x=x))
[[1, 2, 3], [1, 2, 3], [1, 2, 3]]


---

Bug reproduced on:
- Windows 7: Python 3.4, 3.7
- Red Hat Enterprise Linux Server: Python 3.4

Works without errors on:
- Windows 7: Python 2.7
- Red Hat Enterprise Linux Server: Python 2.7

----------
components: Interpreter Core
messages: 323965
nosy: alexb
priority: normal
severity: normal
status: open
title: eval() raises NameError: name '...' is not defined
versions: Python 3.4, Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34483>
_______________________________________

From report at bugs.python.org  Thu Aug 23 17:14:39 2018
From: report at bugs.python.org (Mark Dickinson)
Date: Thu, 23 Aug 2018 21:14:39 +0000
Subject: [New-bugs-announce] [issue34484] Unicode HOWTO incorrectly refers
 to Private Use Area for surrogateescape
Message-ID: <1535058879.5.0.56676864532.issue34484@psf.upfronthosting.co.za>


New submission from Mark Dickinson <dickinsm at gmail.com>:

The Unicode HOWTO currently has contains this text in the "Files in an Unknown Encoding" section [1]:

> The surrogateescape error handler will decode any non-ASCII bytes as code
> points in the Unicode Private Use Area ranging from U+DC80 to U+DCFF. These
> private code points will then be turned back into the same bytes when the
> surrogateescape error handler is used when encoding the data and writing it
> back out.

Unless I'm missing something, the subrange U+DC80 to U+DCFF of the low surrogates is *not* a Private Use Area. There *is* a kinda-sorta PUA in the high surrogates from U+DB80 to U+DBFF (because the only valid codepoints that use these surrogates in their UTF-16 encoding are the codepoints in planes 15 and 16, which are almost entirely PUA codepoints), but that's not what the surrogateescape handler is using.


[1] https://docs.python.org/3/howto/unicode.html#files-in-an-unknown-encoding

----------
assignee: docs at python
components: Documentation
messages: 323976
nosy: docs at python, mark.dickinson
priority: normal
severity: normal
status: open
title: Unicode HOWTO incorrectly refers to Private Use Area for surrogateescape
versions: Python 3.6, Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34484>
_______________________________________

From report at bugs.python.org  Thu Aug 23 19:44:09 2018
From: report at bugs.python.org (STINNER Victor)
Date: Thu, 23 Aug 2018 23:44:09 +0000
Subject: [New-bugs-announce] [issue34485] _PyCoreConfig: add stdio_encoding
 and stdio_errors
Message-ID: <1535067849.47.0.56676864532.issue34485@psf.upfronthosting.co.za>


New submission from STINNER Victor <vstinner at redhat.com>:

Currently, the code to select the encoding and error handler of sys.stdin, sys.stdout and sys.stderr is run later in Py_Initialize(). I propose to move most of this code into _PyCoreConfig_Read() and add stdio_encoding and stdio_errors to _PyCoreConfig to easily let Python embedders to select the encoding.

Attached PR implements this idea. During Py_Initialize(), the code still reads the LC_CTYPE locale to choose the error handler if stdio_errors is NULL. This check cannot be done earlier, since the LC_CTYPE locale is only set to the user locale in Py_Initialize(): after _PyCoreConfig_Read().

The Py_SetStandardStreamEncoding() function has been added by Nick Coghlan in bpo-16129.

    This new pre-initialization API allows embedding
    applications like Blender to force a particular
    encoding and error handler for the standard IO streams.

----------
messages: 323980
nosy: vstinner
priority: normal
severity: normal
status: open
title: _PyCoreConfig: add stdio_encoding and stdio_errors
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34485>
_______________________________________

From report at bugs.python.org  Fri Aug 24 10:02:35 2018
From: report at bugs.python.org (=?utf-8?q?Stanis=C5=82aw_Skonieczny_=28Uosiu=29?=)
Date: Fri, 24 Aug 2018 14:02:35 +0000
Subject: [New-bugs-announce] [issue34486] "RuntimeError: release unlocked
 lock" when starting a thread
Message-ID: <1535119355.34.0.56676864532.issue34486@psf.upfronthosting.co.za>


New submission from Stanis?aw Skonieczny (Uosiu) <stanislaw.skonieczny at gmail.com>:

Sometimes when thread is starting it raises "RuntimeError: release unlocked lock". That is when signal handler is invoked in the same time.

Traceback (most recent call last):
  File "/usr/lib/python3.6/threading.py", line 551, in wait
    signaled = self._cond.wait(timeout)
  File "/usr/lib/python3.6/threading.py", line 304, in wait
    self._acquire_restore(saved_state)
  File "start_threads.py", line 12, in sighandler
    raise MyException("got signal")
__main__.MyException: got signal

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "start_threads.py", line 28, in <module>
    thread.start()
  File "/usr/lib/python3.6/threading.py", line 851, in start
    self._started.wait()
  File "/usr/lib/python3.6/threading.py", line 552, in wait
    return signaled
  File "/usr/lib/python3.6/threading.py", line 243, in __exit__
    return self._lock.__exit__(*args)
RuntimeError: release unlocked lock

I have attached to files that reproduce the problem. First runs python and starts noop threads in the loop. It uses signal handler that raises exception. Second is a bash script to run python script and send signal to it. After some time, when signal is handled in unfortunate place, python script breaks due to unreleased lock.

----------
components: Library (Lib)
files: start_threads.py
messages: 323998
nosy: Stanis?aw Skonieczny (Uosiu)
priority: normal
severity: normal
status: open
title: "RuntimeError: release unlocked lock" when starting a thread
versions: Python 3.6
Added file: https://bugs.python.org/file47761/start_threads.py

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34486>
_______________________________________

From report at bugs.python.org  Fri Aug 24 10:44:31 2018
From: report at bugs.python.org (Gerrit Holl)
Date: Fri, 24 Aug 2018 14:44:31 +0000
Subject: [New-bugs-announce] [issue34487] enum _sunder_ names mix metaclass
 and enum class attributes
Message-ID: <1535121871.11.0.56676864532.issue34487@psf.upfronthosting.co.za>


New submission from Gerrit Holl <gerrit.holl at gmail.com>:

In the [`enum` module](https://docs.python.org/3/library/enum.html#supported-sunder-names) documentation, some of the `_sunder_` names are on `EnumMeta`, whereas others are on the produced `Enum` class:

> 8.13.15.3.2. Supported `_sunder_` names

> * `_name_` ? name of the member
> * `_value_` ? value of the member; can be set / modified in __new__
> * `_missing_` ? a lookup function used when a value is not found; may be overridden
> * `_ignore_` ? a list of names, either as a list() or a str(), that will not be transformed into members, and will be removed from the final class
> * `_order_` ? used in Python 2/3 code to ensure member order is consistent (class attribute, removed during class creation)
> * `_generate_next_value_` ? used by the Functional API and by auto to get an appropriate value for an enum member; may be overridden

Experimentally, it appears `_name_` and `_value_` are on the `Enum` class, whereas the others are all on the `EnumMeta` class:

In [272]: class Test(enum.Enum): a = 0

In [273]: Test.a._name_
Out[273]: 'a'

In [274]: Test._name
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-274-8d4f758e0fb7> in <module>()
----> 1 Test._name

/group_workspaces/cems2/fiduceo/Users/gholl/anaconda3/envs/FCDR37a/lib/python3.7/enum.py in __getattr__(cls, name)
    344             return cls._member_map_[name]
    345         except KeyError:
--> 346             raise AttributeError(name) from None
    347 
    348     def __getitem__(cls, name):

AttributeError: _name

In [275]: Test.a._value_
Out[275]: 0

In [276]: Test._value
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-276-a1cd5032d83d> in <module>()
----> 1 Test._value

/group_workspaces/cems2/fiduceo/Users/gholl/anaconda3/envs/FCDR37a/lib/python3.7/enum.py in __getattr__(cls, name)
    344             return cls._member_map_[name]
    345         except KeyError:
--> 346             raise AttributeError(name) from None
    347 
    348     def __getitem__(cls, name):

AttributeError: _value

In [277]: Test.a._missing_
Out[277]: <bound method Enum._missing_ of <enum 'Test'>>

In [278]: Test._missing_
Out[278]: <bound method Enum._missing_ of <enum 'Test'>>


This is not clear from the documentation.

----------
assignee: docs at python
components: Documentation
messages: 324003
nosy: Gerrit.Holl, docs at python
priority: normal
severity: normal
status: open
title: enum _sunder_ names mix metaclass and enum class attributes
versions: Python 3.4, Python 3.5, Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34487>
_______________________________________

From report at bugs.python.org  Fri Aug 24 13:55:50 2018
From: report at bugs.python.org (Sergey Fedoseev)
Date: Fri, 24 Aug 2018 17:55:50 +0000
Subject: [New-bugs-announce] [issue34488] improve performance of
 BytesIO.writelines() by avoiding creation of unused PyLongs
Message-ID: <1535133350.73.0.56676864532.issue34488@psf.upfronthosting.co.za>


New submission from Sergey Fedoseev <fedoseev.sergey at gmail.com>:

Currently BytesIO.writelines() uses BytesIO.write() which returns number of written bytes as Python ints, but these ints are not used by BytesIO.writelines(), so avoiding creation of these ints can improve performance of BytesIO.writelines() especially for a large number of small lines.

Benchmarks:

python -m perf timeit --compare-to ~/tmp/cpython-master-venv/bin/python -s "from io import BytesIO; lines = [b'']*10000;" "BytesIO().writelines(lines)"                                          
/home/sergey/tmp/cpython-master-venv/bin/python: ..................... 153 us +- 3 us
/home/sergey/tmp/cpython-venv/bin/python: ..................... 126 us +- 4 us

Mean +- std dev: [/home/sergey/tmp/cpython-master-venv/bin/python] 153 us +- 3 us -> [/home/sergey/tmp/cpython-venv/bin/python] 126 us +- 4 us: 1.22x faster (-18%)

python -m perf timeit --compare-to ~/tmp/cpython-master-venv/bin/python -s "from io import BytesIO; lines = [b'b']*10000;" "BytesIO().writelines(lines)"
/home/sergey/tmp/cpython-master-venv/bin/python: ..................... 164 us +- 2 us
/home/sergey/tmp/cpython-venv/bin/python: ..................... 142 us +- 1 us

Mean +- std dev: [/home/sergey/tmp/cpython-master-venv/bin/python] 164 us +- 2 us -> [/home/sergey/tmp/cpython-venv/bin/python] 142 us +- 1 us: 1.16x faster (-13%)

python -m perf timeit --compare-to ~/tmp/cpython-master-venv/bin/python -s "from io import BytesIO; lines = [b'b'*10]*10000;" "BytesIO().writelines(lines)"
/home/sergey/tmp/cpython-master-venv/bin/python: ..................... 387 us +- 6 us
/home/sergey/tmp/cpython-venv/bin/python: ..................... 365 us +- 8 us

Mean +- std dev: [/home/sergey/tmp/cpython-master-venv/bin/python] 387 us +- 6 us -> [/home/sergey/tmp/cpython-venv/bin/python] 365 us +- 8 us: 1.06x faster (-5%)

python -m perf timeit --compare-to ~/tmp/cpython-master-venv/bin/python -s "from io import BytesIO; lines = [b'b'*100]*10000;" "BytesIO().writelines(lines)"                                                    
/home/sergey/tmp/cpython-master-venv/bin/python: ..................... 319 us +- 5 us
/home/sergey/tmp/cpython-venv/bin/python: ..................... 305 us +- 16 us

Mean +- std dev: [/home/sergey/tmp/cpython-master-venv/bin/python] 319 us +- 5 us -> [/home/sergey/tmp/cpython-venv/bin/python] 305 us +- 16 us: 1.04x faster (-4%)

python -m perf timeit --compare-to ~/tmp/cpython-master-venv/bin/python -s "from io import BytesIO; lines = [b'b'*1000]*10000;" "BytesIO().writelines(lines)"
/home/sergey/tmp/cpython-master-venv/bin/python: ..................... 1.13 ms +- 0.02 ms
/home/sergey/tmp/cpython-venv/bin/python: ..................... 988 us +- 88 us

Mean +- std dev: [/home/sergey/tmp/cpython-master-venv/bin/python] 1.13 ms +- 0.02 ms -> [/home/sergey/tmp/cpython-venv/bin/python] 988 us +- 88 us: 1.14x faster (-12%)

python -m perf timeit --compare-to ~/tmp/cpython-master-venv/bin/python -s "from io import BytesIO; lines = [b'b'*10000]*10000;" "BytesIO().writelines(lines)"
/home/sergey/tmp/cpython-master-venv/bin/python: ..................... 38.7 ms +- 0.1 ms
/home/sergey/tmp/cpython-venv/bin/python: ..................... 38.2 ms +- 0.1 ms

Mean +- std dev: [/home/sergey/tmp/cpython-master-venv/bin/python] 38.7 ms +- 0.1 ms -> [/home/sergey/tmp/cpython-venv/bin/python] 38.2 ms +- 0.1 ms: 1.01x faster (-1%)

----------
components: Extension Modules
messages: 324011
nosy: sir-sigurd
priority: normal
severity: normal
status: open
title: improve performance of BytesIO.writelines() by avoiding creation of unused PyLongs
type: performance
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34488>
_______________________________________

From report at bugs.python.org  Fri Aug 24 15:44:10 2018
From: report at bugs.python.org (Sergey G. Brester)
Date: Fri, 24 Aug 2018 19:44:10 +0000
Subject: [New-bugs-announce] [issue34489] subprocess: execution of
 batch-files (.cmd/.bat) is vulnerable in python for windows / insufficient
 escape
Message-ID: <1535139850.17.0.56676864532.issue34489@psf.upfronthosting.co.za>


New submission from Sergey G. Brester <serg.brester at sebres.de>:

There is a vulnerability "insufficient escape of special chars for quoting of arguments by exec process" for python-language, if executing windows batch-files (bat/cmd).

### What version of python is affected?
All

### Does this issue reproduce with the latest master?
Yes

### What did you do?
Execution of batch-file using `subprocess` module with arguments containing some special meta-characters.

A recipe for reproducing the error as well as more extensive PoC with additional info (and more lang's affected also):
- https://github.com/sebres/PoC/blob/master/SB-0D-001-win-exec/README.md
A complete runnable program:
- https://github.com/sebres/PoC/blob/master/SB-0D-001-win-exec/test-dump-inv.py

#### A simple example:
```diff
 # invoke exe-file:
 >>> import subprocess
 >>> subprocess.call(['test-dump.exe', 'test&whoami'])
+    `test-dump.exe? `test&whoami?
 # invoke cmd-file:
 >>> subprocess.call(['test-dump.CMD', 'test&whoami'])
-    `test-dump.exe? `test?my_domain\sebres
```
For more "broken" cases, see the result of my test-suite:
- https://github.com/sebres/PoC/blob/master/SB-0D-001-win-exec/results/python.diff

### What did you expect to see?
Arguments are escaped/quoted properly.

### What did you see instead?
Arguments are insufficient escaped/quoted, so it is vulnerable currently.

### Solution:

For possible solution see the github-PR#8906:
- https://github.com/python/cpython/pull/8906 
For algorithm description:
- https://github.com/sebres/PoC/blob/master/SB-0D-001-win-exec/SOLUTION.md
resp. how it was fixed in TCL (C-code):
- https://core.tcl-lang.org/tcl/vdiff?from=core-8-5-branch&to=0-day-21b0629c81 (see the function `BuildCommandLine`)

### Possible similar issues:
bpo-33515

----------
components: Library (Lib), Windows
messages: 324013
nosy: paul.moore, sebres, steve.dower, tim.golden, zach.ware
priority: normal
pull_requests: 8376
severity: normal
status: open
title: subprocess: execution of batch-files (.cmd/.bat) is vulnerable in python for windows / insufficient escape
type: security
versions: Python 2.7, Python 3.4, Python 3.5, Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34489>
_______________________________________

From report at bugs.python.org  Fri Aug 24 16:46:20 2018
From: report at bugs.python.org (Michael Felt)
Date: Fri, 24 Aug 2018 20:46:20 +0000
Subject: [New-bugs-announce] [issue34490]
 transport.get_extra_info('sockname') of test_asyncio fails on AIX
Message-ID: <1535143580.27.0.56676864532.issue34490@psf.upfronthosting.co.za>


New submission from Michael Felt <aixtools at felt.demon.nl>:

Inspired by msg211764, issue20682

In short: 
   For address family of AF_UNIX or AF_UNIX_CCSID, getsockname() returns 0 if issued before a bind(). The address length is 0. This is always the case for sockets created by socketpair().

That is at least one possible explanation I found.

Have tested on AIX 6.1 and AIX 7.1 - transport.get_extra_info('sockname') returns None

----------
components: Tests
messages: 324017
nosy: Michael.Felt
priority: normal
severity: normal
status: open
title: transport.get_extra_info('sockname') of test_asyncio fails on AIX
type: behavior
versions: Python 3.4, Python 3.5, Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34490>
_______________________________________

From report at bugs.python.org  Fri Aug 24 17:15:18 2018
From: report at bugs.python.org (Zackery Spytz)
Date: Fri, 24 Aug 2018 21:15:18 +0000
Subject: [New-bugs-announce] [issue34491] _bsddb.c: Missing Py_DECREF() in
 DB_join()
Message-ID: <1535145318.12.0.56676864532.issue34491@psf.upfronthosting.co.za>


Change by Zackery Spytz <zspytz at gmail.com>:


----------
components: Extension Modules
nosy: ZackerySpytz
priority: normal
severity: normal
status: open
title: _bsddb.c: Missing Py_DECREF() in DB_join()
type: resource usage
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34491>
_______________________________________

From report at bugs.python.org  Fri Aug 24 18:37:17 2018
From: report at bugs.python.org (Alexey Izbyshev)
Date: Fri, 24 Aug 2018 22:37:17 +0000
Subject: [New-bugs-announce] [issue34492] Python/coreconfig.c: Missing NULL
 check in _Py_wstrlist_copy()
Message-ID: <1535150237.85.0.56676864532.issue34492@psf.upfronthosting.co.za>


New submission from Alexey Izbyshev <izbyshev at ispras.ru>:

The return value of PyMem_RawMalloc() is not checked for NULL at https://github.com/python/cpython/blob/ef8861c112ed1dac9351958c121bc24ca4ecdb08/Python/coreconfig.c#L71.

Reported by Svace static analyzer.

----------
components: Interpreter Core
messages: 324021
nosy: berker.peksag, izbyshev, serhiy.storchaka, vstinner
priority: normal
severity: normal
status: open
title: Python/coreconfig.c: Missing NULL check in _Py_wstrlist_copy()
type: behavior
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34492>
_______________________________________

From report at bugs.python.org  Fri Aug 24 19:05:49 2018
From: report at bugs.python.org (Alexey Izbyshev)
Date: Fri, 24 Aug 2018 23:05:49 +0000
Subject: [New-bugs-announce] [issue34493] Objects/genobject.c: Missing NULL
 check in compute_cr_origin()
Message-ID: <1535151949.16.0.56676864532.issue34493@psf.upfronthosting.co.za>


New submission from Alexey Izbyshev <izbyshev at ispras.ru>:

The return value of PyTuple_New() is not checked for NULL at https://github.com/python/cpython/blob/ef8861c112ed1dac9351958c121bc24ca4ecdb08/Objects/genobject.c#L1130 and then dereferenced.

----------
components: Interpreter Core
messages: 324022
nosy: berker.peksag, izbyshev, njs, serhiy.storchaka
priority: normal
severity: normal
status: open
title: Objects/genobject.c: Missing NULL check in compute_cr_origin()
type: behavior
versions: Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34493>
_______________________________________

From report at bugs.python.org  Sat Aug 25 01:26:30 2018
From: report at bugs.python.org (Tyler Reddy)
Date: Sat, 25 Aug 2018 05:26:30 +0000
Subject: [New-bugs-announce] [issue34494] simple "sequence" class ignoring
 __len__
Message-ID: <1535174790.07.0.56676864532.issue34494@psf.upfronthosting.co.za>


New submission from Tyler Reddy <treddy at dal.ca>:

Downstream in NumPy we've noticed that a "sequence" object defined as below will hang (infinite __getitem__ calls) if we try to turn it into an array. The same holds in CPython for converting it to a list:

class OneList:

    def __len__(self):
        # this won't be checked by
        # PySequence_Fast and several
        # over C API calls
        return 1

    def __getitem__(self, x):
        # called indefinitely by
        # i.e., PySequence_Fast
        return 1

Just to confirm -- this is intentional / desired behavior:

list(OneList()) should hang in CPython?

related: 
https://github.com/numpy/numpy/issues/8912
https://github.com/numpy/numpy/pull/11815
https://stackoverflow.com/a/43566241/2942522

----------
messages: 324037
nosy: treddy
priority: normal
severity: normal
status: open
title: simple "sequence" class ignoring __len__
type: behavior
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34494>
_______________________________________

From report at bugs.python.org  Sat Aug 25 02:22:15 2018
From: report at bugs.python.org (Tatyana)
Date: Sat, 25 Aug 2018 06:22:15 +0000
Subject: [New-bugs-announce] [issue34495] excess_args asserts if args ==
 nullptr
Message-ID: <1535178135.11.0.56676864532.issue34495@psf.upfronthosting.co.za>


New submission from Tatyana <tatyana at synopsys.com>:

Since version 3.7 PyTuple_GET_SIZE performs assert(PyTuple_Check(args)); Due to this change, object_new and object_init cannot be called with args == nullptr.

Such behavior breaks existing user's code, particularly a code generated with Swig.

----------
components: Library (Lib)
messages: 324041
nosy: tkrasnukha
priority: normal
pull_requests: 8390
severity: normal
status: open
title: excess_args asserts if args == nullptr
type: crash
versions: Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34495>
_______________________________________

From report at bugs.python.org  Sat Aug 25 02:51:30 2018
From: report at bugs.python.org (fralau)
Date: Sat, 25 Aug 2018 06:51:30 +0000
Subject: [New-bugs-announce] [issue34496] Argparse library: parse --set type
Message-ID: <1535179890.87.0.56676864532.issue34496@psf.upfronthosting.co.za>


New submission from fralau <laur2035 at yahoo.co.uk>:

With argparse, an common requirement is is that a user may want to pass a series of key/value pairs (a sort of equivalent of **args on the command line). Here is a solution that could be implemented in the standard library with little effort.

This can be done so, e.g.:
    command --set foo=hello bar="hello world" baz=5

With the benefit of experience, this is easy and very intuitive.

This does not require any modification, as the '--set' option can be defined as:  
   nargs='+',
   metavar="KEY=VALUE"

The next step is to parse the resulting list.

See github gist for a trivial implementation: https://gist.github.com/fralau/061a4f6c13251367ef1d9a9a99fb3e8d). 

It might be useful to integrate a helper method to the standard library together with an explanation in the doc on how to do it.

----------
components: Library (Lib)
messages: 324043
nosy: fralau
priority: normal
severity: normal
status: open
title: Argparse library: parse --set type
type: enhancement

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34496>
_______________________________________

From report at bugs.python.org  Sat Aug 25 07:37:26 2018
From: report at bugs.python.org (Dan Snider)
Date: Sat, 25 Aug 2018 11:37:26 +0000
Subject: [New-bugs-announce] [issue34497] Remove needless set operator
 restriction
Message-ID: <1535197046.82.0.56676864532.issue34497@psf.upfronthosting.co.za>


New submission from Dan Snider <mr.assume.away at gmail.com>:

I only just now realized that `dict.fromkeys('abc').keys() - 'bc'` returns {'a'} instead of raising an error like {*'abc'} - 'bc' would, which is really quite handy. 

The __xor__, __and__, and __sub__ methods of dict_keys (and items, assuming no there are no unhashable values) work just as set.symmetric_difference, set.intersection, and set.difference do, respectively.

>>> a, b, c, d = [*map(dict.keys, map(dict.fromkeys, 'abcd'))]
>>> ((a | 'a') | (b & 'b') | (c ^ 'c')) - d
{'b', 'a'}
>>> a, b, c, d = [*map(dict.items, map(dict.fromkeys, 'abcd'))]
>>> ((a | 'a') | (b & 'b') | (c ^ 'c')) - d
{'c', ('a', None), 'a', ('c', None)}

However, set objects are arbitrarily restricted to taking a set object for the second argument on these functions. As for the first example here, there is even code specifically there to handle a dictionary as the second argument, but it is unreachable when called through the dunder version. 

{<class 'list'>, <class 'dict'>, <class 'set'>}
>>> {list, set} | dict.fromkeys((dict, set))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: unsupported operand type(s) for |: 'set' and 'dict'

>>> {*'abc'}.difference('cde')
{'b', 'a'}
>>> {*'abc'} - set('cde')
{'b', 'a'}
>>> {*'abc'} - 'cde'
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: unsupported operand type(s) for -: 'set' and 'str'

>>> {1,2,3}.symmetric_difference(b'\x00')
{0, 1, 2, 3}
>>> {1,2,3} ^ b'\x00'
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: unsupported operand type(s) for ^: 'set' and 'bytes'

The sources of set_and, set_sub, and set_xor all look like this. All they do is add a check that the second argument is a set and then simply call the same function their respective non-dunder method uses. They're so identical in fact that set_xor actually calls the exact same C function used in the PyMethodDef for set.symmetric_update:

static PyObject *
set_xor(PySetObject *so, PyObject *other)
{
    if (!PyAnySet_Check(so) || !PyAnySet_Check(other))
        Py_RETURN_NOTIMPLEMENTED;
    return set_symmetric_difference(so, other);
}
static PyMethodDef set_methods[] = {
/* ... */
{"symmetric_difference",(PyCFunction)set_symmetric_difference, 
 METH_O, symmetric_difference_doc},
/* ... */
};

All that's needed to fix this is to remove a total of 106 characters from setobject.c (4 x " || !PyAnySet_Check(other)").

----------
components: Interpreter Core, Library (Lib)
messages: 324060
nosy: bup
priority: normal
severity: normal
status: open
title: Remove needless set operator restriction
type: enhancement
versions: Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34497>
_______________________________________

From report at bugs.python.org  Sat Aug 25 08:19:29 2018
From: report at bugs.python.org (Dutcho)
Date: Sat, 25 Aug 2018 12:19:29 +0000
Subject: [New-bugs-announce] [issue34498] Python 3.7 breaks on
 singledispatch_function.register(pseudo_type), which Python 3.6 accepted
Message-ID: <1535199569.5.0.56676864532.issue34498@psf.upfronthosting.co.za>


New submission from Dutcho <dutcho at ziggo.nl>:

In Python 3.6, the argument to the register() method of a single-dispatch function (i.e. a function decorated by @functools.singledispatch) can be a 'type-like' class (pseudo-type?) from e.g. the typing module that supports isinstance().

The below demonstrates it works in Python 3.6:

$ py -3.6
Python 3.6.6 (v3.6.6:4cf1f54eb7, Jun 27 2018, 03:37:03) [MSC v.1900 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> from functools import singledispatch
>>> from typing import Sequence
>>> isinstance([1,2,3], Sequence) # 'pseudo-type' okay with isinstance
True
>>> @singledispatch
... def f(arg):
...     print('unqualified', arg)
...
>>> @f.register(Sequence) # 'pseudo-type' okay with register
... def _(arg):
...     print('sequence', *arg)
...
>>> f(1)
unqualified 1
>>> f([1,2,3])
sequence 1 2 3

This code breaks in Python 3.7:

$ py
Python 3.7.0 (v3.7.0:1bf9cc5093, Jun 27 2018, 04:59:51) [MSC v.1914 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> from functools import singledispatch
>>> from typing import Sequence
>>> isinstance([1,2,3], Sequence) # 'pseudo-type' okay with isinstance
True
>>> @singledispatch
... def f(arg):
...     print('unqualified', arg)
...
>>> @f.register(Sequence)  # 'pseudo-type' NOT okay with register
... def _(arg):
...     print('sequence', *arg)
...
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "D:\Program Files\Python\Python37\lib\functools.py", line 801, in register
    f"Invalid first argument to `register()`: {cls!r}. "
TypeError: Invalid first argument to `register()`: typing.Sequence. Use either `@register(some_class)` or plain `@register` on an annotated function.

While agreeing a check on the register() arg IS required, the current check isinstance(cls, type) seems overly restrictive. This is especially true when considering the (welcome!) Python 3.7 use of annotations for single dispatch functions.

----------
components: Library (Lib)
messages: 324061
nosy: Dutcho
priority: normal
severity: normal
status: open
title: Python 3.7 breaks on singledispatch_function.register(pseudo_type), which Python 3.6 accepted
type: crash
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34498>
_______________________________________

From report at bugs.python.org  Sat Aug 25 08:38:48 2018
From: report at bugs.python.org (Dutcho)
Date: Sat, 25 Aug 2018 12:38:48 +0000
Subject: [New-bugs-announce] [issue34499] Extend registering of
 single-dispatch functions to parametrized generic pseudo-types
Message-ID: <1535200728.53.0.56676864532.issue34499@psf.upfronthosting.co.za>


New submission from Dutcho <dutcho at ziggo.nl>:

While issue #34498 signalled a break in Python 3.7 of legal Python 3.6 code that registers a single-dispatch function with a 'pseudo-type' like typing.Sequence, both Python 3.6 and Python 3.7 don't work with a 'parametrized pseudo-type' like typing.Sequence[int].
However, to fully benefit from Python 3.7's use of annotations for registering single-dispatch functions, these 'parametrized pseudo-types' should also be acceptable.

E.g. in Python 3.6:
$ py -3.6
Python 3.6.6 (v3.6.6:4cf1f54eb7, Jun 27 2018, 03:37:03) [MSC v.1900 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> from functools import singledispatch
>>> from typing import Iterable, Sequence, T
>>>
>>> @singledispatch
... def last(iterable: Iterable[T]) -> T:
...     for last in iterable:
...             pass
...     return 'slow', last
...
>>> @last.register(Sequence)
... def _(sequence: Sequence[T]) -> T:
...     return 'fast', sequence[-1]
...
>>> print(*last(iter([1, 2, 3])))
slow 3
>>> print(*last([1, 2, 3]))
fast 3

Note in Python 3.6 no match results if the 'parametrized pseudo-type' Sequence[T] is specified as argument to last.register() as isinstance(..., Sequence[T]) fails.

To fully benefit from Python 3.7's use of annotations for single-dispatch functions, it should
(1) accept 
>>> @last.register
... def _(sequence: Sequence[T]) -> T:
...     return 'fast', sequence[-1]
and (2) remove the T parameter of Sequence for matching when dispatching. I recognize this could potentially create a clash between e.g. Sequence[int] and Sequence[str]

----------
components: Library (Lib)
messages: 324063
nosy: Dutcho
priority: normal
severity: normal
status: open
title: Extend registering of single-dispatch functions to parametrized generic pseudo-types
type: enhancement
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34499>
_______________________________________

From report at bugs.python.org  Sat Aug 25 11:09:38 2018
From: report at bugs.python.org (=?utf-8?q?Micka=C3=ABl_Schoentgen?=)
Date: Sat, 25 Aug 2018 15:09:38 +0000
Subject: [New-bugs-announce] [issue34500] Fix ResourceWarning in difflib.py
Message-ID: <1535209778.67.0.56676864532.issue34500@psf.upfronthosting.co.za>


New submission from Micka?l Schoentgen <contact at tiger-222.fr>:

Fix 2 unclosed files in the documentation and the difflib.py script:

diff.py:77: ResourceWarning: unclosed file <_io.TextIOWrapper name='/tmp/tmpnby8pemn' mode='U' encoding='UTF-8'>
    fromlines = open(fromfile, 'U').readlines()
diff.py:78: ResourceWarning: unclosed file <_io.TextIOWrapper name='/tmp/tmpnby8pemm' mode='U' encoding='UTF-8'>
    self.tolines = open(tofile, 'U').readlines()

----------
assignee: docs at python
components: Demos and Tools, Documentation
messages: 324071
nosy: David Bieber, Tiger-222, docs at python
priority: normal
severity: normal
status: open
title: Fix ResourceWarning in difflib.py
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34500>
_______________________________________

From report at bugs.python.org  Sat Aug 25 11:26:40 2018
From: report at bugs.python.org (Alexey Izbyshev)
Date: Sat, 25 Aug 2018 15:26:40 +0000
Subject: [New-bugs-announce] [issue34501] PyType_FromSpecWithBases:
 spec->name is dereferenced before checking for NULL
Message-ID: <1535210800.55.0.56676864532.issue34501@psf.upfronthosting.co.za>


New submission from Alexey Izbyshev <izbyshev at ispras.ru>:

In the following snippet from PyType_FromSpecWithBases() in Objects/typeobject.c, spec->name is dereferenced by strrchr() but then is checked for NULL:

    /* Set the type name and qualname */
    s = strrchr(spec->name, '.');
    if (s == NULL)
        s = (char*)spec->name;
    else
        s++;

    [snip]

    type->tp_name = spec->name;
    if (!type->tp_name)
        goto fail;

This was reported by Svace static analyzer.

If I were to check spec->name first, what error should I report to the caller? Is something like the following OK?

    if (spec->name == NULL) {
        PyErr_SetString(PyExc_SystemError,
                       "Type spec does not define the name field.");
        goto fail;
    }

----------
components: Interpreter Core
messages: 324073
nosy: berker.peksag, izbyshev, serhiy.storchaka
priority: normal
severity: normal
status: open
title: PyType_FromSpecWithBases: spec->name is dereferenced before checking for NULL
type: behavior
versions: Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34501>
_______________________________________

From report at bugs.python.org  Sat Aug 25 13:54:28 2018
From: report at bugs.python.org (Alexey Izbyshev)
Date: Sat, 25 Aug 2018 17:54:28 +0000
Subject: [New-bugs-announce] [issue34502] Docs of sys.exit() mention
 utf8_mode for an unclear reason
Message-ID: <1535219668.13.0.56676864532.issue34502@psf.upfronthosting.co.za>


New submission from Alexey Izbyshev <izbyshev at ispras.ru>:

Docs for sys.exit() contain the following note added in https://github.com/python/cpython/blame/1871a4a35123b278443b960255cb14314ce196f6/Doc/library/sys.rst#L336

Changed in version 3.7: Added utf8_mode attribute for the new -X utf8 flag.

Does it belong to the adjacent docs for sys.flags (though they mention utf8_mode too)?

----------
assignee: docs at python
components: Documentation
messages: 324086
nosy: docs at python, izbyshev, vstinner
priority: normal
severity: normal
status: open
title: Docs of sys.exit() mention utf8_mode for an unclear reason
versions: Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34502>
_______________________________________

From report at bugs.python.org  Sat Aug 25 16:00:20 2018
From: report at bugs.python.org (Alexey Izbyshev)
Date: Sat, 25 Aug 2018 20:00:20 +0000
Subject: [New-bugs-announce] [issue34503] Reference leak in PyErr_SetObject()
Message-ID: <1535227220.49.0.56676864532.issue34503@psf.upfronthosting.co.za>


New submission from Alexey Izbyshev <izbyshev at ispras.ru>:

'exc_value' reference is leaked at https://github.com/python/cpython/blob/5f79b50763d687aeeed8edcb4efcc7ac9f8fa186/Python/errors.c#L113

----------
components: Interpreter Core
messages: 324099
nosy: benjamin.peterson, berker.peksag, izbyshev, serhiy.storchaka
priority: normal
severity: normal
status: open
title: Reference leak in PyErr_SetObject()
type: resource usage
versions: Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34503>
_______________________________________

From report at bugs.python.org  Sat Aug 25 17:05:49 2018
From: report at bugs.python.org (Alexey Izbyshev)
Date: Sat, 25 Aug 2018 21:05:49 +0000
Subject: [New-bugs-announce] [issue34504] PySequence_Check: argument is
 dereferenced and then checked for NULL
Message-ID: <1535231149.7.0.56676864532.issue34504@psf.upfronthosting.co.za>


New submission from Alexey Izbyshev <izbyshev at ispras.ru>:

The argument of PySequence_Check() is dereferenced at https://github.com/python/cpython/blob/5f79b50763d687aeeed8edcb4efcc7ac9f8fa186/Objects/abstract.c#L1511 and then checked for NULL.

Reported by Svace static analyzer.

Since other PySomething_Check() functions seem to expect a non-NULL argument, I suggest to remove the NULL check instead of moving it.

----------
components: Interpreter Core
messages: 324101
nosy: benjamin.peterson, berker.peksag, izbyshev, serhiy.storchaka
priority: normal
severity: normal
status: open
title: PySequence_Check: argument is dereferenced and then checked for NULL
type: behavior
versions: Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34504>
_______________________________________

From report at bugs.python.org  Sat Aug 25 21:24:24 2018
From: report at bugs.python.org (Balibax)
Date: Sun, 26 Aug 2018 01:24:24 +0000
Subject: [New-bugs-announce] [issue34505] urllib2 fails for proxy
 credentials that contains a '/' character
Message-ID: <1535246664.68.0.56676864532.issue34505@psf.upfronthosting.co.za>


New submission from Balibax <gamekela1 at gmail.com>:

How can solve this<a href="https://www.pinterest.com/CoreTricks/">?</a>

----------
components: Build
messages: 324111
nosy: Balibax5445
priority: normal
severity: normal
status: open
title: urllib2 fails for proxy credentials that contains a '/' character
type: crash

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34505>
_______________________________________

From report at bugs.python.org  Sun Aug 26 03:44:30 2018
From: report at bugs.python.org (=?utf-8?b?SHJ2b2plIE5pa8WhacSH?=)
Date: Sun, 26 Aug 2018 07:44:30 +0000
Subject: [New-bugs-announce] [issue34506] Traceback logged when SSL
 handshake fails
Message-ID: <1535269470.94.0.56676864532.issue34506@psf.upfronthosting.co.za>


New submission from Hrvoje Nik?i? <hniksic at gmail.com>:

When an SSL handshake fails in asyncio, an exception traceback is logged to standard error even if the application code catches the exception. This logging cannot be suppressed, except by providing a custom exception handler for the whole event loop. The question was raised on StackOverflow in https://stackoverflow.com/q/52012488/1600898

To reproduce the issue, run the attached minimal example (taken from the SO question). Expected behavior is for "Error handled" to be printed. Actual behavior is that, in addition to that line, two tracebacks are printed.

It looks like a bug that _fatal_error both logs the exception and calls connection_lost on the protocol (through transport._force_close()). If the idea is for the exception not to get swallowed by protocols that don't implement a sensible connection_lost (e.g. because they've just inherited from BaseProtocol, like the echo server examples), then maybe a protocol that propagates the exception in connection_lost should be able to opt out of the additional logging. That way the stream protocols would avoid spurious output for the suppressed exception by default, and the same opt-out mechanism would be available to user-written protocols.

----------
files: sslerr.py
messages: 324113
nosy: hniksic
priority: normal
severity: normal
status: open
title: Traceback logged when SSL handshake fails
Added file: https://bugs.python.org/file47763/sslerr.py

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34506>
_______________________________________

From report at bugs.python.org  Sun Aug 26 07:00:22 2018
From: report at bugs.python.org (Lior Cohen)
Date: Sun, 26 Aug 2018 11:00:22 +0000
Subject: [New-bugs-announce] [issue34507] Add path expansion interpolation
 in pyvenv.cfg home key
Message-ID: <1535281222.0.0.56676864532.issue34507@psf.upfronthosting.co.za>


New submission from Lior Cohen <chnlior at gmail.com>:

When working with venv virtual environment, python.exe finds its base_prefix and base_exec_prefix by the home key in the pyvenv.cfg file.

If one edits this file with $xyz (or %xyz% in windows), it is not not resolved.

The reason of replacing the absolute path generated by venv to path expand tokens, is to be able to port the venv + its base python to different machine, which is part of my company deployment process.

----------
components: Interpreter Core
messages: 324116
nosy: chnlior
priority: normal
severity: normal
status: open
title: Add path expansion interpolation in pyvenv.cfg home key
type: enhancement
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34507>
_______________________________________

From report at bugs.python.org  Sun Aug 26 10:25:50 2018
From: report at bugs.python.org (Mark Dickinson)
Date: Sun, 26 Aug 2018 14:25:50 +0000
Subject: [New-bugs-announce] [issue34508] return of non-parenthesized
 star-unpacking expression a SyntaxError
Message-ID: <1535293550.9.0.56676864532.issue34508@psf.upfronthosting.co.za>


New submission from Mark Dickinson <dickinsm at gmail.com>:

[From https://stackoverflow.com/q/52026406/270986]

The following is valid, and works as expected:

>>> def f():
...     x = *(1, 2), 3
...     return x
... 
>>> f()
(1, 2, 3)

But the tuple expression can't be used directly in a "return" statement:

>>> def f():
...     return *(1, 2), 3
  File "<stdin>", line 2
    return *(1, 2), 3
           ^
SyntaxError: invalid syntax

It's trivial to work around, by adding an extra pair of parentheses around the return target, but it seems a surprising inconsistency. Would it make sense to allow this? In terms of the grammar,

    return_stmt: 'return' [testlist]

would be replaced with:

    return_stmt: 'return' [testlist_star_expr]

There may be other places in the grammar where "testlist" could reasonably be replaced with "testlist_star_expr", for example:

    for_stmt: 'for' exprlist 'in' testlist ':' suite ['else' ':' suite]

----------
components: Interpreter Core
messages: 324118
nosy: mark.dickinson
priority: normal
severity: normal
status: open
title: return of non-parenthesized star-unpacking expression a SyntaxError
type: enhancement
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34508>
_______________________________________

From report at bugs.python.org  Sun Aug 26 13:14:57 2018
From: report at bugs.python.org (Jun Aruga)
Date: Sun, 26 Aug 2018 17:14:57 +0000
Subject: [New-bugs-announce] [issue34509] Starting to use gcc-8 on CI
Message-ID: <1535303697.92.0.56676864532.issue34509@psf.upfronthosting.co.za>


New submission from Jun Aruga <happy at oh4u.net>:

This ticket is from below conversation on python-dev mailing list.
https://mail.python.org/pipermail/python-dev/2018-August/155011.html

Right now the Python project Travis CI is running on gcc-4.8.4 for coverage testing.
I want to replace it or add the latest version gcc-8 case.

The benefit to use gcc-8 is,

* It is important to run on the old version 4.8 but, it is also meaningful to follow the latest version and care issues such as warnings on the latest version gcc.
* I am working in Fedora project that is using gcc version 8 as a main c compiler. If the Python project care gcc-8, that might be helpful for people who maintain Python Debian/Fedora package or Installer on Mac/Windows.


The demerit is

* Right now the gcc-4.8 is used for the coverage. The total running time on gcc-8 might be longer than current one.
* To run the latest gcc-8, we can add the repository [1] with the way [2][3], but I am not sure whether the repository is stable.

I would show you 2 cases to use gcc-8 on Linux.

[1] https://launchpad.net/~ubuntu-toolchain-r/+archive/ubuntu/test
[2] https://docs.travis-ci.com/user/installing-dependencies/#installing-packages-from-a-custom-apt-repository
[3] https://docs.travis-ci.com/user/languages/c/#gcc-on-os-x
[4] Ruby
   https://travis-ci.org/junaruga/ruby/builds/418242410
   https://github.com/junaruga/ruby/blob/feature/ci-new-gcc/.travis.yml
   https://github.com/ruby/ruby/pull/1937
[5] Trinity
   Example for gcc-8 on both Linux and Mac.
   https://travis-ci.org/trinityrnaseq/trinityrnaseq
   https://github.com/trinityrnaseq/trinityrnaseq/blob/master/.travis.yml

----------
messages: 324125
nosy: junaruga
priority: normal
severity: normal
status: open
title: Starting to use gcc-8 on CI
type: enhancement
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34509>
_______________________________________

From report at bugs.python.org  Sun Aug 26 16:05:27 2018
From: report at bugs.python.org (Collin Anderson)
Date: Sun, 26 Aug 2018 20:05:27 +0000
Subject: [New-bugs-announce] [issue34510] Add add HTTPConnection.settimeout()
Message-ID: <1535313927.12.0.56676864532.issue34510@psf.upfronthosting.co.za>


Change by Collin Anderson <cmawebsite at gmail.com>:


----------
components: Library (Lib)
nosy: collinanderson
priority: normal
pull_requests: 8422
severity: normal
status: open
title: Add add HTTPConnection.settimeout()
type: enhancement

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34510>
_______________________________________

From report at bugs.python.org  Sun Aug 26 16:49:49 2018
From: report at bugs.python.org (harobed)
Date: Sun, 26 Aug 2018 20:49:49 +0000
Subject: [New-bugs-announce] [issue34511] I suggest to add documentation
 about "method" parameter of urllib.request.Request class
Message-ID: <1535316589.49.0.56676864532.issue34511@psf.upfronthosting.co.za>


New submission from harobed <stephane at harobed.org>:

Hi,

I see "method=None" parameter in urllib.request.Request constructor (https://github.com/python/cpython/blob/master/Lib/urllib/request.py#L327) but I don't see "method" parameter in documentation: https://docs.python.org/3.2/library/urllib.request.html#urllib.request.Request

I suggest to add documentation about "method".

Best regards,
St?phane

----------
assignee: docs at python
components: Documentation, Library (Lib)
messages: 324134
nosy: docs at python, harobed
priority: normal
severity: normal
status: open
title: I suggest to add documentation about "method" parameter of urllib.request.Request class
type: enhancement
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34511>
_______________________________________

From report at bugs.python.org  Sun Aug 26 17:06:22 2018
From: report at bugs.python.org (Alexey Izbyshev)
Date: Sun, 26 Aug 2018 21:06:22 +0000
Subject: [New-bugs-announce] [issue34512] Document platform-specific
 strftime() behavior for non-ASCII format strings
Message-ID: <1535317582.77.0.56676864532.issue34512@psf.upfronthosting.co.za>


New submission from Alexey Izbyshev <izbyshev at ispras.ru>:

If a format string contains code points outside of ASCII range, time.strftime() can behave in four different ways depending on the platform, the current locale and the code points:

* raise a UnicodeEncodeError
* return an empty string
* for surrogates in \uDC80-\uDCFF range, replace them with different code points in the output (potentially mangling nearby parts of the output as well)
* round-trip them correctly

Some examples:

* Linux (glibc 2.27):
Python 3.6.4 (default, Jan 03 2018, 13:52:55) [GCC] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import time, locale
>>> locale.getlocale()
('en_US', 'UTF-8')
>>> time.strftime('\x80')
'\x80'
>>> time.strftime('\u044f')
'?' # '\u044f'
>>> time.strftime('\ud800')
'\ud800'
>>> time.strftime('\udcff')
'\udcff'
>>> locale.setlocale(locale.LC_CTYPE, 'C')
'C'
>>> time.strftime('\x80')
'\x80'
>>> time.strftime('\u044f')
'?' # '\u044f'
>>> time.strftime('\ud800')
'\ud800'
>>> time.strftime('\udcff')
'\udcff'

* macOS 10.13.6 and FreeBSD 11.1:
Python 3.7.0 (default, Jul 23 2018, 20:22:55)
[Clang 9.1.0 (clang-902.0.39.2)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import time, locale
>>> locale.getlocale()
('en_US', 'UTF-8')
>>> time.strftime('\x80')
'\x80'
>>> time.strftime('\u044f')
'?' # '\u044f'
>>> time.strftime('\ud800')
''
>>> time.strftime('\udcff')
''
>>> locale.setlocale(locale.LC_CTYPE, 'C')
'C'
>>> time.strftime('\x80')
'\x80'
>>> time.strftime('\u044f')
''
>>> time.strftime('\ud800')
''
>>> time.strftime('\udcff')
''

* Windows 8.1:
Python 3.7.0 (v3.7.0:1bf9cc5093, Jun 27 2018, 04:59:51) [MSC v.1914 64 bit (AMD64)] on win32
>>> import time, locale
>>> locale.getlocale()
(None, None)
>>> time.strftime('\x80')
'\x80'
>>> time.strftime('\u044f')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
UnicodeEncodeError: 'locale' codec can't encode character '\u044f' in position 0: encoding error
>>> time.strftime('\ud800')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
UnicodeEncodeError: 'locale' codec can't encode character '\ud800' in position 0: encoding error
>>> time.strftime('\udcff')
'y' # '\xff'
>>> locale.setlocale(locale.LC_CTYPE, '')
'Russian_Russia.1251'
>>> time.strftime('\x80')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
UnicodeEncodeError: 'locale' codec can't encode character '\x80' in position 0: encoding error
>>> time.strftime('\u044f')
'?' # '\u044f'
>>> time.strftime('\ud800')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
UnicodeEncodeError: 'locale' codec can't encode character '\ud800' in position 0: encoding error
>>> time.strftime('\udcff')
'?' # '\u044f'

The reasons of such differences are the following:
* Reliance on either wcsftime() or strftime() from the C library depending on the platform.
* For strftime(), the input is encoded into the charset of the current locale with 'surrogateescape' error handler, and the output is decoded back in the same way.
* Different handling of code points which are unrepresentable in the charset of the current locale by glibc and macOS/FreeBSD.

I suggest to at least document that the format string, despite being an 'str', requires special care if it contains non-ASCII code points.

The 'datetime' module docs warn about the locale-dependent output, but only with regard to particular format specifiers [1].

I'll submit a draft PR. Suggestions are welcome.

[1] https://docs.python.org/3.7/library/datetime.html#strftime-and-strptime-behavior

----------
assignee: docs at python
components: Documentation
messages: 324136
nosy: belopolsky, docs at python, izbyshev, p-ganssle, taleinat
priority: normal
severity: normal
status: open
title: Document platform-specific strftime() behavior for non-ASCII format strings
type: enhancement
versions: Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34512>
_______________________________________

From report at bugs.python.org  Sun Aug 26 20:40:23 2018
From: report at bugs.python.org (Pablo Galindo Salgado)
Date: Mon, 27 Aug 2018 00:40:23 +0000
Subject: [New-bugs-announce] [issue34513] test_multiprocessing_spawn fails
 on x86 Windows7 3.7 buildbot
Message-ID: <1535330423.47.0.56676864532.issue34513@psf.upfronthosting.co.za>


New submission from Pablo Galindo Salgado <pablogsal at gmail.com>:

test_multiprocessing_spawn fails on x86 Windows7 3.7 buildbot:

https://buildbot.python.org/all/#/builders/111/builds/500

Timeout (0:15:00)!
Thread 0x00000c64 (most recent call first):
  File "D:\cygwin\home\db3l\buildarea\3.7.bolen-windows7\build\lib\multiprocessing\connection.py", line 306 in _recv_bytes
  File "D:\cygwin\home\db3l\buildarea\3.7.bolen-windows7\build\lib\multiprocessing\connection.py", line 250 in recv
  File "D:\cygwin\home\db3l\buildarea\3.7.bolen-windows7\build\lib\multiprocessing\pool.py", line 470 in _handle_results
  File "D:\cygwin\home\db3l\buildarea\3.7.bolen-windows7\build\lib\threading.py", line 865 in run
  File "D:\cygwin\home\db3l\buildarea\3.7.bolen-windows7\build\lib\threading.py", line 917 in _bootstrap_inner
  File "D:\cygwin\home\db3l\buildarea\3.7.bolen-windows7\build\lib\threading.py", line 885 in _bootstrap
Thread 0x000000b4 (most recent call first):
  File "D:\cygwin\home\db3l\buildarea\3.7.bolen-windows7\build\lib\multiprocessing\pool.py", line 422 in _handle_tasks
  File "D:\cygwin\home\db3l\buildarea\3.7.bolen-windows7\build\lib\threading.py", line 865 in run
  File "D:\cygwin\home\db3l\buildarea\3.7.bolen-windows7\build\lib\threading.py", line 917 in _bootstrap_inner
  File "D:\cygwin\home\db3l\buildarea\3.7.bolen-windows7\build\lib\threading.py", line 885 in _bootstrap
Thread 0x000008d4 (most recent call first):
  File "D:\cygwin\home\db3l\buildarea\3.7.bolen-windows7\build\lib\multiprocessing\pool.py", line 413 in _handle_workers
  File "D:\cygwin\home\db3l\buildarea\3.7.bolen-windows7\build\lib\threading.py", line 865 in run
  File "D:\cygwin\home\db3l\buildarea\3.7.bolen-windows7\build\lib\threading.py", line 917 in _bootstrap_inner
  File "D:\cygwin\home\db3l\buildarea\3.7.bolen-windows7\build\lib\threading.py", line 885 in _bootstrap
Thread 0x00000b88 (most recent call first):
  File "D:\cygwin\home\db3l\buildarea\3.7.bolen-windows7\build\lib\threading.py", line 296 in wait
  File "D:\cygwin\home\db3l\buildarea\3.7.bolen-windows7\build\lib\threading.py", line 552 in wait
  File "D:\cygwin\home\db3l\buildarea\3.7.bolen-windows7\build\lib\multiprocessing\pool.py", line 648 in wait
  File "D:\cygwin\home\db3l\buildarea\3.7.bolen-windows7\build\lib\multiprocessing\pool.py", line 651 in get
  File "D:\cygwin\home\db3l\buildarea\3.7.bolen-windows7\build\lib\multiprocessing\pool.py", line 261 in apply
  File "D:\cygwin\home\db3l\buildarea\3.7.bolen-windows7\build\lib\test\_test_multiprocessing.py", line 2224 in test_apply
  File "D:\cygwin\home\db3l\buildarea\3.7.bolen-windows7\build\lib\unittest\case.py", line 615 in run
  File "D:\cygwin\home\db3l\buildarea\3.7.bolen-windows7\build\lib\unittest\case.py", line 663 in __call__
  File "D:\cygwin\home\db3l\buildarea\3.7.bolen-windows7\build\lib\unittest\suite.py", line 122 in run
  File "D:\cygwin\home\db3l\buildarea\3.7.bolen-windows7\build\lib\unittest\suite.py", line 84 in __call__
  File "D:\cygwin\home\db3l\buildarea\3.7.bolen-windows7\build\lib\unittest\suite.py", line 122 in run
  File "D:\cygwin\home\db3l\buildarea\3.7.bolen-windows7\build\lib\unittest\suite.py", line 84 in __call__
  File "D:\cygwin\home\db3l\buildarea\3.7.bolen-windows7\build\lib\unittest\suite.py", line 122 in run
  File "D:\cygwin\home\db3l\buildarea\3.7.bolen-windows7\build\lib\unittest\suite.py", line 84 in __call__
  File "D:\cygwin\home\db3l\buildarea\3.7.bolen-windows7\build\lib\unittest\runner.py", line 176 in run
  File "D:\cygwin\home\db3l\buildarea\3.7.bolen-windows7\build\lib\test\support\__init__.py", line 1882 in _run_suite
  File "D:\cygwin\home\db3l\buildarea\3.7.bolen-windows7\build\lib\test\support\__init__.py", line 1972 in run_unittest
  File "D:\cygwin\home\db3l\buildarea\3.7.bolen-windows7\build\lib\test\libregrtest\runtest.py", line 175 in test_runner
  File "D:\cygwin\home\db3l\buildarea\3.7.bolen-windows7\build\lib\test\libregrtest\runtest.py", line 179 in runtest_inner
  File "D:\cygwin\home\db3l\buildarea\3.7.bolen-windows7\build\lib\test\libregrtest\runtest.py", line 140 in runtest
  File "D:\cygwin\home\db3l\buildarea\3.7.bolen-windows7\build\lib\test\libregrtest\main.py", line 286 in rerun_failed_tests
  File "D:\cygwin\home\db3l\buildarea\3.7.bolen-windows7\build\lib\test\libregrtest\main.py", line 570 in _main
  File "D:\cygwin\home\db3l\buildarea\3.7.bolen-windows7\build\lib\test\libregrtest\main.py", line 531 in main
  File "D:\cygwin\home\db3l\buildarea\3.7.bolen-windows7\build\lib\test\libregrtest\main.py", line 584 in main
  File "D:\cygwin\home\db3l\buildarea\3.7.bolen-windows7\build\lib\test\__main__.py", line 2 in <module>
  File "D:\cygwin\home\db3l\buildarea\3.7.bolen-windows7\build\lib\runpy.py", line 85 in _run_code
  File "D:\cygwin\home\db3l\buildarea\3.7.bolen-windows7\build\lib\runpy.py", line 193 in _run_module_as_main
Process SpawnPoolWorker-184:
Traceback (most recent call last):
  File "D:\cygwin\home\db3l\buildarea\3.7.bolen-windows7\build\lib\multiprocessing\pool.py", line 127, in worker
    put((job, i, result))
  File "D:\cygwin\home\db3l\buildarea\3.7.bolen-windows7\build\lib\multiprocessing\queues.py", line 361, in put
    self._writer.send_bytes(obj)
  File "D:\cygwin\home\db3l\buildarea\3.7.bolen-windows7\build\lib\multiprocessing\connection.py", line 200, in send_bytes
    self._send_bytes(m[offset:offset + size])
  File "D:\cygwin\home\db3l\buildarea\3.7.bolen-windows7\build\lib\multiprocessing\connection.py", line 280, in _send_bytes
    ov, err = _winapi.WriteFile(self._handle, buf, overlapped=True)
BrokenPipeError: [WinError 232] The pipe is being closed
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "D:\cygwin\home\db3l\buildarea\3.7.bolen-windows7\build\lib\multiprocessing\process.py", line 297, in _bootstrap
    self.run()
  File "D:\cygwin\home\db3l\buildarea\3.7.bolen-windows7\build\lib\multiprocessing\process.py", line 99, in run
    self._target(*self._args, **self._kwargs)
  File "D:\cygwin\home\db3l\buildarea\3.7.bolen-windows7\build\lib\multiprocessing\pool.py", line 132, in worker
    put((job, i, (False, wrapped)))
  File "D:\cygwin\home\db3l\buildarea\3.7.bolen-windows7\build\lib\multiprocessing\queues.py", line 361, in put
    self._writer.send_bytes(obj)
  File "D:\cygwin\home\db3l\buildarea\3.7.bolen-windows7\build\lib\multiprocessing\connection.py", line 200, in send_bytes
    self._send_bytes(m[offset:offset + size])
  File "D:\cygwin\home\db3l\buildarea\3.7.bolen-windows7\build\lib\multiprocessing\connection.py", line 280, in _send_bytes
    ov, err = _winapi.WriteFile(self._handle, buf, overlapped=True)
BrokenPipeError: [WinError 232] The pipe is being closed
program finished with exit code 1
elapsedTime=4600.317000

----------
components: Tests, Windows
messages: 324137
nosy: pablogsal, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: test_multiprocessing_spawn fails on x86 Windows7 3.7 buildbot
type: behavior
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34513>
_______________________________________

From report at bugs.python.org  Sun Aug 26 22:21:09 2018
From: report at bugs.python.org (Victor Engmark)
Date: Mon, 27 Aug 2018 02:21:09 +0000
Subject: [New-bugs-announce] [issue34514] assertEqual doesn't use maxDiff
 when comparing dictionaries
Message-ID: <1535336469.7.0.56676864532.issue34514@psf.upfronthosting.co.za>


New submission from Victor Engmark <victorengmark.catalyst at gmail.com>:

According to the documentation `assertDictEqual()` "will be used by default to compare dictionaries in calls to assertEqual()." Great, since `maxDiff` applies to assertDictEqual(). However, `maxDiff` doesn't apply to `assertEqual()` when comparing dictionaries.

----------
components: Tests
messages: 324140
nosy: Victor Engmark2
priority: normal
severity: normal
status: open
title: assertEqual doesn't use maxDiff when comparing dictionaries
type: behavior
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34514>
_______________________________________

From report at bugs.python.org  Mon Aug 27 02:47:47 2018
From: report at bugs.python.org (monson)
Date: Mon, 27 Aug 2018 06:47:47 +0000
Subject: [New-bugs-announce] [issue34515] lib2to3: support non-ASCII
 identifiers
Message-ID: <1535352467.5.0.56676864532.issue34515@psf.upfronthosting.co.za>


New submission from monson <holymonson at gmail.com>:

Python 3.0 introduces additional characters from outside the ASCII range (see PEP 3131). see https://docs.python.org/3/reference/lexical_analysis.html#identifiers

But lib2to3 can't tokenize them corretly.
```
$ echo '? = 1' | python3.7 -m lib2to3.pgen2.tokenize
1,0-1,1:	ERRORTOKEN	'?'
1,2-1,3:	OP	'='
1,4-1,5:	NUMBER	'1'
1,5-1,6:	NEWLINE	'\n'
2,0-2,0:	ENDMARKER	''
```
'?' should be tokenized as NAME instead of ERRORTOKEN.

----------
components: Library (Lib)
messages: 324148
nosy: monson
priority: normal
severity: normal
status: open
title: lib2to3: support non-ASCII identifiers
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34515>
_______________________________________

From report at bugs.python.org  Mon Aug 27 02:52:34 2018
From: report at bugs.python.org (chen wu)
Date: Mon, 27 Aug 2018 06:52:34 +0000
Subject: [New-bugs-announce] [issue34516] httplib sets unbefitting "Host" in
 request header when requests an ipv6 format url.
Message-ID: <1535352754.06.0.56676864532.issue34516@psf.upfronthosting.co.za>


New submission from chen wu <wuchen12 at huawei.com>:

when I tried to request a url like "https://[fc00:0a08::2]:35357"? I got 400. 
The code is like:
    import requests
    requests.get("https://[fc00:0a08::2]:35357", verify=False)
And the apache logs:
    vhost.c(889): [client fc00:ac1c::9a5:58692] AH00550: Client sent malformed Host header: [[fc00::0a08::2]]:35357
If user no set "Host" in header, httpslib will pase it from url and set it.
The paser function is urllib3.util.url.pase_url. When url is "https://[fc00:0a08::2]:35357", we got host [fc00:0a08::2].
And then httplib sets host in putrequest, "[" and "]" will be added to [fc00:0a08::2], which is not a valid format for host.
The part of codes are:
974                    # Wrap the IPv6 Host Header with [] (RFC 2732)
975                    if host_enc.find(':') >= 0:
976                        host_enc = "[" + host_enc + "]"

maybe the judgement condition for wrap ipv6 host header with [] is not very well?

----------
components: Library (Lib)
messages: 324149
nosy: visionwun
priority: normal
severity: normal
status: open
title: httplib sets unbefitting "Host" in request header when requests an ipv6 format url.
type: behavior
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34516>
_______________________________________

From report at bugs.python.org  Mon Aug 27 04:42:26 2018
From: report at bugs.python.org (Alex Mashianov)
Date: Mon, 27 Aug 2018 08:42:26 +0000
Subject: [New-bugs-announce] [issue34517] Error referencing local variables
 in dict comprehensions inside class definitions
Message-ID: <1535359346.38.0.56676864532.issue34517@psf.upfronthosting.co.za>


New submission from Alex Mashianov <mashianov at gmail.com>:

This code:
class A:
    a = 1
    b = {str(x): x for x in range(5) if x != a}

Produces following error:
Traceback (most recent call last):
  File "<input>", line 1, in <module>
  File "<input>", line 3, in A
  File "<input>", line 3, in <dictcomp>
NameError: name 'a' is not defined


Which i think it shouldn't produce. Issue occurs only in dict comprehensions inside class definitions referencing class attributes(which are in local scope during definition).

----------
components: Interpreter Core
messages: 324156
nosy: Alex Mashianov
priority: normal
severity: normal
status: open
title: Error referencing local variables in dict comprehensions inside class definitions
type: behavior
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34517>
_______________________________________

From report at bugs.python.org  Mon Aug 27 04:44:48 2018
From: report at bugs.python.org (=?utf-8?b?SHJ2b2plIE5pa8WhacSH?=)
Date: Mon, 27 Aug 2018 08:44:48 +0000
Subject: [New-bugs-announce] [issue34518] Documentation for coroutine objects
Message-ID: <1535359488.51.0.56676864532.issue34518@psf.upfronthosting.co.za>


New submission from Hrvoje Nik?i? <hniksic at gmail.com>:

Coroutine objects have public methods such as send, close, and throw, which do not appear to be documented. For example, at https://stackoverflow.com/q/51975658/1600898 a StackOverflow user asks how to abort an already created (but not submitted) coroutine without a RuntimeWarning, with the answer being to use the close() method. The user asked where does one find the close method.

Currently methods only appear to be documented in PEP 492, referring to generator documentation for details. The glossary entry for coroutine (object) links to PEP 492 and to the async def statement. Various places in the documentation, e.g. the index, link to https://docs.python.org/3/library/asyncio-task.html#coroutine but that page is mostly concerned with the usage of coroutines within asyncio, where the methods on individual coroutine objects should not be used.

I would expect to find documentation on coroutine objects under built-in types, https://docs.python.org/3/library/stdtypes.html .

In comparison, generator-iterator methods are documented in the language reference:

https://docs.python.org/3/reference/expressions.html#generator-iterator-methods

----------
assignee: docs at python
components: Documentation
messages: 324157
nosy: docs at python, hniksic
priority: normal
severity: normal
status: open
title: Documentation for coroutine objects
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34518>
_______________________________________

From report at bugs.python.org  Mon Aug 27 09:39:51 2018
From: report at bugs.python.org (Michael Osipov)
Date: Mon, 27 Aug 2018 13:39:51 +0000
Subject: [New-bugs-announce] [issue34519] Add additional aliases for HP
 Roman 8
Message-ID: <1535377191.85.0.56676864532.issue34519@psf.upfronthosting.co.za>


New submission from Michael Osipov <1983-01-06 at gmx.net>:

HP Roman 8 is also known as 'hp-roman8', 'cp1051' and 'ibm1051'. They are missing from the aliases.py.

----------
components: Library (Lib)
messages: 324178
nosy: michael-o
priority: normal
severity: normal
status: open
title: Add additional aliases for HP Roman 8
type: enhancement
versions: Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34519>
_______________________________________

From report at bugs.python.org  Mon Aug 27 12:22:27 2018
From: report at bugs.python.org (Pablo Galindo Salgado)
Date: Mon, 27 Aug 2018 16:22:27 +0000
Subject: [New-bugs-announce] [issue34520] test_asyncio leaked [2, 2,
 2] references, sum=6 in AMD64 Windows8.1 Refleaks 3.7
Message-ID: <1535386947.0.0.56676864532.issue34520@psf.upfronthosting.co.za>


New submission from Pablo Galindo Salgado <pablogsal at gmail.com>:

test_asyncio is potentially leaking resources in AMD64 Windows8.1 Refleaks 3.7:

https://buildbot.python.org/all/#/builders/132/builds/241

----------------------------------------------------------------------
Ran 1782 tests in 115.342s
OK (skipped=78)
.
test_asyncio leaked [2, 2, 2] references, sum=6
test_asyncio leaked [2, 3, 2] memory blocks, sum=7
3 tests failed again:
    test_asyncio test_smtplib test_trace
== Tests result: FAILURE then FAILURE ==
383 tests OK.

----------
components: Tests, Windows, asyncio
messages: 324187
nosy: asvetlov, pablogsal, paul.moore, steve.dower, tim.golden, yselivanov, zach.ware
priority: normal
severity: normal
status: open
title: test_asyncio leaked [2, 2, 2] references, sum=6 in AMD64 Windows8.1 Refleaks 3.7
type: behavior
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34520>
_______________________________________

From report at bugs.python.org  Mon Aug 27 13:12:58 2018
From: report at bugs.python.org (Pablo Galindo Salgado)
Date: Mon, 27 Aug 2018 17:12:58 +0000
Subject: [New-bugs-announce] [issue34521]
 test_socket.RecvmsgIntoSCMRightsStreamTest fails on AMD64 FreeBSD CURRENT
 Debug 3.x
Message-ID: <1535389978.32.0.56676864532.issue34521@psf.upfronthosting.co.za>


New submission from Pablo Galindo Salgado <pablogsal at gmail.com>:

test_socket.RecvmsgIntoSCMRightsStreamTest fails on AMD64 FreeBSD CURRENT Debug 3.x:

https://buildbot.python.org/all/#/builders/60/builds/248


======================================================================
FAIL: testFDPassSeparateMinSpace (test.test_socket.RecvmsgIntoSCMRightsStreamTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/home/pablo/cpython/Lib/test/test_socket.py", line 2881, in testFDPassSeparateMinSpace
    maxcmsgs=2, ignoreflags=socket.MSG_CTRUNC)
  File "/usr/home/pablo/cpython/Lib/test/test_socket.py", line 2798, in checkRecvmsgFDs
    self.assertEqual(len(fds), numfds)
AssertionError: 1 != 2

----------------------------------------------------------------------
Ran 18 tests in 0.084s

FAILED (failures=1)
test test_socket failed
test_socket failed

== Tests result: FAILURE ==

1 test failed:
    test_socket


I was able to reproduce the bug in the buildbot but this same test fails in 3.6 and 3.5 in the buildbot so it can be some configuration in the buildbot or some regression in FreeBSD.

----------
components: FreeBSD, Tests
messages: 324188
nosy: koobs, pablogsal
priority: normal
severity: normal
status: open
title: test_socket.RecvmsgIntoSCMRightsStreamTest fails on AMD64 FreeBSD CURRENT Debug 3.x
type: behavior

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34521>
_______________________________________

From report at bugs.python.org  Mon Aug 27 16:25:41 2018
From: report at bugs.python.org (Eddie Elizondo)
Date: Mon, 27 Aug 2018 20:25:41 +0000
Subject: [New-bugs-announce] [issue34522] PyTypeObject's tp_base
 initialization bug
Message-ID: <1535401541.74.0.56676864532.issue34522@psf.upfronthosting.co.za>


New submission from Eddie Elizondo <eduardo.elizondorueda at gmail.com>:

>From the documentation, it says that PyType_Ready should be called on `ALL` type objects to finish their initialization (https://docs.python.org/3/c-api/type.html#c.PyType_Ready). This means that a PyTypeObject's ob_type should always be set by PyType_Ready.

It turns out that this is not actually followed by all the core types in CPython. This leads to the usage of types that were not initialized through PyType_Ready.

This fix modifies PyVarObject_HEAD_INIT to default the type to NULL so that all objects have to be fully initialized through PyType_Ready.

Plus:
* It initializes all the objects that were not being initialized through PyType_Ready.
* Modifies PyType_Ready to special case the ob_type initialization of PyType_Type and PyBaseObject_Type.
* It modifies the edge case of _Py_FalseStruct and _Py_TrueStruct.

Read more: https://mail.python.org/pipermail/python-dev/2018-August/154946.html

----------
components: Interpreter Core
messages: 324195
nosy: eelizondo
priority: normal
severity: normal
status: open
title: PyTypeObject's tp_base initialization bug
type: behavior
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34522>
_______________________________________

From report at bugs.python.org  Mon Aug 27 19:44:03 2018
From: report at bugs.python.org (STINNER Victor)
Date: Mon, 27 Aug 2018 23:44:03 +0000
Subject: [New-bugs-announce] [issue34523] Choose the filesystem encoding
 earlier in Python initialization (add _PyCoreConfig.filesystem_encoding)
Message-ID: <1535413443.13.0.56676864532.issue34523@psf.upfronthosting.co.za>


New submission from STINNER Victor <vstinner at redhat.com>:

Currently, the Python filesystem encoding is get in the middle of the Python initialization. initfsencoding() is only called when the import machinery is ready. Technically, only getting the Python codec name requires a working import machinery.

We can get the locale encoding even before Py_Initialize(), but update the encoding to the Python codec name (ex: replace "ANSI_X3.4-1968" with "ascii") once the import machinery is ready.

With my change, for an application embedding Python, the filesystem encoding can now be forced using _PyCoreConfig.filesystem_encoding. If it's set, Python will use it (and ignore the locale encoding).

Attached PR implements this idea and adds _PyCoreConfig.filesystem_encoding.


The change move all code to select encoding and error handler inside config_init_fs_encoding(), rather than relying on Py_FileSystemDefaultEncoding constant (its default value is set using #ifdef) and initfsencoding() (to get the locale encoding at runtime).


With this change, it becomes more obvious than the interpreter core configuration is mutable. initfsencoding() modify the encoding/errors during Python initialization, and sys._enablelegacywindowsfsencoding() even modify it at runtime. Previously, I wanted to expose the full core_config at the Python level, as something like sys.flags. But since it's mutable, I'm not longer sure that it's a good idea. To *get* the filesystem encoding/errors, they are already sys.getfilesystemencoding() and sys.getfilesystemencodeerrors(). Modifying the filesystem encoding at runtime is a very bad idea tried in the past and it caused many issues.


For the long term, I would like to remove Py_HasFileSystemDefaultEncoding, since it's really an implementation detail and there is no need to expose it. More generally, I don't see the purpose of exposing directly the encoding and error handler at the C level: Py_FileSystemDefaultEncoding and Py_FileSystemDefaultEncodeErrors. These symbols are constant constant, they cannot be modified when Python is embedded. But well, it's just a remark, technically, these 3 symbols don't cause any kind of trouble.


Commit message:
---
bpo-34485: Add _PyCoreConfig.filesystem_encoding

_PyCoreConfig_Read() is now responsible to choose the filesystem
encoding and error handler. Using Py_Main(), the encoding is now
selected even before calling Py_Initialize().

_PyCoreConfig.filesystem_encoding is now the reference instead of
Py_FileSystemDefaultEncoding for the Python filesystem encoding.

Changes:

* Add filesystem_encoding and filesystem_errors to _PyCoreConfig
* _PyCoreConfig_Read() now reads the locale encoding for the file
  system encoding. Coerce temporarly the locale or set temporarly the
  LC_CTYPE locale to the user preferred locale.
* PyUnicode_EncodeFSDefault() and PyUnicode_DecodeFSDefaultAndSize()
  now use the interpreter configuration rather than
  Py_FileSystemDefaultEncoding and Py_FileSystemDefaultEncodeErrors
  global configuration variables.
* Add _Py_SetFileSystemEncoding() and _Py_ClearFileSystemEncoding()
  private functions to only modify Py_FileSystemDefaultEncoding and
  Py_FileSystemDefaultEncodeErrors in coreconfig.c.
---

----------
components: Interpreter Core
messages: 324206
nosy: eric.snow, inada.naoki, ncoghlan, vstinner
priority: normal
severity: normal
status: open
title: Choose the filesystem encoding earlier in Python initialization (add _PyCoreConfig.filesystem_encoding)
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34523>
_______________________________________

From report at bugs.python.org  Mon Aug 27 20:35:58 2018
From: report at bugs.python.org (Richard Evans)
Date: Tue, 28 Aug 2018 00:35:58 +0000
Subject: [New-bugs-announce] [issue34524] Format conversion documentation
 example don't match comment
Message-ID: <1535416558.65.0.56676864532.issue34524@psf.upfronthosting.co.za>


New submission from Richard Evans <rarevans at hotmail.com>:

When reading the documentation for string formats I found that the conversion examples had comments that didn't match the example.

----------
assignee: docs at python
components: Documentation
messages: 324207
nosy: Richard Evans, docs at python
priority: normal
severity: normal
status: open
title: Format conversion documentation example don't match comment
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34524>
_______________________________________

From report at bugs.python.org  Mon Aug 27 22:10:42 2018
From: report at bugs.python.org (Sebastian Rittau)
Date: Tue, 28 Aug 2018 02:10:42 +0000
Subject: [New-bugs-announce] [issue34525] smtplib's authobject return value
 wrongly documented
Message-ID: <1535422242.76.0.56676864532.issue34525@psf.upfronthosting.co.za>


New submission from Sebastian Rittau <srittau at rittau.biz>:

smtplib's authobject() callback is supposed to return bytes, according to both the documentation at https://docs.python.org/3/library/smtplib.html as well as the docstring of SMTP.auth(). This does neither match the implementation of auth(), which immediately calls encode() on the return value, nor that of the auth_cram_md5() et al, which return str.

----------
assignee: docs at python
components: Documentation
messages: 324209
nosy: docs at python, srittau
priority: normal
severity: normal
status: open
title: smtplib's authobject return value wrongly documented
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34525>
_______________________________________

From report at bugs.python.org  Tue Aug 28 04:42:29 2018
From: report at bugs.python.org (Antony Lee)
Date: Tue, 28 Aug 2018 08:42:29 +0000
Subject: [New-bugs-announce] [issue34526] Path.relative_to() taking multiple
 arguments could be better documented
Message-ID: <1535445749.23.0.56676864532.issue34526@psf.upfronthosting.co.za>


New submission from Antony Lee <anntzer.lee at gmail.com>:

Currently, the docs for Path.relative_to read

    PurePath.relative_to(*other)
    Compute a version of this path relative to the path represented by other. If it?s impossible, ValueError is raised: (examples follow)

It's a bit confusing why other is a star-args, especially as no example actually passes more than one argument to relative_to.

The docstring is a tiny bit clearer:

    Return the relative path to another path identified by the passed
    arguments.  If the operation is not possible (because this is not
    a subpath of the other path), raise ValueError.

Effectively, a Path is constructed from all *other args and used as base for the computation of the relative path.  It looks a bit like a misfeature to me, but at least it could be better documented (e.g. by adding `Path("/tmp/foo/bar").relative_to("/tmp", "foo") == Path("bar")` as example in the docs).

----------
assignee: docs at python
components: Documentation
messages: 324224
nosy: Antony.Lee, docs at python
priority: normal
severity: normal
status: open
title: Path.relative_to() taking multiple arguments could be better documented
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34526>
_______________________________________

From report at bugs.python.org  Tue Aug 28 05:35:24 2018
From: report at bugs.python.org (STINNER Victor)
Date: Tue, 28 Aug 2018 09:35:24 +0000
Subject: [New-bugs-announce] [issue34527] On FreeBSD,
 Python 3 doesn't support support the POSIX locale
Message-ID: <1535448924.9.0.56676864532.issue34527@psf.upfronthosting.co.za>


New submission from STINNER Victor <vstinner at redhat.com>:

vstinner at freebsd$ env -i LC_ALL=POSIX ./python -m test -v test_cmd_line_script  -m test_non_ascii
== CPython 3.8.0a0 (heads/master:096329f0b2, Aug 28 2018, 09:30:55) [Clang 4.0.0 (tags/RELEASE_400/final 297347)]
== FreeBSD-11.1-RELEASE-amd64-64bit-ELF little-endian
== cwd: /usr/home/vstinner/prog/python/master/build/test_python_13073
== CPU count: 2
== encodings: locale=US-ASCII, FS=ascii
Run tests sequentially
0:00:00 load avg: 0.48 [1/1] test_cmd_line_script
test_non_ascii (test.test_cmd_line_script.CmdLineTest) ... FAIL

======================================================================
FAIL: test_non_ascii (test.test_cmd_line_script.CmdLineTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/home/vstinner/prog/python/master/Lib/test/test_cmd_line_script.py", line 512, in test_non_ascii
    'stdout=%r stderr=%r' % (stdout, stderr))
AssertionError: "'./@test_13073_tmp\\udce7w\\udcf0.py'" != "'<filename encoding error>'"
- './@test_13073_tmp\udce7w\udcf0.py'
+ '<filename encoding error>'
 : stdout=b"'<filename encoding error>'\n" stderr=b"Failed checking if argv[0] is an import path entry\nUnicodeEncodeError: 'ascii' codec can't encode character '\\xe7' in position 17: ordinal not in range(128)"

----------------------------------------------------------------------
Ran 1 test in 0.093s

FAILED (failures=1)
test test_cmd_line_script failed
test_cmd_line_script failed

== Tests result: FAILURE ==

1 test failed:
    test_cmd_line_script

Total duration: 170 ms
Tests result: FAILURE


By the way, the UTF-8 Mode is not enabled by the POSIX locale:

vstinner at freebsd$ env -i LC_ALL=POSIX ./python -c 'import sys; print(sys.flags.utf8_mode)'
0

It contradicts the PEP 540 which says the the UTF-8 Mode is enabled by the POSIX locale:

https://www.python.org/dev/peps/pep-0540/#abstract

> This mode is off by default, but is automatically activated when using the "POSIX" locale.

----------
components: Interpreter Core
messages: 324230
nosy: ncoghlan, vstinner
priority: normal
severity: normal
status: open
title: On FreeBSD, Python 3 doesn't support support the POSIX locale
versions: Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34527>
_______________________________________

From report at bugs.python.org  Tue Aug 28 06:49:59 2018
From: report at bugs.python.org (Cabu)
Date: Tue, 28 Aug 2018 10:49:59 +0000
Subject: [New-bugs-announce] [issue34528] Windows installer: Failed to run
 untrusted mode.
Message-ID: <1535453399.16.0.56676864532.issue34528@psf.upfronthosting.co.za>


New submission from Cabu <cabuzelth at yahoo.com>:

At work, we are using Windows 7 Enterprise. I have local admin rights on a separate account.

When trying to install Python 3.6.3 or 3.7.0 for amd64, the installer stop after a couple of seconds without any message/popup. But, I have found the attched log in a temp folder.

It seems that the installer try to run a copy of itself in a folder where even the local admin can't run executables by company policy.

I didn't find any issue with the same error code in the tracker.

----------
components: Installation
files: Setup_20180828111438_Failed.txt
messages: 324235
nosy: Cabu, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: Windows installer: Failed to run untrusted mode.
type: crash
versions: Python 3.6, Python 3.7
Added file: https://bugs.python.org/file47768/Setup_20180828111438_Failed.txt

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34528>
_______________________________________

From report at bugs.python.org  Tue Aug 28 08:47:12 2018
From: report at bugs.python.org (ron)
Date: Tue, 28 Aug 2018 12:47:12 +0000
Subject: [New-bugs-announce] [issue34529] add the option for json.dumps to
 return newline delimited json
Message-ID: <1535460432.73.0.56676864532.issue34529@psf.upfronthosting.co.za>


New submission from ron <ron at mailinator.com>:

Many service providers such as Google BigQuery do not accept Json.
They accept newline delimited Json.

https://cloud.google.com/bigquery/docs/loading-data-cloud-storage-json#limitations

please allow to receive this format directly from the dump.

----------
messages: 324244
nosy: ronron
priority: normal
severity: normal
status: open
title: add the option for json.dumps to return newline delimited json
type: enhancement
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34529>
_______________________________________

From report at bugs.python.org  Tue Aug 28 08:47:31 2018
From: report at bugs.python.org (STINNER Victor)
Date: Tue, 28 Aug 2018 12:47:31 +0000
Subject: [New-bugs-announce] [issue34530] distutils: find_executable() fails
 if the PATH environment variable is not set
Message-ID: <1535460451.64.0.56676864532.issue34530@psf.upfronthosting.co.za>


New submission from STINNER Victor <vstinner at redhat.com>:

Example:

$ env -i ./python -c 'import distutils.spawn; print(distutils.spawn.find_executable("true"))'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/home/vstinner/prog/python/master/Lib/distutils/spawn.py", line 176, in find_executable
    path = os.environ['PATH']
  File "/home/vstinner/prog/python/master/Lib/os.py", line 672, in __getitem__
    raise KeyError(key) from None
KeyError: 'PATH'


Attached PR fixes the issue for the master branch.

----------
components: Library (Lib)
messages: 324245
nosy: vstinner
priority: normal
severity: normal
status: open
title: distutils: find_executable() fails if the PATH environment variable is not set
versions: Python 2.7, Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34530>
_______________________________________

From report at bugs.python.org  Tue Aug 28 11:31:27 2018
From: report at bugs.python.org (=?utf-8?q?Andr=C3=A9s_Delfino?=)
Date: Tue, 28 Aug 2018 15:31:27 +0000
Subject: [New-bugs-announce] [issue34531] doc Move comment about list
 sorting behavour outside impl-detail directive
Message-ID: <1535470287.16.0.56676864532.issue34531@psf.upfronthosting.co.za>


New submission from Andr?s Delfino <adelfino at gmail.com>:

As this text is phrased:

CPython implementation detail: While a list is being sorted, the effect of attempting to mutate, or even inspect, the list is undefined. The C implementation of Python makes the list appear empty for the duration, and raises ValueError if it can detect that the list has been mutated during a sort.

I take it that the first sentence is a general comment for all implementations, as the second sentence does define a specific behavior in CPython, so it's not undefined there.

I propose taking the first sentence out of the impl-detail directive.

----------
assignee: docs at python
components: Documentation
messages: 324259
nosy: adelfino, docs at python
priority: normal
severity: normal
status: open
title: doc Move comment about list sorting behavour outside impl-detail directive
type: enhancement
versions: Python 2.7, Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34531>
_______________________________________

From report at bugs.python.org  Tue Aug 28 13:12:38 2018
From: report at bugs.python.org (Brendan Gerrity)
Date: Tue, 28 Aug 2018 17:12:38 +0000
Subject: [New-bugs-announce] [issue34532] Python launcher on Windows exits
 with error when requesting list of available versions
Message-ID: <1535476358.72.0.56676864532.issue34532@psf.upfronthosting.co.za>


New submission from Brendan Gerrity <bgerrity at ucdavis.edu>:

When any of the list available pythons arguments is used (e.g. `py -0`), the launcher exits with the error code 103 `RC_NO_PYTHON`.

This is inconsistent with the launcher's `-h` help argument, which exits with a 0.

----------
components: Windows
messages: 324265
nosy: bgerrity, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: Python launcher on Windows exits with error when requesting list of available versions
type: behavior
versions: Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34532>
_______________________________________

From report at bugs.python.org  Tue Aug 28 13:45:04 2018
From: report at bugs.python.org (Eddie Elizondo)
Date: Tue, 28 Aug 2018 17:45:04 +0000
Subject: [New-bugs-announce] [issue34533] Apply PEP384 to _csv module
Message-ID: <1535478304.23.0.56676864532.issue34533@psf.upfronthosting.co.za>


New submission from Eddie Elizondo <eduardo.elizondorueda at gmail.com>:

This applies the heap type refactoring from PEP 384 to the _csv module.

----------
components: Extension Modules
messages: 324268
nosy: eelizondo
priority: normal
severity: normal
status: open
title: Apply PEP384 to _csv module
type: enhancement
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34533>
_______________________________________

From report at bugs.python.org  Tue Aug 28 15:00:56 2018
From: report at bugs.python.org (Girts Folkmanis)
Date: Tue, 28 Aug 2018 19:00:56 +0000
Subject: [New-bugs-announce] [issue34534] importlib.resources does not work
 with packages that have no __init__.py
Message-ID: <1535482856.6.0.56676864532.issue34534@psf.upfronthosting.co.za>


New submission from Girts Folkmanis <opensource at girts.me>:

importlib.resources does not seem to work with packages that don't have __init__.py present. Since 3.3+ generally there is no need to create empty __init__.py, as directories are automatically treated as packages even without the file present. So my expectation would be that importlib.resources would follow the same suit. 

Repro, where I expect that both "mod1" and "mod2" would return the files when contents() is called:

$ find .
.
./mod2
./mod2/data.txt
./test.py
./mod1
./mod1/__init__.py
./mod1/data.txt

$ cat test.py
import importlib.resources

print('mod1:', list(importlib.resources.contents('mod1')))
print('mod2:', list(importlib.resources.contents('mod2')))

$ python3.7 test.py
mod1: ['__init__.py', '__pycache__', 'data.txt']
mod2: []    # <---- here I would expect to see files too


Looking at the source of Lib/importlib/resources.py, there is the following code in contents():

    [..]
    # Is the package a namespace package?  By definition, namespace packages
    # cannot have resources.  We could use _check_location() and catch the
    # exception, but that's extra work, so just inline the check.
    elif package.__spec__.origin is None or not package.__spec__.has_location:
        return ()
    [..]

This is the branch that is taken - but it's not necessarily a namespace package here, at least to my understanding. Is there a way to make the code here distinguish between namespace packages and implicit non-namespace packages here, and allow contents() (and other functions) to work?

----------
components: Library (Lib)
messages: 324275
nosy: girtsf
priority: normal
severity: normal
status: open
title: importlib.resources does not work with packages that have no __init__.py
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34534>
_______________________________________

From report at bugs.python.org  Tue Aug 28 15:20:21 2018
From: report at bugs.python.org (Paul Overing)
Date: Tue, 28 Aug 2018 19:20:21 +0000
Subject: [New-bugs-announce] [issue34535] queue.Queue(timeout=0.001) avg
 delay Windows:14.5ms, Ubuntu: 0.063ms
Message-ID: <1535484021.65.0.56676864532.issue34535@psf.upfronthosting.co.za>


New submission from Paul Overing <paulovering at gmail.com>:

I have found that using timeout with a python queue.Queue() results in an average delay of  14.5ms in Windows over Ubuntu in python 3.6.5/3.7, I subtracted the 1 ms timeout.

I have also tried datetime.datetime.now() to measure the time, with similar results. 

Does that make sense?


import queue
import time

max=0
min=10000
num = 10000
sum = 0
for x in range(10000):
    q = queue.Queue()
    start = time.perf_counter()
    try:
        msg = q.get(block=True, timeout=.001)
    except:
        end = time.perf_counter()

    chng = end-start -.001

    if chng > max:
        max = chng
    if chng < min:
        min = chng
    sum = sum + chng

print(f"Avg: {sum/num}")
print(f"Min: {min}")
print(f"Max: {max}")

Results:

Window 10, Python 3.6.5/3.7
Avg: 0.014549868429997701
Min: 0.001674142000000046
Max: 0.13890251500000284

Ubuntu 16.04, Python 3.6.5/3.7
Avg: 6.275181290839012e-05
Min: 4.89290034456644e-05
Max: 8.690999695681965e-05

----------
components: Windows
messages: 324277
nosy: Gammaguy, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: queue.Queue(timeout=0.001) avg delay Windows:14.5ms, Ubuntu: 0.063ms
type: performance
versions: Python 3.6, Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34535>
_______________________________________

From report at bugs.python.org  Tue Aug 28 15:37:44 2018
From: report at bugs.python.org (Paul Pinterits)
Date: Tue, 28 Aug 2018 19:37:44 +0000
Subject: [New-bugs-announce] [issue34536] Enum._missing_ doesn't raise
 TypeError when a non-Enum object is a returned
Message-ID: <1535485064.24.0.56676864532.issue34536@psf.upfronthosting.co.za>


New submission from Paul Pinterits <rawing7 at gmail.com>:

In a conversation with Ethan Furman (the author of the enum module) I've been informed that Enum automatically checks whether the _missing_ method has returned an instance of the correct class, and raises a TypeError if not. (Link: https://stackoverflow.com/a/52064774/1222951)

However, this simply does not happen.

```
import enum

class MyEnum(enum.Enum):
    FOO = "foo"

    @classmethod
    def _missing_(cls, value):
        return 5

print(MyEnum('bar'))  # output: 5
```

----------
components: Library (Lib)
messages: 324282
nosy: Paul Pinterits
priority: normal
severity: normal
status: open
title: Enum._missing_ doesn't raise TypeError when a non-Enum object is a returned
type: behavior
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34536>
_______________________________________

From report at bugs.python.org  Tue Aug 28 16:05:29 2018
From: report at bugs.python.org (STINNER Victor)
Date: Tue, 28 Aug 2018 20:05:29 +0000
Subject: [New-bugs-announce] [issue34537] test_gdb fails with LC_ALL=C
Message-ID: <1535486729.44.0.56676864532.issue34537@psf.upfronthosting.co.za>


New submission from STINNER Victor <vstinner at redhat.com>:

On Python 3.7 and master, test_strings() of test_gdb fails if gdb uses Python 3.6 (before UTF-8 Mode and C locale coercion):

* gdb uses ASCII encoding for stdout
* tested Python uses UTF-8 thanks to the UTF-8 Mode and/or C locale coercion


vstinner at apu$ LC_ALL=C ./python -m test -v test_gdb -m test_strings
== CPython 3.7.0+ (heads/3.7:65ef7425a3, Aug 28 2018, 16:29:30) [GCC 8.1.1 20180712 (Red Hat 8.1.1-5)]
== Linux-4.17.6-200.fc28.x86_64-x86_64-with-fedora-28-Twenty_Eight little-endian
== cwd: /home/vstinner/prog/python/3.7/build/test_python_21689
== CPU count: 8
== encodings: locale=UTF-8, FS=utf-8
Run tests sequentially
0:00:00 load avg: 0.35 [1/1] test_gdb
GDB version 8.1:
    GNU gdb (GDB) Fedora 8.1.1-3.fc28
    Copyright (C) 2018 Free Software Foundation, Inc.
    License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
    This is free software: you are free to change and redistribute it.
    There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
    and "show warranty" for details.
    This GDB was configured as "x86_64-redhat-linux-gnu".
    Type "show configuration" for configuration details.
    For bug reporting instructions, please see:
    <http://www.gnu.org/software/gdb/bugs/>.
    Find the GDB manual and other documentation resources online at:
    <http://www.gnu.org/software/gdb/documentation/>.
    For help, type "help".
    Type "apropos word" to search for commands related to "word".
test_strings (test.test_gdb.PrettyPrintTests)
Verify the pretty-printing of unicode strings ... FAIL

======================================================================
FAIL: test_strings (test.test_gdb.PrettyPrintTests)
Verify the pretty-printing of unicode strings
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/vstinner/prog/python/3.7/Lib/test/test_gdb.py", line 331, in test_strings
    check_repr('\u2620')
  File "/home/vstinner/prog/python/3.7/Lib/test/test_gdb.py", line 323, in check_repr
    self.assertGdbRepr(text)
  File "/home/vstinner/prog/python/3.7/Lib/test/test_gdb.py", line 271, in assertGdbRepr
    % (gdb_repr, exp_repr, gdb_output)))
AssertionError: "'\\u2620'" != "'?'"
- '\u2620'
+ '?'
 : "'\\u2620'" did not equal expected "'?'"; full output was:
Breakpoint 1 at 0x52fc1b: file Python/bltinmodule.c, line 1216.
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".

Breakpoint 1, builtin_id (self=, v='\u2620') at Python/bltinmodule.c:1216
1216	    return PyLong_FromVoidPtr(v);
#0  builtin_id (self=, v='\u2620') at Python/bltinmodule.c:1216


----------------------------------------------------------------------
Ran 1 test in 1.030s

FAILED (failures=1)
test test_gdb failed
test_gdb failed

== Tests result: FAILURE ==

1 test failed:
    test_gdb

Total duration: 1 sec 422 ms
Tests result: FAILURE

----------
components: Tests, Unicode
messages: 324284
nosy: ezio.melotti, vstinner
priority: normal
severity: normal
status: open
title: test_gdb fails with LC_ALL=C
versions: Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34537>
_______________________________________

From report at bugs.python.org  Tue Aug 28 16:40:51 2018
From: report at bugs.python.org (Nathaniel Manista)
Date: Tue, 28 Aug 2018 20:40:51 +0000
Subject: [New-bugs-announce] [issue34538] Remove encouragement to author a
 base class for all Exception subclasses in a module
Message-ID: <1535488851.13.0.56676864532.issue34538@psf.upfronthosting.co.za>


New submission from Nathaniel Manista <nathaniel at google.com>:

https://docs.python.org/3.8/tutorial/errors.html (and all other versions of that page back at least as far as 2.7) currently contain the guidance "When creating a module that can raise several distinct errors, a common practice is to create a base class for exceptions defined by that module, and subclass that to create specific exception classes for different error conditions: <code example>".

It may have seemed like a good idea at the time, but we now know from years of experience that this is an experiment that didn't pan out and we can consider this guidance a Now-Contraindicated Best Practice Of Yesteryear?.

Modules define subclasses of Exception for lots of reasons. Some of those subclasses have no relation to one another except that they are subclasses of Exception. Some of those subclasses define Exception instances that are never raised by code in the module, but that are expected to be raised by code passed to and called by the module.

Yes, there are times when a common base class may be appropriate - such as when an inheritance hierarchy and polymorphism that satisfy the Liskov Substitution Principle make sense for the Exception subclasses, and when the base class itself is used (such as when the base class is an item in the Raises: section of a function's doc string). But these cases aren't so common as to justify the advice being offered generally about all Exception subclass definitions.

Exception subclasses are just classes. Advising that authors may wish to define a common base class for all Exception subclasses in a module is like advising authors that they may wish to define a common base class for all object subclasses in a module - it's very, very, very occasionally a good idea in the particular circumstances of a particular module's implementation, but very generally not.

----------
assignee: docs at python
components: Documentation
messages: 324286
nosy: Nathaniel Manista, docs at python
priority: normal
severity: normal
status: open
title: Remove encouragement to author a base class for all Exception subclasses in a module
type: enhancement
versions: Python 2.7, Python 3.4, Python 3.5, Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34538>
_______________________________________

From report at bugs.python.org  Wed Aug 29 00:20:50 2018
From: report at bugs.python.org (=?utf-8?q?=C3=81lvaro_Justen?=)
Date: Wed, 29 Aug 2018 04:20:50 +0000
Subject: [New-bugs-announce] [issue34539] namedtuple's exec() throws
 segmentation fault
Message-ID: <1535516450.51.0.56676864532.issue34539@psf.upfronthosting.co.za>


New submission from ?lvaro Justen <alvarojusten at gmail.com>:

I was working on a library called rows[https://github.com/turicas/rows] when a segmentation fault was thrown in the moment I've tried to read a CSV file. Since this part of the code is implemented completely in Python, I thought it could be a bug in Python itself.
Investigating the code I found it was faulting at exec()'s line inside namedtuple code (Lib/collections/__init__.py). After some time testing, I've came up with this piece of code to reproduce it:

```
try:
    float('?xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx')
except:
    from collections import namedtuple
    namedtuple('T', 'f')
```

The code works if I:
- Run passing `PYTHONMALLOC=debug`
- Place 'from collections import namedtuple' into the first line
- Remove any char from the string in the 'float(...)' line
- Remove the '?' from the string in the 'float(...)' line

I've tested the code on Python 3.6.6, 3.7.0, 3.7-dev and 3.8-dev (versions installed using pyenv on a Debian GNU/Linux machine) and the problem happened only in 3.7.0.
This issue seems to be related to https://bugs.python.org/issue34087 but I preferred to create a new one since I don't know if there are automated tests for this specific case.

----------
components: Library (Lib)
files: namedtuple_bug.py
messages: 324300
nosy: turicas
priority: normal
severity: normal
status: open
title: namedtuple's exec() throws segmentation fault
type: crash
versions: Python 3.7
Added file: https://bugs.python.org/file47772/namedtuple_bug.py

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34539>
_______________________________________

From report at bugs.python.org  Wed Aug 29 01:10:18 2018
From: report at bugs.python.org (Benjamin Peterson)
Date: Wed, 29 Aug 2018 05:10:18 +0000
Subject: [New-bugs-announce] [issue34540] shutil._call_external_zip should
 use subprocess
Message-ID: <1535519418.39.0.56676864532.issue34540@psf.upfronthosting.co.za>


New submission from Benjamin Peterson <benjamin at python.org>:

disutils.spawn isn't very good at quoting command lines (#8987). We need to use subprocess to avoid shell injection.

----------
components: Library (Lib)
messages: 324301
nosy: benjamin.peterson
priority: normal
severity: normal
status: open
title: shutil._call_external_zip should use subprocess
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34540>
_______________________________________

From report at bugs.python.org  Wed Aug 29 04:20:21 2018
From: report at bugs.python.org (Paul Pinterits)
Date: Wed, 29 Aug 2018 08:20:21 +0000
Subject: [New-bugs-announce] [issue34541] pathlib.Path.iterdir doesn't throw
 an exception until you start iterating
Message-ID: <1535530821.14.0.56676864532.issue34541@psf.upfronthosting.co.za>


New submission from Paul Pinterits <rawing7 at gmail.com>:

The fact that `Path.iterdir()` only throws exceptions once you start iterating over it makes it very difficult to write correct code.

Let's look at an example: We'll iterate over all children of a directory and print their file size.

If we try to do it like this, the `try...except` has no effect whatsoever:

try:
    children = path_that_doesnt_exist.iterdir()
except FileNotFoundError:
    print("directory doesn't exist")
for path in children:
    print(path.stat().st_size)

If we explicitly check whether the path exists (and is a directory), we end up with a race condition:

if path_that_doesnt_exist.is_dir():
    print("directory doesn't exist")
else:
    for path in children:
        print(path.stat().st_size)

We can wrap the whole loop in a `try...except`, but then we might end up catching exceptions we didn't intend to catch. (For example, the exception that's thrown when we try to get the size of a broken symlink.)

try:
    for path in path_that_doesnt_exist.iterdir():
        print(path.stat().st_size)  # this can also throw FileNotFoundError
except FileNotFoundError:
    print("directory doesn't exist")

We can manually call `next` on the iterator inside of a `try..except` block, but that's awfully verbose and requires knowledge about iterators and the `next` function:

children = iter(path_that_doesnt_exist.iterdir())
while True:
    try:
        path = next(children)
    except FileNotFoundError:
        print("directory doesn't exist")
        break
    print(path.stat().st_size)

Or we can turn the iterator into a list inside of a `try...except`, which seems to be the best option, but completely defeats the point of having an iterator:

try:
    children = list(path_that_doesnt_exist.iterdir())
except FileNotFoundError:
    print("directory doesn't exist")
else:
    for path in children:
        print(path.stat().st_size)

As you can see, writing correct (and good) code with `iterdir` is more difficult than it has any right to be. Please change this behavior so that exceptions are thrown immediately when `iterdir` is called.

----------
components: Library (Lib)
messages: 324307
nosy: Paul Pinterits
priority: normal
severity: normal
status: open
title: pathlib.Path.iterdir doesn't throw an exception until you start iterating
type: behavior
versions: Python 3.4, Python 3.5, Python 3.6, Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34541>
_______________________________________

From report at bugs.python.org  Wed Aug 29 10:17:16 2018
From: report at bugs.python.org (Christian Heimes)
Date: Wed, 29 Aug 2018 14:17:16 +0000
Subject: [New-bugs-announce] [issue34542] [TLS] Update test certs to future
 proof settings
Message-ID: <1535552236.3.0.56676864532.issue34542@psf.upfronthosting.co.za>


New submission from Christian Heimes <lists at cheimes.de>:

In bug #34399, I updated all RSA keys to 2048. However that not sufficient for future proof settings. Fedora's FUTURE crypto policy requires 3072bit RSA keys. Further more, I forgot to update the signature algorithm, too.

* RSA >= 3072bits
* finite field DH >= 3072bits
* signature algorithm with SHA2-256 or SHA2-384 PKCSv1 #1.5 (I don't think RSASSA-PSS works with OpenSSL 1.0.2 or TLS < 1.0)

----------
assignee: christian.heimes
components: SSL
messages: 324324
nosy: alex, christian.heimes, dstufft, janssen
priority: normal
severity: normal
stage: test needed
status: open
title: [TLS] Update test certs to future proof settings
type: behavior
versions: Python 2.7, Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34542>
_______________________________________

From report at bugs.python.org  Wed Aug 29 12:18:52 2018
From: report at bugs.python.org (DeKrain)
Date: Wed, 29 Aug 2018 16:18:52 +0000
Subject: [New-bugs-announce] [issue34543] _struct.Struct: calling functions
 without calling __init__ results in SystemError
Message-ID: <1535559532.49.0.56676864532.issue34543@psf.upfronthosting.co.za>


New submission from DeKrain <dawidkrainski8 at gmail.com>:

>>> from _struct import Struct
>>> s = Struct.__new__(Struct)
>>> s.unpack_from(b'asdf')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
SystemError: /Objects/tupleobject.c:84: Bad argument to internal function

In Modules/_struct.c:

static PyObject *
s_unpack_internal(PyStructObject *soself, const char *startfrom) {
...
PyObject *result = PyTuple_New(soself->s_len);
// soself->s_len is -1, set in Struct.__new__

----------
components: Extension Modules
messages: 324330
nosy: DeKrain
priority: normal
severity: normal
status: open
title: _struct.Struct: calling functions without calling __init__ results in SystemError
type: behavior
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34543>
_______________________________________

From report at bugs.python.org  Wed Aug 29 19:37:46 2018
From: report at bugs.python.org (STINNER Victor)
Date: Wed, 29 Aug 2018 23:37:46 +0000
Subject: [New-bugs-announce] [issue34544] FreeBSD: Fatal Python error:
 get_locale_encoding: failed to get the locale encoding:
 nl_langinfo(CODESET) failed
Message-ID: <1535585866.04.0.56676864532.issue34544@psf.upfronthosting.co.za>


New submission from STINNER Victor <vstinner at redhat.com>:

It seems like my recent work on Python initialization broke Python on FreeBSD. Many test_c_locale_coerce tests now fail with:

"Fatal Python error: get_locale_encoding: failed to get the locale encoding: nl_langinfo(CODESET) failed"

Example of failed buildbot build:

https://buildbot.python.org/all/#/builders/87/builds/1340

It seems like this buildbot started to fail at this build:

https://buildbot.python.org/all/#/builders/87/builds/1332

This build has been triggered by this change:

commit 177d921c8c03d30daa32994362023f777624b10d
Author: Victor Stinner <vstinner at redhat.com>
Date:   Wed Aug 29 11:25:15 2018 +0200

    bpo-34485, Windows: LC_CTYPE set to user preference (GH-8988)
    
    On Windows, the LC_CTYPE is now set to the user preferred locale at
    startup: _Py_SetLocaleFromEnv(LC_CTYPE) is now called during the
    Python initialization. Previously, the LC_CTYPE locale was "C" at
    startup, but changed when calling setlocale(LC_CTYPE, "") or
    setlocale(LC_ALL, "").
    
    pymain_read_conf() now also calls _Py_SetLocaleFromEnv(LC_CTYPE) to
    behave as _Py_InitializeCore(). Moreover, it doesn't save/restore the
    LC_ALL anymore.
    
    On Windows, standard streams like sys.stdout now always use
    surrogateescape error handler by default (ignore the locale).

test_c_locale_coerce tests invalid locales like LANG="invalid.ascii". It seems like Python supported such configuration before my changes.

My recent work:

* bpo-34523: filesystem encoding/errors
* bpo-34485: stdio encoding/errors

----------
components: Interpreter Core
messages: 324361
nosy: vstinner
priority: normal
severity: normal
status: open
title: FreeBSD: Fatal Python error: get_locale_encoding: failed to get the locale encoding: nl_langinfo(CODESET) failed
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34544>
_______________________________________

From report at bugs.python.org  Thu Aug 30 00:42:05 2018
From: report at bugs.python.org (Noah Haasis)
Date: Thu, 30 Aug 2018 04:42:05 +0000
Subject: [New-bugs-announce] [issue34545] error in the repl due to
 indentation
Message-ID: <1535604125.2.0.56676864532.issue34545@psf.upfronthosting.co.za>


New submission from Noah Haasis <haasis_noah at yahoo.de>:

Pressing the tab key in the python REPL in my command line (cmder on windows 10 32bit) causes an exception:
>>> Readline internal error                                                                                                                       
Traceback (most recent call last):                                                                                                                
  File "C:\Users\leinw\AppData\Local\Programs\Python\Python36-32\lib\site-packages\pyreadline\console\console.py", line 768, in hook_wrapper_23   
    res = ensure_str(readline_hook(prompt))                                                                                                       
  File "C:\Users\leinw\AppData\Local\Programs\Python\Python36-32\lib\site-packages\pyreadline\rlmain.py", line 571, in readline                   
    self._readline_from_keyboard()                                                                                                                
  File "C:\Users\leinw\AppData\Local\Programs\Python\Python36-32\lib\site-packages\pyreadline\rlmain.py", line 536, in _readline_from_keyboard    
    if self._readline_from_keyboard_poll():                                                                                                       
  File "C:\Users\leinw\AppData\Local\Programs\Python\Python36-32\lib\site-packages\pyreadline\rlmain.py", line 556, in _readline_from_keyboard_pol
    result = self.mode.process_keyevent(event.keyinfo)                                                                                            
  File "C:\Users\leinw\AppData\Local\Programs\Python\Python36-32\lib\site-packages\pyreadline\modes\emacs.py", line 243, in process_keyevent      
    r = self.process_keyevent_queue[-1](keyinfo)                                                                                                  
  File "C:\Users\leinw\AppData\Local\Programs\Python\Python36-32\lib\site-packages\pyreadline\modes\emacs.py", line 286, in _process_keyevent     
    r = dispatch_func(keyinfo)                                                                                                                    
  File "C:\Users\leinw\AppData\Local\Programs\Python\Python36-32\lib\site-packages\pyreadline\modes\basemode.py", line 257, in complete           
    completions = self._get_completions()                                                                                                         
  File "C:\Users\leinw\AppData\Local\Programs\Python\Python36-32\lib\site-packages\pyreadline\modes\basemode.py", line 200, in _get_completions   
    r = self.completer(ensure_unicode(text), i)                                                                                                   
  File "C:\Users\leinw\AppData\Local\Programs\Python\Python36-32\lib\rlcompleter.py", line 80, in complete                                        
    readline.redisplay()                                                                                                                          
AttributeError: module 'readline' has no attribute 'redisplay'                                                                              

This happens even if python expects indentation.

----------
components: Windows
messages: 324366
nosy: noah.haasis, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: error in the repl due to indentation
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34545>
_______________________________________

From report at bugs.python.org  Thu Aug 30 04:40:25 2018
From: report at bugs.python.org (=?utf-8?b?5aSn6YeO6ZqG5byY?=)
Date: Thu, 30 Aug 2018 08:40:25 +0000
Subject: [New-bugs-announce] [issue34546] Zipfile encryption function
Message-ID: <1535618425.84.0.56676864532.issue34546@psf.upfronthosting.co.za>


New submission from ???? <oono0114 at gmail.com>:

from https://mail.python.org/pipermail/python-ideas/2018-August/053081.html

I would like to use zipfile encryption as python standard library.
https://github.com/python/cpython/blob/master/Lib/zipfile.py

Below document says "currently" cannot.
https://github.com/python/cpython/blob/master/Doc/library/zipfile.rst
"but it currently cannot create an encrypted file."

Current pythonians like me have to use 3rd party like below, but I believe it is worth to include.
https://pypi.org/project/pyminizip/
https://github.com/wllm-rbnt/py-zipcrypt

----------
components: Library (Lib)
messages: 324372
nosy: ????
priority: normal
severity: normal
status: open
title: Zipfile encryption function
type: enhancement

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34546>
_______________________________________

From report at bugs.python.org  Thu Aug 30 06:17:15 2018
From: report at bugs.python.org (Petter S)
Date: Thu, 30 Aug 2018 10:17:15 +0000
Subject: [New-bugs-announce] [issue34547] Wsgiref server does not handle
 closed connections gracefully
Message-ID: <1535624235.67.0.56676864532.issue34547@psf.upfronthosting.co.za>


New submission from Petter S <petter.strandmark at gmail.com>:

The server in the wsgiref module is actually used a lot. For example, it is the server Django uses for development.

A very common thing that happens during Django development is that the web browser closes the connection for some reason. Then very long stack traces appear in the Django logs:

    [30/Aug/2018 12:10:38] "POST /login/ HTTP/1.1" 200 3964
    Traceback (most recent call last):
      File "d:\python37\Lib\wsgiref\handlers.py", line 138, in run
        self.finish_response()
      File "d:\python37\Lib\wsgiref\handlers.py", line 180, in finish_response
        self.write(data)
      File "d:\python37\Lib\wsgiref\handlers.py", line 274, in write
        self.send_headers()
      File "d:\python37\Lib\wsgiref\handlers.py", line 332, in send_headers
        self.send_preamble()
      File "d:\python37\Lib\wsgiref\handlers.py", line 255, in send_preamble
        ('Date: %s\r\n' % format_date_time(time.time())).encode('iso-8859-1')
      File "d:\python37\Lib\wsgiref\handlers.py", line 453, in _write
        result = self.stdout.write(data)
      File "d:\python37\Lib\socketserver.py", line 796, in write
        self._sock.sendall(b)
    ConnectionAbortedError: [WinError 10053] An established connection was aborted by the software in your host machine
    [30/Aug/2018 12:10:38] "POST /login/ HTTP/1.1" 500 59
    ----------------------------------------
    Exception happened during processing of request from ('127.0.0.1', 50112)
    Traceback (most recent call last):
      File "d:\python37\Lib\wsgiref\handlers.py", line 138, in run
        self.finish_response()
      File "d:\python37\Lib\wsgiref\handlers.py", line 180, in finish_response
        self.write(data)
      File "d:\python37\Lib\wsgiref\handlers.py", line 274, in write
        self.send_headers()
      File "d:\python37\Lib\wsgiref\handlers.py", line 332, in send_headers
        self.send_preamble()
      File "d:\python37\Lib\wsgiref\handlers.py", line 255, in send_preamble
        ('Date: %s\r\n' % format_date_time(time.time())).encode('iso-8859-1')
      File "d:\python37\Lib\wsgiref\handlers.py", line 453, in _write
        result = self.stdout.write(data)
      File "d:\python37\Lib\socketserver.py", line 796, in write
        self._sock.sendall(b)
    ConnectionAbortedError: [WinError 10053] An established connection was aborted by the software in your host machine

    During handling of the above exception, another exception occurred:

    Traceback (most recent call last):
      File "d:\python37\Lib\wsgiref\handlers.py", line 141, in run
        self.handle_error()
      File "D:\Virtualenvs\ledev-X6wd5Q8f\lib\site-packages\django\core\servers\basehttp.py", line 86, in handle_error
        super().handle_error()
      File "d:\python37\Lib\wsgiref\handlers.py", line 368, in handle_error
        self.finish_response()
      File "d:\python37\Lib\wsgiref\handlers.py", line 180, in finish_response
        self.write(data)
      File "d:\python37\Lib\wsgiref\handlers.py", line 274, in write
        self.send_headers()
      File "d:\python37\Lib\wsgiref\handlers.py", line 331, in send_headers
        if not self.origin_server or self.client_is_modern():
      File "d:\python37\Lib\wsgiref\handlers.py", line 344, in client_is_modern
        return self.environ['SERVER_PROTOCOL'].upper() != 'HTTP/0.9'
    TypeError: 'NoneType' object is not subscriptable

    During handling of the above exception, another exception occurred:

    Traceback (most recent call last):
      File "d:\python37\Lib\socketserver.py", line 647, in process_request_thread
        self.finish_request(request, client_address)
      File "d:\python37\Lib\socketserver.py", line 357, in finish_request
        self.RequestHandlerClass(request, client_address, self)
      File "d:\python37\Lib\socketserver.py", line 717, in __init__
        self.handle()
      File "D:\Virtualenvs\ledev-X6wd5Q8f\lib\site-packages\django\core\servers\basehttp.py", line 154, in handle
        handler.run(self.server.get_app())
      File "d:\python37\Lib\wsgiref\handlers.py", line 144, in run
        self.close()
      File "d:\python37\Lib\wsgiref\simple_server.py", line 35, in close
        self.status.split(' ',1)[0], self.bytes_sent
    AttributeError: 'NoneType' object has no attribute 'split'
    ----------------------------------------


Obviously, this is a bit annoying when developing with Django daily. Since Django simply uses the wsgiref server, I though the best solution was to handle closed connections more gracefully there.

I think the best solution is to simply add another except clause here: https://github.com/python/cpython/blob/e6dac0077996b1e1f886f036d6f2606237fa4c85/Lib/wsgiref/handlers.py#L142 . We should catch ConnectionAbortedError here and return from the function.

I am happy to create a PR if this is what we want to do.

----------
components: Library (Lib)
messages: 324375
nosy: Petter S
priority: normal
severity: normal
status: open
title: Wsgiref server does not handle closed connections gracefully
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34547>
_______________________________________

From report at bugs.python.org  Thu Aug 30 08:39:32 2018
From: report at bugs.python.org (Tal Einat)
Date: Thu, 30 Aug 2018 12:39:32 +0000
Subject: [New-bugs-announce] [issue34548] IDLE: Make TextView use the
 configured theme colors
Message-ID: <1535632772.52.0.56676864532.issue34548@psf.upfronthosting.co.za>


Change by Tal Einat <taleinat at gmail.com>:


----------
assignee: taleinat
components: IDLE
nosy: taleinat, terry.reedy
priority: low
severity: normal
status: open
title: IDLE: Make TextView use the configured theme colors
versions: Python 2.7, Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34548>
_______________________________________

From report at bugs.python.org  Thu Aug 30 10:48:07 2018
From: report at bugs.python.org (Nick)
Date: Thu, 30 Aug 2018 14:48:07 +0000
Subject: [New-bugs-announce] [issue34549] unittest docs could use another
 header
Message-ID: <1535640487.61.0.56676864532.issue34549@psf.upfronthosting.co.za>


New submission from Nick <manifes7o at gmail.com>:

I find myself reaching for a list of the assert methods in the unittest library often. There are several methods but no clear way to link a URL via a header if you intend to bookmark or send it out.

I have been using the method above the section in the meantime

https://docs.python.org/3.7/library/unittest.html#unittest.TestCase.debug

Would be great to have an `Assert Methods` header or some other equivalent.

Thank you!

----------
assignee: docs at python
components: Documentation
messages: 324385
nosy: docs at python, napsterinblue
priority: normal
severity: normal
status: open
title: unittest docs could use another header
type: enhancement
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34549>
_______________________________________

From report at bugs.python.org  Thu Aug 30 11:55:48 2018
From: report at bugs.python.org (Tao Chen)
Date: Thu, 30 Aug 2018 15:55:48 +0000
Subject: [New-bugs-announce] [issue34550] UnicodeDecodeError when invoke
 method configure() of Menu instance
Message-ID: <1535644548.51.0.56676864532.issue34550@psf.upfronthosting.co.za>


New submission from Tao Chen <sbellct at 163.com>:

[Brief Description]
Python 3.7.0 IDLE. After create one Menu instance, and invoke it's config() or configure() method without any parameter, python print UnicodeDecodeError.


[Walk around]
menubar = Menu(root)
menubar.config(font=('Arial'))



[LOG]
Python 3.7.0 (v3.7.0:1bf9cc5093, Jun 27 2018, 04:59:51) [MSC v.1914 64 bit (AMD64)] on win32
Type "copyright", "credits" or "license()" for more information.
>>> from tkinter import *
>>> root = Tk()
>>> menubar = Menu(root)
>>> menubar.config()
Traceback (most recent call last):
  File "<pyshell#3>", line 1, in <module>
    menubar.config()
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xc8 in position 2: invalid continuation byte


>>> menubar.cget('font')
Traceback (most recent call last):
  File "<pyshell#15>", line 1, in <module>
    menubar.cget('font')
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xc8 in position 2: invalid continuation byte

>>> platform.uname()
uname_result(system='Windows', node='xxx', release='7', version='6.1.7601', machine='AMD64', processor='Intel64 Family 6 Model 78 Stepping 3, GenuineIntel')

----------
components: Tkinter
messages: 324389
nosy: sbellct
priority: normal
severity: normal
status: open
title: UnicodeDecodeError when invoke method configure() of Menu instance
type: resource usage
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34550>
_______________________________________

From report at bugs.python.org  Thu Aug 30 13:27:21 2018
From: report at bugs.python.org (Eric Lippert)
Date: Thu, 30 Aug 2018 17:27:21 +0000
Subject: [New-bugs-announce] [issue34551] Redundant store can be removed
 from _PyFunction_FastCallDict
Message-ID: <1535650041.35.0.56676864532.issue34551@psf.upfronthosting.co.za>


New submission from Eric Lippert <ericlippert at gmail.com>:

In _PyFunction_FastCallDict we have local nk assigned to be the size of a dictionary, and then local i is assigned to twice the size of the same dictionary, and then nk is assigned to half of i, which it already is:

   nk = (kwargs != NULL) ? PyDict_GET_SIZE(kwargs) : 0;
   if (nk != 0) {
     ...
     pos = i = 0;
     while (PyDict_Next(kwargs, &pos, &k[i], &k[i+1])) {
       ...
       i += 2;
     }
     nk = i / 2;

I am attempting to understand the performance characteristics of this hot path, and I spent far too long trying to figure out why nk was being assigned a value it already has. :)

I propose that the redundant store be replaced with an assertion that i/2 is equal to nk.

I will submit a pull request presently.

----------
components: Interpreter Core
messages: 324395
nosy: Eric Lippert
priority: normal
severity: normal
status: open
title: Redundant store can be removed from _PyFunction_FastCallDict
type: enhancement
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34551>
_______________________________________

From report at bugs.python.org  Fri Aug 31 01:37:52 2018
From: report at bugs.python.org (Windson Yang)
Date: Fri, 31 Aug 2018 05:37:52 +0000
Subject: [New-bugs-announce] [issue34552] Clarify built-in types comparisons
Message-ID: <1535693872.67.0.56676864532.issue34552@psf.upfronthosting.co.za>


New submission from Windson Yang <wiwindson at gmail.com>:

In https://docs.python.org/3.5/library/stdtypes.html#comparisons 

"some types (for example, function objects) support only a degenerate notion of comparison where any two objects of that type are unequal."

We had some discussion at zulipchat already:

"more seriously: I think we can just delete that sentence. The next sentence makes clear that < and friends arne't always defined, and the sentence after that notes that == defaults to is if there's nothing better." - by Nathaniel J. Smith

IMO, I think we should also clarify the relationship between "==", "is" and "__eq__".

----------
assignee: docs at python
components: Documentation
messages: 324403
nosy: Windson Yang, docs at python, gvanrossum, njs, zach.ware
priority: normal
severity: normal
status: open
title: Clarify built-in types comparisons
type: enhancement
versions: Python 2.7, Python 3.4, Python 3.5, Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34552>
_______________________________________

From report at bugs.python.org  Fri Aug 31 03:47:43 2018
From: report at bugs.python.org (Vijay)
Date: Fri, 31 Aug 2018 07:47:43 +0000
Subject: [New-bugs-announce] [issue34553] Python Crashes when trying to
 access any date related fields in MailItem
Message-ID: <1535701663.99.0.56676864532.issue34553@psf.upfronthosting.co.za>


New submission from Vijay <vijay.ramdass at gmail.com>:

MailItem.SentOn or MailItem.ReceivedTime, Python crashes with windows showing "Python has stopped working" dialog window. What can be the cause of this issue?

import win32com.client
import os
import datetime
from datetime import timedelta
outlook = win32com.client.Dispatch("Outlook.Application").GetNamespace("MAPI")
inbox = outlook.GetDefaultFolder(6)
messages=inbox.Items
message = messages.GetLast()

print(type(message))

t = message.ReceivedTime   ---- crashes when the script is reached this line.

Attached the screenshot for your reference.

----------
files: python crash.PNG
messages: 324406
nosy: vijay
priority: normal
severity: normal
status: open
title: Python Crashes when trying to access any date related fields in MailItem
type: crash
versions: Python 3.7
Added file: https://bugs.python.org/file47774/python crash.PNG

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34553>
_______________________________________

From report at bugs.python.org  Fri Aug 31 10:45:23 2018
From: report at bugs.python.org (mokhalid)
Date: Fri, 31 Aug 2018 14:45:23 +0000
Subject: [New-bugs-announce] [issue34554] add match built in function
Message-ID: <1535726723.83.0.56676864532.issue34554@psf.upfronthosting.co.za>


New submission from mokhalid <mohammedkhaildson at gmail.com>:

match built-in function feature request:

what is the match function do?
the match function is doing something like if but with quick easy syntax that even beginners can use.

possible syntax:

match(list or string or dictionary):
  condition: event here
  condition: event here
#all conditions are applied at the same time as if not elif and else
#but you use elif or else.

ex:
match(list or string or dictionary):
  condition: event here
  condition: event here
    elif condition: event here
    else: event here
 

example of what I mean:
match([2,5,4,'Hello']):
  2: print('here is 2')
  6: print('here is 6')
    elif 'Hello': ('Hello world')
  8:print('here is 8')
    elif 'python': ('Hello world')
    else: print('MoKhalid')

the output should look like:
here is 2
Hello world
MoKhalid

----------
components: Regular Expressions
messages: 324425
nosy: MoKhaild, ezio.melotti, mrabarnett
priority: normal
severity: normal
status: open
title: add match built in function
type: enhancement
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34554>
_______________________________________

From report at bugs.python.org  Fri Aug 31 11:12:14 2018
From: report at bugs.python.org (Thomas Herzog)
Date: Fri, 31 Aug 2018 15:12:14 +0000
Subject: [New-bugs-announce] [issue34555] AF_VSOCK not unset because of
 wrong nesting
Message-ID: <1535728334.14.0.56676864532.issue34555@psf.upfronthosting.co.za>


New submission from Thomas Herzog <tom at therzog.net>:

If ./configure runs with the following result...

checking for linux/vm_sockets.h... no
checking for sockaddr_alg... no

...then the result of the first check is treated as if it was "yes". This is because the logic for disabling the vm_sockets functionality is nested inside an #ifdef HAVE_SOCKADDR_ALG.

This leads to compilation errors:

In file included from ./Modules/socketmodule.c:283:0:
./Modules/socketmodule.h:206:24: error: field ?vm? has incomplete type
     struct sockaddr_vm vm;
                        ^

----------
components: Extension Modules
messages: 324427
nosy: mcduke
priority: normal
severity: normal
status: open
title: AF_VSOCK not unset because of wrong nesting
type: compile error
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34555>
_______________________________________

From report at bugs.python.org  Fri Aug 31 11:25:51 2018
From: report at bugs.python.org (Cooper Lees)
Date: Fri, 31 Aug 2018 15:25:51 +0000
Subject: [New-bugs-announce] [issue34556] Add --upgrade to venv module
Message-ID: <1535729151.14.0.56676864532.issue34556@psf.upfronthosting.co.za>


New submission from Cooper Lees <me at cooperlees.com>:

I'd like to propose add a `--upgrade` to venv module to automatically update pip and potentially setuptools if supplied. This will allow ensuring you have the latest and greatest of these core components.

Example Usage:
python3 -m venv --upgrade /tmp/awesome_venv

This will allow me to skip my next command that I usually always do:
/tmp/awesome_venv/bin/pip install --upgrade pip setuptools

Thoughts? If people are happy, I will look at doing the PR. 

I would envision I'll either import (which seems to be a no no from pip maintainers) the upgrade code or just fork pip ...

----------
components: Library (Lib)
messages: 324431
nosy: cooperlees
priority: normal
severity: normal
status: open
title: Add --upgrade to venv module
type: enhancement
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34556>
_______________________________________

From report at bugs.python.org  Fri Aug 31 14:00:53 2018
From: report at bugs.python.org (James Campbell)
Date: Fri, 31 Aug 2018 18:00:53 +0000
Subject: [New-bugs-announce] [issue34557] When sending binary file to a
 Microsoft FTP server over FTP TLS, the SSL unwind method hangs
Message-ID: <1535738453.08.0.56676864532.issue34557@psf.upfronthosting.co.za>


New submission from James Campbell <james at printtapp.com>:

When using the FTP library to transfer a binary file to a Microsoft FTP server using TLS, then the library will hang when unwinding the connection until it finally times out.

The storbinary method calls conn.unwind which seems to have an issue with SSL connections with a Microsoft server. If we terminate the connection early the file is successfully transferred so it's just the unwind procedure that crashes and hangs our server until it times out.


We are able to work around it by creating our own version of the storbinary method which just closes the connection and doesn't do the unwind step.

It's not clear why the library does this step since we never need to drop down to an unencrypted connection so it should be enough to just close it once done.

You can read more information on this by somebody else with Python 3.2
http://www.sami-lehtinen.net/blog/python-32-ms-ftps-ssl-tls-lockup-fix

----------
components: Library (Lib)
messages: 324440
nosy: James Campbell2
priority: normal
severity: normal
status: open
title: When sending binary file to a Microsoft FTP server over FTP TLS, the SSL unwind method hangs
type: crash
versions: Python 2.7, Python 3.4, Python 3.5, Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34557>
_______________________________________

From report at bugs.python.org  Fri Aug 31 15:24:42 2018
From: report at bugs.python.org (Michael Felt)
Date: Fri, 31 Aug 2018 19:24:42 +0000
Subject: [New-bugs-announce] [issue34558] ctypes.find_library processing -
 missing parenthesis prevents subprocess from getting a returncode
Message-ID: <1535743482.34.0.56676864532.issue34558@psf.upfronthosting.co.za>


New submission from Michael Felt <aixtools at felt.demon.nl>:

With AIX I have seen the following messages repeatedly:

root at x066:[/data/prj/python/python3-3.8]./python ../git/*3.8/Lib/ctypes/test/test_loading.py
/data/prj/python/git/python3-3.8/Lib/subprocess.py:852: ResourceWarning: subprocess 11796734 is still running
  ResourceWarning, source=self)
/data/prj/python/git/python3-3.8/Lib/subprocess.py:852: ResourceWarning: subprocess 11796480 is still running
  ResourceWarning, source=self)

There is python2 to python3 missed change typo that is preventing the returncode from ever being determined

diff --git a/Lib/ctypes/_aix.py b/Lib/ctypes/_aix.py
index 463f60a284..190cac6507 100644
--- a/Lib/ctypes/_aix.py
+++ b/Lib/ctypes/_aix.py
@@ -115,7 +115,7 @@ def get_ld_headers(file):
         else:
             break
     p.stdout.close()
-    p.wait
+    p.wait()
     return ldr_headers

 def get_shared(ld_headers):

PR will be posted.

----------
components: Library (Lib), Tests
messages: 324441
nosy: Michael.Felt
priority: normal
severity: normal
status: open
title: ctypes.find_library processing - missing parenthesis prevents subprocess from getting a returncode
type: behavior
versions: Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34558>
_______________________________________

From report at bugs.python.org  Fri Aug 31 18:16:52 2018
From: report at bugs.python.org (natedogith1)
Date: Fri, 31 Aug 2018 22:16:52 +0000
Subject: [New-bugs-announce] [issue34559] multiprocessing
 AuthenticationError when nesting with non-default authkey
Message-ID: <1535753812.06.0.56676864532.issue34559@psf.upfronthosting.co.za>


New submission from natedogith1 <natedogith1 at gmail.com>:

If you nest shared objects on a manager that doesn't use the default authkey, you get an AuthenticationError.

import multiprocessing.managers
a = multiprocessing.managers.SyncManager(authkey=b'')
a.start()
b = a.list()
b.append(a.list())
_ = b[0]

----------
components: Library (Lib)
messages: 324447
nosy: natedogith1
priority: normal
severity: normal
status: open
title: multiprocessing AuthenticationError when nesting with non-default authkey
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34559>
_______________________________________

From report at bugs.python.org  Fri Aug 31 18:33:42 2018
From: report at bugs.python.org (Riccardo Mottola)
Date: Fri, 31 Aug 2018 22:33:42 +0000
Subject: [New-bugs-announce] [issue34560] Backport of uuid1() failure fix
Message-ID: <1535754822.12.0.56676864532.issue34560@psf.upfronthosting.co.za>


New submission from Riccardo Mottola <multix at gmail.com>:

Backport proposal of fix for closed issue 32502 

The patch was directly adapted from
https://github.com/python/cpython/commit/d69794f4df81de731cc66dc82136e28bee691e1e


applied to 2.7 and tested working for me.

Discussion ongoing MacPorts requested to share said patch upstream

https://github.com/macports/macports-ports/pull/2456

----------
components: Library (Lib)
files: uuid-64bit.patch
keywords: patch
messages: 324449
nosy: Riccardo Mottola
priority: normal
severity: normal
status: open
title: Backport of uuid1() failure fix
type: crash
versions: Python 2.7
Added file: https://bugs.python.org/file47776/uuid-64bit.patch

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34560>
_______________________________________

From report at bugs.python.org  Fri Aug 31 21:22:04 2018
From: report at bugs.python.org (Tim Peters)
Date: Sat, 01 Sep 2018 01:22:04 +0000
Subject: [New-bugs-announce] [issue34561] Replace list sorting
 merge_collapse()?
Message-ID: <1535764924.6.0.56676864532.issue34561@psf.upfronthosting.co.za>


New submission from Tim Peters <tim at python.org>:

The invariants on the run-length stack are uncomfortably subtle.  There was a flap a while back when an attempt at a formal correctness proof uncovered that the _intended_ invariants weren't always maintained.  That was easily repaired (as the researchers suggested), but it remains hard to grasp why it's always correct now.  The Java variant didn't take the suggested fix, instead boosting its stack size.  But it turns out that's still not enough, because the original researchers made a different mistake in their own paper ;-)

http://drops.dagstuhl.de/opus/volltexte/2018/9467/

Buss & Knop recently published results on different ways of managing the stack, and I think they're worth investigating for "real life" use:

https://arxiv.org/abs/1801.04641

Offhand, their "2-merge" looks quite appealing to me.  The only invariant it maintains is that each run on the stack is at least twice the length of the run one above it on the stack, which can be maintained just by checking the topmost two stack entries in the loop.  So, e.g., where merge_collapse() is currently happy with runs of lengths 150 and 100 ending the stack, 2-merge would force a merge (it's not the case that 150 >= 2*100).  Both are happy if the stack ends with runs of lengths 200 and 100.

This is much easier to reason about, simpler to code, and would allow reducing the size of the stack (worst-case size is proportional to the log (of the largest possible array) to the base 2 instead of to the current base phi (~1.62)).  They developed some formal measures showing that its "merge cost" (an overall measure abstracting some from real-life behavior) is significantly better than timsort's current strategy.  And a little thought convinced me it preserves that, for random data, it would continue to strongly tend towared triggering perfectly balanced merges (merging runs of the same size).

When I was developing this code, virtually all the time was spent making merging itself as fast as possible; the stack-management code was just the first thing I tried that "worked well".  But turns out that's the only part researchers care about ;-) I'd be pleased if it could be replaced with something both better and simpler, or even just simpler but no worse.  But the devil's in the details ...

----------
messages: 324455
nosy: tim.peters
priority: normal
severity: normal
stage: needs patch
status: open
title: Replace list sorting merge_collapse()?
type: performance
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34561>
_______________________________________